Parallel QA environments for two releases in the same week
A hotfix for the release customers run arrives on Tuesday, while the next release is partway through its regression cycle on the team's one QA stack. Loading the hotfix build onto that stack ends the cycle in progress. Holding the hotfix until the cycle closes leaves customers on the bug for another week. A second QA stack lets both tests run, as long as the two stacks share nothing that keeps state. The database comes first, and six smaller items follow it.
One schema version per database
Take a hypothetical vendor of permit tracking software for county offices, Dunmoor Permit Systems. Its customers run release 8.3. Release 9.0 is in regression, and it carries schema migrations that 8.3 does not contain.
Migration tools keep their own record inside the database they manage. The Rails guide says Rails tracks which migrations have run through the schema_migrations table in the database. Flyway's documentation says Flyway adds a schema history table to the schema, holding each applied migration with its checksum and whether it succeeded. A database migrated for 9.0 therefore belongs to the 9.0 build. Pointing the 8.3 hotfix build at it tests 8.3 code against 9.0 tables, a combination no customer runs.
Rolling one shared database back for the hotfix and forward again for 9.0 costs a restore in each direction. The first restore discards whatever the regression cycle had built up, including the test accounts created along the way and the failures waiting for a retest.
Give the hotfix stack a database server of its own, loaded with the base dataset at the 8.3 schema. That dataset has to exist. Keep the seed script or backup for each release customers still run beside the build for that release. Reproduce a customer bug on their exact version covers what to archive. For one of our customers, a QA stack is an application server and a SQL server, reachable from outside so a tester can open the application in a browser and run it in test mode. A second stack for that team is a second pair of servers.
What each stack keeps to itself
Two stacks built in a hurry end up tied together through whatever was quickest to reuse. Check these six before the first tester logs in.
| Shared item | What goes wrong | What each stack gets |
|---|---|---|
| Cache and job queue | A worker on the 9.0 stack picks up a job the 8.3 stack queued, in a format it was never built to read | Its own cache and queue, or its own namespace on a shared one |
| Payment sandbox | Test customers and webhook events from both runs land in one account, and a tester cannot tell which stack created a charge | Its own sandbox. Stripe's documentation calls a sandbox an isolated test environment and says data and actions in one are completely isolated from other sandboxes |
| Sign in callback | RFC 9700, the IETF's current security practice for OAuth 2.0, requires an authorization server to compare redirect addresses by exact string match. A stack whose callback is unregistered is refused at sign in, and a stack that reuses the first stack's address sends its testers to the first stack | Its own hostname, with that hostname's callback registered at the identity provider |
| Scheduled jobs | Both stacks poll one inbox or one file drop, and whichever runs first takes the file | Its own inbox and drop folder, or the job switched off on one stack |
| Outbound email | Messages from both builds mix in one inbox, and a tester checks a notice the other build sent | A mail catcher inside each stack |
| Licensed components | A second installation of another vendor's component may need its own license or activation, and that vendor's terms decide | A read of the license terms before the second stack is built |
Some partners issue a single sandbox credential or allowlist one address. For that integration the two stacks take turns, and the turn schedule goes in the test plan so a failure can be traced to the build that caused it.
Label every result with its stack and build
Once two stacks are up, a bug report that says "QA" could mean either one. Put the release where nobody has to ask for it.
The application shows its version and commit on the sign in page or an about page, written there by the build. Each stack has a hostname that carries the release, such as qa-83 and qa-90 in this example. The name of the test run in the team's test management tool repeats both.
A failure logged against 9.0 can then be retried on 8.3. If it fails there too, the bug is already in production and belongs in the hotfix.
Two builds on one approved configuration
On Petrichor, we rebuild a QA stack from your saved environment configuration before each test cycle, and nobody changes that configuration without your QA lead's approval. Parallel stacks are among the setups most teams start with, so two releases can test at once.
The second stack should differ from the first in the application build and in nothing the team did not choose. Building both from one configuration does that. Our provisioning tool builds a stack in a fixed order: servers with the right operating systems, the installed software, the environment settings, your application, then the base users and dataset. For the hotfix stack, the last two steps take the 8.3 build and the 8.3 dataset, and the steps before them stay as approved.
Anything that has to differ, such as the older schema or a sandbox key, goes into the written spec, which covers servers, network, data, access and how long the environment should live. A hotfix stack can be the short lived kind. Some environments stay up all year, and others live for one release and come down when testing ends.
A new environment starts with a ticket and an agreed spec. Both stacks sit in your own network segment, which every customer gets. Our engineers can gate access by team, so a Windows test team sees only Windows stacks.
When one stack and a reset are enough
A second stack is a second set of servers to keep patched, and it earns that upkeep only when two builds are under test in the same days.
Teams that ship from one main branch behind feature flags, and support only the current release, have no older build to hotfix. Where the regression cycle finishes inside a day, the hotfix can wait for it. If releases never overlap, one stack can be reset between them, and we set up a one step reset back to baseline for that case so nobody rebuilds by hand. Resetting a QA environment between test runs compares the reset methods.
Parallel stacks also leave the head count where it was. The same testers now cover two cycles, and somebody still has to decide which stack gets them first.
Talk to a consultant and bring next quarter's release calendar, with each week where two builds overlap marked.