← All resources
Cloud Platform

Reproducible Infrastructure

Time to teardown as a maturity model: five levels from hand-built pets to environments you can destroy and rebuild from code, and the shortest path between them.

Read the full white paper

Tell us where to send it. No spam, just the paper.

Thanks. Enjoy the paper.

By submitting you agree we may email you about Simcha Solutions. Unsubscribe anytime.

Ask a team how available their cloud environment is and you will get a number. Ask how long it would take to rebuild that environment from nothing, and you will usually get a pause. That pause is the most honest signal in infrastructure, because the ability to tear an environment down and stand it back up from code is a proxy for everything else you want to be true: that the infrastructure really is defined in version control, that the data has a tested restore path, that there are no hand-applied fixes and no server only one person understands.

We call the metric time to teardown: the elapsed time from destroying an environment to serving traffic again from a fresh build. Nobody tears down production for sport. The point is that the capability, measured occasionally and honestly, forces every hidden dependency into the open. This paper turns that metric into a maturity model and a practical sequence for climbing it.

Why drift is the enemy

Environments rot in a specific way. A hotfix is applied directly in production during an incident. A staging database gets an index that never makes it into the migration history. A security patch is applied by hand on two of three hosts. Each change is small, sensible, and unrecorded, and each one widens the gap between what the code says the environment is and what the environment actually is.

Drift is invisible until it bites, and it bites at the worst moments: the deploy that worked in staging fails in production; the disaster-recovery runbook restores a system that no longer matches reality; the rebuild that should take an hour takes a week of archaeology. Reproducibility is not an aesthetic preference. It is the property that makes recovery, scaling, and change boring.

The maturity model

LevelNameReality on the groundTime to teardown
1PetsHand-built servers, tribal knowledge, console changesUnknown, effectively never
2Documented petsRunbooks describe the hand-built stateDays to weeks, high risk
3Partial codeCore stack in IaC, edges still hand-managedDays, with surprises
4RebuildableEverything in code, restores tested occasionallyHours
5Routinely rebuiltEphemeral environments, restores exercised on scheduleMinutes to an hour, proven

Two observations from applying this model. First, most teams believe they are a level higher than they are, because belief is measured against the happy path and teardown is measured against reality. Second, the expensive jump is not adopting infrastructure-as-code; it is closing the last twenty percent: DNS zones, secrets, IAM grants, TLS material, the queue configuration someone clicked into existence, and above all the data layer.

Climbing: the sequence that works

Inventory the unmanaged edges. Diff what the code declares against what the cloud account contains. Every resource that exists only in the console is a rebuild landmine. The tooling for this is mundane; the discipline of running it is the hard part.

Bring the edges under code, ugliest first. Secrets management, IAM, DNS, and certificates are usually the stragglers, and they are exactly the pieces that stall a rebuild at hour two. Import them, or recreate them through code and retire the originals.

Treat data as part of the environment. Compute is easy to rebuild; state is not. A reproducible environment needs automated backups plus a tested restore path with a measured duration. An untested backup is a hope, not a capability. Restore into a scratch environment on a schedule and record how long it took.

Kill the manual step, not the person who documents it. Every runbook line that says "then click" is future drift. The runbook for a rebuilt environment should converge toward a single entry point: run the pipeline, wait, verify.

Rebuild something regularly. Ephemeral preview and staging environments are the forcing function: if staging is destroyed and recreated weekly, staging cannot drift, and the rebuild path stays warm. Production earns the same confidence by inheritance, because it is built from the same code paths that are exercised constantly.

The quarterly question Once a quarter, answer in writing: if this environment vanished now, how long to serve traffic again, and what would we lose. The first honest answer is usually uncomfortable. The trend line matters more than the starting point.

What changes at the top

Teams at level four and five stop having a class of incident. Configuration mysteries shrink because configuration has one source of truth. Disaster recovery becomes an engineering exercise with a measured duration instead of a leap of faith. Onboarding accelerates because the environment is legible from the repository. Security reviews get easier because what runs is what is written.

And a quieter benefit compounds underneath: change gets cheaper. When rebuilding is safe, experiments are safe. When environments are disposable, upgrades stop being feared. The teams that can tear it all down are, not coincidentally, the teams that ship the fastest.

You do not have to rebuild your world every week. You do have to know, with evidence, that you could.