Deterministic Simulation Testing Is For Everyone

20 Nov 2026

Deterministic Simulation Testing is often seen as a technique just for distributed systems. I argue that it's a testing technique for everyone and all applications.

Why We Test

Why do we use automated testing when building software?

To Prevent Bugs

We test because we want to find issues internally and not have them be found for us by users (who may not even report them at all).

We do this reactively; when an implementation bug is found, a failing test we think corresponds to the bug is written, and the system is changed to make the test pass. If the bug can no longer be reproduced, we have proven to ourselves that the test corresponds to the bug, and we keep the test to guard against further regressions.

We can also do this defensively; predicting where and what the bugs might be, and using tests as guardrails ensuring they do not occur in development.

To Remove Doubt

We test because we are plagued with self-doubt.

Doubting every development decision we make is exhausting. Can this feature be added? Can this dependency be removed? Can these modules be re-arranged? Can this all be done without breaking what we already have? Second guessing brings decision fatigue. Doubt kills productivity. If we are confident the machine is checking this for us, then our confidence grows.

To Fulfil Obligations

We test because we ought to test, in the same way a small child brushes their teeth - they may not see the point but they know someone else will make their lives hard if they don't.

In Test-Driven Development, each bit of code must be preceded by a failing test. Tests are written under TDD because without them nothing at all can be written. It follows that these tests tend to be numerous, brittle, numerous, limited in utility and numerous. Obligation is an external pressure, and if that external pressure is gone (i.e., the senior TDD champion leaves the company), the obligation leaves with it.

I see limited utility in testing processes driven by obligation.

The Elements of a Test

System Under Test (SUT)
The unit of code under test.
Test Logic
Assertions and associated setup which check that properties are upheld.
Test Seed
Used to seed any random values needed for the test. By default this is chosen randomly, but can be overridden by the user to replay specific test runs.

What Makes a Test Useful?

The utility of a test can be evaluated across three dimensions.

Scope

Tests have a certain scope. This scope may be anything from an individual function to an entire program. The greater the scope, the more interesting the properties we can reason about. If the scope is large enough, properties start corresponding to the requirements; that is we can test things relevant to the real world.

Determinism

A test is run many times. A non-deterministic or "flaky" test can produce different results even when the SUT, test logic, and test seed remain the same. Such tests are incredibly frustrating for the developer - how can they tell if a bug is fixed if their tests don't produce consistent results?

If a test is deterministic, then the same code with the same test will return the same failure. Always.

Thoroughness

In some tests, specific examples are thought up by the programmer, and specific states or responses are tested for. In others the inputs to the SUT are described declaratively and generated randomly, so that each run generates multiple different examples. This lets us explore a lot more of the state space. Let's call these tests thorough.

Testing Approaches

Different testing approaches ask different kinds of questions:

End to End Testing
Can a user login, sign up, and place an order?
Unit Testing
Does 1 + 1 = 2?
Property-Based Testing
Does n + 0 = n hold for every integer n?
Black-Box Fuzzing
Can any input crash the SUT?
Deterministic Simulation Testing
If we simulate running this for a year, will our properties hold?

The different testing approaches can be analysed according to the criteria discussed above:

Widely Scoped Deterministic Thorough
End-to-End Testing Yes Almost never; most systems do IO internally No
Unit Testing No Deterministic units considered a best practice No
Property-Based Testing Usually no, though scope can widen using model based PBT, or if the pure function is large enough. SUT is typically deterministic; PBT comes from the FP world. Yes
Black-Box Fuzzing Yes Only if the SUT is deterministic for all inputs A black-box cannot bias interesting states.
Deterministic Simulation Testing Most of the program Yes Yes

Deterministic Simulation Testing

In the above analysis Deterministic Simulation Testing (henceforth DST) stands out. So how does it work? In short they combine a simulated environment with a deterministic machine. Let's look at these terms in detail.

Deterministic

The whole program, hence the SUT, is a deterministic machine. This means it's single-threaded (or at least deterministically scheduled) and does not make direct IO calls; so you're left with input-driven deterministic state machines, making the same state transitions for the same inputs every single time.

If you're interested in how to do this, see my previous article on designing programs around a deterministic core .

Simulation

The environment is simulated. This includes all relevant real world phenomena that are shared with the deterministic machine, for example:

Simulating the environment turns a source of non-determinism into something deterministic. Given the same seed, the simulated environment always produces the same outputs. It also allows you to dial up the chaos: weird inputs, failures and coincidences that will "never happen" can all be dialled up. A good simulation is an exercise in amplifying the deviousness of those parts of the environment we cannot control.

Testing

Typically the test is its own executable. A seed is provided, and a test loop is run using the simulated environment and the deterministic machine, checking that properties hold each loop. If a property does not hold the loop exits and the executable returns the failing seed. This makes errors reproducible.

Some DST implementations act like a game loop, using ticks to simulate time; just as an "in-game" day might take 24 minutes of real time to represent 24 hours of game time, a simulation test might take 1 month of sim time but complete in one hour of real time. Others model progression through the test as distinct events, jumping from one to the next. In either case, because all network and storage delay is simulated, the test never has to truly wait, and things can progress much faster than they would under a real environment. This minimises "time to bug", in effect compressing time, and surfacing issues far more quickly than it would ever take them to appear naturally in production.

Isn't This Just for Distributed Systems?

When people think of DST, a few products come to mind: Dropbox, FoundationDB, TigerBeetle, Turso; all very clever systems engineering products made by people who are essentially wizards.

But conceptually, all software programs have an environment and a machine. If you make the machine deterministic, and you define an interface for the environment so it can be simulated, then you can do DST.

Here are some examples of how DST can apply to different kinds of software:

Simulated Environment Deterministic Machine Properties
Distributed Databases Physical Time, Network, Storage, Crashes Sync Algorithm, Consensus Protocol, Storage Engine Consistency level upheld, handles network splits and replica crashes.
Web Applications User actions (clicking, typing, navigation), third party API responses Business Logic, View State, Routing Can't check out empty cart, prices aren't negative, fields can't be empty.
Embedded Systems Sensors, Interrupts, Timers Control loop, State machines Fail safe on sensor faults, real-time deadlines met.

DST Is Inevitable

DST is not driven by obligation, it does not plague developers with doubt; it lets them build without fear, by finding as many bugs in development as possible. Development is the best place to find bugs.

DST improves architecture by maximising the amount of code in the deterministic context, and carefully controlling exposure to the environment.

DST is under your control; you do not have to rely on someone else to simulate the entire world for you - just the environment you care about.

My goal in this article is not to dissuade you from unit tests, property-based tests, or any other testing technique. My goal is to persuade you that DST is relevant for you and your team. Don't let a technique being associated with one domain stop you from considering using it in yours.