The Power of Property-Based Testing

17 Aug 2026

(First version originally posted on my homepage)

I'm somewhat obsessed with randomised testing. AFAIK, I'm the first consultant to do Deterministic Simulation Testing (DST) for a client. But when I am approached about implementing DST we quickly realize their existing codebase isn't ready for it; DST requires significant refactoring to isolate testable components from external dependencies.

A much easier starting point is Property-Based Testing, which is more amenable to smaller slices of deterministic code (smaller slices are about all you get in legacy codebases). While narrower in scope than DST it offers massive benefits over traditional example-based tests, and is relatively easy to get started.

Definitions

Let's first define a unit test as a test of internal logic that does not interact with, mock, or simulate anything external like a database or remote API. (For some this may overlap with your definition of integration test; I ask you to bear with me)

Unit tests are typically example-based - they use concrete actual and expected values. An example test might look something like this: 

expect(1 + 1).toBe(2)
expect(2 + 2).toBe(4)
expect(1**2).toBe(1)
expect(sin(0)).toBe(0)
expect(cos(0)).toBe(1)

Something that generalises examples is called a Property-based test, or PBT. Instead of giving hand-picked examples, you instead describe the kinds of examples you are interested in. The test runner then uses a seeded RNG value to sample from the range of values you specify, and checks if any trigger your assertions. 

assert(
  property(double(), n => 0 + n === n)
)

assert(
  property(double(), n => 1**n === 1)
)

assert(
  property(double(), t => sin(t)**2 + cos(t)**2 === 1)
)

assert(
  property(double(), x => -1 <= sin(x) && sin(x) <= 1)
)

assert(
  property(
    tuple(double(), double(), double()),
    ([x, y, z]) => (x * y) * z === x * (y * z)
  )
)

A mathematical analogy

I've chosen mathematical tests deliberately (and I hope keener eyed readers will forgive me playing fast and loose with floating point issues in the service of concise examples). Example based tests are statements without variables, while property based tests are statements with variables. The variables themselves are sampled randomly; generated for each test run (seeded so a test can be repeated).

A Business Logic Example

Let's look beyond basic mathematics, to a shopping cart implementation in Typescript. 

export type Item = {
  id: string
  name: string
  price: number
  quantity: number
}

export class ShoppingCart {
  #items: Map = new Map()

  add(item: Item): void {
    const existing = this.#items.get(item.id)
    if (existing) {
      existing.quantity += item.quantity
    } else {
      this.#items.set(item.id, item)
    }
  }

  remove(id: string, quantity: number): void {
    const item = this.#items.get(id)
    if (!item) return

    item.quantity -= quantity
    if (item.quantity <= 0) {
      this.#items.delete(id)
    }
  }

  getItem(id: string): Item | undefined {
    return this.#items.get(id)
  }

  get items(): MapIterator {
    return this.#items.values()
  }

  get totalPrice(): number {
    return this.items.reduce(
      (price, item) => price + item.price * item.quantity,
      0,
    )
  }

  get itemCount(): number {
    return this.items.reduce(
      (total, item) => total + item.quantity,
      0,
    )
  }

  clear(): void {
    this.#items.clear()
  }

  get empty(): boolean {
    return this.#items.size === 0
  }
}

And here's some example based unit tests to verify the behaviour (imports elided for brevity): 

test("adding items to the cart", () => {
  let cart = new ShoppingCart()
  const item = {
    id: "1",
    name: "Apple",
    price: 1.5,
    quantity: 2 
  }
  cart.add(item)
  expect(cart.getItem("1")).toEqual(item)
})

test("calculating totals", () => {
  let cart = new ShoppingCart()
  cart.add({
    id: "1",
    name: "Apple",
    price: 1.5, 
    quantity: 2
  })
  cart.add({
    id: "2", 
    name: "Banana",
    price: 0.75,
    quantity: 3
  })
  expect(cart.totalPrice).toBe(5.25)
})

test("removing items from cart", () => {
  let cart = new ShoppingCart()
  cart.add({
    id: "1",
    name: "Apple",
    price: 1.5,
    quantity: 3
  })
  cart.remove("1", 1)
  expect(cart.getItem("1")?.quantity).toBe(2)
})

test("empty cart should behave like it's empty", () => {
  let cart = new ShoppingCart()
  expect(cart.empty).toBe(true)
  expect(cart.totalPrice).toBe(0)
  expect(cart.itemCount).toBe(0)
})

Wow, static typing AND unit tests? This code is indestructible... or is it!?

Enter Property-Based Testing

One of the great things about PBT is how they immediately demonstrate how broken almost any piece of code is. Here we use fast-check, a property based testing library for JS/TS, to define some tests: 

const arbItem: fc.Arbitrary = fc.record({
  id: fc.string(),
  name: fc.string(),
  price: fc.double({ noDefaultInfinity: false }),
  quantity: fc.double({ noDefaultInfinity: false }),
})

test("total price should never be negative", () => {
  fc.assert(
    fc.property(fc.array(arbItem), items => {
      const cart = new ShoppingCart()

      for (const item of items) {
        cart.add(item)
      }

      expect(cart.totalPrice).toBeGreaterThanOrEqual(0)
    })
  )
})

The test - which generates a random array of random Items - checks if the following property holds true: is the total price of the shopping cart always greater than or equal to 0?

It does not! Our PBT finds a simple example of an input that will trigger a failure: 

{
  "id": "",
  "name": "",
  "price": -1.1890419245983166e-66,
  "quantity": 2.0775787447871207e-258
}

Our shopping cart has no defense against items with negative prices.

Note how "weird" the example is. Machines are fantastic at thinking up inputs we ourselves would not even think of. Most PBT libraries will heavily bias weird values (tiny numbers, zero, empty strings etc) that show up in bug reports.

Let's update the add to method reject negative prices. And being clever, let's also foresee that quantity must also be positive: 

  add(item: Item): void {
    if (0 > item.price) {
      throw new Error("Item price cannot be negative")
    }
    if (0 > item.quantity) {
      throw new Error("Item quantity cannot be negative")
    }
    const existing = this.#items.get(item.id)
    if (existing) {
      existing.quantity += item.quantity
    } else {
      this.#items.set(item.id, item)
    }
  }

And let's update our PBT to make sure a failed add does not modify the state of the cart: 

test("total price should never be negative", () => {
  fc.assert(
    fc.property(fc.array(arbItem), items => {
      const cart = new ShoppingCart()

      for (const item of items) {
        const empty = cart.empty
        const totalPrice = cart.totalPrice
        const itemCount = cart.itemCount

        try {
          cart.add(item)
        } catch {
          expect(cart.empty).toBe(empty)
          expect(cart.totalPrice).toBe(totalPrice)
          expect(cart.itemCount).toBe(itemCount)
        }
      }

      expect(cart.totalPrice).toBeGreaterThanOrEqual(0)
    }),
  )
})

Is all well? ROFL no. Here's the next value that breaks the code.

{
  "id": "",
  "name": "",
  "price": 0,
  "quantity": Number.POSITIVE_INFINITY
}

At this point, I would probably use a schema to validate any potential items, so let's replace our if statements with a superstruct schema, and add some more logic for quantity: 

import * as ss from "superstruct"

const ItemSchema = ss.object({
  id: ss.string(),
  name: ss.string(),
  price: ss.refine(ss.number(), "non-negative", n => n >= 0),
  quantity: ss.refine(ss.integer(), "positive", n => n > 0),
})

export type Item = ss.Infer

And now add reads like this:

add(item: Item): void {
  ss.assert(item, ItemSchema, "Invalid Item")
  const existing = this.#items.get(item.id)
  if (existing) {
    existing.quantity += item.quantity
  } else {
    this.#items.set(item.id, item)
  }
}

Finally, our property-based test is successful!

Findings

Let's consider what we've uncovered by using PBT:

Think also about how much more confident we are in our shopping cart implementation. And how any future production bugs we get can now be generalised as a property and added as a PBT (keen eyed readers may note we do not handle duplicate Item IDs).

Confident developers iterate faster than developers riddled with doubt. PBT brings us one step closer to verifying behaviour in terms of requirements. If you'd rather find issues before your users do, PBT is a fantastic start. Most languages have a PBT library - I encourage you to experiment.