Property-Based Testing in Python with Hypothesis: Automating Edge Case Detection

The Blind Spot in Example-Based Testing

Most engineering teams rely on example-based tests. We write a function, sketch out three or four typical inputs, assert the expected return values, and ship. The flaw is obvious: edge cases inevitably escape into production. Subtle bugs like unhandled Unicode surrogate pairs, empty payloads, signed negative zeros (-0.0), or off-by-one errors rarely show up in hand-rolled fixtures.

We naturally design unit tests that validate our expectations. When testing a string parser, we type "hello" or "user_123" instead of strings filled with null bytes (\x00) or right-to-left override marks. Hypothesis replaces this guesswork with property-based testing. Instead of hardcoding isolated examples, you define the general rules—the invariants—that your code must always respect. The engine then throws hundreds of generated permutations at your code to find the break point.

Approach Comparison: Example-Based vs. Property-Based Testing

Pros and Cons of Hypothesis

Switching from static examples to invariant modeling brings clear advantages along with real computational costs.

Advantages

• Automated Shrinking: If a test fails on an 80-element list containing deeply nested strings, Hypothesis does not dump all 80 items on you. It discards irrelevant values until it uncovers the exact trigger—often reducing the payload to a single empty string "" or a lone 0.
• Exhaustive Boundary Discovery: It relentlessly evaluates corner cases. Empty sets, subnormal floating-point numbers, whitespace combinations, and maximum integer thresholds are fed into your functions automatically.
• Deterministic Regression Cache: When Hypothesis catches a failure, it records the minimal reproduction into a local .hypothesis directory. Every subsequent test run verifies that exact regression first, even across different machines.

Trade-offs

• Higher Execution Latency: A standard unit test runs in roughly 0.4 ms. Running 100 generated cases per test with Hypothesis can push that duration to 30–120 ms.
• Conceptual Shift: You cannot lean on simple value comparisons. Teams must learn to assert structural relationships, such as round-trip encoding symmetry, idempotency, or mathematical bounds.

Recommended Setup

Install both packages into your active virtual environment:

pip install pytest hypothesis

To keep local feedback fast while running exhaustive checks on your pull requests, register dedicated profiles in your root conftest.py:

# conftest.py
import os
from hypothesis import settings

settings.register_profile("ci", max_examples=250, deadline=None)
settings.register_profile("dev", max_examples=40, deadline=300)

# Switch profile via environment variable, defaulting to dev locally
current_profile = os.getenv("HYPOTHESIS_PROFILE", "dev")
settings.load_profile(current_profile)

Implementation Guide

Consider an e-commerce price calculator that handles base costs, percentage discounts, and sales tax. Subtle rounding errors or unusual decimal inputs frequently cause billing discrepancies in these pipelines.

1. The Business Logic

Our calculation lives inside pricing.py, using Python’s Decimal module to avoid binary floating-point errors:

# pricing.py
from decimal import Decimal, ROUND_HALF_UP

def calculate_final_price(
    base_price: Decimal,
    discount_percent: Decimal,
    tax_percent: Decimal,
) -> Decimal:
    if base_price < Decimal("0.00"):
        raise ValueError("Base price cannot be negative")
    if not (Decimal("0.00") <= discount_percent <= Decimal("100.00")):
        raise ValueError("Discount must be between 0 and 100")
    if tax_percent < Decimal("0.00"):
        raise ValueError("Tax cannot be negative")

    discount_factor = Decimal("1.00") - (discount_percent / Decimal("100.00"))
    tax_factor = Decimal("1.00") + (tax_percent / Decimal("100.00"))
    
    total = base_price * discount_factor * tax_factor
    return total.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)

2. Writing Invariant Properties

Forget hardcoded dollar values like $19.99. Frame the core requirements as universal truths:

1. Zero-Discount Floor: If the discount is 0% and tax is positive or zero, the final price must never fall below the base price.
2. Full-Discount Guarantee: A 100% discount must always result in exactly Decimal("0.00"), regardless of tax or starting cost.
3. Price Monotonicity: For any base price and tax rate, raising the discount must never increase the final price.

The resulting test suite in test_pricing.py expresses these three rules directly:

# test_pricing.py
from decimal import Decimal
from hypothesis import given, strategies as st
import pytest
from pricing import calculate_final_price

# Limit values to realistic operational ranges
valid_money = st.decimals(min_value=Decimal("0.01"), max_value=Decimal("50000.00"), places=2)
valid_percentage = st.decimals(min_value=Decimal("0.00"), max_value=Decimal("100.00"), places=2)

@given(base_price=valid_money, tax=valid_percentage)
def test_zero_discount_preserves_or_increases_price(base_price, tax):
    result = calculate_final_price(base_price, Decimal("0.00"), tax)
    assert result >= base_price

@given(base_price=valid_money, tax=valid_percentage)
def test_full_discount_yields_zero(base_price, tax):
    result = calculate_final_price(base_price, Decimal("100.00"), tax)
    assert result == Decimal("0.00")

@given(
    base_price=valid_money,
    d1=valid_percentage,
    d2=valid_percentage,
    tax=valid_percentage,
)
def test_higher_discount_lowers_or_equals_price(base_price, d1, d2, tax):
    low_discount, high_discount = sorted([d1, d2])
    price_with_low = calculate_final_price(base_price, low_discount, tax)
    price_with_high = calculate_final_price(base_price, high_discount, tax)
    assert price_with_high <= price_with_low

3. Executing the Test Suite

Run the suite with verbose output:

pytest test_pricing.py -v

If someone alters the rounding order or neglects an input bound, Hypothesis does not just report a failure. It identifies the exact boundary values that tripped the assertion and shrinks the traceback to its minimal input.

Adopt property-based testing on core parsing logic, financial calculations, and data serialization. Example-based tests confirm that your happy paths work; Hypothesis proves that your domain rules hold everywhere else.