Property testing: verify laws, not examples
You wrote a function to parse a configuration file. You tested it with a valid file. You tested it with a missing file. You even tested it with a file that has a typo. All tests pass. You ship the code. Three days later, a user sends a config with a specific combination of unicode characters and the parser crashes. You missed the edge case.
Unit tests check specific examples. You pick inputs, you predict outputs, you verify the match. If you test five inputs, you only know the code works for those five inputs. Property testing checks rules that must hold for all inputs. You describe a law about your code, and the library generates thousands of random inputs to try to break it. If the law holds, you gain confidence. If the law breaks, the library finds the smallest input that causes the failure and shows it to you.
Property testing shifts your focus from "Does this work?" to "What must always be true?" This change catches bugs that unit tests miss. It finds edge cases you never thought to write. It forces you to think about the boundaries of your logic.
How property testing works
Property testing relies on three concepts: strategies, assertions, and shrinking.
A strategy defines how to generate random data. Strategies know how to create values of a type and how to reduce those values when a test fails. The proptest crate provides strategies for primitive types, strings, collections, and custom structs. You combine strategies to build complex inputs.
An assertion checks a property. Instead of comparing against a hardcoded expected value, you check a relationship between inputs and outputs. For example, "reversing a string twice returns the original string" or "the length of a sorted vector equals the length of the original vector." These properties must hold for every input the strategy generates.
Shrinking is the magic. When a test fails, proptest doesn't just stop. It tries to find a smaller input that also fails. If a test fails with a 4000-character string, shrinking reduces it to the minimal string that still breaks the property. Often this is a single character. Shrinking turns a mysterious failure into a reproducible, debuggable bug.
Unit tests verify examples. Property tests verify laws. Use property testing to check invariants, round-trips, and mathematical properties. Use unit tests to check specific behavior for known inputs. Both tools belong in your test suite.
Minimal example
Add proptest as a dev-dependency in Cargo.toml. Define a test function with the #[test] attribute. Use the proptest! macro to run a strategy and assert a property.
use proptest::prelude::*;
/// Verifies that addition order does not change the result.
#[test]
fn addition_is_commutative() {
// proptest! generates random pairs of i32 values.
// It runs the closure many times with different inputs.
proptest!(|(a in any::<i32>(), b in any::<i32>())| {
// prop_assert! captures the values if the assertion fails.
// This lets proptest report exactly what broke the rule.
prop_assert_eq!(a + b, b + a);
});
}
Run the test with cargo test. The output shows the number of cases tested and the seed used for random generation. If the test passes, you see a summary like "Success: 256 passed; 0 failed; 0 ignored; 0 filtered out."
The any::<T>() strategy uses the Arbitrary trait to generate random values of type T. For i32, it generates values across the full range, including edge cases like 0, 1, -1, i32::MAX, and i32::MIN. This coverage is automatic. You don't need to write separate tests for boundary values.
Convention aside: always use prop_assert! or prop_assert_eq! inside proptest! closures. If you use the standard assert! macro, the test panics on failure. The panic destroys the context of the input values. Proptest loses the ability to report the failing input and to shrink it. The community treats this as a hard rule. prop_assert! is the only way to get shrinking and value reporting.
Run this test and watch proptest hammer your code with random numbers. It will never fail, because math is reliable.
Strategies control the chaos
Strategies are the heart of property testing. A strategy tells proptest how to generate data and how to shrink it. You can use built-in strategies or compose your own.
The any::<T>() strategy works for any type that implements Arbitrary. Derive Arbitrary for your own structs to use any::<MyStruct>().
use proptest::prelude::*;
#[derive(Arbitrary, Debug)]
struct User {
name: String,
age: u32,
}
/// Verifies that user age is within a reasonable range.
#[test]
fn user_age_is_reasonable() {
proptest!(|(user in any::<User>())| {
// This property is false by default.
// any::<User>() generates ages up to u32::MAX.
// The test will fail quickly.
prop_assert!(user.age < 150);
});
}
This test fails because any::<User>() generates ages across the full u32 range. You need a strategy that generates valid data. Use range strategies to constrain values.
use proptest::prelude::*;
#[derive(Arbitrary, Debug)]
struct User {
name: String,
age: u32,
}
/// Generates users with valid ages.
fn valid_user() -> impl Strategy<Value = User> {
(".*", 0..150u32).prop_map(|(name, age)| User { name, age })
}
/// Verifies that user age is within the generated range.
#[test]
fn user_age_is_reasonable() {
proptest!(|(user in valid_user())| {
prop_assert!(user.age < 150);
});
}
The prop_map combinator transforms generated values. The tuple strategy (".*", 0..150u32) generates a string and an age. prop_map combines them into a User. This pattern is common. Generate primitive values, then map them into your domain types.
Use prop_compose! for reusable strategy functions. This macro creates a named strategy that you can use in multiple tests.
use proptest::prelude::*;
proptest::prop_compose! {
/// Generates a user with a non-empty name and valid age.
fn valid_user() -> User {
name in "[a-z]+",
age in 0..150u32
User { name, age }
}
}
Convention aside: keep strategies small and focused. A strategy should generate valid data for a single concept. If you need complex constraints, use prop_filter to reject invalid values, but be careful. Filtering reduces the effective randomness and can cause tests to time out if the filter rejects too many values. Prefer generating valid data directly over filtering invalid data.
Write strategies that generate valid data, or your tests will spend more time filtering than testing.
Realistic example: finding a bug
Property testing shines when you test round-trip operations. A round-trip test checks that transforming data and then reversing the transformation returns the original data. This property should hold for all inputs. If it doesn't, you have a bug.
Consider a function that splits a string by whitespace and joins it back. You might assume this is a no-op. It isn't.
/// Splits a string by whitespace and joins with a single space.
fn split_and_join(s: &str) -> String {
s.split_whitespace()
.collect::<Vec<_>>()
.join(" ")
}
/// Verifies that split_and_join is a round-trip.
#[test]
fn split_join_roundtrip() {
proptest!(|(s in ".*")| {
// This property is false.
// " a b " becomes "a b".
// Proptest will find a counter-example.
prop_assert_eq!(split_and_join(&s), s);
});
}
Run this test. It fails. Proptest reports the counter-example. The output looks like this:
thread 'tests::split_join_roundtrip' panicked at 'Code: 1
PropTestFailure { cause: Asserts: 1, Abort: 0, Fail: 1, Seed: 0x123456789abcdef0,
Failure: "prop_assert_eq!(split_and_join(&s), s)" failed:
left: `"a b"`,
right: `" a b "`
Minimal counterexample: s = " a b "
Proptest found the input " a b " and shrunk it to the minimal failing case. The bug is clear. The function normalizes whitespace, so it's not a round-trip. You can fix the function or change the property. Maybe the property should be "the result contains the same words as the input, in the same order."
/// Verifies that split_and_join preserves words.
#[test]
fn split_join_preserves_words() {
proptest!(|(s in ".*")| {
let words = s.split_whitespace().collect::<Vec<_>>();
let result = split_and_join(&s);
let result_words = result.split_whitespace().collect::<Vec<_>>();
prop_assert_eq!(words, result_words);
});
}
This property holds. The test passes. You've verified that the function preserves the semantic content of the string, even if it changes the formatting.
Let the test fail. The counter-example proptest finds is more valuable than a passing test. It shows you exactly where your assumption was wrong.
Pitfalls and compiler errors
Property testing introduces new failure modes. Understanding these pitfalls saves time.
If you try to use a type in a strategy without implementing Arbitrary, the compiler rejects you with E0277 (trait bound not satisfied). You must derive Arbitrary or implement it manually. The derive macro is the standard approach. Add proptest to your dev-dependencies and use #[derive(Arbitrary)].
use proptest::prelude::*;
// This fails to compile.
// error[E0277]: the trait bound `MyStruct: Arbitrary` is not satisfied
#[test]
fn test_my_struct() {
proptest!(|(s in any::<MyStruct>())| {
prop_assert!(true);
});
}
Add the derive to fix it.
#[derive(Arbitrary, Debug)]
struct MyStruct {
value: i32,
}
Strategies can generate data that your function cannot handle. If your function expects a sorted vector, but any::<Vec<i32>>() generates unsorted vectors, the test will fail or panic. You need a strategy that generates sorted vectors. Use prop_filter or a custom strategy.
/// Generates a sorted vector of i32.
fn sorted_vec() -> impl Strategy<Value = Vec<i32>> {
proptest::collection::vec(any::<i32>(), 0..100).prop_map(|mut v| {
v.sort();
v
})
}
Property tests are non-deterministic. They use random data. A test might pass locally and fail in CI. This is normal. Proptest prints the seed used for each test run. You can reproduce a failure by setting the seed. Use the PROPTEST_SEED environment variable.
PROPTEST_SEED=0x123456789abcdef0 cargo test
Convention aside: treat failing property tests as bugs until proven otherwise. If a test fails intermittently, investigate the failure. Don't just rerun the test until it passes. The failure indicates a real issue with your code or your property. Shrinking helps you find the root cause quickly.
Non-determinism is a feature, not a bug. It ensures your code works for inputs you haven't thought of. Embrace the randomness.
Property testing doesn't replace unit tests. It complements them. Use both to build a safety net that catches logic errors and edge cases alike.
When to use property testing
Use property testing when you need to verify invariants across a wide range of inputs. Use property testing when you want to discover edge cases you haven't thought of. Use property testing for round-trip checks, commutativity, associativity, and idempotence. Use property testing when the input space is large and manual testing is impractical.
Use unit testing when you need to verify specific behavior for known inputs. Use unit testing when the cost of generating valid inputs is high. Use unit testing when you need to check exact output values where the logic is deterministic and narrow. Use unit testing for mocking external dependencies that property testing cannot handle.
Reach for property testing when the rule matters more than the example. Reach for unit testing when the example is the specification.
Property testing doesn't replace unit tests. It complements them. Use both to build a safety net that catches logic errors and edge cases alike.