During round-4 aggregation (2026-06-01), an initial JVM run reported 1 failure across 710 tests / 1581 assertions. Two subsequent runs of the identical command (same code, different test.check seed) reported 0 failures. The CLJS cross-host run was clean (721/1609/0) on every attempt.

A single property fired on a single seed and never reproduced in the session window. The shrunk counterexample was not captured – the output had scrolled past before the seed could be pinned.

This is the kind of event a casual reader would dismiss as "flake." It is not flake; it is signal. The remediation is not "ignore" – it is "capture next time it fires."

When defspec or tc/quick-check finds a counterexample, the generator framework shrinks it: repeatedly tries smaller values until it can no longer produce a failure. The smallest still-failing value is the shrunk counterexample.

Example: a property (prop/for-all [n gen/nat] (< (* n 2) 1000)) catches n=500, then shrinks to n=500 (can't go smaller without passing), reports the smallest value that breaks the claim.

The output looks like:

{:result false,
 :seed 1780279059130,
 :failing-size 47,
 :num-tests 48,
 :fail [{:n 500}],
 :shrunk {:total-nodes-visited 9,
          :depth 6,
          :result false,
          :smallest [{:n 500}]}}

The :seed is the reproducer key. The :smallest is the shrunk counterexample.

Most of our defspec forms pass no :seed option and take the framework default, which test.check derives from the current time. Trade-off:

We pick the non-deterministic seed deliberately. The whole point of property tests is to find the edge cases we didn't think to write. A fixed seed turns property tests into expensive deterministic tests.

The cost is: a property that holds for 999 seeds and fails on 1 will fire occasionally in CI. That's not a bug; it's discovery.

When a property test fails – even once – the right response is:

1. Capture the seed and the shrunk counterexample. Both are in the defspec output. Save them immediately; they will scroll away.

2. Reproduce locally with the seed:

      (require '[clojure.test.check :as tc]
               '[wal-sh.adtech.<m>.core-prop-test :as p])
      (tc/quick-check 100 p/<the-prop> :seed 1780279059130)
   

   If it reproduces – fix the impl. If it doesn't – the bug is non-determinism in the impl, which is a real bug in its own right.

3. Promote the shrunk counterexample to an example test in core_test.cljc:

      (deftest pinned-from-shrink-2026-06-01
        ;; Originally found by property test, seed 1780279059130.
        ;; Shrunk counterexample: {:n 500}.
        (is (not (your-property-pred {:n 500}))))
   

   This means the case is now in CI deterministically, regardless of whether the property's random seed ever hits it again.

4. Leave the property in place. The shrunk example pins THIS case; the property still hunts for other cases.

We could enforce "every defspec records its last 10 seeds" or similar machinery. We don't, because:

• The pin-on-find pattern only needs the one seed that found a real bug; tracking every seed is overhead with no payoff.
• Most properties pass on every seed; the few that don't deserve manual investigation, not automation.
• The defspec forms in test/pocket_es/token_test.cljc pass only a trial count (100) and no :seed option, so they run on the framework default seed. That is the working norm; deviating per-module would create inconsistency.

The seed wasn't captured before scroll-back. Three options:

1. Wait – if the failure repeats, capture it then. (Defensible: single occurrence in ~700 tests across 5 runs is below noise floor.)
2. Re-run with verbose output to file until it fires: for i in 1..50; do clojure -M:test ... > /tmp/run-$i.txt; done then grep for :result false. Catches the seed eventually.
3. Bisect by module – if you suspect a specific port introduced it, run that module's prop-tests in isolation many times to surface it.

For this event we chose (1). The CLJS cross-host gate is clean, so the bug – if real – is JVM-specific, which narrows the search space. JVM-only test.check bugs are usually in java.util.Random seeding interaction with bit-and masks; the recent FNV-1a hash work (round-3 ab-engine, round-4 referral-program / content-gate extensions) is the most likely surface.

If it repeats, we'll capture and pin. No recurrence recorded as of 2026-09-10: the commit history under test/ since 2026-06-01 carries no reference to this failure, and no pinned-from-shrink test exists in the tree.

Property tests find bugs you didn't write tests for. Some of those bugs are subtle enough that ONE seed in a thousand exposes them.

A flake in a property test is a successful test – it found something. The failure mode is not "property tests are unreliable"; it's "we didn't have a process for capturing the discovery when it happened."

This doc IS that process.

The full catalog lives in _drafts/adtech-cljc/EXPERIENCE-REPORT.org (unpublished).

• Round 2 attribution: Shapley float drift at n=7 channels ~ε=1e-15 – pinned a per-model tolerance in the property assertion.
• Round 3 prebid-lite: granularity-bucket IEEE-754 failure – (floor (/ 0.30 0.10)) = 2.0 not 3.0; switched to integer thousandths.
• Round 4 referral-program: agent pre-wrote a guessed cross-host vector literal; JVM caught the mismatch on first run. Write the test, run it, pin from the output – don't guess golden values.

All three are pin-on-find applied to specific shrinks. The 2026-06-01 event is the first where the shrink fired without being captured – hence this doc.