Refinement Loop Across Heterogeneous Systems: Liquidus Overlay Experience Report
Table of Contents
- 1. Decision: report the method, not the artifact
- 2. The systems in play
- 3. The epistemic shape: proofs and refutations
- 4. The governance loop
- 5. The method under the reflection-and-evaluation lens
- 5.1. Intrinsic self-correction fails; external signals do not
- 5.2. The Generator-Critic loop must be tool-grounded
- 5.3. Capabilities want extraction, but extraction is untested
- 5.4. This document is Experience Replay, at three grains
- 5.5. Self-Heal closes on deterministic failures only
- 5.6. The evaluation stack, five layers deep
- 6. Reconstruction from the contract
- 7. Invariants of the style
- 8. Refutation conditions
- 9. How to reproduce it
- 10. Provenance
1. Decision: report the method, not the artifact
In one sitting a contract-only specification became a shipped, public artifact
(aygp-dr/liquidus-overlay, one of the build repos grown from the spec-first
grind described in the PRD grind note):
an injectable command overlay for the Solidus
admin, its command surface derived mechanically from a pinned OpenAPI contract,
its state machine checked in TLA+, its history mined from git, hosted on
Cloudflare, exercised against three different backends. The artifact is
incidental. What is worth preserving is the method – a Lakatosian
proofs-and-refutations loop run across a dozen unlike systems, steered by a human
who supplied the terministic screen and let the machine hold the context.
This is that method, stated so it can be reproduced. Every claim below names its refutation condition; a method that cannot fail is not a method, it is a mood.
2. The systems in play
Eleven systems, no two speaking the same protocol, each a potential contract violation surface:
| System | Role | Protocol |
|---|---|---|
| the spec repo (contract) | the source of truth; no implementation | org + pinned SHA |
| the build repo | where implementation is permitted to exist | git |
| Prism | contract mock, stateless, presence-only auth | OpenAPI → HTTP |
| node harness / mock | LAN-reachable stand-ins for a live admin | HTTP + fixtures |
| real Solidus 4.7 | stateful backend; Ruby/Rails/Postgres/puma | /api + api_key |
| Cloudflare | hosting for the injectable asset | Workers/Pages |
| TLA+ / TLC | the state machine's proof engine | temporal logic |
| research subagents | git-history archaeology of the contract | gh api + diff |
| Chrome (automated) | the overlay's actual runtime | DOM + fetch |
| GitHub | the public provenance surface | repo + gist |
| the human | the governance tuple; supplies what matters | interjection |
The interesting property is not that these connect. It is that the boundary between any two of them is where the work actually happened – and every boundary lied at least once.
3. The epistemic shape: proofs and refutations
Lakatos's thesis is that mathematics advances not by deduction from axioms but by conjecture, attempted proof, refutation by counterexample, and refinement of the conjecture to survive it. The same shape held here, except the counterexamples arrived as HTTP status codes and native-extension load errors. A catalogue of the refutations, because the refutations are the content:
- Conjecture: the pinned contract exposes 103 paths under
/api/v2/platformand/api/v2/storefront. Refutation: the git history of the OpenAPI file shows 25 flat, unversioned/apipaths and nov2anywhere – the split is Spree's lineage, not Solidus's. Refinement: correct the contract's own provenance prose; the bare paths were right all along. - Conjecture: the overlay cannot be exercised without a live store (the spec said so, and made it a non-goal). Refutation: it ran against a stateless mock and against fixtures the same afternoon. Refinement: the non-goal's stated mechanical reason falls; a narrower boundary survives – see the next entry.
- Conjecture: "jump to record R" is testable against the mock. Refutation: Prism is stateless; TLC returns a six-state counterexample where the mock renders a different record than the one searched. Refinement: jump-correctness is a theorem against a live contract and a refuted invariant against a stateless one. The product's supported boundary, discharged by a checker rather than asserted in a README.
- Conjecture:
gem install rails -v '~> 7.2' bundler. Refutation: the package manager rejects a version constraint against two packages. Refinement: split the invocation. (Not every aporia is profound. Some are just a CLI.) - Conjecture: a real backend bound to
0.0.0.0is reachable on the LAN. Refutation: the TCP handshake completes but the HTTP response never arrives – only for the Ruby process, only on the non-loopback interface. Refinement: a node reverse-proxy to loopback; the boundary was in the host's packet filter, not the bind. - Conjecture: the session cookie authenticates the admin's own API. Refutation:
requires_authenticationdefaults true, so the contract answers 401 and ignores the cookie. Refinement: supply the running user's api_key; provenance of the call is the user, but the credential is explicit.
Each refinement is cheap only because the context that produced the conjecture survived the refutation. The machine held every prior boundary in memory while the human moved the frontier. That is the whole trick.
4. The governance loop
The human interjected roughly twenty times, almost always mid-task: change the
config vocabulary to SPREE_URL=/=SPREE_TOKEN, cut the self-congratulatory copy,
stack the install snippets, stand up a real store, make it public, add topics.
None of these were corrections of error. Each was a re-selection of what
mattered – Burke's terministic screen, applied live. A screen that foregrounds
one contract backgrounds another; the human owns the screen, the machine owns the
consequences.
This is the load-bearing asymmetry. The machine is congruent across systems: it will as happily mine a git history as boot Postgres as model-check a temporal formula. It has no preference. Preference is the human's contribution, and the interjection is its interface. Treating the redirections as interruptions is the category error; they are the specification arriving in instalments.
5. The method under the reflection-and-evaluation lens
The liquidus grind's own self-audit, docs/reflection-self-audit.md in
aygp-dr/liquidus (public),
organises the same territory into six moves; that document in turn applies a
"Reflection and evaluation" chapter it does not name further. The session is a
worked example of each move – and, more usefully, of where each has a
refutation condition.
5.1. Intrinsic self-correction fails; external signals do not
The load-bearing negative result. Left to its own assessment the generator was
wrong, confidently, repeatedly: the browser could reach the local harness (it
could not), the API was up (it was 500-ing on a missing native library),
X-Spree-Token would authenticate (401), the session cookie would suffice
(requires_authentication says otherwise). A generator's self-critique is
congruent with its own blind spots – it cannot see the boundary it cannot see.
Each of those beliefs was corrected only by a signal from outside the generator:
a status code, an nc handshake, a config value read off the running system. The
lesson is not "reflect harder." It is "route the critique through a system that
does not share the generator's screen."
5.2. The Generator-Critic loop must be tool-grounded
The refutation catalogue above is a Generator-Critic loop with the critic held by
a tool, not a second opinion. curl, TLC, gh api, lsof – each a critic that
cannot be argued with and cannot be flattered. The LoadError naming vips.42
is the critic pointing at the exact line; no model critiquing its own output
produces that. Refutation: a Generator-Critic loop whose critic is another
instance of the generator inherits the generator's blind spots and launders
confidence into consensus. If the critic cannot fail the generator on evidence
the generator lacks, it is theatre.
5.3. Capabilities want extraction, but extraction is untested
Three skills did real work this session – deep research, this style pack,
frontend design – each a packaged capability with its own contract (a SKILL.md,
in that ecosystem's terms). The session also produced capabilities not yet
extracted: boot-a-Solidus-from-cold, proxy-around-a-filtered-port,
registry-from-a-pinned-OAS. The GNUmakefile is their provisional home – a skill
package with the lid off. Refutation: a capability is reusable only once it
survives removal from the context that produced it. Until the Solidus recipe runs
on a machine that is not this one, "reusable" is a conjecture, not a property.
5.4. This document is Experience Replay, at three grains
Learning across sessions needs the trace to outlive the session. It survives here at three abstraction levels, each a different replay: the raw command output (what the systems actually returned), the refutation/refinement pairs (the tactic that survived), and the reproduction protocol (the method with the artifact subtracted). Read the lowest level to debug, the middle to reuse, the top to teach. A replay that keeps only one level is either unteachable or unverifiable.
5.5. Self-Heal closes on deterministic failures only
The deterministic failures healed cheaply: the two-package version constraint,
the binstub off the PATH, the missing image library, the taxonomy name clash –
each a fix-and-re-run cycle against a reproducible error, the build script's own
fail markers closing the loop. The non-deterministic boundary – a host
filtering one process's packets on one interface – did not heal. It needed a
human-supplied reframe: route through a process the filter trusts. Boundary
condition: Self-Heal is a function of reproducibility. A failure you cannot
re-trigger on demand you cannot heal; you can only route around it, and routing
around is the human's move, not the loop's.
5.6. The evaluation stack, five layers deep
Scattered checks become a methodology when they stack. The session's, bottom to top:
| Layer | What it answers | Instrument this session |
|---|---|---|
| unit | does one call return the right shape | curl /api/products → 200, 3 |
| contract | are the fixture obligations met | F4 401→200; F5 provenance tuple |
| formal | do the invariants hold over all states | TLC, 476 states, 4 invariants |
| integration | does it work end-to-end in the runtime | overlay lookup renders records |
| provenance | are the claims themselves audited | git-confirmed schema history |
The contract and formal rows are backed by artifacts in aygp-dr/liquidus-overlay.
F4 is the verify target in the repo's GNUmakefile (the no-bearer probe asserts
401; the bearer-carrying probe must return rows, which implies 200), and the
README.org status line records the observed pair:
verify: ## Smoke-test a running harness (expects `make serve` in another shell)
@set -e; B="Authorization: Bearer liquidus-dev-stub"; \
curl -fsS http://localhost:$(PORT)/health >/dev/null && echo "health OK"; \
test $$(curl -s -o /dev/null -w '%{http_code}' http://localhost:$(PORT)/products) = 401 \
&& echo "F4 no-bearer -> 401 OK"; \
test $$(curl -s -H "$$B" http://localhost:$(PORT)/products | node -e "...JSON.parse(d).data.length...") -ge 1 \
&& echo "lookup /products returns rows OK"
README.org: Verified via the harness: registry builds (33 tags -> 38 commands),
F4 bearer-presence enforced (401->200), spec-derived lookup paths
return rows, bundle self-contained.
The TLC figures come from formal/README.org and formal/OverlayStates.cfg. The
config names four Inv_* invariants plus a type invariant; the raw TLC transcript
is not checked in, so the 476-state count is the repo's recorded result rather
than a log this note re-ran:
formal/OverlayStates.cfg:
CONSTANTS Records = {r1, r2} Backend = "Live" Bearer = TRUE MaxSeq = 3
INVARIANT TypeInvariant
INVARIANT Inv_Provenance
INVARIANT Inv_Credential
INVARIANT Inv_MonotoneSeq
INVARIANT Inv_JumpCorrect
PROPERTY Live_Returnable
formal/README.org, Results (TLC, checked):
Backend = "Live", Bearer = TRUE All invariants + liveness hold.
476 distinct reachable states.
Backend = "Stateless" (Prism mock) Inv_JumpCorrect REFUTED; 6-state
counterexample: type a query, the mock
renders a different record, the CSA
opens the wrong one.
A lower layer passing tells you nothing about a higher one: unit-green endpoints sat beneath a formal invariant that a stateless backend refutes. The stack is ordered by what each layer cannot see below it. Evaluate up the stack, and distrust any layer reporting green while the layer above it has not run.
6. Reconstruction from the contract
The reproduction question has a sharper form: if the artifact were lost, what
would an agent need to regrow it? The answer is small – spec.org and a pinned
SHA – and the smallness is the point. The contract is not documentation of the
system; it is the seed the system regrows from.
6.1. The seed
spec.org carries what a rebuild cannot infer: the invariants (a governance
tuple on every response, presence-only auth, a bearer that is never a real
token), the fixtures F1–F5 as executable obligations, the anti-goals (do not
teach the mock to be stateful; do not expose the token), and the pinned commit.
Everything else is derived, mechanically, from that commit: reify the upstream
OpenAPI at the SHA, and the command registry, the mock, and the fixtures all fall
out of it – thirty-three tags become thirty-eight commands with no hand-curation.
The translation being mechanical is what makes the seed a seed rather than a
sketch. An agent that hand-curates what the tags already encode has stopped
rebuilding and started forking.
6.2. The three seams
Stitching the pieces is a matter of three seams, and each is where a rebuild tears if the version is wrong:
- Core system ↔ spec version. The pinned SHA binds one Solidus commit's contract. The running store and the mocked contract must be the same version or the seam parts – a consumer built against paths the live system renamed gets silence, not an error. The drift study exists to measure the seam's tolerance: for this contract, near-total (25 paths unchanged across six years), so pinning is cheap insurance rather than a tax.
- Contract ↔ tooling. Every tool hangs off the contract as a spoke – Prism mocks it, the TLA+ model consumes it, the overlay's registry derives from it, the fixtures grade against it. The contract is the hub; move the SHA and the spokes regenerate rather than drift. A tool that does not regenerate from the contract is not additional tooling, it is a second source of truth, and two sources of truth is zero.
- Mock ↔ live. The presence-only stub and the authenticity-checked real key
meet here, and this is the seam the session spent most of its refutations on:
the mock's statelessness (a lookup cannot return the record it was asked for),
requires_authenticationrefusing the session cookie, the host filtering the live port. The contract holds on both sides; the behaviour at the boundary does not, and no measure of contract fidelity papers over it. Instrument this seam or be surprised by it.
6.3. For the next rebuild
An agent handed spec.org and a SHA, aimed at related work, needs three things
this session had to discover the hard way. A reification step that turns the
commit into the derived surfaces – registry, mock, fixtures – rather than
authoring them. A version-match check across the core↔spec seam before trusting
any live call. And the mock↔live boundary treated as a first-class object with
its own tests, not an implementation detail. Given those, the artifact is
reconstructible from the seed. Absent them, the seed grows something that compiles
and lies.
7. Invariants of the style
Properties that held across every system and must hold for the method to reproduce:
- Provenance on every result. No datum crossed a boundary bare. Mock responses
carried
source · spec-SHA · backend; the overlay stamped its panels; the research agent separated git-confirmed from inferred. When you cannot trust a boundary, you can at least label what came through it. - Falsification before belief. Nothing was "working" until a command produced
the evidence.
curlbefore claim, TLC before "verified",gh apibefore "the spec says". The refutation catalogue above exists because the belief was always provisional. - One writable repo. Implementation lived only where implementation was
permitted; the contract stayed copy-only. The three spec files were relocated
to
spec-input/rather than edited, so a build could never silently fork the thing it was grading. - Long-running work detached, not awaited. The Solidus build, the vips install, the research agent – all backgrounded, watched for a terminal signal, never blocked on. The human kept steering while the slow systems churned.
8. Refutation conditions
This method is wrong, and this report should be revised, if:
- A boundary is crossed with no way to instrument it – if provenance cannot be attached, the falsification-first stance degrades to trust, and the loop is running on faith.
- The human's interjections stop re-selecting what matters and start correcting the machine's errors. That inverts the asymmetry: it means the machine is no longer congruent across systems, and the method is subsidising its incompetence with human attention.
- The context does not survive the refutations – if each counterexample forces re-establishing prior boundaries, the cheap-refinement property is gone and the loop is just expensive trial and error with extra steps.
9. How to reproduce it
Not a checklist to be filled but a set of preconditions to arrange:
- Pin the contract, separate it from the implementation. A source of truth that cannot be edited by the thing consuming it. Copy it in; feed changes back through a graded report, never a silent edit.
- Give the machine one context and many tools. One conversation holding every boundary; ambient access to shell, git, browser, a model checker, cloud CLIs, and subagents for the archaeology you do not want in the main context.
- Make every claim produce evidence. Wire the loop so "it works" is unspeakable without a command that shows it. Prefer a counterexample-producing checker to a test suite that only confirms.
- Background the slow systems; watch for a terminal signal. Never await a build
in the foreground. Poll for
Listening onorFAILED, keep the frontier moving. - Let the human supply the screen, in instalments. Do not demand a complete specification up front. The interjections are the spec; the method's job is to keep the context coherent enough that each one is a cheap refinement, not a restart.
The artifact will differ every time. The method is the invariant.
10. Provenance
Written from a single session, 2026-07-01, that produced aygp-dr/liquidus-overlay
(public) – and written as a reference for related future work, the method rather
than the artifact, so an agent handed only spec.org and a SHA can regrow the
rest. Refutation catalogue is drawn from that session's command output – HTTP
codes, TLC counterexamples, git diffs – not reconstructed from memory. Contract-
history claims are git-confirmed against solidusio/solidus; see the build repo's
docs/solidus-api-schema-drift.org. The spec-side history of the same grind is in
the PRD grind note.