Refinement Loop Across Heterogeneous Systems: Liquidus Overlay Experience Report

Table of Contents

1. Decision: report the method, not the artifact

In one sitting a contract-only specification became a shipped, public artifact (aygp-dr/liquidus-overlay, one of the build repos grown from the spec-first grind described in the PRD grind note): an injectable command overlay for the Solidus admin, its command surface derived mechanically from a pinned OpenAPI contract, its state machine checked in TLA+, its history mined from git, hosted on Cloudflare, exercised against three different backends. The artifact is incidental. What is worth preserving is the method – a Lakatosian proofs-and-refutations loop run across a dozen unlike systems, steered by a human who supplied the terministic screen and let the machine hold the context.

This is that method, stated so it can be reproduced. Every claim below names its refutation condition; a method that cannot fail is not a method, it is a mood.

2. The systems in play

Eleven systems, no two speaking the same protocol, each a potential contract violation surface:

System Role Protocol
the spec repo (contract) the source of truth; no implementation org + pinned SHA
the build repo where implementation is permitted to exist git
Prism contract mock, stateless, presence-only auth OpenAPI → HTTP
node harness / mock LAN-reachable stand-ins for a live admin HTTP + fixtures
real Solidus 4.7 stateful backend; Ruby/Rails/Postgres/puma /api + api_key
Cloudflare hosting for the injectable asset Workers/Pages
TLA+ / TLC the state machine's proof engine temporal logic
research subagents git-history archaeology of the contract gh api + diff
Chrome (automated) the overlay's actual runtime DOM + fetch
GitHub the public provenance surface repo + gist
the human the governance tuple; supplies what matters interjection

refinement-loop.png

The interesting property is not that these connect. It is that the boundary between any two of them is where the work actually happened – and every boundary lied at least once.

3. The epistemic shape: proofs and refutations

Lakatos's thesis is that mathematics advances not by deduction from axioms but by conjecture, attempted proof, refutation by counterexample, and refinement of the conjecture to survive it. The same shape held here, except the counterexamples arrived as HTTP status codes and native-extension load errors. A catalogue of the refutations, because the refutations are the content:

  • Conjecture: the pinned contract exposes 103 paths under /api/v2/platform and /api/v2/storefront. Refutation: the git history of the OpenAPI file shows 25 flat, unversioned /api paths and no v2 anywhere – the split is Spree's lineage, not Solidus's. Refinement: correct the contract's own provenance prose; the bare paths were right all along.
  • Conjecture: the overlay cannot be exercised without a live store (the spec said so, and made it a non-goal). Refutation: it ran against a stateless mock and against fixtures the same afternoon. Refinement: the non-goal's stated mechanical reason falls; a narrower boundary survives – see the next entry.
  • Conjecture: "jump to record R" is testable against the mock. Refutation: Prism is stateless; TLC returns a six-state counterexample where the mock renders a different record than the one searched. Refinement: jump-correctness is a theorem against a live contract and a refuted invariant against a stateless one. The product's supported boundary, discharged by a checker rather than asserted in a README.
  • Conjecture: gem install rails -v '~> 7.2' bundler. Refutation: the package manager rejects a version constraint against two packages. Refinement: split the invocation. (Not every aporia is profound. Some are just a CLI.)
  • Conjecture: a real backend bound to 0.0.0.0 is reachable on the LAN. Refutation: the TCP handshake completes but the HTTP response never arrives – only for the Ruby process, only on the non-loopback interface. Refinement: a node reverse-proxy to loopback; the boundary was in the host's packet filter, not the bind.
  • Conjecture: the session cookie authenticates the admin's own API. Refutation: requires_authentication defaults true, so the contract answers 401 and ignores the cookie. Refinement: supply the running user's api_key; provenance of the call is the user, but the credential is explicit.

Each refinement is cheap only because the context that produced the conjecture survived the refutation. The machine held every prior boundary in memory while the human moved the frontier. That is the whole trick.

4. The governance loop

The human interjected roughly twenty times, almost always mid-task: change the config vocabulary to SPREE_URL=/=SPREE_TOKEN, cut the self-congratulatory copy, stack the install snippets, stand up a real store, make it public, add topics. None of these were corrections of error. Each was a re-selection of what mattered – Burke's terministic screen, applied live. A screen that foregrounds one contract backgrounds another; the human owns the screen, the machine owns the consequences.

This is the load-bearing asymmetry. The machine is congruent across systems: it will as happily mine a git history as boot Postgres as model-check a temporal formula. It has no preference. Preference is the human's contribution, and the interjection is its interface. Treating the redirections as interruptions is the category error; they are the specification arriving in instalments.

5. The method under the reflection-and-evaluation lens

The liquidus grind's own self-audit, docs/reflection-self-audit.md in aygp-dr/liquidus (public), organises the same territory into six moves; that document in turn applies a "Reflection and evaluation" chapter it does not name further. The session is a worked example of each move – and, more usefully, of where each has a refutation condition.

5.1. Intrinsic self-correction fails; external signals do not

The load-bearing negative result. Left to its own assessment the generator was wrong, confidently, repeatedly: the browser could reach the local harness (it could not), the API was up (it was 500-ing on a missing native library), X-Spree-Token would authenticate (401), the session cookie would suffice (requires_authentication says otherwise). A generator's self-critique is congruent with its own blind spots – it cannot see the boundary it cannot see. Each of those beliefs was corrected only by a signal from outside the generator: a status code, an nc handshake, a config value read off the running system. The lesson is not "reflect harder." It is "route the critique through a system that does not share the generator's screen."

5.2. The Generator-Critic loop must be tool-grounded

The refutation catalogue above is a Generator-Critic loop with the critic held by a tool, not a second opinion. curl, TLC, gh api, lsof – each a critic that cannot be argued with and cannot be flattered. The LoadError naming vips.42 is the critic pointing at the exact line; no model critiquing its own output produces that. Refutation: a Generator-Critic loop whose critic is another instance of the generator inherits the generator's blind spots and launders confidence into consensus. If the critic cannot fail the generator on evidence the generator lacks, it is theatre.

5.3. Capabilities want extraction, but extraction is untested

Three skills did real work this session – deep research, this style pack, frontend design – each a packaged capability with its own contract (a SKILL.md, in that ecosystem's terms). The session also produced capabilities not yet extracted: boot-a-Solidus-from-cold, proxy-around-a-filtered-port, registry-from-a-pinned-OAS. The GNUmakefile is their provisional home – a skill package with the lid off. Refutation: a capability is reusable only once it survives removal from the context that produced it. Until the Solidus recipe runs on a machine that is not this one, "reusable" is a conjecture, not a property.

5.4. This document is Experience Replay, at three grains

Learning across sessions needs the trace to outlive the session. It survives here at three abstraction levels, each a different replay: the raw command output (what the systems actually returned), the refutation/refinement pairs (the tactic that survived), and the reproduction protocol (the method with the artifact subtracted). Read the lowest level to debug, the middle to reuse, the top to teach. A replay that keeps only one level is either unteachable or unverifiable.

5.5. Self-Heal closes on deterministic failures only

The deterministic failures healed cheaply: the two-package version constraint, the binstub off the PATH, the missing image library, the taxonomy name clash – each a fix-and-re-run cycle against a reproducible error, the build script's own fail markers closing the loop. The non-deterministic boundary – a host filtering one process's packets on one interface – did not heal. It needed a human-supplied reframe: route through a process the filter trusts. Boundary condition: Self-Heal is a function of reproducibility. A failure you cannot re-trigger on demand you cannot heal; you can only route around it, and routing around is the human's move, not the loop's.

5.6. The evaluation stack, five layers deep

Scattered checks become a methodology when they stack. The session's, bottom to top:

Layer What it answers Instrument this session
unit does one call return the right shape curl /api/products → 200, 3
contract are the fixture obligations met F4 401→200; F5 provenance tuple
formal do the invariants hold over all states TLC, 476 states, 4 invariants
integration does it work end-to-end in the runtime overlay lookup renders records
provenance are the claims themselves audited git-confirmed schema history

The contract and formal rows are backed by artifacts in aygp-dr/liquidus-overlay. F4 is the verify target in the repo's GNUmakefile (the no-bearer probe asserts 401; the bearer-carrying probe must return rows, which implies 200), and the README.org status line records the observed pair:

verify: ## Smoke-test a running harness (expects `make serve` in another shell)
	@set -e; B="Authorization: Bearer liquidus-dev-stub"; \
	curl -fsS http://localhost:$(PORT)/health >/dev/null && echo "health OK"; \
	test $$(curl -s -o /dev/null -w '%{http_code}' http://localhost:$(PORT)/products) = 401 \
	  && echo "F4 no-bearer -> 401 OK"; \
	test $$(curl -s -H "$$B" http://localhost:$(PORT)/products | node -e "...JSON.parse(d).data.length...") -ge 1 \
	  && echo "lookup /products returns rows OK"

README.org: Verified via the harness: registry builds (33 tags -> 38 commands),
            F4 bearer-presence enforced (401->200), spec-derived lookup paths
            return rows, bundle self-contained.

The TLC figures come from formal/README.org and formal/OverlayStates.cfg. The config names four Inv_* invariants plus a type invariant; the raw TLC transcript is not checked in, so the 476-state count is the repo's recorded result rather than a log this note re-ran:

formal/OverlayStates.cfg:
  CONSTANTS  Records = {r1, r2}  Backend = "Live"  Bearer = TRUE  MaxSeq = 3
  INVARIANT TypeInvariant
  INVARIANT Inv_Provenance
  INVARIANT Inv_Credential
  INVARIANT Inv_MonotoneSeq
  INVARIANT Inv_JumpCorrect
  PROPERTY  Live_Returnable

formal/README.org, Results (TLC, checked):
  Backend = "Live", Bearer = TRUE      All invariants + liveness hold.
                                       476 distinct reachable states.
  Backend = "Stateless" (Prism mock)   Inv_JumpCorrect REFUTED; 6-state
                                       counterexample: type a query, the mock
                                       renders a different record, the CSA
                                       opens the wrong one.

A lower layer passing tells you nothing about a higher one: unit-green endpoints sat beneath a formal invariant that a stateless backend refutes. The stack is ordered by what each layer cannot see below it. Evaluate up the stack, and distrust any layer reporting green while the layer above it has not run.

6. Reconstruction from the contract

The reproduction question has a sharper form: if the artifact were lost, what would an agent need to regrow it? The answer is small – spec.org and a pinned SHA – and the smallness is the point. The contract is not documentation of the system; it is the seed the system regrows from.

6.1. The seed

spec.org carries what a rebuild cannot infer: the invariants (a governance tuple on every response, presence-only auth, a bearer that is never a real token), the fixtures F1–F5 as executable obligations, the anti-goals (do not teach the mock to be stateful; do not expose the token), and the pinned commit. Everything else is derived, mechanically, from that commit: reify the upstream OpenAPI at the SHA, and the command registry, the mock, and the fixtures all fall out of it – thirty-three tags become thirty-eight commands with no hand-curation. The translation being mechanical is what makes the seed a seed rather than a sketch. An agent that hand-curates what the tags already encode has stopped rebuilding and started forking.

6.2. The three seams

Stitching the pieces is a matter of three seams, and each is where a rebuild tears if the version is wrong:

  • Core system ↔ spec version. The pinned SHA binds one Solidus commit's contract. The running store and the mocked contract must be the same version or the seam parts – a consumer built against paths the live system renamed gets silence, not an error. The drift study exists to measure the seam's tolerance: for this contract, near-total (25 paths unchanged across six years), so pinning is cheap insurance rather than a tax.
  • Contract ↔ tooling. Every tool hangs off the contract as a spoke – Prism mocks it, the TLA+ model consumes it, the overlay's registry derives from it, the fixtures grade against it. The contract is the hub; move the SHA and the spokes regenerate rather than drift. A tool that does not regenerate from the contract is not additional tooling, it is a second source of truth, and two sources of truth is zero.
  • Mock ↔ live. The presence-only stub and the authenticity-checked real key meet here, and this is the seam the session spent most of its refutations on: the mock's statelessness (a lookup cannot return the record it was asked for), requires_authentication refusing the session cookie, the host filtering the live port. The contract holds on both sides; the behaviour at the boundary does not, and no measure of contract fidelity papers over it. Instrument this seam or be surprised by it.

6.3. For the next rebuild

An agent handed spec.org and a SHA, aimed at related work, needs three things this session had to discover the hard way. A reification step that turns the commit into the derived surfaces – registry, mock, fixtures – rather than authoring them. A version-match check across the core↔spec seam before trusting any live call. And the mock↔live boundary treated as a first-class object with its own tests, not an implementation detail. Given those, the artifact is reconstructible from the seed. Absent them, the seed grows something that compiles and lies.

7. Invariants of the style

Properties that held across every system and must hold for the method to reproduce:

  • Provenance on every result. No datum crossed a boundary bare. Mock responses carried source · spec-SHA · backend; the overlay stamped its panels; the research agent separated git-confirmed from inferred. When you cannot trust a boundary, you can at least label what came through it.
  • Falsification before belief. Nothing was "working" until a command produced the evidence. curl before claim, TLC before "verified", gh api before "the spec says". The refutation catalogue above exists because the belief was always provisional.
  • One writable repo. Implementation lived only where implementation was permitted; the contract stayed copy-only. The three spec files were relocated to spec-input/ rather than edited, so a build could never silently fork the thing it was grading.
  • Long-running work detached, not awaited. The Solidus build, the vips install, the research agent – all backgrounded, watched for a terminal signal, never blocked on. The human kept steering while the slow systems churned.

8. Refutation conditions

This method is wrong, and this report should be revised, if:

  • A boundary is crossed with no way to instrument it – if provenance cannot be attached, the falsification-first stance degrades to trust, and the loop is running on faith.
  • The human's interjections stop re-selecting what matters and start correcting the machine's errors. That inverts the asymmetry: it means the machine is no longer congruent across systems, and the method is subsidising its incompetence with human attention.
  • The context does not survive the refutations – if each counterexample forces re-establishing prior boundaries, the cheap-refinement property is gone and the loop is just expensive trial and error with extra steps.

9. How to reproduce it

Not a checklist to be filled but a set of preconditions to arrange:

  1. Pin the contract, separate it from the implementation. A source of truth that cannot be edited by the thing consuming it. Copy it in; feed changes back through a graded report, never a silent edit.
  2. Give the machine one context and many tools. One conversation holding every boundary; ambient access to shell, git, browser, a model checker, cloud CLIs, and subagents for the archaeology you do not want in the main context.
  3. Make every claim produce evidence. Wire the loop so "it works" is unspeakable without a command that shows it. Prefer a counterexample-producing checker to a test suite that only confirms.
  4. Background the slow systems; watch for a terminal signal. Never await a build in the foreground. Poll for Listening on or FAILED, keep the frontier moving.
  5. Let the human supply the screen, in instalments. Do not demand a complete specification up front. The interjections are the spec; the method's job is to keep the context coherent enough that each one is a cheap refinement, not a restart.

The artifact will differ every time. The method is the invariant.

10. Provenance

Written from a single session, 2026-07-01, that produced aygp-dr/liquidus-overlay (public) – and written as a reference for related future work, the method rather than the artifact, so an agent handed only spec.org and a SHA can regrow the rest. Refutation catalogue is drawn from that session's command output – HTTP codes, TLC counterexamples, git diffs – not reconstructed from memory. Contract- history claims are git-confirmed against solidusio/solidus; see the build repo's docs/solidus-api-schema-drift.org. The spec-side history of the same grind is in the PRD grind note.