Grinding a PRD until it breaks
A contract-only Solidus toolkit spec stress-tested by 10 parallel implementations, then hit with a real live backend

I wanted to build a toolkit for the Solidus API (CLI, TUI, MCP server), but the target had a curious constraint: there was no live Solidus. Just an OpenAPI spec sitting in the upstream repo and a mock server (Prism) I could point it at. The axiom was stated up front:

This repo never talks to a running Solidus instance. Contract in, contract out.

The interesting question was not "can I build the toolkit?" It was: how do I tell whether the spec I write is good enough to build against? You cannot A/B a PRD; you can only try to falsify it. So the plan became: write the spec, then fork 10 isolated build repos across 6 languages, and let each one grade the spec by producing a FEEDBACK.md against it. Every FEEDBACK triggers a spec revision. Version bumps follow the crowsnest style guide: major for wire breaks, minor for additions. See if the spec survives 10 stresses.

The public spec has three normative files: README.org, spec.org, CLAUDE.md. Later I let two structural files in: a GNUmakefile that reifies the upstream OAS at a pinned SHA, and a .gitignore that keeps the reified copy out of the tree. And a docs/ directory for design artifacts that would bloat the spec proper.

Every grind repo (liquidus-001 through -010) consumes only the current public-spec commit and produces one implementation of one component:

Each repo took 5–15 minutes to build. The whole cycle from v0 to v1.9 (10 builds + 8 spec revisions) took 97 minutes.

The load-bearing finding of the whole exercise came from the first build. The v0 spec claimed paths like /api/v2/storefront/products (the modern Solidus JSONAPI). The reified OAS at the pinned SHA turned out to be the classic v1 Spree API, with flat paths like /products, /orders/mine. The mock served the v1 shape; the fixtures asserted the v2 shape. Nothing matched.

This was a v0 → v1.0 major bump, wire-breaking against consumers built to v0's stated paths. It would have been avoided by 30 seconds of curl | head against the pinned SHA before writing fixtures. The reify-first discipline became normative for the rest of the cycle: if the spec references an upstream artifact, run the fetch target and read the actual bytes before writing any assertion against them.

Every subsequent revision was minor: additive, non-wire-breaking. That was the whole point. Nine minor revisions later the spec was stable enough that a Rust thin-CLI author (liquidus-009) implemented against v1.8 and produced only clarification-class FEEDBACK, no correction-class.

The one durable technique across the cycle: every constraint in the spec was written as a bulleted refutation:

- If <observable>, then this spec is wrong and MUST be updated.

Not "the spec assumes X." Not "the spec asserts X." Explicitly: the exact observation that would prove the spec wrong. Two spec-level refutations fired during the first grind alone:

1. Tag → CLI-subcommand mapping. The v0 spec said each Solidus tag maps to one CLI subcommand family. The Orders tag actually spans two customer-flow stages (cart-during-checkout and post-order lookup). Refutation fired, §Component: CLI got rewritten around a flow-driven surface instead.
2. F3 statelessness surprise. The v0 spec called the session shim optional. Even the mocking-framework grind hit the "cart-add then cart-show returns divergent bodies" problem, which meant the shim was required at the cart stage, not (as v0 said) "later." §Session shim got promoted from optional to required.

The pattern: a refutation that fires is a load-bearing pin the spec was missing. If you write a PRD without them, you can only find those pins by having every consumer walk into them independently and complain. The refutation-condition regime turns "consumer complained" into "consumer verified a pre-registered failure mode", which is testable, gradeable, and cheap.

Prism in dynamic mode (prism mock -d) returns schema-shaped random Latin: strings become enim aute sit fugiat nulla, integers become random-int, and that's technically contract-conformant but useless for a demonstration. The spec had prohibited shipping code in the public repo. But the moment I wanted to wire an MCP server into a Claude Code session to actually show the toolkit working, I needed a mock that returned believable data: real product names, real prices, cart-add followed by cart-show returning the same body.

So v1.8 landed a deliberate constraint violation: mocks/msw/ (curated MSW mock with cart-consistency), demos/ (asciinema+agg scripts producing GIFs), and .mcp.json (wired the reference MCP server into Claude Code). Documented as an exception with its scope explicitly bounded. The mock's data must match the reified OAS shape; new endpoints not present in the OAS are forbidden; if the exception grows the spec is wrong, not the exception.

Naming what breaks the discipline is the way to preserve it. Silently letting scope creep happen is the failure mode. Publishing the exception with its own refutation conditions ("if it grows past mocks/ + demos/ + .mcp.json, that is a spec bug, not scope creep") kept the boundary honest.

At around v1.9 the person maintaining the mock team's infra spun up a real Solidus 4.7 instance on the LAN and handed me an admin API key, a URL, and a truth table:

Authorization: Bearer <key>   → 200 ✓
X-Spree-Token: <key>          → 401 ✗
?token=<key> (query param)    → 200 ✓
Authorization: Bearer <bad>   → 401 (validated, not presence-only)

The v2 spec draft I had been sketching in docs/spec-v2-draft.md had already proposed a LIQUIDUS_AUTH_HEADER env var to select between Bearer and X-Spree-Token. This truth table killed that speculation flat: Solidus 4.7 does not accept X-Spree-Token at all. The draft got rewritten in place; the env var was deleted; the ?token query param was folded in as a genuine second channel.

But the interesting part came from actually pointing the reference CLI (liquidus-006, Ruby / Thor / Faraday) at the live URL through a Node proxy on port :3002. The CLI got a 200 response with a body that jq refused to parse.

I initially framed the failure as "the CLI needs a patch." A push-back from the mock team ("I think it's a spec or mock or live issue") was correct. Two bugs, both spec-level, both would silently break every downstream consumer:

The content-type matcher. I had written the SDK's Faraday middleware with /\bjson$/ as the content-type regex. Real Solidus returns application/json; charset=utf-8. The regex anchored on end-of-string and did not match, so Faraday left the body as a raw string. Prism also returned application/problem+json on 200 responses (which is a Prism config bug; RFC 7807 reserves problem+json for errors), also not matched. The fix is one character, /\bjson\b/, which is word-bounded and matches all three cases. This is a normative rule every SDK in every language needs.

The base-URL path stripping. Faraday, treating a leading / in a resource path as absolute-from-host, quietly stripped the /api prefix out of SPREE_URL=http://host/api. The request went to /products (the storefront) instead of /api/products (the API). Real Solidus's storefront returned HTML with status 200. My CLI got HTML and thought it was fine. The fix: either append a trailing / to the base URL and use relative paths, or parse SPREE_URL and reassemble explicitly. Same normative rule for every Faraday-style client library (requests-python has the same trap; Node's fetch does not).

These two rules landed as spec v1.14 and v1.15. Both are changelog rows in spec.org, but v1.14 was never tagged; its content shipped in the single commit tagged v1.15 (the tag sequence is v1.13, v1.15). They are the reason I keep coming back to the spec is not what tells you how to build; the spec is what tells you what will silently break if you don't build carefully. Neither of these bugs is caught by any language's type system, any linter, any static analyzer. They fire only against a specific class of backend and only when the response media-type or base-URL happens to match a particular shape. The CLI works fine against Prism. Ships a happy-path test suite. Silently breaks against production the first time it's pointed at one.

The vocabulary I started the cycle with was: "is the spec correct?" That question has no answer. Specs are always at least partially wrong; the question is whether they name their own wrongness.

Better vocabulary: does the spec name enough tripwires that a wrong implementation will fire one before it gets to production? Under that question:

• v0 named 3 refutation conditions. Two fired in liquidus-001. One never fired at all (Prism doesn't validate token authenticity).
• v1.0 added the Customer-flow refutation family: 5 more conditions, one per flow stage. One fired in every subsequent consumer build.
• v1.4 added the MCP F5 acceptance-test rules. One fired in liquidus-005.
• v1.7 added the shim-required-at-cart rule and the shim+live-Solidus data-corruption rule. The latter has not fired yet, because liquidus-008's config.ru actively refuses to launch against a live URL.
• v1.14 (untagged) and v1.15 added SDK content-type and base-URL rules that surfaced from a live probe, not from a grind. This is different: an observation, not an axiom. Both survived being folded into refutation form: if a Content-Type header contains "json" as a word-bounded substring and the SDK rejects the body as non-JSON, the SDK is wrong.

That last category, observations promoted into refutations, is the one I did not have in my initial vocabulary. It's what makes a spec grow toward correctness rather than just accumulating clarifications. A spec that never sees a live backend can only encode what the author guessed would matter. A spec that gets hit by a real backend once, honestly, gets to encode what actually mattered.

Ten grind repos, but only two were promoted to public "reference implementation" status after the cycle:

• aygp-dr/liquidus-006: Ruby CLI + TUI (Thor + tty-prompt + Faraday). Solidus's native ecosystem; the fullest coverage of the customer-flow surface; the SDK layer verified end-to-end against real Solidus 4.7 at v0.1.3.
• aygp-dr/liquidus-005: Python MCP server (mcp SDK, F5 provenance wrapping). The reference for the MCP tool/resource/template surface.

The other 8 stayed private: grind artifacts, not references. Each produced its FEEDBACK.md, its version bump got folded, its author was done.

The rank order of "would I actually ship this to production" was interesting and not the same as the rank order of "which one was hardest to write":

• Ruby (liquidus-006): production-ready floor, because Solidus is Ruby.
• Rust (liquidus-009): the compiler catches the bearer-tri-state at build time better than any other language. If I were paranoid about correctness, Rust would win.
• Zig (liquidus-010): an honest thin wrapper that shells out to curl, because Zig 0.15's stdlib HTTP client would break me on the next release. Ecosystem-shape drives the implementation choice.
• Everything else is defensible for specific niches.

The one durable finding from the language survey: the best language for a consumer of a contract-only PRD is the one whose ecosystem shape matches the PRD's own discipline. Ruby matched Solidus. Python matched MCP's protocol ergonomics. Rust matched the bearer-tri-state's typing needs. Zig matched the thin-wrapper pattern's "just shell out" honesty.

If everything else in this writeup got compressed into a single sentence:

A PRD without refutation conditions cannot fail; a PRD that can fail cannot mislead.

The style guide, the version-bump policy, the fixtures, the feedback structure, the mock exception, the reference implementations, and the live probe are all scaffolding around that one idea. Everything else is variations on how to make the scaffolding structurally sound.

• Public spec: https://github.com/aygp-dr/liquidus (current head v1.15)
• Long-form retrospective: aygp-dr/liquidus/docs/writeup.md
• CLI + TUI reference: https://github.com/aygp-dr/liquidus-006 (public)
• MCP server reference: https://github.com/aygp-dr/liquidus-005 (public)
• v2 draft (live-Solidus mode): aygp-dr/liquidus/docs/spec-v2-draft.md
• The 8 grind repos: private (aygp-dr/liquidus-001 through -004, -007 through -010). Available on request; each is 1 KLOC + FEEDBACK.md.