When Hardening Comes Due
Six triggers for agent-built work, the context they were earned in, and a re-trigger for work already running
Table of Contents
Overview
Vibe coding means constant change with nobody managing it: no sprint, no gate review, no launch date to hang a checklist on. Checklists such as the OWASP Top 10 already say what to secure. What is missing is when: which event, noticed as it happens, makes a given safeguard due.
The answer here is that hardening comes due when work moves to a new context. A move raises the work's audience or its lifetime, or changes the authority it runs with. A move is an event an agent can detect, where a launch date or a user count is not.
The evidence is one repository's: experiments 016 and 017 of a Bombadil and Clojure lab, where a ticket-sales app called eventstr was built and broken on purpose. One repository is enough evidence for a skill; a synthesis needs two. See Limits.
Where agent work runs
Live mutants sorted agent work by Getting Things Done context. For hardening, what matters is less what each context can build than what it starts with.
| context | starts with | secrets | outlives the session | provided / owed |
|---|---|---|---|---|
| 1. chat, clean session | nothing; throwaway code execution | none to leak | no; the platform hosts artifacts | deployable comes free; contract and handoff are owed when a scaffold or spec leaves the chat; secure is owed when a published page gets a second viewer |
| 2. Desktop, clean environment | no working directory, no mounted secrets | none, except what connectors grant | no | as 1; a connector that can write is mounted authority and brings back secure |
| 3. Claude Code in the working directory | your shell, files, keychain, git push, long jobs |
ambient: reachable whether or not declared | yes, as background jobs (capped at 2 h here) | nothing comes free; all six triggers apply, plus keeping secrets out |
| 4. harness in a clean sandbox (sbx, Docker, Bedrock) | an image and declared configuration | injected by name; environments are built in | the harness owns the lifecycle | secrets and part of deployable come from the platform; contract, reachable, observable and handoff are still owed |
All the evidence below comes from context 3, the one with the most ambient authority and the fewest built-in safeguards. The triggers are written for it.
Three rules about the direction of a move fall out of the table:
- Moving into context 3 is a trigger in itself. A scaffold from chat that lands in your working directory gains authority it never had, so keeping secrets out of it is due then, not later.
- Moving from 3 to 4 is the cheapest hardening there is. A clean sandbox makes every secret, path and dependency nobody declared fail loudly. The move takes the inventory for you.
- Moving up the table needs no hardening, only a reseal. When evidence goes back into a chat or a spec, write down the rule the run taught you.
Six triggers
Each trigger is a fact about the new context, not a judgement about quality, and each was earned by a failure.
| harden | due when, in the new context | the failure that earned it |
|---|---|---|
| deployable | it outlives the session or runs with nobody watching | a live dashboard, run as a background job, was killed twice at the 2-hour job limit while it was being watched |
| contract | two components exchange data | the server sent kebab-case keys (expires-at), the page read snake_case (expires_at); each side was fine alone, and only the first run through both found it |
| reachable | it is reached through a real entry point | an unbalanced paren in routes.clj kept the server from starting; the unit tests never loaded the routes, so they passed |
| secure | a second person or agent acts on it, or relies on its state | a view that showed every user's ticket holds to every user passed all ten server invariants; a check on what the page shows caught it in 82 ms (a planted mutant; the blindness was real) |
| observable | a check's silence will be read as "fine" | an accessibility check starved behind a debounce, and a layout-shift check went blind because the page re-rendered by replacing its HTML; both reported clean |
| handoff | someone else ships, runs or reads it | a research page merged to main but was never deployed: deploy belonged to another lane, and no owner was named |
For each true trigger the minimum evidence is a safeguard seen to fire: kill the process, rename the field, unmount the route, leak a record into the view, blind the check, skip the deploy. A safeguard nobody has watched fail is a hope.
The triggers ship as a staging skill, harden-on-context-change, in the house skills
repository, with five behavior evals not yet run.
A re-trigger for work already running
The triggers fire at moves. Work that has already moved keeps decaying (Lehman's point, below), and nothing re-fires them. SRE's error budget is the mechanism built for this: a service promises a level, measures against it, and when the budget is spent, reliability work takes priority over features.
A coordinator agent working in context 3 can carry the same thing, but not as a reminder in its prompt. "Consider error budgets" is a prose rule of the "be careful with secrets" kind, and context 3 keeps nothing across sessions except files. The error budget's working parts are mechanical:
- Declare the promise at the move. When the triggers are checked, write one ledger row in the
repository: what the work promises, where that is measured, the budget, the owner. For
example: "the eventstr clean run stays silent; source
runs/; one unexplained failure per 20 runs; owner: me". Rows are written only at moves, so the rule against hardening what isn't moving still holds. - Measure from evidence that already exists. Agent work has its own measures: false alarms in clean runs, checks that report they never ran, "waiting for" items older than N days, background jobs that hit the cap.
- Have a policy that preempts. When a budget is spent, the coordinator opens an issue and starts no feature work on that item until it is closed. That rule is what makes a budget a budget and not a dashboard.
The check belongs in a script the coordinator runs at session start or weekly review, with the ledger in the repository. A watcher in context 3 dies with the session, which is the dashboard failure again.
What the literature says
The parts are all well studied. Less common is the trigger: the literature mostly keys hardening to a date (a launch, a gate review) or a scale (users, a spent budget).
When to seal a spec
Boehm's anchor points (LCO, LCA, IOC; IEEE Software, 1996) commit at milestones where the remaining risk is judged small enough. Poppendieck's last responsible moment (Lean Software Development, 2003) defers a decision until not deciding costs more. Cooper's stage-gate (1990) puts a review between phases. They agree with resealing a spec, but tie it to risk or a meeting; here it is tied to a rule a run has taught.
When to harden for production
The production readiness review and launch checklists in Beyer et al., Site Reliability Engineering (2016), cover everything the six triggers do and more, and assume a service team and a launch date. The same book's error budgets are the one measured trigger, borrowed above. Hamilton's "On Designing and Deploying Internet-Scale Services" (LISA 2007) is an operations checklist. Beck's Explore / Expand / Extract (essays and talks, around 2016) is the closest match: practices that are harmful while exploring become necessary when the bottleneck moves. Wardley's evolution stages make the same point about components.
Moving to a different context
Cynefin (Snowden and Boone, Harvard Business Review, 2007) ties method to domain: checklists start to pay when a problem moves from complex to complicated. Lehman's laws (1980) say software in a changing world must keep changing, and its quality declines unless someone works to keep it, which is why the re-trigger is needed. Parnas's "Software Aging" (ICSE 1994), Cunningham's debt metaphor (1992) and Sculley et al. on machine-learning systems (NeurIPS 2015) describe the decay. Hyrum's law (Software Engineering at Google, 2020) explains why a second user is a trigger rather than a scale threshold: once someone depends on a behaviour, it is part of the contract.
Meta-reflection
Schön (The Reflective Practitioner, 1983) separates reflection in action from reflection on it. Argyris and Schön's double-loop learning (1978) changes the governing rule rather than the action; resealing a spec is double-loop, patching the instance is single-loop. Blameless postmortems (Allspaw, 2012; the SRE book) are the team form. The drift literature (Rasmussen, 1997; Vaughan on the normalization of deviance, 1996; Cook, "How Complex Systems Fail", 1998; Dekker, Drift into Failure, 2011) is the theory behind the observable trigger: a check that has gone quiet becomes the accepted normal. Agent self-reflection methods such as Reflexion (Shinn et al., 2023) and Self-Refine (Madaan et al., 2023) reflect inside a task and have no notion of the work leaving its context.
Checklists come after the trigger
Practitioner guides say what to secure. Engadget's beginner list for vibe coding (2026-11-01) is typical: write a specific spec and a rules file, keep keys out of chat and code, parameterize queries and use row-level security, rate-limit, and review. Against the six triggers it adds three gaps (secrets, supply chain, rate limiting) that nothing in the lab repository has yet failed on. It also disagrees on one: having a model review its own work, with no planted fault, is the check that reports clean. Use a checklist once a trigger has fired, and take its numbers from primary sources; the guide's "four times as productive, ten times the security issues" comes from a security vendor.
Moving an app to another context
Most hardening work in practice is a transfer: an app built in one context is moved to another. A prototype from a hosted agent platform goes into a repository, or a repository's service goes into a sandbox or onto a cloud platform. The rule for a transfer follows from the contexts table: whatever the source context gave for free becomes owed, and whatever the target gives for free can be dropped.
A hosted agent platform hands over a running app with deploy, secrets and a URL already solved, and nothing that says what the app is supposed to do. Moving it into a repository loses the first three and gains nothing on the fourth unless someone writes it down. The transfer, in order:
- Inventory what the source provided. Deploy, secrets, a database, auth, logs. Each becomes a line in the new repository's README with who now provides it, or a trigger that is now true.
- Write the spec the app never had. What it promises, its invariants, its boundaries. On a hosted platform the agent's chat history is the only record of intent, and it does not travel.
- Find any second implementation. A companion that re-implements another system's rules (a visualizer that recomputes what a simulator computes, a client that re-derives a server's pricing) is a contract between two implementations with no test. Either share conformance fixtures between them, or make the companion replay the original's output so there is only one implementation.
- Run the triggers against the target context, and plant the fault for each one that is true.
- Move the credentials last. Bringing a credential into the target is a move in its own right (see the next section). In context 3 it becomes ambient; in context 4 it is injected by name.
Hosting platforms and clouds cover the generic rows: deploy, secrets, telemetry, and infrastructure fault injection (killing instances, adding latency). The re-trigger is covered too, once the app is deployed: managed SLOs with burn-rate alerts. None of them covers the rows that depend on what the app means: the contract, the invariants, faults in the app's own logic, and checks on the checks. A platform agent that "tests its own app" sits on the wrong side of the observable trigger: a model reviewing its own output, with no planted fault.
An open trigger: acting on the world
A credential that can spend money or start something physical (a payment, an order, a fabrication job) adds a move the six triggers don't cover. Every other mistake above can be rolled back; an order that has shipped cannot. Nothing in the lab repository has failed this way, so under the skills rule it stays a candidate until one does:
| harden | minimum evidence |
|---|---|
| quote, don't order | the default path stops at a quote or a dry run; ordering is a separate step a human runs |
| checks before submitting | the design, file or payload passes its own checks before it leaves the machine, so the provider's checks are a second line, not the first |
| idempotency | each order carries a key derived from what is being ordered, so a retry cannot order twice (eventstr's replay mutant, with real money) |
| cap and ledger | a spend limit per order and per month; one ledger row per quote and per order |
| credential scope | the ordering credential is never in the agent's shell; where a provider separates quoting from ordering keys, only the quoting key goes near an agent |
The classic planted fault is units: a part designed in inches and read as millimetres comes out 25.4 times too small. An invariant on the expected size catches it before a quote is requested.
By kind of application
| application | dominant trigger | fault to plant | the platform covers | you own |
|---|---|---|---|---|
| visualizer or companion app | contract | the other system changes a rule on its side only | hosting | replay or shared fixtures, not a second implementation |
| interactive web app (eventstr) | secure, contract | a view that leaks other users' records; a stale response painted over a fresh one | deploy, secrets, telemetry | page and server invariants |
| data pipeline | contract, observable | a column renamed upstream; a day of zero rows | the scheduler, retries | schema checks, a row-count invariant |
| agent holding tool credentials | secure, acts on the world | an injected instruction to order more | key storage | scope, a cap, a human approval gate |
| ordering or fabrication flow | acts on the world, contract | the units mistake; a missing file in the bundle | quoting, the provider's own checks | checks before submitting, idempotency, the ledger |
Porting, not greenfield
Most business software is not written from nothing. It is ported: a spreadsheet becomes an application, a manual process becomes a service, a codebase moves to another language or runtime. A port has something greenfield work never has, an existing system that is already right about most things, including things nobody wrote down. The hardening question for a port is what plays the oracle, and how to keep it from moving.
There are two starting points, and they need opposite first steps.
No tests: the oracle has to be built
A spreadsheet, a mail-merge, a month-end checklist: software or process with no tests at all. The existing system is the spec, bugs included, and its behaviour lives in places a port will not look: a hidden column, a manually overridden cell, a formula that differs on one row, a step someone does because "it always breaks otherwise". Spreadsheets are known to carry errors at rates that make this matter (Panko, Journal of End User Computing, 1998); a range that missed five rows changed a widely cited economics result (Herndon, Ash and Pollin, 2013).
- Capture before changing anything. Run the existing system on real inputs and record its outputs as golden files: characterization tests in the sense Feathers gives them (Working Effectively with Legacy Code, 2004). Record the bugs too; deciding which are bugs comes later.
- Inventory the manual parts. Every override, hidden range and human step is a line in the inventory, as in the transfer steps. Each one is either ported, made explicit, or dropped on purpose.
- Lock the golden set, then plant faults the port must not make: a rounding mode changed, a range one row short, a date read in the wrong locale. The golden set has to catch each one.
- Run old and new side by side on fresh inputs until they agree: differential testing (McKeeman, Digital Technical Journal, 1998). Disagreements are either a port bug or an old bug; the second kind goes to the owner and stays out of the port.
Existing tests: the oracle has to be locked
A codebase with a test suite has an oracle already. The danger is that an agent, asked to make the port pass, changes the oracle instead: an agent's goal is green, and the cheapest green is a test edited to match the code. That is specification gaming, and the lock has to be mechanical, not a sentence in the prompt.
- Only boundary tests are the oracle. Tests that reach into internals have to be ported along with the code, so they move with it and cannot judge it. The end-to-end tests at the public boundary (the CLI, the API, the file format) are the ones that hold still.
- Make the oracle read-only to the porting agent. Deny rules on the test directory, code owners on it, branch protection. Close the escape hatches too: a CI override label or a "skip" flag that a human may apply is one an agent may apply.
- Port one-to-one, bugs included. A port that also fixes bugs cannot be checked against the old behaviour. Fixes are separate changes, after the port, each with its own test. An agent that "helpfully" tightens a check mid-port has made a change nobody asked for and the oracle can't see.
- Delete the old copy as each piece is ported. If the source keeps changing while the port runs, a change to a file that has already been ported should fail loudly as a conflict, not land silently in a copy nobody runs. This is the second-implementation contract from the transfer steps, closed by having only one implementation at a time.
- Expect the scope to move. A source that others are still adding to grows while it is ported, and code that looked like a separate layer turns out to be tangled with the part being moved. Count the source again at each milestone; the first estimate is a lower bound.
A trigger particular to ports: the execution model changes
A port that changes how code runs gains failures that neither the old tests nor the new compiler see. A single-threaded event loop becomes threads, a spreadsheet's recalculation order becomes a batch job's statement order, a synchronous call becomes a queue. Every assumption the old model made for free (one thing at a time, this cell before that one) is now owed. Plant the fault: run two requests at once, reorder two steps, deliver a message twice. The stale response in eventstr, painted over a fresh one after a tab switch, is the small version of this.
Limits
- One repository. Every failure above comes from one lab repository in one context. The three-tier rule for these notes asks for two repositories that agree; this note is ahead of that, and should be revised when the triggers are applied to a second project.
- Citations not yet audited. The references in What the literature says were written from memory, and the search budget ran out before they could be checked. Venues and years may be wrong; chapter numbers are left out on purpose.
- Context 4 is untested here. What a sandbox provides is inferred from how such harnesses are built; no run in this repository used one.
- The re-trigger is a design. No ledger has been kept yet.
- The transfer steps and the acting-on-the-world trigger are unearned. No app has been moved through them yet, and no order has gone wrong.
- The porting steps are drawn from practice reported elsewhere, not from a port done here.