wal.sh
home > research > 2026-q4-hardening-small-apps · Oct 29, 2026

From Spreadsheet to Service

application hardeningshadow itvibe codingcisosecrets managementdeployment tiersloggingobservabilitypiiecommerceproperty-based testingelenctic reviewagentic development

From Spreadsheet to Service
A guide for hardening small apps (deployment, debugging, logging and secrets), written for the CISO who has to sign off

Table of Contents

1. The situation

Every mid-sized ecommerce company runs on things nobody approved. A promotions spreadsheet decides which discount codes go live. A returns sheet with a macro approves refunds under a threshold. A script someone's agent wrote over a weekend rewrites the product feed before it goes to the marketplaces. A small internal app reconciles payouts. They work, people depend on them, and each one holds a credential, touches customer data, or moves money.

Agents have made this worse and better at once. Worse, because building such a tool now takes an afternoon, so there are more of them. Better, because the same agents can do most of the hardening, if someone says what "hardened" means and checks that it happened.

This guide is that definition. It is deliberately modest: the target is not a full software development life cycle (SDLC) for every sheet. It is a small app that a CISO can answer five questions about, with evidence rather than reassurance.

2. The five questions

Question Control Evidence that counts Evidence that does not
Where are its secrets, and who can read them? Secrets a list of secret names per environment, and the store they live in "they're in the environment"; a .env file in the repo
How does a change reach production? Deployment the path from commit to production, with the step that needs a human "we deploy from my laptop"
What does it record, and is any of it personal? Logging the logging schema, and a test that personal data cannot enter it "we don't log PII" (with no test)
When it breaks, how do we find out why? Debugging a failure reproduced from its record, by someone who was not there "we'd check the logs"
Does it do what the business thinks it does? The ladders the strongest rule it enforces, written as a property, with a run that exercised it "the tests are green"

The last row is the one that gets skipped, and it is the one an agent is worst at judging about its own work. It gets its own section: Green is not evidence.

3. Two ladders

The elenctic review introduced a correctness ladder: each level adds one kind of evidence that the app computes the right thing. Hardening needs a second ladder beside it, for whether the app can be run, changed and investigated safely. They are independent. A spreadsheet with property tests can still email its credentials around, and a perfectly deployed app can still accept a negative discount.

Level Correctness (does it compute the right thing?) Operations (can it be run, changed and investigated safely?)
0 manual checks runs on one person's machine
1 examples: two tests, the happy path and one edge case reproducible: anyone can start it from the repository with one command
2 contracts: inputs and outputs validated at the boundaries secrets out of code: named, tiered, loaded from a store; nothing secret on disk
3 properties: the strongest business rule, tested generatively tiered deployment: local, staging, production; production needs a deliberate step
4 proofs, where a rule warrants it (most never do) observable: structured records with personal data excluded by construction; failures reproducible

Each level assumes the ones below it. Most internal tools should reach 3 on both ladders, and many will stop at 2 on correctness, which is fine as long as everyone knows they have.

The rule carried over from the elenctic review: recommend exactly one next step. An owner can take one step this week. A roadmap tends to get filed and forgotten.

4. The four controls

4.1. Secrets

The failure mode in small apps is not exotic. A key is pasted into the code or a committed config file, copied to a teammate in chat, and never rotated because nobody knows everywhere it is.

The pattern that holds up:

  • Name every secret. Every secret has a structured name: project, environment tier, service, kind, purpose. The repository commits the list of names and nothing else. A reviewer can read that list, and it is the inventory the CISO asks for.
  • Tiers are separate. Local, development, test, staging and production each have their own entries. A test run cannot read a production key, because it asks for a different name, not because someone was careful.
  • Loaded at the door. Values come from the secret store into the process environment when a shell or service starts, and never land in a file. A per-checkout config file is for non-secret settings only (ports, feature flags).
  • Production takes an explicit, dated step. Pointing a working copy at staging or production requires an environment setting plus a dated confirmation in that copy's local config. A forgotten setting from last month does not count as a decision made today.
  • Agents get their own seats. Each automated worker gets its own scoped credential, so its access can be revoked and its actions attributed separately from a person's.
  • Redact by mechanism. Logs, status pages and dead-letter files hold a function that fetches the key, never the key itself, so there is no value to print by accident.

The CISO-level test: delete the secret store's entry for one tier and confirm that only that tier stops working.

4.2. Deployment

Three environments, and the rules for moving between them:

Environment Purpose Who can change it Must never
local (break-glass) development, and the fallback when everything else is down the owner depend on staging or production being up
staging the production shape with test data automated pipeline, after the gates pass hold production data or production credentials
production the business pipeline plus one deliberate human step be reachable by a change that skipped staging

Two lessons from building a small lab app this way, both easy to miss:

  • Isolate the data as well as the ports. Two copies of an app on one machine were given separate network ports, and the second still wrote to the first one's database, because the database path was relative to where it was started. In a small app the data path is where isolation tends to fail. Name databases, queues and buckets from the environment, the same way the ports are.
  • Assets and code deploy separately, even on one machine. A rebuilt front-end bundle went live while the server still ran the old code, and every request from the new page failed. Version them together, or restart them together.

Promotion should change the endpoint, never the names. A secret, a database or a queue is called the same thing in staging and production, apart from the tier field, so the configuration that passed in staging is the one that runs in production.

4.3. Logging

For ecommerce, the analytics layer is usually the largest unreviewed data flow in the company. Every page pushes events into a tag manager, and the tag manager forwards them to vendors. Small apps copy that pattern.

What holds up:

  • Logs are a projection, with a schema. The app's events (what it believed, what it did) are projected into records by one function with a written schema. Every sink reads that projection: a local file, an analytics vendor, a security information and event management (SIEM) system. Nothing writes ad hoc strings.
  • Personal data is excluded by construction. The projection reduces an email address to its domain, and the vendor-facing shape drops even that (GA4's own policy forbids email addresses). Any other @ in a value makes the record invalid instead of quietly passing it through. This is a tested property of the projection, not a clean-up job afterwards.
  • Every record carries its origin. Each record says which run or session produced it (a person, a test, an automated explorer) and carries an idempotency key, so a retry never double-counts and a test run never reaches the production numbers.
  • A failing sink never blocks the others. Each destination has its own queue, retries with backoff and a dead-letter file. Vendor tags are routinely blocked in the field, by ad blockers and privacy tools, and the app has to keep working when they are.
  • Order by sequence number. Records arrive out of order once there is more than one request in flight. Consumers sort by the sequence number in the record.

4.4. Debugging

A small app is debuggable when someone who was not there can reproduce a failure from what the app recorded. That needs the same evidence a test needs:

Evidence In a small app Why it matters
source the repository, at a known commit what was supposed to happen
state the page or screen as the user saw it what the user saw, which is not always what the code meant
records the structured log above, with run and sequence ids what the app says happened
data the rows the action wrote what actually changed
telemetry traces and metrics, joined to the above by a trace id the only evidence that survives into production, where nobody can attach a debugger

The rule: a claim about the system is checked by joining two or more of these, never by reading one. A green status page with no matching rows is not a success. Nor is a row with no record of where it came from.

Telemetry is worth a sentence of its own. Once the app is deployed, its source, its database and an interactive session are out of reach. Telemetry, read by trace id, is what an engineer or an agent on call has to go on. Reading production telemetry should be an ask-first action, and span attributes follow the same no-personal-data rule as logs.

5. Green is not evidence

This section is the most important one for a CISO, and the least intuitive.

In a lab project built for this guide, an automated explorer (Bombadil, a property-based tester for web pages) ran 99 sessions against 33 signup forms: 55,731 page states, zero violations, all green on the dashboard. Then we asked what the green had checked. The answer was nothing about the forms. The suite carried only the tool's default properties (no crash, no console error, no failed request), and none of them read a field, a rule or an error. When faults were planted on purpose (an error shown on the wrong field, a form that submits while invalid, a confirm field that is never compared) the old suite caught 0 of 30.

Four properties that actually described the forms caught 19 of 30. Adding a generator that could finish a form, so the success path was reached at all, raised that to every fault that could apply (26 of 26). The same forms, the same tool and the same machine gave a very different result, because the definition of "checked" changed.

What to ask for instead of "the tests pass":

  • Which rule does it check, and how often was that rule exercised? A property whose trigger never occurred passed vacuously. Ask for the count, and require a run to fail when a rule was never exercised.
  • Has it caught a planted fault? Break the rule on purpose in a copy. If the checks stay green, they are not checking that rule.
  • Was the result read by someone? The first violations an automated checker reports are often the specification's fault rather than the app's. Someone has to read them before anyone counts them.

6. Where the effort goes

The expectation is that hardening is work on the application. With agents doing the building, it mostly is not. The same lab project kept a catalogue of every incident that blocked, crashed or conflicted: 71 so far, of which 51 were outside the application under test. They were in isolation (ports, databases, test litter), process (two workers editing one file, a test-only dependency breaking the production start), observability (dashboards reading "no news" as "good news") and the specification itself.

For a CISO that changes where to look. A small app's risk sits less in its business logic than in what surrounds it: the credential it was given, the machine it runs on, what it logs, and whether anyone can tell when it is wrong. Those are also the parts an agent will not harden unless asked, and the parts where its work is hardest to check.

7. A worked example: the promotions sheet

A merchandising team keeps a spreadsheet of discount codes: code, percentage or amount, start and end dates, eligible categories, a stacking flag. A macro exports it nightly to the storefront. Someone asks an agent to "turn it into an app". The steps below follow both ladders, one step at a time, starting from correctness 0 and operations 0.

Step Ladder Action What it closes
1 operations 1 the app starts from the repository with one command; the storefront credential moves out of the macro the sheet no longer contains the key
2 correctness 1 two tests: a 10% code on an eligible item; a code past its end date the obvious mistakes
3 operations 2 secrets named and tiered; the test tier points at a storefront sandbox a test can no longer publish a real code
4 correctness 2 validation at the boundary: no negative amounts, no end before start, no percentage over 100 the input errors that become pricing incidents
5 operations 3 staging, plus a deliberate step for production publication an unreviewed change cannot publish codes
6 correctness 3 one property: no combination of stackable codes takes an order below the margin floor the expensive mistake, found by search rather than by a customer
7 operations 4 every publication is a record (who, which codes, from which commit); codes are logged, customers never are "who published this code?" has an answer

Step 6 is where generative testing earns its keep. Nobody writes the example with four stacked codes on a clearance item, but a property tester tries it in seconds. Then plant the fault: remove the margin check in a copy and confirm the property fails.

8. How to run it as a programme

  • Inventory first. Ask each team for the sheets and small tools they could not work a day without. For each one, record the two ladder levels and the single next step. That table is the programme.
  • Spec, review, decide, build. Each conversion starts with a short written spec: what must hold, what would show it does not, what the agent may and may not do. An independent reviewer reads the spec, not the author's reasoning. Decisions the business owns (cost, data retention, who may publish) go into one tracked issue and are answered there, so work that does not depend on them is not held up.
  • Three-tier boundaries for the agents. Always (read code, run tests), ask first (touch staging, read production telemetry), never (read production secrets, change the deploy pipeline, store customer data in logs). Write them down per app. Ambiguity goes to ask-first.
  • Count incidents as well as defects. Keep a log of everything that blocked, crashed or conflicted during the conversion. It shows where the programme is really spending its time.

9. Questions to bring to the first review

  1. What are the secret names for each environment, and where is the store?
  2. Which step in the path to production needs a person, and is that step recorded?
  3. Show me the logging schema, and the test that keeps personal data out of it.
  4. Pick a failure from last month. Can someone who was not there reproduce it from the records?
  5. What is the strongest business rule this app enforces? Show me the property, how many times the last run exercised it, and a planted fault it caught.
  6. What is the single next step on each ladder?

10. Limits

  • The numbers come from one lab project with rebuilt historical forms, not from a production ecommerce system. They show the shape of the problem, not its size at any particular company.
  • Level 3 on both ladders is not a security certification. It is a floor, below which an app should not hold a production credential or customer data.
  • None of this replaces threat modelling for apps that move money at scale. It is meant for the long tail of small tools that will never get one.

11. See also