A skill is a unit under test. Before you trust one, give it what any experiment needs: a boundary, an oracle that has failed, a blind, and held variables.

Agent skills are being shipped the way scripts were in 1995: written, run once, trusted. Treat one as an experiment instead and four controls fall out. Scope fixes the unit: least-privilege tools, a declared environment, and a trigger description that has near-misses it must not fire on. Evals are the oracle, and an oracle is trusted only after it has been seen to fail on known-bad input. A sealed spec is the blind: the builder gets the document and nothing else, and the grader keeps a holdout the builder never saw. The environment is the variable you forgot to hold: one task, one 4B model, three harnesses, and the pass rate went 3/5, 0/5, 0/5, for reasons that had nothing to do with the skill. Five minutes, four slides of evidence, one checklist.

(133 words. Shorter cut for forms with a 500-character limit:)

An agent skill is a unit under test. Four controls make one trustworthy: scope (least-privilege, near-miss triggers), evals that have been seen to fail, a sealed spec with a hidden holdout, and a held environment. One task, one model, three harnesses: 3/5, 0/5, 0/5. The skill didn't change.

1. The claim (0:30). "Works on my machine" is a result with no controls.
2. Scope (0:50). One SKILL.md, its allowed-tools, and the near-miss query it must not fire on. Implicit guardrails: the action was never on the menu.
3. Evals (1:00). A gate that passes a hardcoded solution. Fix the gate; watch it go red. The calibration set is the negative control.
4. Sealed spec (0:50). What "sealed" means; the hidden holdout; why findings go to the next draft and never back into the sealed one.
5. Environment (1:10). The three-harness table. Claude Code wrote no file four times out of five because the model put code in prose and thinking was on by default; Aider's test command covered one mode of two. The skill was constant; the room wasn't.
6. Checklist (0:40). Four lines, one per control, and the QR code to the note.

A 20-minute cut adds the protocol: preregister the hypothesis, calibrate the gate, run, log deviations before reading results.

Audience

Anyone shipping agent skills, prompts or eval suites; QA, SRE and test engineers who have not looked at agents and will recognise every control.

Format

5-minute lightning first. Cuts to 10 (Babashka Conf, Clojure lightning slots) and 20 (MinuteCon lightning, SPLASH workshop).

Where, and the angle for each

Venue	Slot	Angle
MinuteCon 2027, AI Security track	20-minute lightning	scope as capability: a skill's tool list is a sandbox boundary; the environment control as a security property
AI Engineer World's Fair 2027	lightning or Evals track	the practitioner cut; goes one step past "ship evals" to "calibrate them"
SPLASH 2027, SpecOps or LMPL	workshop talk or short paper	the sealed spec as a blinding protocol; holdouts and spec drafts as experimental design
European Lisp Symposium 2027	lightning	the one-slide version with a REPL-run gate going red
Dutch Clojure Days or Babashka Conf 2027	10-minute lightning	skills written and gated in bb; the calibration demo live

CFP dates for all but MinuteCon are unchecked; see conferences still ahead.

The committee will ask what this adds. The talks it sits beside:

• Philipp Schmid, Don't Ship Skills Without Evals, and Agents Without Code: How Skills, YAML, and Filesystems Replaced Python (AI Engineer World's Fair 2026). The nearest competitor. This talk takes the eval as given and asks whether it can fail.
• Vinoth Govindarajan, Your Agent Didn't Fail. Your Harness Did., and Ameya Bhatawdekar, Your Agent Evolved. Your Evals Didn't. (same event). The environment and drift controls, argued from production.
• Maxime Rivest and Isaac Miller, The Unreasonable Effectiveness of Separating the Task from the Model (same event). The task–model–harness seam that the hold-the-model run measures.
• Nate Aune, Does Your Coding Harness Actually Do Anything? Measuring It, Fairly (AI Tinkerers Boston, 2026-06-29). Holds harness and model, varies the skill framework, blind judge. The mirror image of the run here, which holds the model and varies the harness.
• Michele Tufano et al., Grounding AI Agents in Contracts, and Mark Marron, Specifications for Humans, Agents, and Tooling (SPLASH 2026 SpecOps). Specs as agent input; neither seals the spec from the builder.
• Shuvendu Lahiri, Intent Formalization (SPLASH 2026 SpecOps). How good AI-written specs are; the reverse direction.
• Aviv Donenfeld, LGTM: Bypassing an LLM Build Gate When Prompt Injection Fails (DEF CON 34). A judge gate defeated without injection: the uncalibrated oracle, from the offensive side.
• Datura and Red-Teaming Coding Agents from a Tool-Invocation Perspective (ISSTA at SPLASH 2026). The scope control, attacked.

Not yet reviewed. The question to expect: "this is just testing." Yes; the claim is that skills are skipping it, and the hold-the-model table is the evidence that the uncontrolled variable is large. The risk is breadth: four controls in five minutes leaves one slide each, so the environment slide has to carry the talk. If a venue gives only five minutes, drop the sealed spec to one sentence and keep the table.

Overlaps: absorbs The Failure That Looks Like Success as control two; do not submit both to the same venue. Points at What Can Your Agent See? for the scope control.

Notes on this site:

• 2026 Q2 Skills: Context, Caching, and the Sandbox Lens (scope)
• Identity Store Specification: A Stack of Drafts (sealed spec)
• Rebuilding from the spec (one oracle, N rebuilds)
• CLI Coding Agents, Q2 2026 (sandbox, harnesses)

Not public yet: the skill promotion contract and its evals (private repository), the eval-construction checklist and the Q4 harness note (drafts), and the hold-the-model run (private experiment repository). The talk can be given from them; the abstract should not link them until they are out.