Experiments
Table of Contents
The site runs a client-side feature-gate SDK modeled on Statsig's client
payload shape. There is no server: the gate registry is a static EDN file
served from /static/experiments.json (exported from data/experiments.edn by gmake experiments-json) and evaluated in the browser.
The user id is a persistent uuid in localStorage under wal-sh.exp.uid
— the local analogue of Statsig's analyticsAnonymousId.
Why
Two use cases pulled the same shape:
- Tool rollouts. Ship a new
/tools/SPA behind a gate, ramp the percentage, and check nothing regresses before flipping to 100%. - Content variants. Split-test a page's layout, copy, or ordering with the same primitives, and record which arm the reader saw.
The 5 tools scaffolded from RubyConf 2026 (message-format, mutation,
fib, gc-viz, fair-queue) each ship as a feature gate. Turn one off
and its dropdown/link in site/tools/index disappears at render time.
Shape
Two record types, both keyed by string id:
feature-gates— return a boolean. Wrong id =false(fail-closed).dynamic-configs— return a structured value. Wrong id =nil.
Both types share a :rules list. Each rule has:
| Field | Meaning |
|---|---|
:id |
rule name (opaque; used for exposure attribution) |
:percentage |
0-100. Omitted = 100. |
:salt |
changes the bucketing partition without renaming gates |
:value |
the value returned when this rule matches |
:secondary-exposures |
list of { :gate id :value bool } dependency checks |
Rules are matched top-to-bottom; the first that matches wins. A
:default rule (no percentage) at the bottom is the fallthrough.
Bucketing
Bucket assignment is deterministic on
djb2(user-id ":" salt ":" gate-id) mod 10000. Djb2 was picked to match
Statsig's hash_used: "djb2" — same-user-same-gate always land in the
same bucket, which means a 50% rollout is stable across page loads.
(require '[wal-sh.experiments.core :as e])
(def user {:analytics-anonymous-id "aaaa-bbbb"})
(e/bucket (:analytics-anonymous-id user) "tool.gc-viz" "gc")
That 8642 falls above 5000, so the 50% gc-viz gate returns false
for this user. Change the salt from gc to gc2 and the bucketing
partition rotates — the user's arm changes without renaming the gate.
Currently gated
Sourced from data/experiments.edn; gmake deploy-experiments-json ships it. Change there, no bundle rebuild
required: reload the page and the browser client refetches.
| Gate | Rollout | Depends on | Purpose |
|---|---|---|---|
tool.message-format |
100% | — | RubyConf 2026 — Fung |
tool.mutation |
100% | — | RubyConf 2026 — Schirp |
tool.fib |
100% | — | RubyConf 2026 — d'Oliveira |
tool.gc-viz |
50% | tool.mutation |
RubyConf 2026 — Zhu (canary) |
tool.fair-queue |
100% | — | RubyConf 2026 — Baygeldin |
tool.display |
100% | — | 16-colour grid sink, spec v0.3.1 |
tool.pasted-file-editor |
100% | — | editor with a paste threshold (after Simon Willison) |
tool.dinner-split |
0% | — | scaffold: core and tests, no UI yet |
tool.gas-town-tla |
0% | — | scaffold: core and tests, no UI yet |
tool.mono-watch |
0% | — | scaffold: core and tests, no UI yet |
tool.order-state-flow |
0% | — | scaffold: core and tests, no UI yet |
tool.spect8r |
0% | — | scaffold: core and tests, no UI yet |
ui.tools-nav-experiments-section |
100% | — | Show RubyConf-adjacent bucket |
The tool.gc-viz canary demonstrates two features at once:
percentage-based rollout (50%) and secondary exposure (only users
already in the tool.mutation arm are eligible).
Flags
A second, simpler mechanism sits next to the gates above: a flat per-browser
on/off switch, not a rollout. src/wal_sh/flags/core.cljc +
src/wal_sh/flags/browser.cljs declare each flag once in a registry and
resolve it by one fixed precedence: url-param > localStorage > env >
default. A decisive ?param=on|off also writes itself into the flag's
localStorage key, so it sticks on later visits without the param.
| Flag | Param | localStorage key |
Default | Purpose |
|---|---|---|---|---|
ecommerce |
?shop |
wal_shop_flag |
off | site-wide commerce/checkout |
verdict-badges |
?verdict |
wal_verdict_badge_flag |
off | show verification badges (verdict · checker · date) under headings |
verdict-badges is hand-written vanilla JS
(site/static/js/verdict-badge-flag.js), not the compiled flags.js
bundle the ecommerce flag uses — same precedence and naming convention,
but the compiled bundle is ~21KB gzipped (Closure/cljs runtime) against
every other sitewide postamble script being under 3KB, and this flag needs
no fetch, so there was nothing to gain from the bundle.
To turn verdict badges on for yourself: visit any page once with
?verdict=on appended — it sticks via localStorage, no need to repeat the
param. Equivalent from the devtools console:
localStorage.setItem('wal_verdict_badge_flag', 'on'), then reload. Reverse
with ?verdict=off or the matching setItem(..., 'off'); remove the key
entirely to fall back to the site default (hidden).
Exposures
The client records an exposure event whenever gate? or config-value
resolves. Events are POSTed to the local crowsnest receiver at
localhost:8127/sightings when it's running; otherwise buffered in
sessionStorage. See crowsnest for the receiver.
Statsig shape reference
The record shape came from watching the wire on
events.atlassian.com/teameurope/agenda — Atlassian ships Statsig's
Java server SDK (v3.1.3) with djb2 hashing and analyticsAnonymousId
as the primary id. We keep the same primitives but replace the service
call with a static file, which is enough for a single-author site.
Related
- tools — SPAs that read gates
- crowsnest — exposure receiver
- Team '26 Europe — the Statsig payload came from that event's client