Browser adtech infrastructure — tracking, A/B, feature flags, sponsored slots
Table of Contents
- 1. Purpose
- 2. Ownership map
- 3. Client-side tracking
- 4. A/B testing (existing ab-engine)
- 5. Feature flags
- 6. Beacons
- 7. Attribution-audit (v5)
- 8. Intent-signals (v6)
- 9. Exit-intent (v6)
- 10. Stuffing-detectors (v5 addendum)
- 11. Sponsored formats
- 12. Plate generation pipeline (v3, partial rewrite)
- 13. Slot placement contract (v4 draft)
- 14. Search + testing on load / dwell / scroll / slots
- 15. Search-mount v7 spec (captured, not landed)
- 16. Open work
- 17. Cross-references
1. Purpose
wal.sh runs a full client-side adtech stack — as parody, as research instrumentation, and as the ambient setting the site is written for. This note is the reference: what module owns what, where the boundaries sit, and where each piece of behaviour is implemented in the tree.
Two teams, one contract:
- Creative management — owns plate generation, prompts, versioning,
labeling. Writes manifests to
site/static/img/sponsored/. - Render + serve — owns HTML/CLJS surfaces, tracker, interleave logic. Reads manifests.
The interface between them is manifest.json. Neither team calls into
the other's implementation.
Two economic mechanics are demonstrated (not monetised):
- Implicit auction — bid × quality = price, priced against a stable
bucket. Documented via
Demo.explainon sponsored-research and theexp-104"Auction Disclosure" experiment. The/go/resolver'sOnClickledger entry is the analogue of click-based settlement, minus the money. - Affiliate tracking —
/go/:tokeninattribution-auditIS the affiliate-link infra, C1-safeguarded to same-origin only. Referral-program + affiliate-engine exist as raw JS; port pending.
2. Ownership map
| Concern | Files | Team |
|---|---|---|
| Passive-capture tracker | src/wal_sh/site/tracker/{core.cljc,browser.cljs} |
render |
| Sponsored-research (four archetypes) | src/wal_sh/adtech/sponsored_research/{core.cljc,browser.cljs} |
render |
| Sponsored-display (banner format) | src/wal_sh/adtech/sponsored_display/browser.cljs |
render |
| Attribution-audit (v5, provenance + holdout) | src/wal_sh/adtech/attribution_audit/{core.cljc,browser.cljs} |
render |
| Intent-signals (v6, engagedTime + score) | src/wal_sh/adtech/intent_signals/{core.cljc,browser.cljs} |
render |
| Exit-intent (v6, blended-grid overlay) | src/wal_sh/adtech/exit_intent/browser.cljs |
render |
| Stuffing-detectors (v5 addendum) | src/wal_sh/adtech/stuffing_detectors/{core.cljc,browser.cljs} |
render |
| Search + sponsored interleave | src/pocket_es/ui.cljs |
render |
| Search index thumbnails | src/pocket_es/indexer.clj |
render |
| A/B bucketing | src/wal_sh/adtech/ab_engine/ (existing) |
render |
| Feature flags | publish.el — wal-sh/adtech-systems registry |
render |
| Payload envelope | src/wal_sh/adtech/beacon/core.cljc |
render |
| Plate prompts (tone spec) | src/wal_sh/adtech/plates/prompts.cljc |
creative |
| Plate generation (flux2-klein via ollama) | src/wal_sh/adtech/plates/gen.clj (bb) |
creative |
| Plate manifest | site/static/img/sponsored/manifest.json |
creative writes, render reads |
3. Client-side tracking
The tracker is a single browser module (wal-sh.site.tracker.browser)
that installs one document-level listener per event class. Envelope is
built via wal-sh.adtech.beacon.core/build — same envelope shape as the
adtech beacon module, so downstream consumers see one JSON schema.
3.1. Event catalogue
| Event | Trigger | Sampling |
|---|---|---|
pageview |
init | 100 % |
page.dwell |
visibilitychange=hidden or pagehide | 100 % |
scroll.depth |
25 / 50 / 75 / 100 % milestones | 100 % (4 max) |
scroll.reverse |
up-scroll ≥200 px after 50 % depth | first-per-session |
element.impression |
IntersectionObserver ≥50 % for ≥1 s | 100 % |
element.dwell |
element leaves viewport after impression | 100 % |
element.click |
pointerup on <a> / <button> / [data-track-id] |
100 % |
element.hover |
pointerover with ≥500 ms dwell | 100 % |
sponsored.impression |
as element.impression, restricted to data-track-kind=sponsored-* |
100 % |
sponsored.click |
as element.click, restricted to sponsored | 100 % |
form.submit |
form submit event | 100 % |
form.focus |
focusin on input/textarea/select | 100 % |
Distinct event names for sponsored vs organic; both carry elKind for
downstream discrimination. The tracker's sponsored? predicate branches
on data-track-kind.
3.2. Transport (v6: CSP-safe pixel)
Was: POST beacon.termbox.org/webhook via sendBeacon / fetch
keepalive. Blocked by connect-src 'self' in the deployed CSP.
Now: Image().src GET to beacon.termbox.org/pixel per
docs/beacon-termbox-spec.json. Piggybacks on img-src. Payload
encoded as query params; clipped at tracker.core/max-query-len.
Response is a 1×1 GIF.
3.3. Gating
- Bot UA +
navigator.webdriver→ drop ?tracking=offURL param → drop- Query >=max-query-len= → truncated before send
- No cross-session cookie;
sessionIdis opaque + rotating per page-life
3.4. Public surface
window.__wal_tracker with .send(evt, extra), .rescan() (for
dynamic DOM after a mount), .session().
4. A/B testing (existing ab-engine)
Bucketing hash is djb2 seed 5381 over ${userId}:${expId},
_ab_uid in localStorage, exported as window._abTesting. The four
sponsored-research experiments (exp-101..104) reuse this hash so a
reader in variant-a for one experiment is in variant-a everywhere.
Source of truth: src/wal_sh/adtech/ab_engine/. The sponsored-research
core copies djb2 verbatim (sponsored-research.core/djb2) to avoid a
circular require. The attribution-audit holdout arm uses the same hash —
same reader, same arm across the surface.
Current experiments:
| ID | Purpose | Variants |
|---|---|---|
exp-101 |
Sponsored slot archetype | product / listing / local / arbitrage |
exp-102 |
Interleave position | slot-0 reserved / interleave-at-3 |
exp-103 |
Idf tail weighting | hapax / inverse |
exp-104 |
Auction disclosure | label-only / show-auction |
5. Feature flags
Flags are Emacs defvar in publish.el sourced from environment. Each
adtech module has a USE_<MODULE> flag; the postamble include is
emitted only when the flag is truthy.
Registry: wal-sh/adtech-systems in publish.el — an ordered list of
(name toggle-var include-file) tuples. Adding a module means:
- New CLJS module + shadow-cljs build
- New
includes/adtech-<name>.html(one<script src>line) - New
(defvar wal-sh/use-<name> ...) - Append to
wal-sh/adtech-systems
Current tracked modules (post 2026-08-16): tracker, sponsored-research, sponsored-display, attribution-audit, intent-signals, exit-intent, stuffing-detectors, plus the round-1–4 adtech ports (ab-engine, attribution, beacon, bot-greeting, ceddl, clean-room, content-gate, influencer-links, metered-access, native-ads, prebid, pricing-engine, promo-engine, referral-program, search-ads, sponsored-products, tag-manager, affiliate-engine).
Load order (registry-enforced):
- sponsored-research before sponsored-display (
Displayattaches to the former'swindow._sponsoredResearch). - attribution-audit before intent-signals (appends to its ledger) and before exit-intent (reads the arm).
- stuffing-detectors depends on attribution-audit's ledger at run time.
6. Beacons
Two beacons: the adtech beacon module (explicit window._beacon.pageView())
and the passive-capture tracker (auto-firing on scroll/click). Both
target beacon.termbox.org via the shared envelope; both are now
pixel-transport (see 3.2).
Payload shape (verbatim from wal-sh.adtech.beacon.core/payload):
{
"event": "sponsored.impression",
"ts": 1697000000000,
"session": "s-abc123",
"url": "https://wal.sh/search?q=dafny",
"referrer": "",
"userAgent": "...",
"screen": "1920x1080",
"timezone": "America/New_York",
"pageID": "",
"elId": "sponsored-sr-001",
"elKind": "sponsored-product",
"creativeId":"sr-001",
"unit": "product",
"position": 3
}
Extra keys (elId, elKind, creativeId, unit, position,
sponsor, pageIdx, poolIdx, isDup, dwellMs, depthPct, …)
are merged by the tracker's element-payload builder before query-encoding.
7. Attribution-audit (v5)
Sits on top of wal-sh.adtech.attribution (six-model engine). Adds
what the engine lacks:
- Touchpoint provenance — closed set
#{observed asserted}(R18). Enforced attouchpointconstructor (C3). - Credit-conservation —
claimedCreditRatio; honest = 1.0, assertion inflates it (R19, R20). - Incrementality holdout —
holdoutarm suppresses ads/overlays and measures the lift delta. The only falsifying measurement in the stack (R21, R22). Default 10 %. - go:token resolver — same-origin only.
DESTINATIONSallowlist incore(C1); merchant networks structurally unreachable. Every resolve appends anobservedOnClicktouchpoint (R23). Demo.stuff()— asserts a phantom touchpoint; ratio inflates while incrementality stays flat. The asymmetry IS the demonstration.
Public: window._srAttribution with {Ledger, Audit, Go, Demo}.
Toggle: USE_ATTRIBUTION_AUDIT. Pure logic core.cljc is REPL-testable
under :clj.
8. Intent-signals (v6)
Chartbeat-style engagedTime — visible AND recently active, NOT
wall-clock (R25). Every signal enters the v5 ledger as observed (R24).
Signals + weights (core/weights):
| Signal | Weight | Trigger |
|---|---|---|
engagedTick |
+1 | 1 s tick, visible AND input <5 s ago |
scrollQuartile |
+1/quartile | 25/50/75/100 % (once each) |
readComplete |
+3 | footer IntersectionObserver ≥50 % |
codeCopy |
+5 | copy inside <pre> / <code> |
textCopy |
+2 | copy elsewhere (never records selection — R27) |
outbound |
+4 | click on off-site <a> |
repoClick |
+6 | click on github/gitlab/sourcehut/codeberg |
returnVisit |
+5 | localStorage-tracked recurring visitor |
searchRefine |
+2 | /search query refinement |
deadClick |
−1 | click resolved nothing (frustration ≠ engagement) |
idle |
−2 | ≥30 s no input while visible |
Score is a weighted convention reported alongside the raw signal vector — never instead of it (R26).
Public: window._srIntent. Toggle: USE_INTENT_SIGNALS. No-ops if
attribution-audit's Ledger is missing.
9. Exit-intent (v6)
Fortune-inspired "Before you leave" blended-grid overlay. Owned + paid units in one grid.
Gating (all must hold):
intent.eligible?— engagedTime ≥ 15 s AND scrollDepth ≥ 50 % (R29)arm !holdout= (R31)- Not already shown this session; not in 7-day dismiss window (R30)
Triggers: desktop cursor within 20 px of viewport top; mobile upward scroll velocity ≥ 1.5 px/ms.
Divergences from source (deliberate):
- Every card carries its own provenance label (R32) — source hides disclosure behind a container-level ad-choices icon.
- Dismissible via Escape / backdrop / close button with focus return (R33).
- Never traps the back button. R28 is permanent.
Public: window._srExitIntent. Toggle: USE_EXIT_INTENT.
10. Stuffing-detectors (v5 addendum)
Four distributional detectors on the v5 ledger. Every detector returns
{:baseline :rows} (R35) — a verdict without a baseline is an
assertion. R34: NEVER reads :provenance during detection; reading
:provenance is validation (ground truth), never detection.
| ID | Detector | Flags |
|---|---|---|
| D1 | latency | median latency < 10 % of population median |
| D2 | orphaned | claimed touch with no preceding signal in 10 min |
| D3 | density | claimant's touches-per-session > 3× population |
| D4 | overlap | co-occurrence with other claimants above threshold |
D2 is the strongest single indicator. R36: validate() checks recall
against :provenance — the demonstration is precisely that post-hoc
detection cannot substitute for a boolean recorded at write time.
Public: window._srDetectors with {Suite: {run, validate, report},
Detectors}. Toggle: USE_STUFFING_DETECTORS.
11. Sponsored formats
Two formats coexist. Both keyed on the same ab-engine bucket, both non-displacing (organic order is never permuted).
11.1. Format 1: Sponsored research (native, in-list)
Four archetypes selected per session by exp-101:
| Archetype | Modeled on | Unit | Displaces organic? |
|---|---|---|---|
| product | Amazon SP | SKU in the grid | yes (ACOS-priced, in-list) |
| listing | Redfin | Card in the feed | yes but labeled |
| local | Yelp | Slot-0 reserved | no (above result 1) |
| arbitrage | Yahoo related searches | A query, not a result | n/a (outbound click) |
Arbitrage creatives are minted deterministically from (git_sha, query)
against the corpus's own idf tail (max-idf hapax). The other three read
static catalogs in wal-sh.adtech.sponsored-research.core.
11.2. Mixed-archetype sampling
sponsored-research.core/sample-mixed samples across all four
archetypes per session rather than picking one for the whole set. Each
card carries an :_archetype tag; deterministic per (git-sha, query).
Used by search-page interleave for visual variety.
Trade-off: exp-101 archetype measurement is weaker under mixed serving.
The single-archetype path lives on as ads-for-archetype, used by the
sponsored-display leaderboard banner.
11.3. Interleave
20 % ratio, non-displacing splice. One sponsored slot after every
⌈n/⌈n·0.20⌉⌉ organic hits. Implementation:
sponsored-research.browser/interleave-hits. When the async idf-tail
fetch hasn't landed, emits :placeholder entries so DOM slots are
reserved and the swap-in on wal:sponsored-ready doesn't reflow.
11.4. Runtime contract for sponsored slots
pocket-es no longer hard-requires wal-sh.adtech.sponsored-research.browser.
src/pocket_es/ui.cljs checks window._sponsoredResearch at render
time via string-keyed aget (advanced-compilation-safe). Setting
USE_SPONSORED_RESEARCH=nil genuinely disables it; the search index
and ranking stay untouched.
11.5. Format 2: Sponsored display (banner)
Criteo onsite-display shape — a banner with build-time PNG plate + DOM
text overlay. The plate carries no legible copy (flux2-klein can't
spell; the disclosure label must be selectable text for
data-nosnippet + a11y anyway; two constraints, one architecture).
Provenance surfaced via a ? affordance showing model / sha256 / seed /
reproducible flag / synthetic-true.
Implementation: wal-sh.adtech.sponsored-display.browser. Attaches
Display onto window._sponsoredResearch.
11.6. Sponsored slot markup contract
Every sponsored surface carries:
data-track-id— stable per creativedata-track-kind— one ofsponsored-product,chumbox-card,sponsored-placeholder, or the equivalentsearch-resultfor organicdata-track-creative— creative IDdata-track-unit— archetype (product/listing/local/arbitrage/onsite_display)data-track-sponsor— advertiser name if presentdata-track-page-idx/data-track-pool-idx— for infinite-feed positional analysis
The tracker's el-attrs reader picks these up verbatim. Adding a new
attr means updating tracker.core/element-payload to whitelist it.
12. Plate generation pipeline (v3, partial rewrite)
Owned by the creative team. Runbook lives at
docs/plate-generation-workflow.md.
12.1. Requirements (locked)
- R1
- Every
_dirhas a<dir>-master.png, 1024×1024 - R2
- Every (dir, slot) has a
<dir>-<slot>.pngat IAB dimensions - R3
- Greyscale only
- R4
- Manifest carries model, prompt, sha256, width, height, alt, synthetic
- R5
- sha256 matches the file on disk
- R6
- Derived plate ≤ per-slot
:max-bytescap (below) - R7
- >1 unique colour after conversion (else deleted as degenerate)
- R8
- Idempotent — masters cached,
--force-mastersblows the cache - R9
- Alt describes the plate, not the article, never the headline
12.2. Prompts
Chumbox-register prompts keyed on _dir taxonomy. See
src/wal_sh/adtech/plates/prompts.cljc. :cljc so JVM tests can grep
the prompt text without a JS host.
12.3. Slot geometry + curves
Masters (1024×1024) crop-then-derive into IAB slots via two curves:
| Slot | Output | Crop from master | Curve | :max-bytes |
|---|---|---|---|---|
leaderboard |
728×90 | 1024×260 | dither (posterize+o8x8) | 80 K |
billboard |
970×250 | 1024×500 | dither | 250 K |
mrec |
300×250 | 1024×853 | sigmoidal | 60 K |
mobile |
320×50 | 1024×400 | dither | 30 K |
Dither for anything under 128 px tall (1-bit is smaller and reads as
deliberate); sigmoidal for mrec, tall enough to hold gray. Per-slot
:max-bytes caps because dithered PNG doesn't compress like sigmoidal.
12.4. Current output
20 plates in site/static/img/sponsored/*.png (5 dirs × 4 slots) plus
manifest.json. ~1.1 MB derived, masters gitignored. Dirs: research,
current, tools, events, _.
12.5. Invocation
bb -e "(require '[wal-sh.adtech.plates.gen :as g]) \
(g/-main \"--dirs\" \"research,current,tools,events,_\" \
\"--slots\" \"leaderboard,mrec,billboard,mobile\")"
bb -x parses args as a babashka.cli map and breaks the pipeline; prefer bb -e. --force-masters re-runs Ollama for all masters.
13. Slot placement contract (v4 draft)
Not implemented, spec captured. Grammar is ours to name — the
observed source (ScienceAlert's Purch_{D|Y}_{L|C|R}_{tier}_{n}) is
one convention; ours can be any stable, greppable pattern. The
essentials:
- R10 — stable slot IDs (naming is our choice)
- R11 — container + mount pair, ID triplicated on class + class +
id - R12 — reserved dimensions on the container (CLS=0 on fill)
- R13 — in-content and search slots server-rendered by the org exporter, not client-injected
- R14 — call sites push to
window._sr.cmd, typeof-guarded - R15 — placement names come from a registry, not string literals
- R16 — segment targeting set-once, guarded on current value
- R17 — slot density is a build parameter; default 3, ceiling 14
Observed ScienceAlert density: 14 slots against ~15 paragraphs. The observed ceiling, not a target — wal.sh's default should be far lower.
14. Search + testing on load / dwell / scroll / slots
Pocket-es is the ClojureScript BM25 search UI at /search.
14.1. Sponsored interleave on results
render-results! (in src/pocket_es/ui.cljs) calls the runtime
provider (window._sponsoredResearch) with the raw hit list, current
query, and 20 % ratio. Returns a mixed vector of :organic /
:sponsored / :placeholder entries. Every access uses string-keyed
aget for Closure advanced-compilation safety.
:organic— standard.pes-hitwithdata-track-kind=search-result:sponsored—.pes-hit--sponsored(yellow left border, "Sponsored" badge, rel=nofollow href=#, click-prevented):placeholder— same slot dimensions, shimmer skeleton, replaced in-place onwal:sponsored-ready
After rendering, pocket-es.ui calls window.__wal_tracker.rescan() so
the tracker's IntersectionObserver picks up newly-mounted slots.
14.2. Result thumbnails
pocket-es.indexer emits :thumb per doc when the note's dir has
thumbnail.png / card.png / banner.png (in that order — from the
banner skill). Rendered at 48×48 with hover-zoom-to-4× on sponsored
slots. Only DIR-form notes ship thumbnails.
14.3. Testing signals surfaced on /search
Every organic hit and every sponsored slot emits impression, dwell, click, and hover events (see 3.1). Downstream we can compute:
- Per-position CTR (organic vs sponsored, faceted by archetype)
- Dwell distribution per hit-kind
- Reverse-scroll rate as an intent proxy
- Sponsored displacement (should always be zero — non-displacement
invariant on
interleave-hits)
14.4. Bare /search default (proposed, not landed)
Landing on /search with no ?q shows an empty placeholder. Proposed:
match_all sort:date desc size:20 labelled "Recent changes — type to
search →". Suppress sponsored interleave (no intent signal yet).
14.5. Infinite scroll (chumbox-tier, deferred)
Chumbox-style infinite sponsored feed is spec'd but not landed. Design constraints:
- Feed is generated (CFG over lexicon with seeded PRNG), not authored
data-nosnippeton the container is non-negotiable- Duplicate re-serving is a feature, not a bug (
dupRate=0.18) - Every third page is a "Related Searches" arbitrage unit
?chumbox=offis the kill switch
Cardinality is the payload: the meter reads "article N words · inventory M creatives", and M is the punchline.
15. Search-mount v7 spec (captured, not landed)
MutationObserver adapter that INVERTS the pocket-es→sponsored dependency: sponsored WATCHES pocket-es and injects after render.
- R37 — per-query
page-uid(one observer run per query) - R38 — clean teardown between queries (disconnect, remove DOM, clear listeners)
- R39 — view-dedup: no duplicate impressions for the same
(page-uid, slot)pair on rerender - R40 — non-permutation of organic (invariant from v6)
16. Open work
[ ]Search-mount v7 adapter (spec captured, not landed)[ ]Cut over the 21 remaining raw-JS adtech includes (mostly compiled outputs missing + include paths pointing at raw)[ ]Hiccup / hiccups-lite for the accumulated inline HTML strings[ ]Batching for the tracker transport (1 s debounce, flush on visibility change or ≥8 KB) — pixel switch made this less urgent but not moot[ ]Bare-/searchdefault → recent changes[ ]Chumbox infinite feed (spec in hand, not tangled)[ ]Slot placement contract v4 (grammar + registry + org emitter)[ ]Manifest schema v2: per-plate sidecar JSON, thin index, rollback, A/B on plates[ ]Archetype table v2: numeric-keyed, bidirectional dir index,:archetype/sourceprovenance,:archetype/grayscale-riskfor regen priority[ ]REPL-driven tests for the pure fns intracker/core.cljc,sponsored-research/core.cljc,plates/prompts.cljc,attribution_audit/core.cljc,intent_signals/core.cljc,stuffing_detectors/core.cljc[ ]Portreferral-program+affiliate-enginefrom raw JS to CLJS
17. Cross-references
src/wal_sh/site/tracker/— tracker (v6 pixel transport)src/wal_sh/adtech/sponsored_research/— native format + interleave +sample-mixedsrc/wal_sh/adtech/sponsored_display/— banner formatsrc/wal_sh/adtech/attribution_audit/— v5 provenance ledger +/go/resolver + holdoutsrc/wal_sh/adtech/intent_signals/— v6 engagedTime + score + ledger emitsrc/wal_sh/adtech/exit_intent/— v6 blended-grid overlaysrc/wal_sh/adtech/stuffing_detectors/— v5 addendum, four distributional detectorssrc/wal_sh/adtech/plates/— prompts + generation (v3)src/wal_sh/adtech/beacon/— envelopesrc/pocket_es/ui.cljs— search UI + sponsored integration (runtime aget)src/pocket_es/indexer.clj— search index +:thumbemissionpublish.el— feature-flag registry + postamble injectionshadow-cljs.edn— module build definitionsdocs/plate-generation-workflow.md— creative-team runbookdocs/beacon-termbox-spec.json— pixel-transport OpenAPI spec