Browser adtech infrastructure — tracking, A/B, feature flags, sponsored slots

Table of Contents

1. Purpose

wal.sh runs a full client-side adtech stack — as parody, as research instrumentation, and as the ambient setting the site is written for. This note is the reference: what module owns what, where the boundaries sit, and where each piece of behaviour is implemented in the tree.

Two teams, one contract:

  • Creative management — owns plate generation, prompts, versioning, labeling. Writes manifests to site/static/img/sponsored/.
  • Render + serve — owns HTML/CLJS surfaces, tracker, interleave logic. Reads manifests.

The interface between them is manifest.json. Neither team calls into the other's implementation.

Two economic mechanics are demonstrated (not monetised):

  • Implicit auction — bid × quality = price, priced against a stable bucket. Documented via Demo.explain on sponsored-research and the exp-104 "Auction Disclosure" experiment. The /go/ resolver's OnClick ledger entry is the analogue of click-based settlement, minus the money.
  • Affiliate tracking/go/:token in attribution-audit IS the affiliate-link infra, C1-safeguarded to same-origin only. Referral-program + affiliate-engine exist as raw JS; port pending.

2. Ownership map

Concern Files Team
Passive-capture tracker src/wal_sh/site/tracker/{core.cljc,browser.cljs} render
Sponsored-research (four archetypes) src/wal_sh/adtech/sponsored_research/{core.cljc,browser.cljs} render
Sponsored-display (banner format) src/wal_sh/adtech/sponsored_display/browser.cljs render
Attribution-audit (v5, provenance + holdout) src/wal_sh/adtech/attribution_audit/{core.cljc,browser.cljs} render
Intent-signals (v6, engagedTime + score) src/wal_sh/adtech/intent_signals/{core.cljc,browser.cljs} render
Exit-intent (v6, blended-grid overlay) src/wal_sh/adtech/exit_intent/browser.cljs render
Stuffing-detectors (v5 addendum) src/wal_sh/adtech/stuffing_detectors/{core.cljc,browser.cljs} render
Search + sponsored interleave src/pocket_es/ui.cljs render
Search index thumbnails src/pocket_es/indexer.clj render
A/B bucketing src/wal_sh/adtech/ab_engine/ (existing) render
Feature flags publish.elwal-sh/adtech-systems registry render
Payload envelope src/wal_sh/adtech/beacon/core.cljc render
Plate prompts (tone spec) src/wal_sh/adtech/plates/prompts.cljc creative
Plate generation (flux2-klein via ollama) src/wal_sh/adtech/plates/gen.clj (bb) creative
Plate manifest site/static/img/sponsored/manifest.json creative writes, render reads

3. Client-side tracking

The tracker is a single browser module (wal-sh.site.tracker.browser) that installs one document-level listener per event class. Envelope is built via wal-sh.adtech.beacon.core/build — same envelope shape as the adtech beacon module, so downstream consumers see one JSON schema.

3.1. Event catalogue

Event Trigger Sampling
pageview init 100 %
page.dwell visibilitychange=hidden or pagehide 100 %
scroll.depth 25 / 50 / 75 / 100 % milestones 100 % (4 max)
scroll.reverse up-scroll ≥200 px after 50 % depth first-per-session
element.impression IntersectionObserver ≥50 % for ≥1 s 100 %
element.dwell element leaves viewport after impression 100 %
element.click pointerup on <a> / <button> / [data-track-id] 100 %
element.hover pointerover with ≥500 ms dwell 100 %
sponsored.impression as element.impression, restricted to data-track-kind=sponsored-* 100 %
sponsored.click as element.click, restricted to sponsored 100 %
form.submit form submit event 100 %
form.focus focusin on input/textarea/select 100 %

Distinct event names for sponsored vs organic; both carry elKind for downstream discrimination. The tracker's sponsored? predicate branches on data-track-kind.

3.2. Transport (v6: CSP-safe pixel)

Was: POST beacon.termbox.org/webhook via sendBeacon / fetch keepalive. Blocked by connect-src 'self' in the deployed CSP.

Now: Image().src GET to beacon.termbox.org/pixel per docs/beacon-termbox-spec.json. Piggybacks on img-src. Payload encoded as query params; clipped at tracker.core/max-query-len. Response is a 1×1 GIF.

3.3. Gating

  • Bot UA + navigator.webdriver → drop
  • ?tracking=off URL param → drop
  • Query >=max-query-len= → truncated before send
  • No cross-session cookie; sessionId is opaque + rotating per page-life

3.4. Public surface

window.__wal_tracker with .send(evt, extra), .rescan() (for dynamic DOM after a mount), .session().

4. A/B testing (existing ab-engine)

Bucketing hash is djb2 seed 5381 over ${userId}:${expId}, _ab_uid in localStorage, exported as window._abTesting. The four sponsored-research experiments (exp-101..104) reuse this hash so a reader in variant-a for one experiment is in variant-a everywhere.

Source of truth: src/wal_sh/adtech/ab_engine/. The sponsored-research core copies djb2 verbatim (sponsored-research.core/djb2) to avoid a circular require. The attribution-audit holdout arm uses the same hash — same reader, same arm across the surface.

Current experiments:

ID Purpose Variants
exp-101 Sponsored slot archetype product / listing / local / arbitrage
exp-102 Interleave position slot-0 reserved / interleave-at-3
exp-103 Idf tail weighting hapax / inverse
exp-104 Auction disclosure label-only / show-auction

5. Feature flags

Flags are Emacs defvar in publish.el sourced from environment. Each adtech module has a USE_<MODULE> flag; the postamble include is emitted only when the flag is truthy.

Registry: wal-sh/adtech-systems in publish.el — an ordered list of (name toggle-var include-file) tuples. Adding a module means:

  1. New CLJS module + shadow-cljs build
  2. New includes/adtech-<name>.html (one <script src> line)
  3. New (defvar wal-sh/use-<name> ...)
  4. Append to wal-sh/adtech-systems

Current tracked modules (post 2026-08-16): tracker, sponsored-research, sponsored-display, attribution-audit, intent-signals, exit-intent, stuffing-detectors, plus the round-1–4 adtech ports (ab-engine, attribution, beacon, bot-greeting, ceddl, clean-room, content-gate, influencer-links, metered-access, native-ads, prebid, pricing-engine, promo-engine, referral-program, search-ads, sponsored-products, tag-manager, affiliate-engine).

Load order (registry-enforced):

  • sponsored-research before sponsored-display (Display attaches to the former's window._sponsoredResearch).
  • attribution-audit before intent-signals (appends to its ledger) and before exit-intent (reads the arm).
  • stuffing-detectors depends on attribution-audit's ledger at run time.

6. Beacons

Two beacons: the adtech beacon module (explicit window._beacon.pageView()) and the passive-capture tracker (auto-firing on scroll/click). Both target beacon.termbox.org via the shared envelope; both are now pixel-transport (see 3.2).

Payload shape (verbatim from wal-sh.adtech.beacon.core/payload):

{
  "event":     "sponsored.impression",
  "ts":        1697000000000,
  "session":   "s-abc123",
  "url":       "https://wal.sh/search?q=dafny",
  "referrer":  "",
  "userAgent": "...",
  "screen":    "1920x1080",
  "timezone":  "America/New_York",
  "pageID":    "",
  "elId":      "sponsored-sr-001",
  "elKind":    "sponsored-product",
  "creativeId":"sr-001",
  "unit":      "product",
  "position":  3
}

Extra keys (elId, elKind, creativeId, unit, position, sponsor, pageIdx, poolIdx, isDup, dwellMs, depthPct, …) are merged by the tracker's element-payload builder before query-encoding.

7. Attribution-audit (v5)

Sits on top of wal-sh.adtech.attribution (six-model engine). Adds what the engine lacks:

  • Touchpoint provenance — closed set #{observed asserted} (R18). Enforced at touchpoint constructor (C3).
  • Credit-conservationclaimedCreditRatio; honest = 1.0, assertion inflates it (R19, R20).
  • Incrementality holdoutholdout arm suppresses ads/overlays and measures the lift delta. The only falsifying measurement in the stack (R21, R22). Default 10 %.
  • go:token resolver — same-origin only. DESTINATIONS allowlist in core (C1); merchant networks structurally unreachable. Every resolve appends an observed OnClick touchpoint (R23).
  • Demo.stuff() — asserts a phantom touchpoint; ratio inflates while incrementality stays flat. The asymmetry IS the demonstration.

Public: window._srAttribution with {Ledger, Audit, Go, Demo}. Toggle: USE_ATTRIBUTION_AUDIT. Pure logic core.cljc is REPL-testable under :clj.

8. Intent-signals (v6)

Chartbeat-style engagedTime — visible AND recently active, NOT wall-clock (R25). Every signal enters the v5 ledger as observed (R24).

Signals + weights (core/weights):

Signal Weight Trigger
engagedTick +1 1 s tick, visible AND input <5 s ago
scrollQuartile +1/quartile 25/50/75/100 % (once each)
readComplete +3 footer IntersectionObserver ≥50 %
codeCopy +5 copy inside <pre> / <code>
textCopy +2 copy elsewhere (never records selection — R27)
outbound +4 click on off-site <a>
repoClick +6 click on github/gitlab/sourcehut/codeberg
returnVisit +5 localStorage-tracked recurring visitor
searchRefine +2 /search query refinement
deadClick −1 click resolved nothing (frustration ≠ engagement)
idle −2 ≥30 s no input while visible

Score is a weighted convention reported alongside the raw signal vector — never instead of it (R26).

Public: window._srIntent. Toggle: USE_INTENT_SIGNALS. No-ops if attribution-audit's Ledger is missing.

9. Exit-intent (v6)

Fortune-inspired "Before you leave" blended-grid overlay. Owned + paid units in one grid.

Gating (all must hold):

  • intent.eligible? — engagedTime ≥ 15 s AND scrollDepth ≥ 50 % (R29)
  • arm ! holdout= (R31)
  • Not already shown this session; not in 7-day dismiss window (R30)

Triggers: desktop cursor within 20 px of viewport top; mobile upward scroll velocity ≥ 1.5 px/ms.

Divergences from source (deliberate):

  • Every card carries its own provenance label (R32) — source hides disclosure behind a container-level ad-choices icon.
  • Dismissible via Escape / backdrop / close button with focus return (R33).
  • Never traps the back button. R28 is permanent.

Public: window._srExitIntent. Toggle: USE_EXIT_INTENT.

10. Stuffing-detectors (v5 addendum)

Four distributional detectors on the v5 ledger. Every detector returns {:baseline :rows} (R35) — a verdict without a baseline is an assertion. R34: NEVER reads :provenance during detection; reading :provenance is validation (ground truth), never detection.

ID Detector Flags
D1 latency median latency < 10 % of population median
D2 orphaned claimed touch with no preceding signal in 10 min
D3 density claimant's touches-per-session > 3× population
D4 overlap co-occurrence with other claimants above threshold

D2 is the strongest single indicator. R36: validate() checks recall against :provenance — the demonstration is precisely that post-hoc detection cannot substitute for a boolean recorded at write time.

Public: window._srDetectors with {Suite: {run, validate, report}, Detectors}. Toggle: USE_STUFFING_DETECTORS.

11. Sponsored formats

Two formats coexist. Both keyed on the same ab-engine bucket, both non-displacing (organic order is never permuted).

11.1. Format 1: Sponsored research (native, in-list)

Four archetypes selected per session by exp-101:

Archetype Modeled on Unit Displaces organic?
product Amazon SP SKU in the grid yes (ACOS-priced, in-list)
listing Redfin Card in the feed yes but labeled
local Yelp Slot-0 reserved no (above result 1)
arbitrage Yahoo related searches A query, not a result n/a (outbound click)

Arbitrage creatives are minted deterministically from (git_sha, query) against the corpus's own idf tail (max-idf hapax). The other three read static catalogs in wal-sh.adtech.sponsored-research.core.

11.2. Mixed-archetype sampling

sponsored-research.core/sample-mixed samples across all four archetypes per session rather than picking one for the whole set. Each card carries an :_archetype tag; deterministic per (git-sha, query). Used by search-page interleave for visual variety.

Trade-off: exp-101 archetype measurement is weaker under mixed serving. The single-archetype path lives on as ads-for-archetype, used by the sponsored-display leaderboard banner.

11.3. Interleave

20 % ratio, non-displacing splice. One sponsored slot after every ⌈n/⌈n·0.20⌉⌉ organic hits. Implementation: sponsored-research.browser/interleave-hits. When the async idf-tail fetch hasn't landed, emits :placeholder entries so DOM slots are reserved and the swap-in on wal:sponsored-ready doesn't reflow.

11.4. Runtime contract for sponsored slots

pocket-es no longer hard-requires wal-sh.adtech.sponsored-research.browser. src/pocket_es/ui.cljs checks window._sponsoredResearch at render time via string-keyed aget (advanced-compilation-safe). Setting USE_SPONSORED_RESEARCH=nil genuinely disables it; the search index and ranking stay untouched.

11.5. Format 2: Sponsored display (banner)

Criteo onsite-display shape — a banner with build-time PNG plate + DOM text overlay. The plate carries no legible copy (flux2-klein can't spell; the disclosure label must be selectable text for data-nosnippet + a11y anyway; two constraints, one architecture).

Provenance surfaced via a ? affordance showing model / sha256 / seed / reproducible flag / synthetic-true.

Implementation: wal-sh.adtech.sponsored-display.browser. Attaches Display onto window._sponsoredResearch.

11.6. Sponsored slot markup contract

Every sponsored surface carries:

  • data-track-id — stable per creative
  • data-track-kind — one of sponsored-product, chumbox-card, sponsored-placeholder, or the equivalent search-result for organic
  • data-track-creative — creative ID
  • data-track-unit — archetype (product / listing / local / arbitrage / onsite_display)
  • data-track-sponsor — advertiser name if present
  • data-track-page-idx / data-track-pool-idx — for infinite-feed positional analysis

The tracker's el-attrs reader picks these up verbatim. Adding a new attr means updating tracker.core/element-payload to whitelist it.

12. Plate generation pipeline (v3, partial rewrite)

Owned by the creative team. Runbook lives at docs/plate-generation-workflow.md.

12.1. Requirements (locked)

R1
Every _dir has a <dir>-master.png, 1024×1024
R2
Every (dir, slot) has a <dir>-<slot>.png at IAB dimensions
R3
Greyscale only
R4
Manifest carries model, prompt, sha256, width, height, alt, synthetic
R5
sha256 matches the file on disk
R6
Derived plate ≤ per-slot :max-bytes cap (below)
R7
>1 unique colour after conversion (else deleted as degenerate)
R8
Idempotent — masters cached, --force-masters blows the cache
R9
Alt describes the plate, not the article, never the headline

12.2. Prompts

Chumbox-register prompts keyed on _dir taxonomy. See src/wal_sh/adtech/plates/prompts.cljc. :cljc so JVM tests can grep the prompt text without a JS host.

12.3. Slot geometry + curves

Masters (1024×1024) crop-then-derive into IAB slots via two curves:

Slot Output Crop from master Curve :max-bytes
leaderboard 728×90 1024×260 dither (posterize+o8x8) 80 K
billboard 970×250 1024×500 dither 250 K
mrec 300×250 1024×853 sigmoidal 60 K
mobile 320×50 1024×400 dither 30 K

Dither for anything under 128 px tall (1-bit is smaller and reads as deliberate); sigmoidal for mrec, tall enough to hold gray. Per-slot :max-bytes caps because dithered PNG doesn't compress like sigmoidal.

12.4. Current output

20 plates in site/static/img/sponsored/*.png (5 dirs × 4 slots) plus manifest.json. ~1.1 MB derived, masters gitignored. Dirs: research, current, tools, events, _.

12.5. Invocation

bb -e "(require '[wal-sh.adtech.plates.gen :as g]) \
       (g/-main \"--dirs\" \"research,current,tools,events,_\" \
                \"--slots\" \"leaderboard,mrec,billboard,mobile\")"

bb -x parses args as a babashka.cli map and breaks the pipeline; prefer bb -e. --force-masters re-runs Ollama for all masters.

13. Slot placement contract (v4 draft)

Not implemented, spec captured. Grammar is ours to name — the observed source (ScienceAlert's Purch_{D|Y}_{L|C|R}_{tier}_{n}) is one convention; ours can be any stable, greppable pattern. The essentials:

  • R10 — stable slot IDs (naming is our choice)
  • R11 — container + mount pair, ID triplicated on class + class + id
  • R12 — reserved dimensions on the container (CLS=0 on fill)
  • R13 — in-content and search slots server-rendered by the org exporter, not client-injected
  • R14 — call sites push to window._sr.cmd, typeof-guarded
  • R15 — placement names come from a registry, not string literals
  • R16 — segment targeting set-once, guarded on current value
  • R17 — slot density is a build parameter; default 3, ceiling 14

Observed ScienceAlert density: 14 slots against ~15 paragraphs. The observed ceiling, not a target — wal.sh's default should be far lower.

14. Search + testing on load / dwell / scroll / slots

Pocket-es is the ClojureScript BM25 search UI at /search.

14.1. Sponsored interleave on results

render-results! (in src/pocket_es/ui.cljs) calls the runtime provider (window._sponsoredResearch) with the raw hit list, current query, and 20 % ratio. Returns a mixed vector of :organic / :sponsored / :placeholder entries. Every access uses string-keyed aget for Closure advanced-compilation safety.

  • :organic — standard .pes-hit with data-track-kind=search-result
  • :sponsored.pes-hit--sponsored (yellow left border, "Sponsored" badge, rel=nofollow href=#, click-prevented)
  • :placeholder — same slot dimensions, shimmer skeleton, replaced in-place on wal:sponsored-ready

After rendering, pocket-es.ui calls window.__wal_tracker.rescan() so the tracker's IntersectionObserver picks up newly-mounted slots.

14.2. Result thumbnails

pocket-es.indexer emits :thumb per doc when the note's dir has thumbnail.png / card.png / banner.png (in that order — from the banner skill). Rendered at 48×48 with hover-zoom-to-4× on sponsored slots. Only DIR-form notes ship thumbnails.

14.3. Testing signals surfaced on /search

Every organic hit and every sponsored slot emits impression, dwell, click, and hover events (see 3.1). Downstream we can compute:

  • Per-position CTR (organic vs sponsored, faceted by archetype)
  • Dwell distribution per hit-kind
  • Reverse-scroll rate as an intent proxy
  • Sponsored displacement (should always be zero — non-displacement invariant on interleave-hits)

14.4. Bare /search default (proposed, not landed)

Landing on /search with no ?q shows an empty placeholder. Proposed: match_all sort:date desc size:20 labelled "Recent changes — type to search →". Suppress sponsored interleave (no intent signal yet).

14.5. Infinite scroll (chumbox-tier, deferred)

Chumbox-style infinite sponsored feed is spec'd but not landed. Design constraints:

  • Feed is generated (CFG over lexicon with seeded PRNG), not authored
  • data-nosnippet on the container is non-negotiable
  • Duplicate re-serving is a feature, not a bug (dupRate=0.18)
  • Every third page is a "Related Searches" arbitrage unit
  • ?chumbox=off is the kill switch

Cardinality is the payload: the meter reads "article N words · inventory M creatives", and M is the punchline.

15. Search-mount v7 spec (captured, not landed)

MutationObserver adapter that INVERTS the pocket-es→sponsored dependency: sponsored WATCHES pocket-es and injects after render.

  • R37 — per-query page-uid (one observer run per query)
  • R38 — clean teardown between queries (disconnect, remove DOM, clear listeners)
  • R39 — view-dedup: no duplicate impressions for the same (page-uid, slot) pair on rerender
  • R40 — non-permutation of organic (invariant from v6)

16. Open work

  • [ ] Search-mount v7 adapter (spec captured, not landed)
  • [ ] Cut over the 21 remaining raw-JS adtech includes (mostly compiled outputs missing + include paths pointing at raw)
  • [ ] Hiccup / hiccups-lite for the accumulated inline HTML strings
  • [ ] Batching for the tracker transport (1 s debounce, flush on visibility change or ≥8 KB) — pixel switch made this less urgent but not moot
  • [ ] Bare-/search default → recent changes
  • [ ] Chumbox infinite feed (spec in hand, not tangled)
  • [ ] Slot placement contract v4 (grammar + registry + org emitter)
  • [ ] Manifest schema v2: per-plate sidecar JSON, thin index, rollback, A/B on plates
  • [ ] Archetype table v2: numeric-keyed, bidirectional dir index, :archetype/source provenance, :archetype/grayscale-risk for regen priority
  • [ ] REPL-driven tests for the pure fns in tracker/core.cljc, sponsored-research/core.cljc, plates/prompts.cljc, attribution_audit/core.cljc, intent_signals/core.cljc, stuffing_detectors/core.cljc
  • [ ] Port referral-program + affiliate-engine from raw JS to CLJS

17. Cross-references

  • src/wal_sh/site/tracker/ — tracker (v6 pixel transport)
  • src/wal_sh/adtech/sponsored_research/ — native format + interleave + sample-mixed
  • src/wal_sh/adtech/sponsored_display/ — banner format
  • src/wal_sh/adtech/attribution_audit/ — v5 provenance ledger + /go/ resolver + holdout
  • src/wal_sh/adtech/intent_signals/ — v6 engagedTime + score + ledger emit
  • src/wal_sh/adtech/exit_intent/ — v6 blended-grid overlay
  • src/wal_sh/adtech/stuffing_detectors/ — v5 addendum, four distributional detectors
  • src/wal_sh/adtech/plates/ — prompts + generation (v3)
  • src/wal_sh/adtech/beacon/ — envelope
  • src/pocket_es/ui.cljs — search UI + sponsored integration (runtime aget)
  • src/pocket_es/indexer.clj — search index + :thumb emission
  • publish.el — feature-flag registry + postamble injection
  • shadow-cljs.edn — module build definitions
  • docs/plate-generation-workflow.md — creative-team runbook
  • docs/beacon-termbox-spec.json — pixel-transport OpenAPI spec