Sponsored research — spec

Table of Contents

1. Purpose

Sponsored-research is the native, in-list sponsored format. Four archetypes model four sponsored-inventory shapes from real sites — Amazon Sponsored Products, Redfin listings, Yelp locals, Yahoo related-searches arbitrage. Each archetype is a catalog shape plus a rendering pattern; the same reader in the same session sees the same archetype (via ab-engine bucketing on exp-101), or, for search-page interleave, sees all four archetypes mixed via sample-mixed.

The four-archetype design is the parody: real sponsored inventory comes in exactly these four shapes, and each shape carries its own disclosure convention. Putting them all on one site — where an Amazon-shape SKU card sits next to a Redfin-shape listing card next to a Yelp-shape slot-0 unit next to a Yahoo-shape query template — makes the visual grammar of native advertising legible in a way that no single archetype does.

ORACLE: src/wal_sh/adtech/sponsored_research/core.cljc (pure — catalogs + interleave + sample-mixed), browser.cljs (window._sponsoredResearch).

As of contract v2 (2026-08-17): the closed-type authority is src/wal_sh/adtech/sponsored_research/contract.cljc. That namespace owns unit (the SINGLE constructor), the unit-keys closed set, the total validate predicate (I1..I7), and assert-non-displacing! (I8). The v-final check-unit in core.cljc now delegates to contract/validate; the gate lives at src/wal_sh/adtech/sponsored_research/gate.clj (invocable via bb -f). Negative fixtures live at src/wal_sh/adtech/sponsored_research/fixtures/negative_units.edn.

2. Requirements

SR1
Four archetypes: product, listing, local, arbitrage. Each has its own catalog shape and rendering.
SR2
Non-displacing interleave. Organic order is never permuted; sponsored slots are inserted between organic hits. interleave-ratio is the invariant contract.
SR3
Bucketing = djb2 seed 5381 over userId:expId, matches the ab-engine hash. A reader in variant-a for one experiment is in variant-a everywhere.
SR4
Arbitrage creatives are minted deterministically from (git_sha, query) against the corpus's own idf tail (max-idf hapax). The other three archetypes read static catalogs.
SR5
sample-mixed samples across all four archetypes per session, deterministic per (git-sha, query). Each returned card carries an :_archetype tag so the renderer knows the shape.
SR6
landing-href resolves every non-arbitrage card to /research/underwriter/<id>/. Arbitrage cards return # (the click is the point, not the destination — the search-arbitrage unit is authored to demonstrate that pattern, not to lead somewhere).
SR7
The mulberry32 PRNG is the JS-reference PRNG; seed is the djb2 of (git-sha, query). Reseeding per keystroke turns incremental search into a slot machine — that is the observed behaviour we are documenting, not fixing.

3. Archetypes

Superseded by v-final (2026-08-17). The archetype dimension is removed. A sponsored unit is now a REAL document from /static/search-index.json, rendered in retail-media visual language; no authored catalog, no per-archetype card shape. The catalog defs (sponsored-products, sponsored-listings, sponsored-local) and the arbitrage templates have been deleted from core.cljc. The authored underwriter pages (/research/underwriter/*) are preserved under _drafts/underwriter-sunset-2026-08-17/ and excluded from publish. This section is retained as a historical record of the pre-v-final design; see the top of core.cljc for the current contract (eligible?, matched-terms, unit-from, check-unit).

Archetype Modeled on Unit Displaces organic?
product Amazon SP SKU in a grid yes (ACOS-priced, in-list)
listing Redfin Card in a feed yes but labeled
local Yelp Slot-0 reserved no (above result 1)
arbitrage Yahoo related searches A query, not a result n/a (outbound click)

The catalog for each is a def in core.cljc: sponsored-products, sponsored-listings, sponsored-local. The arbitrage-templates list is a set of 8 templates with a \{T\} slot; the idf tail supplies real hapax terms from the wal.sh corpus (squoze, syzygies, reuleaux, …).

4. Contract signature

Pure (core.cljc):

(djb2 s)                                          ; -> uint32 (matches ab-engine)
(mulberry32 seed)                                 ; -> fn (fn [] -> double in [0,1))
(archetype-for user-id)                           ; -> "product" | "listing" | "local" | "arbitrage"
(render-arbitrage state query n)                  ; -> [{:text :term :sponsored :advertiser :cpc}]
(sample-mixed state query n)                      ; -> [{:_archetype ...}]
(interleave-ratio organic ads ratio)              ; -> [{:kind :organic|:sponsored ...}]
(landing-href ad)                                 ; -> "/research/underwriter/<id>/" | "#"
(greyscale-thumb-svg id)                          ; -> deterministic 4-band greyscale SVG string

experiments                                       ; [exp-101 exp-102 exp-103 exp-104]
sponsored-products                                ; 5 SKU entries
sponsored-listings                                ; 4 listing entries
sponsored-local                                   ; 4 local entries
arbitrage-templates                               ; 8 templates
tail-fallback                                     ; 20 hapax terms

Browser surface:

window._sponsoredResearch = {
  Format: {
    sample-mixed:  (fn [git-sha query n] -> [cards])
    interleave:    (fn [organic query ratio] -> [{:kind ...}])
    archetype-for: (fn [user-id] -> "product" | ...)
    render:        (fn [ad] -> HTML string / hiccup)
  }
  Display: {                                      ; attached by sponsored-display
    banner:        (fn [] -> DOM node)
  }
}

5. Experiments (exp-101..104)

ID Purpose Variants
exp-101 Sponsored slot archetype product / listing / local / arbitrage
exp-102 Interleave position slot-0 reserved / interleave-at-3
exp-103 Idf tail weighting hapax / inverse
exp-104 Auction disclosure label-only / show-auction

Shared metrics: primary sponsored_ctr, secondary varies (rpm, organic_ctr, query_refinement_rate, disclosure_expand_rate).

6. Related literature

  • Roesner et al.'s NSDI 2012 measurement (Roesner, Kohno, and Wetherall 2012) classifies sponsored-inventory profiles by disclosure pattern; our four archetypes cover the four most-common shapes.
  • Bashir et al. on retargeted-ad tracing (Bashir et al. 2016) documents how sponsored inventory is routed across ad exchanges — the "arbitrage" archetype models the smallest observable slice of that.
  • Taboola/Outbrain publisher documentation (Taboola, n.d.), (Outbrain, n.d.) is the industry reference for the recommendation-widget mechanics the arbitrage archetype models at N=1.
  • ANA 2023 MFA study (Association of National Advertisers 2023) characterises the economic pattern the arbitrage archetype demonstrates: cheap synthesised inventory monetised at scale via clickthrough to another ad surface.
  • rel"sponsored"= is the outbound-link convention the interleave slots use for disclosure (WHATWG, n.d.).
  • IAB Standard Ad Unit Portfolio (Interactive Advertising Bureau 2017) is the reference for the physical dimensions of each archetype's container (leaderboard / mrec / mobile).
  • Zeng, Kohno & Roesner on bad ads (Zeng, Kohno, and Roesner 2020) documents the disclosure-hiding failure mode this spec's per-card disclosure (see exit-intent R32 for the sibling rule) is designed to avoid.

7. Cross-references

  • attribution-engine/spec.org — every sponsored click appends a touchpoint via the audit ledger.
  • ab-engine/spec.orgarchetype-for reuses the djb2 hash; exp-101..104 are the four production experiments.
  • sponsored-display/spec.org — the banner format attaches to window._sponsoredResearch as .Display.
  • sponsored-formats/spec.org — the historical taxonomy of native / product / listing / local / arbitrage / display / exit-intent.
  • pocket-es-integration/spec.org — search results consume sample-mixed at 20 % ratio for interleave.
  • ORACLE:
    • src/wal_sh/adtech/sponsored_research/core.cljc (catalogs + interleave)
    • src/wal_sh/adtech/sponsored_research/browser.cljs (surface)

8. Open questions

  • Catalog rotation. The five product SKUs, four listings, four locals are static literals. A per-week rotation would let us measure novelty vs. familiarity on sponsored CTR; the mechanism is a data change, not a code change.
  • Arbitrage template lexicon. Eight templates × 20 fallback hapax gives 160 unique renderable strings. The IDF tail fetch expands this dramatically at the cost of a network round-trip; a build-time pre-computed tail file would remove the round-trip.
  • Landing-page underwriter shape. /research/underwriter/<id>/ is a placeholder path convention. Real underwriter pages are not yet written — the 404 on click is the honest current state.
Association of National Advertisers. 2023. “Programmatic Media Supply Chain Transparency Study.” ANA. https://www.ana.net/miccontent/show/id/rr-2023-06-ana-programmatic-transparency-study.
Bashir, Muhammad Ahmad, Sajjad Arshad, William Robertson, and Christo Wilson. 2016. “Tracing Information Flows between Ad Exchanges Using Retargeted Ads.” In 25Th Usenix Security Symposium (Usenix Security 16), 481–96. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/bashir.
Interactive Advertising Bureau. 2017. “Iab New Standard Ad Unit Portfolio.” https://www.iab.com/newadportfolio/.
Outbrain. n.d. “Publisher Help Center.” https://www.outbrain.com/help/publishers/.
Roesner, Franziska, Tadayoshi Kohno, and David Wetherall. 2012. “Detecting and Defending against Third-Party Tracking on the Web.” In 9Th Usenix Symposium on Networked Systems Design and Implementation (Nsdi 12), 155–68. https://www.usenix.org/conference/nsdi12/technical-sessions/presentation/roesner.
Taboola. n.d. “Publisher Documentation.” https://help.taboola.com/hc/en-us/categories/115000012407-Publishers.
WHATWG. n.d. “Html Living Standard: Link Types (Rel=Sponsored).” https://html.spec.whatwg.org/multipage/links.html#link-type-sponsored.
Zeng, Eric, Tadayoshi Kohno, and Franziska Roesner. 2020. “Bad News: Clickbait and Deceptive Ads on News and Misinformation Websites.” In Workshop on Technology and Consumer Protection (Conpro). https://badads.cs.washington.edu/.