Stuffing detectors — spec

Table of Contents

1. Purpose

Stuffing detectors are the post-hoc layer over the attribution-audit ledger. Four distributional detectors run over the touchpoint stream and flag claimants whose behaviour is inconsistent with a normal signal timeline. The whole module is a demonstration: post-hoc detection cannot substitute for recording provenance at write time (R36).

The four detectors — latency, orphaned, density, overlap — are picked to give partial coverage of the stuffing threat model (D1..D4). The strongest single indicator is D2 (orphaned), which flags claims with no preceding user signal in the same session window. But even D2 misses cases where a stuffer also arranges a plausible-looking preceding signal.

R36 is the point of the module: run the detectors, then run validate to check recall against the :provenance field the real world does not have. If recall stays below ~0.8 at realistic path lengths, the detectors do not substitute for recording provenance at write time. The demonstration is precisely that they don't.

ORACLE: src/wal_sh/adtech/stuffing_detectors/core.cljc (pure), browser.cljs (window._srDetectors).

2. Requirements

R34
Detectors operate on observed AND asserted touches alike, without reading :provenance. Reading :provenance is validation (ground truth), never detection. Enforced by not passing :provenance to the detector functions.
R35
Every detector returns {:baseline :rows}. A verdict without a baseline is an assertion; the baseline is a summary statistic over the population.
R36
validate checks recall against the :provenance field the real world does not have. Returns a per-detector {:recall :false-positives}. Only meaningful when there are asserted touches in the ledger (test setup runs Demo.stuff first).

3. Detectors

ID Detector Flags claimant when
D1 latency median latency < 10 % of population median
D2 orphaned claimed touch with no preceding signal in 10 min
D3 density claimant appears in > 90 % of conversion paths
D4 overlap > 80 % of touches co-occur with another claimant in 5 min

D2 is the strongest single indicator. D1 + D4 together capture the "burst of claims immediately before a conversion" signature that D2 catches individually. D3 flags indispensability that in a real ledger could be either genuine (a truly-central channel) or inflated — D1 + D2 separate the two cases.

4. Contract signature

Pure (core.cljc):

(latency rows conversion-at)                      ; -> {:baseline {:pop-median-ms}
                                                  ;     :rows [{:claimant :median-latency-ms :ratio :flag}]}
(orphaned rows)                                   ; -> {:baseline {:signal-count}
                                                  ;     :rows [{:claimant :touches :orphans :rate :flag}]}
(density rows)                                    ; -> {:baseline {:total-paths}
                                                  ;     :rows [{:claimant :paths :share :flag}]}
(overlap rows & [{:keys [window-ms]}])            ; -> {:baseline {:window-ms}
                                                  ;     :rows [{:claimant :touches :shared :rate :flag}]}

(run rows conversion-at)                          ; -> {:latency ... :orphaned ... :density ... :overlap ...}
(validate rows conversion-at)                     ; -> {:asserted [claimants] :recall {detector {:recall :false-positives}}}
                                                  ; or {:note "..."} when no asserted touches

Browser surface:

window._srDetectors = {
  Suite: {
    run:      (fn [] -> {detector -> result})
    validate: (fn [] -> recall-report)
    report:   (fn [] -> nil)                      ; console.log the flagged rows
  }
  Detectors: {
    latency, orphaned, density, overlap
  }
}

5. The demonstration

Run _srAttribution.Demo.stuff("bad-actor") then _srDetectors. Suite.validate(). The recall report answers: given a claimant we know (from :provenance"asserted"=) inflated the ledger, which detectors caught it? At the small ledger sizes on wal.sh, D2 catches most single-claim inflations, D1 + D4 catch bursts. As path length grows the recall curve degrades — the exact way it degrades is the empirical output of this module.

6. Related literature

  • Musa & Nithyanand's ATOM (Ad-network Tomography) (Musa and Nithyanand 2022) is the topology-level version of this detection problem: infer trust relationships from external observations without access to provenance.
  • Cook, Nithyanand & Shafiq on header bidding (Cook, Nithyanand, and Shafiq 2020) documents the structural opacity that makes post-hoc detection the only lever available to outside auditors.
  • Vekaria et al. on ad-inventory pooling and misinformation (Vekaria et al. 2022) characterises the scale of inventory misattribution — post-hoc detection at industry scale.
  • Zeng, Kohno & Roesner on bad ads (Zeng, Kohno, and Roesner 2020) catalogues the observed clickbait/chumbox behaviours a distributional detector would flag.
  • ANA 2023/2024 MFA studies (Association of National Advertisers 2023), (Association of National Advertisers 2024) are the industry-scale measurement wal.sh's detectors approximate at N ≈ 1.
  • Bashir et al. on retargeted-ad tracing (Bashir et al. 2016) shows how bursts of touches immediately before a conversion look in a real DSP log — the D1/D4 signature.
  • Ali et al. on delivery discrimination (Ali et al. 2019) frames the audit-visibility argument that motivates R36: recall must be measurable.

7. Cross-references

8. Open questions

  • Detector portfolio expansion. Four detectors is a starting set; the ATOM paper (Musa and Nithyanand 2022) suggests half a dozen more distributional shapes (bipartite-graph cluster coefficients, path-length distribution moments). Adding them is straightforward under the R35 contract but the value shows only at larger ledger sizes than wal.sh generates.
  • Recall vs precision trade-off. Every detector's :flag threshold is a fixed constant. A learned threshold per detector would improve F-score at the cost of a training loop we haven't built. Currently the thresholds match published defaults from the measurement literature.
Ali, Muhammad, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. 2019. “Discrimination through Optimization: How Facebook’s Ad Delivery Can Lead to Biased Outcomes.” In Proceedings of the Acm on Human-Computer Interaction. Vol. 3. CSCW. https://doi.org/10.1145/3359301.
Association of National Advertisers. 2023. “Programmatic Media Supply Chain Transparency Study.” ANA. https://www.ana.net/miccontent/show/id/rr-2023-06-ana-programmatic-transparency-study.
———. 2024. “Made-for-Advertising (Mfa) Second Look and Update.” ANA. https://www.ana.net/miccontent/show/id/rr-2024-01-ana-mfa-second-look.
Bashir, Muhammad Ahmad, Sajjad Arshad, William Robertson, and Christo Wilson. 2016. “Tracing Information Flows between Ad Exchanges Using Retargeted Ads.” In 25Th Usenix Security Symposium (Usenix Security 16), 481–96. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/bashir.
Cook, John, Rishab Nithyanand, and Zubair Shafiq. 2020. “Inferring Tracker-Advertiser Relationships in the Online Advertising Ecosystem Using Header Bidding.” Proceedings on Privacy Enhancing Technologies (Popets) 2020 (1): 65–82. https://doi.org/10.2478/popets-2020-0005.
Musa, Muhammad, and Rishab Nithyanand. 2022. “Atom: Ad-Network Tomography.” Proceedings on Privacy Enhancing Technologies (Popets) 2022 (4): 295–313. https://doi.org/10.56553/popets-2022-0110.
Vekaria, Yash, Eric Zeng, Tadayoshi Kohno, and Franziska Roesner. 2022. “The Inventory Is Dark and Full of Misinformation: Understanding the Abuse of Ad Inventory Pooling in the Ad-Tech Supply Chain.” https://arxiv.org/abs/2210.06654.
Zeng, Eric, Tadayoshi Kohno, and Franziska Roesner. 2020. “Bad News: Clickbait and Deceptive Ads on News and Misinformation Websites.” In Workshop on Technology and Consumer Protection (Conpro). https://badads.cs.washington.edu/.