Vocabulary binding — the taxonomy is the targeting layer
controlled-vocabulary-v1 × sponsored-contract-v2

Table of Contents

1. 1. The relationship

A PLP's ad targeting is bound to the retailer's merchandising taxonomy. Criteo's delivery call carries filters=(price,le,500) — faceted filtering over the retailer's own attributes — and its keyword targeting runs on the retailer's category terms. The ad system does not own the taxonomy; it consumes it.

wal.sh has the same structure. The auction's matched (I6) is doc.keywords ∩ query, so sponsored relevance is a function of vocabulary quality and nothing else. Every pathology in the folksonomy propagates directly into which units can serve.

This was not designed. It was discovered by building the auction on top of an unrevised vocabulary and watching it behave.

2. 2. The two vocabularies are already correctly split

controlled-vocabulary-v1 §3.2 gives #+FILETAGS and #+KEYWORDS different contracts — validated against a term list versus unvalidated free text. That split maps cleanly onto the two things retail media needs, and they are not the same thing:

Field Vocabulary contract Ad system role Retail analogue
#+FILETAGS controlled, 47 terms, build-validated targeting keys — faceted eligibility category tree + attributes
#+KEYWORDS free text, convergence-normalized matching — BM25 recall and matched display search relevance signal

So: eligibility filters on facets, relevance matches on keywords. A sponsored slot scoped to domain:formal-methods draws candidates by FILETAGS; which of those candidates wins is decided by KEYWORDS overlap with the query.

The current implementation conflates them — Auction.quality reads keywords for both jobs, which is why a slot cannot be scoped at all.

3. 3. The objective functions coincide

controlled-vocabulary-v1 §1 sets the operational goal from chip UX: chips that connect 2–25 documents each. A chip resolving to 61 documents is noise; one resolving to a single document is isolation.

The auction independently wants the same band, for a different reason:

Term frequency Chip behaviour Auction behaviour
df = 1 (2,777 terms, 81%) isolation document can essentially never serve — I6 requires non-empty matched
df 2–25 usable discriminable candidate set; quality score separates
df = 61 (clojure) noise 61 candidates tie on relevance; GSP ranks on bid, which is deterministic noise

Two independently derived targets landing on the same interval is worth noticing. The chip goal and the ad-targeting goal are the same objective function, because both are asking a term to partition the corpus informatively. That makes the vocabulary work structural for the ad system rather than adjacent to it.

4. 4. What convergence does and does not threaten

The arbitrage unit samples idf hapax (idf 6.343, ~3,320 alphabetic terms). Reducing the #+KEYWORDS singleton ratio from 81% toward <15% sounds like it would collapse that pool.

It does not. The idf map is over indexed tokens — predominantly body text — while convergence normalizes the #+KEYWORDS field. squoze, folktrash, reuleaux are body-text hapax and survive vocabulary work untouched.

Different consumers, different vocabularies:

Consumer Reads Affected by convergence
Auction.quality / I6 doc.keywords yes — directly
Slot eligibility (proposed) #+FILETAGS yes — it is the target state
Arbitrage sampling idf over body tokens no
_srMarks _id no

Worth stating explicitly, because the intuition runs the other way.

5. 5. New invariants

I15 Targeting validity
a slot's scope, if present, MUST be a facet:term pair in the pinned vocabulary version. A slot scoped to a retired or misspelled term fails closed rather than serving unscoped.
I16 Vocabulary pin
every unit MUST carry vocabularyVersion, and its matched terms MUST validate under that version. Without this, a vocabulary v2 silently changes what serves and no ledger event records why.
I17 Serviceability
a document whose keywords are all df = 1 MUST NOT be admitted as inventory at B1. It cannot satisfy I6 for any realistic query, so admitting it produces a line item that never wins — inventory that is not inventory.

I17 is the one that changes the catalogue. Against the current corpus it would exclude a substantial fraction of the 841 eligible documents, and that exclusion count is a useful vocabulary-health metric in its own right: the number of documents too weakly tagged to advertise.

6. 6. The card made the taxonomy user-visible

The unit renders matched: retrieval, memory. That is an internal taxonomy rendered as a public surface.

Consequences the vocabulary document did not have to consider before:

  • Near-duplicate fragmentation now appears on cards. A unit showing matched: agentic system beside one showing matched: agentic-systems exposes the pathology directly rather than burying it in a chip list.
  • Sentence-length keywords ("best practices for jquery performance") break the card layout, not just the chip row.
  • Uppercase variants render inconsistently where the BM25 tokenizer lowercased them anyway — §6.3's argument, now with a visual symptom.

Cheap mitigation, no vocabulary change required: constrain the rendered matched to terms of ≤3 words and lowercase them at render. But it is a patch over a corpus defect, and §6.3 already prescribes the real fix.

7. 7. Gate ordering

Vocabulary violations are upstream ad defects, so they must fail first. A contract check that passes because the vocabulary is broken is a false green.

.PHONY: gate

gate:
    bb scripts/check-headers
    python3 scripts/check_vocabulary.py          # I15 term list validity
    node bin/check-inventory.mjs                 # I17 serviceability
    node bin/check-contract.mjs                  # I1-I7, I16 pin
    node bin/check-plates.mjs                    # asset acceptance

check_vocabulary.py (controlled-vocabulary-v1 §7) becomes a dependency of the contract gate rather than a sibling of it.

8. 8. What the ad system does not get to ask for

The taxonomy exists for the corpus, not for the auction. Three demands the ad system might make and should be refused:

  • Do not add terms to improve fill. §9's four-part test governs new terms; "a sponsored slot would serve more often" is not one of the four.
  • Do not retire hapax keywords to raise the singleton ratio. §6.2 phase 3 already forbids removing location keywords for search reasons; the same logic covers everything else. Unservable documents are I17's problem, not the vocabulary's.
  • Do not version the vocabulary on the ad system's schedule. §11 states the vocabulary does not version in place — v2 is a new document plus a migration pass. I16 exists so the contract can pin a version and absorb that on its own cadence.

9. 9. Recursion

research/2026-keyword-vocabulary-convergence/controlled-vocabulary-v1 is unit #8 in the sponsored inventory. The document specifying the targeting taxonomy is itself advertised through it, and its own matched terms are drawn from the vocabulary it defines. Its keywords — taxonomy, controlled-vocabulary, folksonomy, information-retrieval, org-mode — are in the 2–25 band, so it serves cleanly. It passes its own test.

Refutation condition. The claim is that chip density and auction discriminability are the same objective, so vocabulary work pays the ad system for free. It fails if the 2–25 band turns out to be right for chips and wrong for the auction — most plausibly at the low end, where a df = 2 term gives a perfectly good chip and a candidate set of two, which the quality score cannot rank meaningfully. Test by computing, for every keyword, chip utility (df in 2–25) against auction utility (candidate-set entropy after quality scoring). If the two curves peak at different df, they are separate objectives that happen to overlap, and I17's threshold must be derived from the auction curve rather than inherited from the chip target.

10. Cross-references