Controlled Vocabulary v2 for wal.sh: Adtech Facet and Bidirectional Matching

v1 was written on 2026-05-31 to convert 492 free-text-keyword docs into a 6-facet, 47-term controlled vocabulary (Controlled Vocabulary v1). Two things have changed since then that make a v2 necessary rather than optional.

First, keyword hygiene is now the sole reason-to-serve gate for the sponsored slot. wal-sh.adtech.sponsored-research.core/matched-terms returns up to three keywords from a candidate doc's :keywords list that share a bidirectional substring with a query token; V-final V5 requires the returned vector to be non-empty. If matched-terms returns [], no sponsored card serves for that doc under that query. Keyword hygiene is no longer just a search-ranking concern — it directly gates ad inventory. The matched-terms implementation lives in src/wal_sh/adtech/sponsored_research/core.cljc (lines 85-110) and the build+contract discipline it sits inside is documented in adtech-contracts-first-process.

Second, the corpus has grown from 623 docs (v1) to 855 docs (this file's Corpus snapshot), and a new content cluster — site/research/browser-adtech-infra/ with 15 spec.org files plus a creative-ops/ subtree with 9 creative pages — has entered the tree carrying keywords that don't belong to any existing facet. v1's six facets (lang, domain, format, era, project, status) do not cover browser-adtech, tracker, pocket-es, plates, beacon, chumbox, creative-ops, sponsored-research. These are neither languages nor domains nor formats nor eras — they are components of an experimental adtech surface the site operates as its own subject matter. v2 adds one facet, adtech, with 12 terms, and formalises a compound-keyword policy whose necessity was invisible before bidirectional matching.

The adtech facet exists because the browser-adtech-infra cluster has 15 spec files with keywords that don't belong to domain:* (they are not what the doc is about in a topical sense — they are the system components the doc specifies). Treating them as domain terms would put browser-adtech in a bucket alongside domain:security and domain:distributed, which is a category error: the browser adtech surface is a thing wal.sh operates, not a territory wal.sh researches. The project=/=domain boundary from v1 is the guide — but the adtech subsystem is large enough (15 specs + 9 creatives + core + gate + browser adapter) that folding it into project:* would eat half the project facet.

Twelve terms. Same earned-term criterion as v1: appears in ≥3 docs AND is irreducible.

Six terms above are essential (appear in ≥3 docs already or are load- bearing for matched-terms): browser-adtech, sponsored-research, creative-ops, pocket-es, tracker, plates. The remaining six are included on the irreducibility criterion (they name distinct spec files that other specs cross-reference) even though several are hapax today — they will grow with the cluster.

This is the v2 addition that has no analogue in v1. v1 assumed keywords were matched by exact string equality (search chips resolve to docs that list the exact term). matched-terms does bidirectional substring matching:

;; Reference impl (matches v-final JS oracle)
(filter (fn [k]
          (some (fn [t]
                  (or (str/includes? k t)
                      (str/includes? t k)))
                query-tokens))
        keywords)

Consequences the vocabulary must plan for:

1. The compound keyword local-first MATCHES query token local (k.includes(t) — the keyword contains the token).
2. Query serving MATCHES keyword sponsored-serving (k.includes(t)).
3. Query sponsored MATCHES keyword sponsored-research, sponsored-display, sponsored-onboarding — one query fans out to every sponsored keyword variant.
4. Query research MATCHES keyword sponsored-research (t.includes(k) is not needed here; k.includes(t) handles the research substring inside sponsored-research).
5. Query bot would MATCH keyword robot (k.includes(t), because "robot" contains "bot"). Silent semantic slippage.

Analysis of site/static/search-index.json as of the run at 2026-08-17T00:34Z. Reproducible via bb (see src/wal_sh/site/ for the readers; the counts below are from a one-off bb script). Caveat: the index does NOT include the 15 browser-adtech-infra/*/spec.org files or the 9 creative-ops/creatives/*.org files yet — the indexer either filters spec.org or has not been re-run since those files landed. The adtech-facet counts under 3 come from the org files on disk, not from this index. See 7.

Ten sampled hapax (arbitrage-unit gold under matched-terms — the queries that hit exactly one doc):

• boeing-speea-contract
• def con 18
• co-located event
• hypersensitivity
• context-repository
• professional development
• open-source-ai-advocacy
• high performance computing
• codex-desktop-linux
• trust building

The v2 proposal is falsified if any of the following holds after a full migration pass:

1. The adtech facet, after Phase 2 review, ends up with fewer than 6 terms actually applied to ≥3 docs. This would prove the facet is speculation rather than a real emergent category. (Fallback: demote the terms to project:* entries and retire the facet.)
2. Any of the compound-keyword rules produces a matched-terms result set that violates the sponsored-research V-final contract — most notably, a query that matches ZERO keywords on a doc that should plausibly serve a sponsored card. This would prove the vocabulary is under-connected relative to the reference JS oracle's substring behaviour. (Fallback: add short-form aliases per the corollary rule more aggressively, and re-measure the empty-matched-terms rate.)
3. Post-migration, the singleton ratio has not dropped below 70% (from 79.7%). The additive-alias approach is designed to lower the hapax share by giving lonely compounds a broader alias. If it doesn't, the vocabulary work has produced no measurable convergence and this file should be superseded rather than iterated. (Fallback: return to the pre-2026 folksonomy view and accept that keyword hygiene is not controllable at this corpus scale.)
4. Any single query token, after the _root adjacency fanout enabled by substring match, exceeds plausibly should serve 3 sponsored cards but returns 30. The compound-keyword policy is supposed to keep matched-terms slicing to ≤3 without gaming the take 3 call. If the take 3 is doing all the work and the pre-slice set is uniformly huge, the vocabulary is not doing its job.
5. The 15 browser-adtech-infra/*/spec.org files remain absent from search-index.json after the re-index step in Phase 5. If the indexer refuses spec files by design, then the adtech:* vocabulary is registering terms that never reach matched-terms — the facet documents system components the search engine can't see. In that case, either the indexer needs to include spec files (preferred) or the adtech:* facet needs to be limited to the 1 index doc + 9 creatives + the tracker/beacon/plates specs that DO reach the index.

• Controlled Vocabulary v1 — the v1 that this file extends
• 2026-keyword-vocabulary-convergence — the convergence method these vocabularies target
• v1.1 consolidation run — 128 canonical terms from gemini-2.5-flash, not yet applied
• sponsored_research/core.cljc — the matched-terms reference impl
• adtech-contracts-first-process — the eight-step recipe every adtech feature crosses
• sponsored-research/spec.org — the V-final contract that requires non-empty matched-terms
• nomenclature-audit.org — L7 audit that flagged the adtech naming drift this file responds to