pocket-es: Query Surface Rollout & Test Plan

Table of Contents

1. Scope

Sequences the structured query surface from the shipped toggle to the deferred lucene grammar. Each phase names its invariants and the mechanism that checks each one: a unit test, a test.check property, a Bombadil LTL property, or a manual step. Companion documents: the contract is query-surface-spec, the wire schemas are in spec, the Bombadil practices are in Bombadil for SPAs.

The verification spine: free mode constructs its IR and is valid by construction; JSON mode is the only surface that can produce invalid IR, so it is the only surface validation fires on. Every claim of "valid by construction" is backed by a property test, not asserted.

2. Phases

phase change new contract? risk
1 (shipped) toggle clarifies the two options no none
2 pocket-es.dsl malli; validate in JSON branch request schema v1 malli gzip
3 textarea + Tab-complete + mode URL sync no DOM state machine
4 (deferred) radio + simple=/=lucene producers no parser correctness

Phases are independently shippable. Each lands with its test plan green before the next starts.

3. Invariants (consolidated)

Drawn from query-surface-spec and the UI state machine in spec.

id invariant phase mechanism
INV-1 every IR reaching search() is schema-valid, or search() is not called and an inline error shows 2 unit + bombadil
INV-2 free/simple modes construct IR; dsl/explain is nil for all their output 2 test.check
INV-3 a query container has exactly one clause key 2 test.check
INV-4 lucene parse/print round-trips on generated IR 4 test.check
INV-5 toggle off ⇒ free input visible, textarea hidden; toggle on ⇒ inverse 1,3 bombadil
INV-6 mode never auto-translates the query text on switch 3 bombadil
INV-7 URL q/d/p/j round-trips: reload at the URL restores identical view 3 unit + bombadil
INV-8 structured mode passes the JS object to search unchanged (no double js->clj) 2 unit
INV-9 rendered hit count ≤ requested size all bombadil
INV-T tokenizer: no stopwords, lowercase, idempotent, bounded tf n/a test.check (exists)

4. test.check properties

These are the property-based tests. INV-T already lives in test/pocket_es/token_test.cljc (25 assertions, 300+ iterations). The new ones land in test/pocket_es/dsl_test.cljc alongside phase 2/4.

4.1. INV-2 – the refutation hook (highest leverage)

The free producer must never emit an IR the schema rejects. If it does, the constructor and the contract have drifted.

(require '[clojure.test.check.clojure-test :refer [defspec]]
         '[clojure.test.check.properties :as prop]
         '[clojure.test.check.generators :as gen]
         '[pocket-es.dsl :as dsl])

(defn free-ir
  "Mirror of run-free!'s IR constructor (CLJS side builds the JS equivalent)."
  [text]
  {:query {:multi_match {:query text
                         :fields ["title^3" "keywords^2" "description"
                                  "headings^2" "terms"]}}
   :size 200})

(defspec free-mode-ir-always-valid 500
  (prop/for-all [s gen/string]
    (nil? (dsl/explain (free-ir s)))))

A failure here is finding #2 from the L7 review: a field in the fan-out (terms) absent from the closed set. The schema's +fields+ and ::boost already include terms; this test is what keeps them in sync.

4.2. INV-3 – single-clause container

(def gen-clause
  (gen/elements
   [{:match {:_all "x"}} {:term {:keywords "x"}} {:prefix {:title "x"}}
    {:match_all {}} {:multi_match {:query "x" :fields ["title^3"]}}]))

(defspec single-clause-required 200
  (prop/for-all [a gen-clause b gen-clause]
    (let [two (merge a b)]                       ; two distinct keys
      (or (= 1 (count two))                       ; collision -> still one key
          (some? (dsl/explain {:query two}))))))  ; else must be rejected

4.3. generator sanity – valid IRs validate

(require '[malli.generator :as mg])

(defspec generated-requests-validate 200
  (prop/for-all [req (mg/generator dsl/SearchRequest)]
    (dsl/validate req)))

4.4. INV-4 – lucene print/parse round-trip (phase 4)

;; malli.generator produces valid IRs in the lucene subset; print renders each
;; to its lucene string; re-parse; assert identity.
(defspec lucene-roundtrip-identity 300
  (prop/for-all [ir (mg/generator dsl/LuceneSubset)]
    (= ir (lucene/parse (lucene/print ir)))))

5. Bombadil LTL check

pocket-es is Bombadil's running example (Bombadil for SPAs). The query surface is a client-side state machine on one URL, which is the case Bombadil's quiescence model handles well. Extractors read the live DOM; verify every selector in a real browser first – most early violations refute the spec, not the site.

5.1. State extractor

mode      = #pes-json-toggle:checked ? "json" : "free"
freeShown = getComputedStyle('#pes-input').display    != "none"
jsonShown = getComputedStyle('#pes-json-input').display != "none"
hitCount  = document.querySelectorAll('#pes-results .pes-hit').length
errShown  = !!document.querySelector('#pes-results .pes-error')
size      = 200 in free mode; parsed .size or engine default in json mode

5.2. Properties

  • INV-5, surface exclusivityalways (mode = "free" implies freeShown and not jsonShown)= and always (mode = "json" implies jsonShown and not freeShown)=.
  • INV-9, bounded resultsalways (hitCount < size)=.
  • Error visibility (the anti-silent-failure property) – in JSON mode, now (textarea holds malformed JSON) implies eventually-within 1s (errShown). This is the property that would have caught the silent-empty-results bug: a malformed-but-parseable IR must surface an error, never render as zero hits.
  • INV-6, no auto-translationnow (toggle flips) implies (query text in the newly active surface is unchanged from its prior value) – i.e. flipping the toggle does not rewrite what the user typed.
  • Livenessalways (typing a non-empty common term in free mode implies eventually (hitCount > 0)) for terms known to be in the corpus.

Bound runs with --time-limit, not a step count; keep the boundary on the search surface origin.

# real-browser drive (watch the run); managed-chromium path also works
chromium --remote-debugging-port=9992 &
bombadil test-external --remote-debugger http://localhost:9992 \
  --create-target --time-limit 120 \
  http://localhost:8000/research/pocket-es/

6. Per-phase test plan

6.1. Phase 1 – toggle clarity (shipped)

Pure UI; no contract. Already verified end-to-end (Playwright): toggle exists, default unchecked, free input visible / textarea hidden, mode swap, ?j=1 URL sync, invalid-JSON inline error, valid JSON query returns hits, Tab-complete (skeleton seed + prefix completion). Bombadil: INV-5 only.

  • [X] gmake cljs builds clean (0 warnings)
  • [X] bundle gzip delta acceptable (32KB → 35KB)
  • [X] manual: toggle sits right of the cluster line, adds no vertical row

6.2. Phase 2 – contract + validation

  1. Author src/pocket_es/dsl.cljc (tangled from query-surface-spec).
  2. Measure malli gzip first (L7 finding): if malli.core + malli.error adds >3KB gzip, ship a hand-rolled valid?=/=explain in CLJS and keep malli build-time-only for the JSON-Schema emit. Record the number here.
  3. Wire run-structured! to validate via dsl/explain in the JSON branch and render inline; pass the JS object to search unchanged (INV-8).
  4. Tests: test.check INV-2, INV-3, generator sanity (clojure -M:test).
  5. Bombadil: add the error-visibility property.
  6. [ ] dsl/explain returns nil on valid IR, humanized errors on invalid
  7. [ ] INV-2 spec passes at 500 iterations (free-mode IR always valid)
  8. [ ] INV-8: a console call pocketES.search(parsed) and the UI path produce identical results (no double-convert regression)
  9. [ ] malformed-but-parseable IR (, ={query:{bool:{}}}) shows an inline error, never zero silent hits

6.3. Phase 3 – textarea, Tab-complete, mode URL sync

  1. mode atom (:free default), URL-synced via j; restore on init + popstate.
  2. Textarea swap; Tab-complete over the closed vocabulary; suppress completion inside JSON string literals (L7 finding: ma| in a value must not become match).
  3. Clear the shared debounce timer on mode switch (L7 finding).
  4. Tests: unit for URL round-trip (INV-7); bombadil INV-5/6/9.
  5. [ ] reload at ?q…&j=1= restores JSON mode + textarea contents
  6. [ ] back/forward restores mode and query
  7. [ ] flipping the toggle does not rewrite typed text (INV-6)
  8. [ ] Tab inside a string value does not corrupt the value

6.4. Phase 4 – radio + lucene (deferred)

Gated on the refutation condition: a recurring hand-typed bool blob. Until then, do not build. When built: simple and lucene producers compile to the same IR; INV-4 print/parse round-trip property; reuse one recursive-descent parser with two configs.

7. Commands

gmake cljs           # build :site, copy to legacy path
gmake index-check    # token property tests + reachability
clojure -M:test -e "(require 'pocket-es.dsl-test) ..."  # phase 2/4 properties
gmake serve          # local preview for Bombadil at :8000

8. Status

Phase 1 shipped and verified. Phase 2 is the next step and is blocked only on the malli gzip measurement. Phases 3–4 follow the contract.