TOC rail + highlighter — data flow, boundaries, and the implicit org to html contract

Table of Contents

1. What this note settles

The right-rail TOC ships (site/static/css/style.css, gated by body:has(#table-of-contents) at ≥1024px). The next feature is the highlighter — as the reader scrolls, mark the current section in the rail. The reducer contract for it landed independently in docs/toc-rail-reducer-contract.org (internal to the repo) and a JS mirror at experiments/toc-nav-simulator/reducer.mjs; the head-less simulator already grades that reducer against real org-publish HTML fixtures.

What is not settled and this note settles:

  1. Data flow between browser events, geometry reads, the pure reducer, and DOM mutations.
  2. Boundary responsibilities — what the reducer owns, what the browser adapter owns, what the CSS owns, what the publisher owns.
  3. The implicit contract the org to html exporter provides — the guarantees the highlighter relies on, and the guarantees the highlighter must not rely on.
  4. Bundle strategy — extend the existing tracker.js so the highlighter ships with the ambient page load, without a new <script> include touched into every template.

The reducer itself is done. This is the wiring diagram + boundary spec + bundle decision.

2. Data flow

Six events, one pure reducer, one DOM effect. The reducer is a function (state, event) → state; the adapter is what turns browser noise into events and what turns state changes into class toggles.

dataflow.png

The reducer inputs are five numbers and one array: scrollY, headerH, viewportH, documentH, headings[]=. Output is one string (activeId) or null. Nothing else crosses the boundary in either direction.

3. User flow — from click to highlight

Two entry points: passive scroll, and explicit navigation. Both go through the same reducer.

userflow.png

The "activeId != last" diamond matters. Without diff-caching, every rAF tick writes to the DOM, which is a 60Hz style recalc for zero visual change. With the cache the write happens exactly at section boundaries.

4. Boundaries — who owns what

Five actors. Each owns a strictly non-overlapping surface. The whole feature works because none of them reach across into another's surface.

boundaries.png

Reading the arrows: every cross-boundary edge is a single-direction contract, and each is small enough to state in one sentence. If a future change forces an arrow to reverse, it's a spec-bump signal.

4.1. Reducer ownership — what it must NOT touch

Forbidden Why
document, window, Math.random Purity requirement — the reducer is graded against a JSON oracle on the JVM
setTimeout, requestAnimationFrame Timing is the adapter's problem
IntersectionObserver Alternative algorithm; the reducer decides from geometry, not from intersection events
Any mutable state outside the state arg Idempotency — same input, same output, always

4.2. Adapter ownership — what it must NOT do

Forbidden Why
Decide which heading is active That is the reducer's one job
Modify heading content or ids Publisher-owned; adapter only READS
Style the .toc-active class CSS-owned; adapter only toggles the class name
Fire tracker events Separate concern — the tracker.js observer can pick up the class change via its own IntersectionObserver if wanted
Scroll the document, or write location.hash The adapter scrolls one element, the rail itself, and only to keep the active entry in frame. Page position belongs to the browser anchor navigation and the reader

4.3. Rail scroll: the one thing the adapter does move

The rail is overflow-y: auto at calc(100vh - 14rem). On a long note the highlighted entry leaves the frame and the class toggle is invisible. The reducer gains a second pure decision, rail-scroll-top, and the adapter applies it after the class toggle. The when and how are Emacs's own scrolling rules with the rail as the window and the active entry as point: scroll-margin (two lines, 40 px), scroll-conservatively (scroll-driven activation moves the rail just enough), recenter on a discontinuous jump (anchor click, hashchange, load with a fragment), and next-screen-context-lines (a minimal move of a near-full frame recenters instead). Reduced motion selects instant over smooth at the adapter; the reducer never knows.

The mapping table, the quoted Emacs docstrings, the signature, and the five properties live in docs/toc-rail-reducer-contract.org under "Rail scroll (keeping the active entry in frame)". The user-flow diagram above still shows "rail entry visible in overflow-y scroll" as a browser built-in; it is now an adapter write, and the diagram is due a regeneration.

5. The implicit org to html contract

Six guarantees the highlighter relies on, and two the highlighter must NOT rely on. These aren't in the org manual as a "contract" per se — they're what org-export-html observably does, and the gap between "observable" and "guaranteed" is where the risk lives.

5.1. Relied-on guarantees

# The exporter guarantees Highlighter use
G1 #+OPTIONS: toc:t emits <div id"table-of-contents"><div id="text-table-of-contents"><ul>= Gate: presence check
G2 Every heading gets an id attribute headings array construction
G3 The <ul> inside #text-table-of-contents has <a href"#<id>">= entries matching heading ids Class-toggle target
G4 Heading order in the ToC matches DOM order activeId scan invariant
G5 :CUSTOM_ID: on a heading overrides the random id Stable-id path (see G7)
G6 The publisher's canonical-URL filter rewrites .html paths but does NOT touch #hash fragments Rail anchors survive canonicalization

5.2. Must-not-rely-on guarantees

# The exporter DOES NOT guarantee Consequence
G7 Ids are stable across rebuilds — org-export-format-reference generates random 7-hex-digit ids per export (three exports of the same file produce three disjoint id sets, verified in experiments/toc-nav-simulator/) Highlighter reads DOM at load time; never caches ids across page-loads. Shared #hash URLs break on republish unless the source uses :CUSTOM_ID: on every heading.
G8 Heading top coordinates are stable across viewport widths or font-loading Adapter recomputes on resize AND on load (after web-fonts settle) rather than trusting a single DOMContentLoaded read

The G7 finding is why the reducer takes headings as an argument rather than computing them from a URL or a manifest — the DOM is the only source of truth for what ids exist right now.

6. Bundle strategy

The site loads six sitewide JS bundles today: tracker.js · page-tags.js · web-vitals-init.js · bot-signal.js · global-pollution.js · event-pixel.js. Any of them could host the highlighter. Options:

6.1. Extend tracker.js (recommended)

  • Add wal-sh.site.toc-rail.core (.cljc, pure reducer, 77-line port of experiments/toc-nav-simulator/reducer.mjs) and wal-sh.site.toc-rail.browser (.cljs, ~50 lines) to the :site-tracker shadow-cljs build.
  • The adapter reuses wal-sh.site.tracker.browser' existing rAF-throttle util (already handles scroll for scroll.depth).
  • Guard: gate the toc-rail.browser/init! call on (and (.querySelector js/document "#table-of-contents") (> (.-innerWidth js/window) 1024))=, so pages without a TOC pay no cost.
  • Bundle-size cost: ~2 KB advanced-compiled (function-level) added to the ~130 KB tracker.js. No new <script> tag anywhere.

6.2. New toc-rail.js bundle (rejected)

  • Requires editing every template's postamble to add a new <script src"/static/js/cljs/toc-rail.js">= include, OR touching publish.el to inject it into the exported HTML head.
  • Adds an HTTP round-trip on every page load.
  • The only reason to prefer it: hard isolation from the tracker's event vocabulary. But the two are ergonomically related — the next feature after "highlight current section" is "fire a section.dwell event when a section is on the reading line ≥30s," which is already the tracker's shape.

6.3. Piggyback on page-tags.js (rejected)

  • page-tags.js renders the keyword-chip footer; unrelated concern.
  • Would couple two independent UI behaviors in one bundle without a shared abstraction.

7. Implementation sketch (follow-on, not this note)

Three files land, all in the :site-tracker shadow build:

src/wal_sh/site/toc_rail/core.cljc      ; ~80 lines, port of reducer.mjs
src/wal_sh/site/toc_rail/browser.cljs   ; ~60 lines, adapter
test/wal_sh/site/toc_rail/core_test.cljc ; grades against fixtures/geometry.json

core.cljc exposes (active-id state) and (reachable-limit state) as pure functions. Test harness reads experiments/toc-nav-simulator/fixtures/geometry.json and asserts the CLJC output byte-matches the JS oracle's output for every step of every scenario.

browser.cljs wires:

  1. init! — gate + build headings array + attach listeners.
  2. on-scroll — rAF-throttle to (active-id ...) to diff-cache to DOM write.
  3. on-resize — recompute headings[].top and documentH.
  4. on-load — one delayed recompute after web-fonts settle, to catch G8.

Ledger block on land, with the reducer contract's own version as the pinned dependency.

8. Refutation

The claim is that this reducer + adapter, added to tracker.js with zero template touches, produces a working highlighter with no regression on any current page. It fails if:

  1. The reducer disagrees with reducer.mjs on any fixture scenario (grading harness catches this in CI).
  2. Rail links stop scrolling to the right position — implicates scroll-padding-top interacting with the class-toggle timing; test by clicking every link on a long note.
  3. Pages without a TOC take a measurable perf hit — the gate at init! should return in <1ms; measure with performance.now() in dev.
  4. The tracker's existing scroll.depth event count changes — the two handlers share the rAF-throttle util; a bug here shows up as event-count drift in the daily snapshot.
  5. On a page with :CUSTOM_ID: on every heading, the highlighter still misses one anchor — implicates G4 (heading order in ToC vs. DOM order). This is the one that would be a real spec bug.

9. Cross-references

  • docs/toc-rail-reducer-contract.org (repo-internal) — the ClojureScript reducer contract this note wraps
  • experiments/toc-nav-simulator/README.org (repo-internal) — the JS mirror + real org-publish fixtures + the G7 anchor-instability finding
  • docs/views-projection.org (repo-internal) — bundle-per-page map that §Extend tracker.js references
  • tracker-rebuild-spec — the tracker whose rAF-throttle util the adapter reuses
  • site/static/css/style.css — the rail CSS (:has() gate + fixed positioning + scroll-padding-top)