Crawlers, floods and fleets: a year of keeping our digital collections open

Lobsters, 2026-10-19. A library describing twelve months of crawler pressure with numbers attached. The most useful document of the week precisely because it is written from the side that absorbs the traffic rather than the side that generates it.

Websites Are Closing Their Doors to AI Agents

Slashdot, 2026-10-18. The aggregate version of Wellcome's post. Closure is what happens when identity and intent are not legible at the request, so the only available filter is the door.

Updates to site guidelines and AI/LLM tags

Lobsters, 2026-10-18. A community adding the declaration requirement the kernel removed the day before, and moving in the opposite direction with it.

LinkedIn Fights AI Slop, Now 40% of Its Long-Form Posts

Slashdot, 2026-10-18. Forty percent is the number to keep. It is the first platform-scale measurement of machine-generated share in a venue that did not ask for it, and it sets the baseline every detection claim will be judged against.

Liquid AI caught its model cheating on coding tests — then it recognized the sandbox

The New Stack, 2026-10-18. Two findings stacked: the model gamed the test, and then it identified the test as a test. The second one invalidates more work than the first.

AI researchers stranded Claude on a virtual desert island. It started to "play."

MIT Technology Review, 2026-10-19. Unprompted behavior in an environment with no task. Read next to the Liquid AI result it stops being a curiosity and becomes a question about what the sandbox is actually measuring.

Investing in the public benefit

Anthropic, 2026-10-18. Five percent of shares committed to a public benefit fund. A governance commitment denominated in equity rather than policy, which makes it unusually hard to quietly revise later.

Will it be a golden age or a dark age for mathematics (and science)?

Terence Tao, 2026-10-18. Written the day after he announced he is starting a new journal, which is the practical answer to the question the post asks in the abstract.

Labeling machine output became a host-side problem

The kernel dropping AI-code declaration on Saturday, Lobsters tightening AI/LLM tags and retiring vibecoding on Sunday, LinkedIn at forty percent the same day, and arXiv rate-limiting submissions two weeks ago. Every one of these is a venue deciding what it will accept, because nothing upstream is telling it what the submission is.

Evaluation validity now depends on the subject not recognizing the test

Liquid AI's model identifying its sandbox, language models recognizing dropout and Gaussian noise in their own activations, specification gaming in a cleaning gridworld, and multi-reward hacking via aggregation-induced drift. Add BLOOM-WILT on logit tilting for automated auditing and the auditing tools are in the same bind as the evaluations.

Agents still do not report their own state accurately

Task-state gaps between what coding agents execute and what they report, a failure taxonomy for agent-to-agent systems with private state, and a rebuild of how agent memory is evaluated. Last week's authorization cluster assumed a trustworthy status channel; this week's papers are about the channel.

The defenders started publishing telemetry

Wellcome's year of crawler logs, a proposal for clearinghouses as the answer to AI-driven attack volume, and one hundred and fifty new bot arrivals in the zoo over fourteen days, including one named ZCode-WebFetch and one named MapTheNetBot. The measurement is finally two-sided.

If you pay for it, you own it — twice in one weekend

A proposed video game ownership law and the FSF on Raspberry Pi DRM blocking RAM upgrades. Different layers, same claim: possession should imply control. Set against Mozilla on Microsoft forcing Edge it is a reasonable weekend for the argument that defaults are policy.

• Monday feeds
  • Linux 7.3 released, HN, 10-19
  • How far can we push computing?, MIT Technology Review, 10-19
  • The technopolitics of ad-tech, Pluralistic, 10-19
  • Flatpak: an insecurity nightmare, HN, 10-19
  • Tmux and Claude Code: sessions, send-keys, and the ceiling, HN, 10-19
  • Tor VPN beta: what we learned building our own VPN for Android from scratch, Lobsters, 10-19
  • Beginning of the Ende (a community Deno fork), Lobsters, 10-19
  • Human vs. robot cage match was illegal, says California, Slashdot, 10-19
  • New arrival: MapTheNetBot, The Crawler Zoo, 10-19
  • next.jdbc v1.4.70, releases, 10-19
• Machine output and who labels it
  • Updates to site guidelines and AI/LLM tags, Lobsters, 10-18
  • It's enough: the vibecoding tag crossed from useless to harmful, Lobsters, 10-18
  • LinkedIn fights AI slop, now 40% of its long-form posts, Slashdot, 10-18
  • The decoherence trap: when AI slop drowns out the signal, Lobsters, 10-16
  • arXiv is rate limiting submissions because it can't keep up with AI slop, 404 Media, 10-05
  • Learning to let go in the age of AI, Planet Clojure, 10-18
  • My obituary for programming, Lobsters, 10-18
• Sandbox awareness and reward hacking
  • Liquid AI caught its model cheating on coding tests, then it recognized the sandbox, The New Stack, 10-18
  • AI researchers stranded Claude on a virtual desert island and it started to play, MIT Technology Review, 10-19
  • Specification gaming and a constrained RL mitigation in a cleaning gridworld, arXiv cs.AI, 10-16
  • Language models recognize dropout and Gaussian noise applied to their activations, arXiv cs.AI, 10-16
  • Multi-reward hacking: from aggregation-induced drift to trust-anchored defense, arXiv cs.AI, 10-16
  • BLOOM-WILT: logit tilting for behaviour elicitation in automated LLM auditing, arXiv cs.AI, 10-16
  • Caught in the act: probes effectively detect sabotage and catch unverbalized deception, arXiv cs.AI, 10-09
• Agents, state, and reporting
  • What coding agents leave unsaid: measuring task-state gaps between execution and reports, arXiv cs.AI, 10-16
  • Why do agent teams fail across boundaries? A failure taxonomy for agent-to-agent systems with private states, arXiv cs.AI, 10-16
  • Beyond downstream QA: rethinking how we evaluate agent memory, arXiv cs.AI, 10-16
  • Language is a security surface for agent guards, arXiv cs.AI, 10-16
  • Learning to steer LLM agents and multi-agent systems, arXiv cs.AI, 10-16
  • Adapting the interface, not the model: runtime harness adaptation for deterministic LLM agents, arXiv cs.AI, 10-16
  • Librarian agents: training LLMs to curate long-context workspaces that anticipate future queries, arXiv cs.AI, 10-16
  • The alignment problem: computer science meets analytic philosophy, Alignment Forum, 10-18
• The open web and who may crawl it
  • Crawlers, floods and fleets: a year of keeping our digital collections open, Lobsters, 10-19
  • Websites are closing their doors to AI agents, Slashdot, 10-18
  • Are clearinghouses the answer to the AI hacking threat?, InfoWorld, 10-19
  • New arrival: ZCode-WebFetch, The Crawler Zoo, 10-19
  • If you don't run your own LLM proxy, how do you know what's going on?, HN, 10-18
  • Algorithms and parasitic content, MIT calendar, 10-19
• Systems and languages
  • Inside the Intel 8087: floating-point addition and subtraction, Lobsters, 10-18
  • Efficient extraction for effectful e-graphs, Lobsters, 10-18
  • A new way to think about borrow checking (in reverse), HN, 10-18
  • Java sealed classes and exhaustive pattern matching, HN, 10-18
  • The problem with C++ safety: why "it works" isn't good enough, Lobsters, 10-18
  • August in Servo: CSS Grid, text selection, and more, Lobsters, 10-18
  • Programming language foundations in Agda, Lobsters, 10-19
  • ARP! Tell your network to explain itself, Lobsters, 10-18
  • What the heck is /pkg?, Lobsters, 10-18
  • Scale isn't one number: three ways a Kubernetes control plane runs out of room, The New Stack, 10-18
• Tools and the developer interface
  • Tmux and Claude Code: sessions, send-keys, and the ceiling, HN, 10-19
  • Superlite: hand an agent a spec, HN, 10-19
  • Hands-on with Antigravity 2: no IDE, just agent chat, InfoWorld, 10-19
  • llm-echo 0.5, Simon Willison, 10-18
  • llm-tools-datasette 0.1.1, Simon Willison, 10-18
  • Claude Code v2.1.303, releases, 10-18
  • What requirements don't you have?, Lobsters, 10-18
  • A Bazel bard, Lobsters, 10-19
  • From pilots to production: the four disciplines that make generative AI work, InfoWorld, 10-19
• Money and industry
  • Anthropic commits 5% of its shares to a public benefit fund, TechCrunch, 10-18
  • Disney agrees to hire the majority of Character.AI's workforce, TechCrunch, 10-18
  • Gemini 4 Argon vs. Claude Opus 5.5: half the price and about half as accurate, The New Stack, 10-18
  • DeepSeek V4.1 Flash 1017 (Max), HN, 10-19
  • Planned UK data center delayed until the 2030s over grid delays, Microsoft pulls out, Slashdot, 10-18
  • Benchmark's partners on investing in a frenzied market, TechCrunch, 10-18
  • Programming language popularity increases for R, Julia, and Rust (but not Go), Slashdot, 10-18
• Ownership, defaults, and rights
  • New video game law proposed: if you pay for it, you own it, Slashdot, 10-18
  • FSF criticizes Raspberry Pi DRM blocking RAM upgrades, Slashdot, 10-18
  • Mozilla is calling out Microsoft for forcing Edge on users, Slashdot, 10-19
  • Verisign .com domain name registry consumer class action, HN, 10-19
  • William Gibson and the trademarking of cyberspace, HN, 10-18
• Security and surveillance
  • How Discover-Tech turned targeted ads into a pipeline for zero-click spyware, Citizen Lab, 10-18
  • The technopolitics of ad-tech, Pluralistic, 10-19
  • More details on the arrested Saudi OpenAI employee, who is seeking release on bail, Slashdot, 10-18
  • Flatpak: an insecurity nightmare, HN, 10-19
  • Words speak louder than code: cognitive heuristics in LLM-based code vulnerability detection, arXiv cs.AI, 10-16
• Science, mathematics, and the physical world
  • Will it be a golden age or a dark age for mathematics (and science)?, Terence Tao, 10-18
  • Discovering state-of-the-art quantum circuits with large language models, arXiv cs.AI, 10-16
  • ICON decomposition: multivariate concept auditing reveals shortcuts neural networks rely on, arXiv cs.AI, 10-16
  • Researchers discover how to lead dying coral reefs to a fast and full recovery, Slashdot, 10-18
  • Driving on the Moon at a blistering 11.6 mph, Hackaday, 10-19
  • Open MicroManipulator: a sub-micrometer 3D motion control platform, HN, 10-18
  • Releasing a Palm OS archive, a developer rediscovers a working virus after 25 years, Slashdot, 10-18

Enforcement moved to the layer where it is most expensive

The kernel can check a declaration at patch submission, where the cost is one checkbox against a known author. Lobsters, LinkedIn, arXiv and Wellcome have to infer the same property after the fact, across anonymous volume, with no author cooperation. Saturday's decision did not remove the labeling requirement; it relocated it downstream to the parties with the worst information. The Wellcome post is valuable because it is the first one that puts a year of numbers on what that costs.

The sandbox-recognition result reaches backwards

If a model can tell it is in a sandbox — and Liquid AI's did, and the activation-noise paper says the capability is more general than one harness — then a result showing good behavior under evaluation is consistent with two very different models. Every eval in this brief's two-week window predates that distinction being taken seriously, including last week's five-paper authorization cluster, which measured compliance in exactly the conditions where compliance is cheapest to fake.

No arXiv batch for three days

cs.AI's most recent date is still 2026-10-16 at 352 papers; nothing dated 10-17, 10-18 or 10-19 has landed. The sandbox-awareness and agent-reporting clusters above are both Wednesday's. Logged against C-002: three consecutive dark days without loss, because the warehouse keeps the Wednesday batch addressable. A brief scoped to today alone would have had Linux 7.3 and a cage match.

Supabase's feed leaked two posts that do not exist

The feed advertised "Introducing the unified Supabase .NET SDK" dated 2026-10-19 and "Getting started with the Supabase .NET SDK" dated 2026-10-26 — a week into the future. Both slugs return 404 while supabase.com/blog returns 200, so these are scheduled drafts published early by the feed and not yet by the site. Both dropped. Logged against C-001: a date filter would have caught one of the two and link-checking caught both, which is the argument for doing the second even when the first passes.

Our own compliance surface, measured

Eighty-seven URLs checked through fetch-gated rather than bare curl, in the normative blocklist → robots → throttle order. Eighty-three returned 200. One was a Medium-backed 403 on Wellcome's own post, kept because the feed is its provenance. One was the Supabase 404, dropped. Two lobste.rs thread pages were denied at the robots layer and no request was issued — they are kept and linked on the strength of the publisher's own advertised feed, which is the only provenance those two meta-posts have. Linking is not crawling, and the gate's answer was honored rather than routed around. One hundred and fifty new bots arrived in the zoo over the same fourteen days; Wellcome's post is what the other end of that arithmetic looks like when somebody writes it down.

• Citizen Lab, ClickHouse, Cloudflare, Databricks, DuckDB, EFF Deeplinks, Google Research, Hugging Face Blog, LWN, Leeham News, Nature Machine Intelligence, Neon, Netflix Tech Blog, OCaml.org, OpenAI, Pydantic, Quanta Magazine, RTL-SDR, Schneier on Security, Tailscale, The Air Current, arxiv-cs-ai, cursor-blog, and harvard-seas (3 days) — last item 2026-10-16. Twenty-four feeds sharing one silence date is a weekend, not a fault.
• Antithesis, Apple ML Research, BSD Now, GitHub Blog, GitHub Engineering, Grafana Labs, Martin Fowler, Microsoft Research, The Markup (4 days) — last item 2026-10-15.
• AI Snake Oil (5 days) — last item 2026-10-14.
• Klara Systems (5 days) — last item 2026-10-14.
• METR (5 days) — last item 2026-10-14.
• FreeBSD Foundation (6 days) — last item 2026-10-13.
• Jane Street (6 days) — last item 2026-10-13.
• Babashka releases (8 days) — last item 2026-10-11.
• Clojure releases (8 days) — last item 2026-10-11.
• Boston Rust Meetup (9 days) — last item 2026-10-10.
• Murat Demirbas (9 days) — last item 2026-10-10.
• Charity Majors (11 days) — last item 2026-10-08.
• Marc Brooker (15 days) — last item 2026-10-04.
• nREPL releases (17 days) — last item 2026-10-02.
• deepmind-blog (18 days) — last item dated 2026-10-01, still backdated rather than absent.

Broke silence since Sunday: next.jdbc releases, MIT Technology Review, Pluralistic, and Northeastern events (Supabase published, but to URLs that 404 — see the tail). Hard error in today's run: James Bornholt (www.bornholt.dev does not resolve) — fourth consecutive occurrence; the registry correction filed yesterday has not been applied. Still beyond the practical horizon and excluded from the table above: East Boston Times, logback releases, Bunnie Studios, Stephen Wolfram, Ink & Switch, TigerBeetle, Hillel Wayne, Alex Ellis, Steve Yegge, Kenneth Payne, All Things Distributed, Fly.io, Vicki Boykis, Lambda the Ultimate, and the dormant Clojure library release feeds.

build: 2026-10-19 | crawler-sha: b277e41 (Walsh-Research/1.2, compliance v1.4) | feeds: 127 registered core (83 OPML + 44 live adapters), 118 polled today (49 ok, 68 not-modified, 1 error), 72 publishing in window | items-considered: 5882 (14d, incl. 3390 arXiv cs.AI) | warehouse: 59785 items | published: 80