REPL-Driven Ticket Triage: A Data-First Agent Tool Backend
Table of Contents
- 1. Thesis
- 2. Source landscape: mock-Python utils vs a live store
- 3. The backend is data, not code
- 4. Boundaries verified at the REPL
- 5. Configuration and analysis are the same surface
- 6. The agentic harness
- 7. The workflow, concretely
- 8. Survey
- 9. Open conjectures
- 10. Claims to test  refutation
- 11. Resources
- 12. References
1. Thesis
The best interface between an agent and a complex system is a REPL. A ticket
backlog is that problem worn as a queue: a store of work items, each with a
status you mutate and a shape you query, that an agent pulls from and pushes
back to. You do not learn the backlog's contract from docs — you evaluate
(next-ticket) and watch the map come back, then (set-status! …) and watch
one line change on disk.
This is the companion to REPL-Driven Feed Crawling and Flight Tracking. Same thesis, different domain. There the divergence between replicas of one system was the signal; here the interesting object is the tool backend itself — and the claim is that when you build it REPL-first, "the agent's tools", "the configuration you edit", and "the dataset you analyze" stop being three layers and become one.
2. Source landscape: mock-Python utils vs a live store
Re-checked live via `gh api repos/anthropics/claude-cookbooks/contents/tool_use/utils`: customer_service_api.py and visualize.py are both still present (alongside customer_service_tools.py, team_expense_api.py, __init__.py). The `@beta_tool` decorator (`from anthropic import beta_tool`) is used at 8 call sites in customer_service_tools.py. Upstream repo unchanged in the respects this section characterizes, since the original 2026-07-26 pass.
Anthropic's cookbooks ship a reference tool backend, tool_use/utils, used by the compaction and programmatic-tool-calling demos. It is a good baseline precisely because every axis is the opposite of the REPL-driven one:
| tool_use/utils (Python) | this backend (Clojure/bb/REPL) |
|---|---|
customer_service_api generates mock tickets |
wal-sh.site.beads reads the real issue store |
tools are @beta_tool decorated functions |
tools are EDN data (tool-defs) + a dispatch fn |
visualize.py renders responses |
the REPL is the visualizer (group-by, frequencies) |
| edit code → re-run the whole script | (next-ticket) / (agent/run …) incrementally in one session |
| domain is a fixture | domain is beads — a git-native issue tracker |
The Python version is the right shape for a tutorial — hermetic, reproducible, no external state. The REPL version is the right shape for doing the work: the backend is the thing you actually triage against, and you reshape it while it runs.
3. The backend is data, not code
The whole tool surface an agent sees is a literal value you can print, diff, and edit at the REPL. No decorator, no registration step:
(require '[wal-sh.site.beads :as b])
b/tool-defs
;;=> [{:name "get_next_ticket"
;; :description "Get the next ready (open, unblocked) ticket..."
;; :input_schema {:type "object" :properties {} :required []}}
;; {:name "update_ticket_status"
;; :input_schema {:properties {:id {...} :status {:enum ["open" "in_progress" "closed" "blocked"]}}
;; :required ["id" "status"]}}]
Because it is data, adding a tool is conj, and constraining one is
assoc-in — evaluated live, no restart. The dispatch side is one pure-ish
function, (handle-tool name input), returning the JSON string the Messages API
wants. That is the entire "SDK": a vector and a case.
4. Boundaries verified at the REPL
Re-verified against the exact commit that wrote this note (f5a7c1fc, 2026-07-26) rather than today's drifted backlog (now 306 records, 150 open, 136 ready --- still 14 dependency-blocked by coincidence, ~9.3%). `git show f5a7c1fc:.beads/issues.jsonl` has exactly 303 records: 147 open, 155 closed, 1 in_progress; computing readiness from that snapshot gives 133 ready / 14 dependency-blocked --- an exact match to this section's 147/133/14/~9% figures and the original 303-line count. A live `bb -cp src` run today against current src/wal_sh/site/beads.clj reproduces the schema (same 15 keys) and all three error-contract behaviors (empty-queue error+progress, unknown-tool, nil-on-bad-id) byte-for-byte, so the contracts --- not just the historical counts --- hold against the current backend (.beads/config.yaml still sets no-db: true, unchanged from this note's description). set-status!'s line-preserving behavior re-confirmed by reading the current function body (not re-executed against the live tracked JSONL, to avoid mutating production issue data).
Every contract below was checked against the live backlog; the shapes are the tooling's, not the backlog's.
The schema an agent's tools resolve against (structure, no content):
(sort (keys (first (b/read-issues))))
;;=> (:acceptance_criteria :comment_count :created_at :created_by :dependencies
;; :dependency_count :dependent_count :description :id :issue_type
;; :owner :priority :status :title :updated_at)
The failure modes — each returns a value, never throws, so the agent loop can keep going:
;; empty queue: get_next_ticket yields an error object with a progress count
(let [h (b/queue-backend [])] (h "get_next_ticket" {}))
;;=> {"error":"No ready tickets in the queue","processed":0,"total":0}
(b/handle-tool "nope" {})
;;=> {"error":"unknown tool: nope"}
;; updating an id that doesn't exist mutates nothing and returns nil
(b/set-status! "does-not-exist-xyz" "closed")
;;=> nil
- Readiness is dependency-aware.
readyexcludes open tickets whose dependencies aren't closed — 14 of 147 open tickets (~9%) were dependency-blocked, soget_next_ticketnever hands the agent work it can't start. set-status!is line-preserving. Setting one ticket's status rewrites only that line of the JSONL; every other line is byte-identical, so the git diff is a single line and the store stays reviewable. (Verified: 303 lines before = 303 after, exactly one changed.)
5. Configuration and analysis are the same surface
The claim that makes this more than "Python but parens": at the REPL the ticket store is simultaneously the config you edit and the dataset you analyze, with no API or visualization layer between.
Analyze — ordinary Clojure over the maps, no visualize.py:
(->> (b/read-issues) (map :priority) frequencies (into (sorted-map)))
;; a histogram across priority buckets
(->> (b/read-issues) (filter #(seq (:dependencies %))) count)
;; how many items carry a dependency edge (the blocked-work risk)
(->> (b/ready-queue) (map :issue_type) frequencies)
;; the shape of what's actually workable right now
Configure — the same values, mutated in place:
(b/set-status! "SYNTHETIC-001" "in_progress") ; advance one item
(b/claim-next!) ; pop the next ready + claim it
In the Python fixture these are different files — an API module to read, a
tools module to act, a visualize module to see. Here group-by is the
dashboard and assoc is the migration. The dataset, the configuration, and the
agent's tools are one store seen three ways.
6. The agentic harness
Re-confirmed today by reading src/wal_sh/site/agent.clj: run-tickets (line 156) takes :compact-threshold/:max-turns, and its underlying `run` defaults handle-fn to beads/queue-backend --- an in-memory cursor, no disk write --- matching "dry-run backend" / "mutating nothing" exactly. The 37%-fewer-tokens figure (57929->36762) still matches the companion note's table (site/research/2026-compaction-against-beads/index.org:118), which that note's own wave-4 annotation (PR #240) independently verified against the git commit message for 8cf7af00 rather than re-trusting the prose --- unchanged since 2026-07-26.
The same backend drives an autonomous loop. wal-sh.site.agent is a Messages-API
tool-use loop (a port of Anthropic's compaction cookbook) that takes any
tool-defs + handle-fn; pointing it at beads is one line:
(require '[wal-sh.site.agent :as a])
(a/run-tickets {:compact-threshold 6000 :max-turns 20}) ; dry-run backend
Run under the dry-run backend (queue-backend), the agent processes the real
ready queue while mutating nothing — the cursor advances in memory. That run is
written up empirically in Compaction Against a Beads Backend (same work, 37%
fewer tokens with client-side compaction). The REPL is where you watch it:
each turn's context size, each tool call, printed as it goes — the loop is not a
black box you submit, it's a form you step.
7. The workflow, concretely
A triage session is a sequence of evaluations, each informing the next — the literate-transcript form this site is built on:
(b/stats (b/read-issues))— how big is the problem.(->> … frequencies)— where is it concentrated (priority, type, blocked).(b/next-ticket)— pull one; read its shape; decide.(b/set-status! id "in_progress")— claim it; one line moves on disk.(a/run-tickets …)— or hand a batch to the agent and audit the transcript with chat.clj.
8. Survey
8.1. Scope
The object is a triage item: a body of text with metadata (title, creation time, reporter, channel) and an implicit contract on how soon something must happen to it (first response, acknowledgement, resolution). Support cases, GitHub issues, incidents, bug reports and email all share this shape. Triage is the decision taken under that contract: classify, prioritise, route, deduplicate, respond or escalate.
Each system is recorded against the same fields:
| field | question |
|---|---|
| item | what is triaged (case, issue, incident, alert, email) |
| metadata used | which fields the agent reads beyond the body |
| decisions | classify / priority / route / dedupe / draft / resolve / escalate |
| time contract | SLA/SLO model; is the clock an input to the agent? |
| agent shape | rules, classifier, LLM call, tool-using agent, multi-agent |
| human in the loop | where a person approves or overrides |
| metrics claimed | deflection, MTTA, MTTR, accuracy; source of the number |
| status | GA / preview / demo / paper, with date |
8.2. CRM and customer support
Customer support is the area with the most triage products and the oldest ones. Every help desk here ships a classifier that writes intent, priority, language or sentiment into ticket fields, and every one ships an SLA engine that runs a clock against those fields. The question for this area is the direction of the arrow: does triage choose the SLA, or does the SLA inform triage?
8.2.1. Salesforce Einstein Case Classification and Case Routing
| field | value |
|---|---|
| item | Service Cloud case |
| metadata used | subject and description of closed cases (training); predicts picklist, checkbox and lookup fields |
| decisions | classify (Case Reason, Priority, Escalated, Language, custom fields); route via assignment rules |
| time contract | entitlements and milestones exist on the case; not an input to the model |
| agent shape | per-org supervised classifier; no LLM |
| human in the loop | three tiers by confidence: recommend top three, preselect for agent to confirm, auto-fill |
| metrics claimed | none; minimum 400 closed cases per predicted field |
| status | GA Spring '19; still documented in 2026 |
Einstein Case Classification trains on closed cases and predicts field values for new ones (Salesforce n.d.-a). It went GA in the Spring '19 release (Salesforce 2019). The design choice worth recording is the confidence ladder: the admin sets thresholds per field, and the more automated the action, the higher the confidence needed. Below the auto-fill threshold Einstein falls back to preselection, then to a top-three recommendation.
Einstein Case Routing is not a separate model. After classification writes the fields, Einstein can rerun case assignment rules or skills-based routing rules on the updated case. Priority is one of the predictable fields, and priority selects the milestone targets downstream, so the classifier chooses the clock; the clock does not inform the classifier.
8.2.2. Salesforce Omni-Channel Acceptance Due Date
| field | value |
|---|---|
| item | Omni-Channel work item (case, chat, messaging session) |
| metadata used | primary and secondary routing priority, Acceptance Due Date, time entered queue |
| decisions | order the queue |
| time contract | Acceptance Due Date may be set from CaseMilestone.TargetDate; earliest date wins within a tier |
| agent shape | deterministic router configured in Omni-Channel Flow; no AI |
| human in the loop | admin writes the flow; agents accept pushed work |
| metrics claimed | none |
| status | Winter '26 (October 2025) |
Not an AI feature, but it is the one place in the Salesforce stack where a
milestone deadline is a routing input. The Route Work flow action takes an
Acceptance Due Date. Omni-Channel orders by primary priority, then secondary
priority, then Acceptance Due Date (earliest first), then time in queue
(Paterson 2025). The date can be a formula
(NOW() + 2/24 for a two-hour tier) or the TargetDate of the open case
milestone, which reuses entitlement processes as they stand. The SLA clock
breaks ties only among items with equal priority; it does not override
priority. The source is a Salesforce product manager's post on
UnofficialSF; the release notes were not checked.
8.2.3. Salesforce Agentforce Service Agent and entitlements
| field | value |
|---|---|
| item | customer conversation (chat, messaging, voice); case created or updated as an action |
| metadata used | CRM records reachable through topics and actions (contact, account, order, case) |
| decisions | resolve, create or update case, escalate to human with transcript |
| time contract | entitlement processes and milestones run on the case; agent use of milestone state unconfirmed |
| agent shape | LLM agent with topics, instructions and actions (Atlas reasoning engine) |
| human in the loop | escalation to a rep in the service console; admins scope topics and actions |
| metrics claimed | none for triage specifically in the GA release |
| status | GA 29 October 2024; successor to Einstein Bots, which has no published end-of-life date |
Agentforce Service Agent went GA on 29 October 2024 with prebuilt topics for case management, order inquiries and account management, and handoff to a human rep with the conversation attached (Salesforce 2024). It is the customer-facing successor to Einstein Bots. A third-party migration guide reports no published retirement date for Einstein Bots, and that the two products share no authoring objects, so dialogs and intent training sets do not carry over (Salesforce Dictionary 2026).
The Implement Agentforce Service trail places SLA management, entitlements and milestones in the same curriculum (Salesforce n.d.-b), but this pass found no documented out-of-box action through which the agent reads milestone time remaining or breach state. An admin can expose that through a custom flow or Apex action; whether Salesforce ships one is unconfirmed. Milestone violation and warning actions remain time-triggered workflow, outside the agent.
8.2.4. Salesforce Data 360 support triage pattern
| field | value |
|---|---|
| item | inbound email via Email-to-Case |
| metadata used | email body; vectorised past cases and knowledge articles |
| decisions | classify intent and reason, attach knowledge with confidence score, set owner/priority |
| time contract | none stated; severity patterns trigger platform events |
| agent shape | LLM classification into Data Model Objects plus vector search |
| human in the loop | not described; enrichment happens before the case reaches an agent |
| metrics claimed | none quantified |
| status | reference architecture blog, 30 April 2026 |
A Salesforce engineering blog describes LLM classification of inbound email into intent ("Feature Request" vs "Critical Outage") and reason, a vector search against knowledge, and a platform event fired on "High-Severity Technical Issue" patterns that sets ownership and priority (Sethuraman, Sarkar, and Chinnangolla 2026). It is a reference pattern with no product SKU. Severity is derived from text; the entitlement clock is not consulted.
8.2.5. Salesforce AI Research: CRMArena and CRMArena-Pro
| field | value |
|---|---|
| item | synthetic Salesforce org: 16 objects incl. case, account, order, knowledge article |
| metadata used | case text, product, agent case history, open-case counts |
| decisions | New Case Routing, Handle Time and Transfer Count Understanding, Policy Violation Identification, others |
| time contract | none; handle time and region closure speed are query targets, not constraints |
| agent shape | LLM agents, ReAct and function calling, against a live Salesforce org |
| human in the loop | none (benchmark) |
| metrics claimed | CRMArena: best (o1) 57.7% ReAct, 64.3% with hand-built tools; CRMArena-Pro: ~58% single-turn, ~35% multi-turn |
| status | papers: CRMArena NAACL 2025; CRMArena-Pro arXiv May 2025 |
CRMArena is the closest thing in the area to a public triage benchmark (Huang, Prabhakar, Dhawan, et al. 2025). Of its nine tasks, New Case Routing is triage: assign an incoming case to the best human agent. The ground truth is a three-tier policy: issue expertise (most closed cases on the most similar issue), then product expertise, then workload (fewest non-closed cases). Neither SLA nor case age enters the policy. Handle time and transfer count appear only as analytic questions about agents.
CRMArena-Pro extends this to 19 tasks over sales, service and CPQ, in B2B and B2C orgs, and adds multi-turn interaction; leading agents drop from about 58% single-turn success to about 35% multi-turn, with near-zero unprompted confidentiality awareness (Huang, Prabhakar, Thorat, et al. 2025). Whether any Pro task includes a time contract is unconfirmed from the abstract.
8.2.6. Zendesk Intelligent Triage
| field | value |
|---|---|
| item | Zendesk Support ticket (email, messaging, web form) |
| metadata used | first message text |
| decisions | classify: Topic (formerly Intent), Sentiment (5 levels), Language (~150), Entities; each with confidence |
| time contract | outputs usable as SLA policy conditions; the SLA is downstream |
| agent shape | Zendesk ML model; not documented as an LLM |
| human in the loop | admins build triggers and views over predictions; agents see fields |
| metrics claimed | none on the docs page |
| status | launched 13 September 2022; Intent renamed Topic for Copilot buyers from 11 June 2026 |
Intelligent triage writes four predictions into ticket fields, each with a confidence field, and those fields become available to "views, triggers, automations, macros, SLAs, queues, and the ticket API" (Zendesk n.d.-a). Classifications are on by default with the Copilot add-on and off otherwise. The launch release is dated 13 September 2022 (Zendesk 2022).
The SLA relationship runs one way. Zendesk SLA policies key every target (first reply, next reply, total resolution, periodic update and others) by ticket priority, with a business-hours flag per metric, and filter conditions select which policy applies (Zendesk n.d.-c). A trigger can set priority from a predicted Topic or Sentiment, which selects the target. The model does not read elapsed time.
8.2.7. Zendesk omnichannel routing by time to SLA breach
| field | value |
|---|---|
| item | ticket in an omnichannel routing queue |
| metadata used | next SLA breach time; priority and routing-eligibility timestamp as fallback |
| decisions | order the queue for assignment |
| time contract | the input: breached first, then nearest breach; SLA tickets ahead of non-SLA |
| agent shape | deterministic router; no AI |
| human in the loop | admin toggles the ordering mode |
| metrics claimed | none |
| status | GA; Team plans and above; release date unconfirmed (December 2024 per index) |
When ordering by SLA is on, "tickets are ordered by nearness to SLA breach, beginning with those that have already breached", and all tickets with an SLA sort ahead of those without, regardless of priority (Zendesk n.d.-b). This is stronger than the Salesforce equivalent: in Zendesk the breach time replaces priority as the sort key, where Salesforce's Acceptance Due Date only breaks ties within a priority. Neither is an AI decision. The clock is a router input in both.
8.2.8. Zendesk AI agents (Ultimate) and Forethought
| field | value |
|---|---|
| item | customer conversation; ticket on handoff |
| metadata used | conversation, knowledge sources, connected back-end systems |
| decisions | resolve, act via integrations, hand off |
| time contract | unconfirmed as an agent input |
| agent shape | LLM agent (AI agents - Advanced, from Ultimate); Forethought self-improving agents |
| human in the loop | handoff to human agents; workflow authoring by admins |
| metrics claimed | vendor-reported: Ultimate automates "up to 80 percent" of requests |
| status | Ultimate acquisition announced 13 March 2024; Forethought acquisition announced 12 March 2026 |
Zendesk bought Ultimate in March 2024 for its autonomous agents (Zendesk 2024) and announced the purchase of Forethought on 12 March 2026, described as its largest acquisition in two decades (The Next Web 2026). Forethought's Triage product is covered separately below. Zendesk bills AI agents per automated resolution; a secondary source reports a 2026 split between LLM-verified resolutions, which are billed, and contained ones, which are not. That split is unconfirmed against Zendesk docs.
8.2.9. Freshworks Freddy AI Auto Triage
| field | value |
|---|---|
| item | Freshdesk ticket |
| metadata used | subject, description, inferred intent and sentiment, historical ticket patterns |
| decisions | classify: Priority, Group, Type, custom dropdown and dependent fields |
| time contract | none; the predicted Priority selects the SLA policy target downstream |
| agent shape | Freddy model; LLM vs classic ML not stated |
| human in the loop | per-field Manual mode (agent clicks Apply) or Automatic mode; automation rules always override |
| metrics claimed | none; 2,000 historical tickets recommended |
| status | GA on Freshdesk Pro/Enterprise and Omni Pro/Enterprise; launch date unconfirmed |
Auto Triage predicts Priority and Group directly, the two fields that most help desks use to select an SLA target and a queue (Freshworks n.d.). The Manual/Automatic choice is made per field, and "Automation Rules always take precedence over AI suggestions", so a rule-set priority wins over a predicted one. The docs say nothing about SLA policies or due dates.
8.2.10. Intercom Fin and Fin Attributes
| field | value |
|---|---|
| item | Intercom conversation or ticket |
| metadata used | conversation text; admin-defined attribute descriptions (Issue type, Sentiment, Urgency, Spam) |
| decisions | classify into Fin Attributes; branch workflows; escalation rules route to teams; resolve |
| time contract | SLA applied by a workflow step; office hours honoured; bot-inbox time can be excluded from the SLA |
| agent shape | LLM agent (Fin) that detects attributes while it handles the conversation |
| human in the loop | escalation rules and guidance; handoff to teammates; inbox views |
| metrics claimed | vendor and secondary sources report resolution rates of roughly 67-76%; priced at $0.99 per resolution |
| status | Fin GA 2023; Fin Attributes documented 2025-2026; legacy AI Category Detection block retired for new flows |
Fin Attributes are admin-defined fields that Fin fills while it handles a conversation; they drive workflow branches and escalation rules (Intercom n.d.-d). Detection happens only after a "Let Fin handle" block runs, not before and not without Fin. Classification is a side effect of the resolve attempt.
Intercom's SLA handling is the most explicit about where the clock starts when an agent goes first. SLAs are applied by a workflow step and honour office hours: a 15-minute first-response target on a message received at 17:50 against 09:00-18:00 hours is due at 09:05 the next working day (Intercom n.d.-c). With the bot inbox enabled, an admin can "exclude the time conversations spend in the bot inbox from SLA targets", so the human SLA timer starts only when Fin hands off (Intercom n.d.-b). That makes the SLA a contract on the human alone. Intercom's own example of combining the two is an inbox view, which a person reads: "Fin Attribute: Issue = Technical + SLA due in < 30 min".
8.2.11. HubSpot Breeze (Help Desk AI and Customer Agent)
| field | value |
|---|---|
| item | Service Hub ticket; customer conversation |
| metadata used | conversation thread, ticket properties, knowledge base, CRM records |
| decisions | name, categorise, detect language, summarise (Help Desk AI); resolve or hand off (Customer Agent) |
| time contract | none stated in either product page |
| agent shape | LLM features plus an LLM agent (Customer Agent) |
| human in the loop | routing rules act on the categories; Customer Agent hands off with context |
| metrics claimed | vendor-reported: 70% average resolution ("top teams hit 90%"); resolution = no human handoff within 72 hours |
| status | GA; categorisation on Service Hub Professional, language detection on Enterprise; Customer Agent at $0.50 per resolution |
HubSpot splits triage into small generative features: descriptive ticket names, categories, language and summaries, which routing rules then consume (HubSpot n.d.-b). The Customer Agent resolves or hands off; HubSpot defines a resolution as no human handoff within 72 hours, the only time window in the HubSpot material, and it measures the agent, not the customer's wait (HubSpot n.d.-a). The vendor's own figures differ by page (65%, 70%, 50% more resolutions); none cites a method.
8.2.12. Kustomer (KustomerIQ and AI Agent Studio)
| field | value |
|---|---|
| item | Kustomer conversation (customer timeline) |
| metadata used | conversation and customer attributes; agent skills and team capacity |
| decisions | classify conversation; route; AI agents resolve, follow up, reroute or escalate stalled conversations |
| time contract | business-hours-aware routing (fall 2025); SLA attainment feeds workforce forecasting, not routing |
| agent shape | ML classifier (KIQ) then LLM AI agent teams (AI Agent Studio) |
| human in the loop | managers assign or reassign conversations to AI agent teams; routing audit logs |
| metrics claimed | vendor-reported "AI Resolution Rate" and "AI Support Rate" exist as metrics; no figures in the release |
| status | KIQ launch date unconfirmed; fall 2025 release published 1 October 2025 |
KustomerIQ introduced ML conversation classification feeding queues that also weigh agent skills and team capacity (Smart Customer Service n.d.). The fall 2025 release adds business-hours-aware routing, follow-up on abandoned conversations, manager reassignment to AI agent teams, and audit logs that record why a routing decision was made (Kustomer 2025). The routing audit log is uncommon in this area. SLA appears only as an input to workforce forecasting.
8.2.13. Gorgias intents, sentiment and AI Agent
| field | value |
|---|---|
| item | e-commerce support ticket (email, chat, social) |
| metadata used | each incoming message; Shopify order context for the AI Agent |
| decisions | classify (23 fixed intents, 3 sentiments); rules tag, auto-reply, set priority, route |
| time contract | none in the intents docs |
| agent shape | fixed-taxonomy classifier plus an LLM AI Agent that fills tags from tag descriptions |
| human in the loop | admin-written rules act on predictions; views by intent and sentiment |
| metrics claimed | none on the docs page |
| status | GA; launch date unconfirmed |
Gorgias uses a fixed e-commerce taxonomy (Order/Cancel, Refund/Request, Shipping/Status, Subscription/Cancel and others) rather than per-tenant training; each message gets one sentiment label (Gorgias n.d.). Decisions are made by rules over those labels. The fixed taxonomy trades fit for zero setup.
8.2.14. Front AI Tagging, Topics and Autopilot
| field | value |
|---|---|
| item | shared-inbox conversation (email, SMS, WhatsApp, chat, portal) |
| metadata used | subject and body; Topics; help content and conversation history |
| decisions | tag (legacy), classify into Topics, branch Playbooks on natural-language questions, reply, hand off |
| time contract | Front rules support SLA timers and unreplied-time conditions; not an Autopilot input |
| agent shape | AI Tagging (few-shot, 10 approved examples per tag) replaced by Topics and Autopilot (LLM agent) |
| human in the loop | admins pick which Topics Autopilot may handle; tag approval in legacy tagging |
| metrics claimed | none in the announcement |
| status | AI Tagging legacy, closed to new users (date unconfirmed); Autopilot announced 30 September 2025 |
Front's AI Tagging required an admin to approve ten example emails per tag before tagging began, up to 50 tags per inbox, and is now closed to new users in favour of Topics and Autopilot branching (Front n.d.). Autopilot, announced 30 September 2025, lets admins choose which Topics the agent may handle and which go to humans (Front 2025). Front's rule engine can start SLA timers and act on unreplied time, but nothing documented passes that timer to Autopilot.
8.2.15. Microsoft Dynamics 365 Customer Service
| field | value |
|---|---|
| item | case, email, conversation (any routing-enabled record) |
| metadata used | up to 10 text attributes (sentiment); trained text field (skill finder); Dataverse history |
| decisions | classify sentiment (7-point), predict required skills, estimate effort; route-to-queue rules act on outputs |
| time contract | Dynamics SLA KPIs exist; effort estimation's relation to SLA unconfirmed |
| agent shape | AI Builder classifiers (2021); Case Management Agent, LLM, GA 31 October 2025 |
| human in the loop | dry run before rules depend on a model; reps correct skills to retrain |
| metrics claimed | none |
| status | skill finder April 2021; sentiment routing still preview in 2026; Case Management Agent GA October 2025 |
Unified routing keeps the model and the decision apart: sentiment prediction and intelligent skill finder write values, and "queue and assignment rules still determine the destination" (Microsoft 2026d). Effort estimation, announced in 2021, generates a work estimate from historical data (Microsoft 2021); a secondary summary says an SLA can serve as the effort definition, which is unconfirmed in Microsoft's text. The Case Management Agent creates and updates cases from chats and email and drafts follow-ups (Microsoft 2025c).
8.2.16. Zoho Desk Zia
| field | value |
|---|---|
| item | Zoho Desk ticket |
| metadata used | ticket text; past tickets |
| decisions | auto-tag (keyword clusters), predict picklist fields incl. priority, sentiment |
| time contract | none in the overview; SLA-breach flagging claimed only in a community post |
| agent shape | clustering and classifiers; reply assistance from knowledge base |
| human in the loop | agents see predictions and drafts |
| metrics claimed | none |
| status | GA; dates unconfirmed |
Zia's overview lists auto-tagging by keyword clusters, field prediction for category and priority, sentiment and tone, and anomaly detection on ticket volume (Zoho n.d.). The anomaly detector watches volume, not per-ticket clocks.
8.2.17. Forethought Triage
| field | value |
|---|---|
| item | ticket in Zendesk, Salesforce, Freshdesk and others |
| metadata used | ticket text; historical tickets |
| decisions | classify intent, priority, sentiment, language, spam; tag for host routing rules |
| time contract | none stated |
| agent shape | prebuilt models (sentiment, language, spam) plus custom supervised models per customer |
| human in the loop | host help-desk rules act on tags |
| metrics claimed | vendor-reported: "up to 98% ticket prediction rate", 100+ languages, 15x ROI |
| status | GA; Forethought founded 2017; acquired by Zendesk, announced 12 March 2026 |
Forethought is the vendor that named a product "Triage". It tags and prioritises so that the host system's rules route the ticket (Forethought n.d.). "Prediction rate" measures coverage, the share of tickets that receive a prediction; the page gives no accuracy figure. The product's future inside Zendesk is unconfirmed.
8.2.18. SupportLogic Escalation Agent
| field | value |
|---|---|
| item | open B2B support case in a host CRM or help desk |
| metadata used | sentiment over interactions, account health and ARR, SLA age, interaction frequency, escalation history |
| decisions | score escalation risk, alert supervisors and CSMs, route by expertise and queue load |
| time contract | an input: "SLA age" feeds a risk score that updates as the case ages |
| agent shape | ML scoring (Escalation, Sentiment, Routing agents); RAG for knowledge; MCP server |
| human in the loop | thresholds alert people; people act |
| metrics claimed | vendor-reported: NICE case study, escalations reduced 35% |
| status | GA |
SupportLogic is the one system in this area whose documented model input includes the SLA clock. The Escalation Agent produces a continuous risk score per open case that "updates as new activity occurs and as the case ages", combining sentiment, account health, ARR and "SLA age" (SupportLogic n.d.). Its framing is a critique of SLA-ordered queues: a high-ARR account with compounding negative interactions can sit behind tickets nearer breach. Here the SLA is one feature among several and does not set queue order on its own.
8.2.19. Plain (Ari) and Pylon
| field | value |
|---|---|
| item | B2B support thread (Slack, Teams, email, chat, Discord) |
| metadata used | customer tier, label, channel, business hours; Pylon adds account health and product usage |
| decisions | answer, hand off, record knowledge gaps (Ari); investigate and act with approval (Pylon) |
| time contract | both platforms carry tiered SLAs and business hours; neither documents the agent reading them |
| agent shape | LLM agents grounded in designated knowledge |
| human in the loop | workflows decide which threads Ari gets; Pylon admins set approval requirements |
| metrics claimed | Ari counts a thread resolved if not reopened within 7 days; Pylon none |
| status | GA; dates unconfirmed |
Plain and Pylon are B2B help desks where tier, contract and SLA sit on the customer record. Plain's workflows assign threads to Ari by tier, label, channel or business hours (example: free-tier chat goes to Ari) (Plain n.d.). That uses the contract to decide whether an agent gets the item at all, which is a different use from ordering a queue. A search snippet claimed Ari raises priority to High for security requests and Urgent for incident reports; the current Ari doc does not say so, so it is unconfirmed. Pylon describes agents with "full account and product context" and approval gates, with no triage or SLA specifics (Pylon n.d.).
8.2.20. Sierra and tau-bench
| field | value |
|---|---|
| item | customer conversation (retail, airline in the benchmark) |
| metadata used | user profile and order/reservation database; written domain policy |
| decisions | act via tools under policy, resolve, transfer to human |
| time contract | none in the benchmark |
| agent shape | LLM tool-using agent; product is an "Agent OS" across chat, voice, email |
| human in the loop | transfer-to-human tool in the benchmark; outcome-based contracts in the product |
| metrics claimed | paper: GPT-4o under 50% of tasks, pass^8 under 25% on retail |
| status | paper June 2024; product GA; outcome-based pricing announced December 2024 |
Sierra's research contribution is tau-bench: a simulated user, typed tools, a written policy, and success judged by comparing the final database state to an annotated goal (Yao et al. 2024). Its pass^k metric (all k trials succeed) measures consistency, which matters more for triage than single-run accuracy because the same ticket should route the same way twice. The benchmark has no clock.
8.2.21. Decagon
| field | value |
|---|---|
| item | customer conversation (chat, voice, email) |
| metadata used | conversation history; back-end systems reached by AOP code |
| decisions | extract intent, select an AOP, act, escalate on guardrails |
| time contract | none stated |
| agent shape | LLM agent driven by Agent Operating Procedures (natural language plus code) |
| human in the loop | CX teams write AOPs; guardrails and escalation intents; supervisor model |
| metrics claimed | vendor case studies: Chime 70% resolution, Duolingo 80% deflection |
| status | GA |
Decagon's triage step is intent extraction followed by AOP selection, with RAG as the fallback when no AOP matches (Decagon n.d.). Escalation Drivers reports which guardrail or AOP produced each handoff, which treats escalation as the measured output of triage.
8.2.22. Ada
| field | value |
|---|---|
| item | customer conversation (chat, social, email, voice) |
| metadata used | conversation; knowledge and integrations |
| decisions | resolve or hand off |
| time contract | idle timeouts only: 24 hours chat and social, 72 hours email |
| agent shape | LLM agent (Reasoning Engine, November 2023; Unified Reasoning Engine, February 2026 per secondary) |
| human in the loop | handoff marks the conversation Not Resolved |
| metrics claimed | automated resolution = relevant, accurate, safe, contained; vendor-reported up to 83% |
| status | GA |
Ada's contribution is a published definition of its headline metric: a conversation counts as automatically resolved only if relevant, accurate, safe and contained, assessed by AI; before 31 July 2024 only a statistical sample was assessed (Ada n.d.). The only clocks are idle timeouts that end a conversation.
8.2.23. Wise.io (historical)
| field | value |
|---|---|
| item | support ticket in Zendesk or Salesforce Service Cloud |
| metadata used | ticket text; history |
| decisions | route (Wise Routing), auto-respond (Wise Response) |
| time contract | unconfirmed |
| agent shape | supervised ML |
| human in the loop | unconfirmed |
| metrics claimed | none found |
| status | acquired by GE November 2016; spun out as SmartAssist.io, August 2017 |
Wise.io sold ML ticket routing and auto-response into Zendesk and Service Cloud before either vendor shipped its own. GE acquired it in November 2016 and spun the support product out as SmartAssist.io, which raised a $5M Series A in August 2017 with Twilio, MailChimp and Zendesk as customers (TechCrunch 2017). Its later status is unconfirmed.
8.2.24. Area synthesis
Three generations coexist. Per-tenant supervised classifiers (Wise.io before 2016, Einstein Case Classification 2019, KustomerIQ, Zendesk intelligent triage 2022, Dynamics skill finder 2021, Freddy, Zia, Forethought Triage) write predictions into ticket fields. Fixed-taxonomy and few-shot classifiers (Gorgias intents, Front AI Tagging) trade fit for setup cost. LLM agents (Agentforce, Fin, Breeze Customer Agent, Zendesk AI agents, Autopilot, Sierra, Decagon, Ada, Ari) resolve first and classify as a side effect; Fin Attributes are detected only while Fin handles the conversation.
The SLA enters triage in four ways, and only one of them is an agent input.
- Triage selects the clock. The commonest pattern. A classifier writes priority (Einstein, Freddy, Zia, Forethought) or a field a trigger maps to priority (Zendesk Topic and Sentiment), and the SLA policy keyed on priority selects the target. The model never reads elapsed time.
- The clock orders the queue. Deterministic routers consume breach time: Zendesk omnichannel routing can sort by nearness to breach ahead of priority; Salesforce Omni-Channel's Acceptance Due Date (Winter '26) breaks ties within a priority, optionally from the case milestone's target date. No AI in either.
- The clock is paused for the agent. Intercom can exclude bot-inbox time from SLA targets, so the contract binds the human after handoff, not the agent. Business hours are honoured by Intercom, Zendesk and Kustomer routing.
- The clock is a model feature. Only SupportLogic documents "SLA age" as an input to a learned score (escalation risk), combined with sentiment and ARR, and it argues against SLA-only ordering.
No LLM agent in this area documents reading the SLA clock (time to breach, business hours remaining) when it decides to resolve, escalate or change priority. Plain uses the contract (tier, business hours) to decide whether the agent gets a thread at all, which is the nearest case.
The public benchmarks have no clock either. CRMArena's New Case Routing ground truth is issue expertise, product expertise, then open-case workload; tau-bench judges final database state under a written policy. Both report low consistency (CRMArena under 55% with function calling; tau-bench pass^8 under 25% on retail), which bears on any triage agent whose routing must be repeatable.
Metrics are vendor-reported and defined differently: Ada and HubSpot publish definitions of "resolution" (contained, or no handoff within 72 hours); Zendesk reportedly bills only LLM-verified resolutions; Forethought's "98% prediction rate" measures coverage and says nothing about accuracy. No independent evaluation of any production triage classifier in this area was found.
Open questions:
- Does any Agentforce, Fin or Zendesk AI agent action expose milestone time remaining to the LLM, out of the box? Not found; worth a direct test in a sandbox org.
- Would an agent that reads time to breach make different escalation decisions than a router that sorts on it? No study in this area tests this.
- How do SLA-paused-for-bot schemes (Intercom) affect reported first response time? The customer's wait includes the bot time; the metric does not.
- Dates still unconfirmed: Zendesk Answer Bot (2017) and Intercom Resolution Bot retirements, DigitalGenius, MonkeyLearn (acquisition by Medallia unconfirmed), Freddy Auto Triage launch, Gorgias intents launch.
8.3. ITSM and incident management
IT service management (ITSM) is the area where the time contract is most formal. An incident record in ServiceNow, Jira Service Management, BMC Helix or Ivanti carries task SLAs with schedules, pause conditions and breach timestamps. Incident tools (PagerDuty, incident.io, Rootly, FireHydrant) work with a looser contract: urgency, escalation timeouts, and the MTTA/MTTR that the vendors report. The question for this area is whether any agent reads that clock when it decides.
8.3.1. ServiceNow Predictive Intelligence (incident)
| field | value |
|---|---|
| item | ITSM incident record |
| metadata used | short description (classification); short description + description (similarity) |
| decisions | classify (category, service, CI), route (assignment group), dedupe/link (similar and major incidents) |
| time contract | Task SLA engine exists on the same record; not an input to any out-of-box solution |
| agent shape | supervised classifiers and similarity models trained per instance; no LLM |
| human in the loop | predictions written to fields or shown as recommendations; business rules decide which |
| metrics claimed | none on the docs page; per-instance precision/coverage set at training time |
| status | GA; predecessor named Agent Intelligence; first-release date unconfirmed (Kingston 2017 per secondary) |
Predictive Intelligence is the classic-ML layer under ServiceNow's later generative products. The out-of-box incident solutions are four classifiers (assignment group, category, service, configuration item), each fed by the short description alone, and a family of similarity solutions over short description and description, including major-incident detection (ServiceNow n.d.-b). The page lists no regression or clustering solution for incident, and none of the solutions reads priority, SLA or elapsed time.
"SLA breach prediction" is not an out-of-box Predictive Intelligence solution on that page. Partners describe training a custom model for it, and the platform supports regression solutions that could predict time to resolution, but a shipped ServiceNow breach-prediction model is unconfirmed. The SLA reaches the AI stack later, and only as something to explain (see 8.3.2).
8.3.2. ServiceNow Now Assist and AI Agents (ITSM)
| field | value |
|---|---|
| item | incident, problem, change, case |
| metadata used | work notes, Virtual Agent chat, caller's assigned assets (categorization agent), Task SLA records (Explain SLA) |
| decisions | summarize, draft resolution notes, classify category/subcategory/CI, explain SLA state |
| time contract | Explain SLA reads the most critical Task SLA: target, elapsed time, pause/resume events, breach likelihood; manual only |
| agent shape | LLM skills (Now Assist); tool-using agents coordinated by AI Agent Orchestrator |
| human in the loop | agent invokes skills; Explain SLA has no trigger and runs on request from the Now Assist panel |
| metrics claimed | none found on the docs pages fetched |
| status | Now Assist for ITSM GA (Vancouver/Washington era, date unconfirmed); AI Agents announced 2025-01-29; Explain SLA in Zurich |
Now Assist for ITSM summarizes incidents, chats and sidebar discussions and drafts resolution notes (ServiceNow n.d.-a). In January 2025 ServiceNow announced AI Agents and an AI Agent Orchestrator; the ITSM set includes an incident categorization agent that infers category, subcategory and the likely CI from the assets assigned to the caller (ServiceNow 2025a). Orchestration is described as specialist agents invoked by a planner. The servicenow.com announcement pages returned 403 to the fetcher, so the agent list is from syndicated copies.
The Explain SLA agentic workflow (Zurich release, docs updated 2026-03-12) is the clearest case in this area of an LLM reading the SLA clock (ServiceNow 2026). It picks the most critical SLA on the task and reports target, elapsed time, breach likelihood, pause/resume events, reassignment patterns and breach causes. It explains; it does not act. It has no trigger, so the SLA is an input to an answer a person asks for, not to routing or priority.
8.3.3. Moveworks Triage (now ServiceNow)
| field | value |
|---|---|
| item | ticket in ServiceNow, Jira, Freshservice (IT and HR) |
| metadata used | short description, description; optional extra fields such as location; source channel filter |
| decisions | classify and route: assignment group, category, subcategory, cmdb_ci, Jira component, HR service |
| time contract | none read; acts 30-60 s after creation (or at creation) |
| agent shape | embedding kNN: top 10 similar historical tickets, majority field value, confidence threshold |
| human in the loop | customer sets confidence threshold and ticket filters; below threshold the ticket is left alone |
| metrics claimed | Equinix: auto-routing on about 65% of tickets, about 96% correct, 32% shorter ticket lifespan (vendor-reported) |
| status | GA; blog 2020-01-10; Moveworks acquired by ServiceNow, closed 2025-12-15 ($2.85B) |
Moveworks Triage intercepts new tickets that pass customer filters, embeds them, retrieves the ten nearest historical tickets by cosine similarity and writes the most common field value if its confidence clears a customer-set threshold (Moveworks n.d.). Phone and walk-up tickets are excluded. The 2020 design post frames the motivation in time terms: an average of five hours before a human first looks at a ticket, against under a minute for the model (Mairena and Kala 2020).
The Equinix numbers come from a devops.com piece and are vendor-reported; the date is unconfirmed. ServiceNow completed the acquisition on 2025-12-15 (ServiceNow 2025b), so Moveworks Triage and Predictive Intelligence assignment now overlap inside one vendor.
8.3.4. Jira Service Management: AI triage and virtual service agent
| field | value |
|---|---|
| item | JSM request (often "Emailed request" with no type) |
| metadata used | summary, description, current request type |
| decisions | classify request type, fill fields for the new type (bulk); virtual agent answers or deflects |
| time contract | JSM SLAs are native to the project; not documented as an input to triage or the virtual agent (unconfirmed) |
| agent shape | LLM suggestions (Atlassian Intelligence, now Rovo); virtual agent = intent flows + LLM answers from KB |
| human in the loop | agent selects tickets and accepts the bulk change; needs "Make bulk changes" permission |
| metrics claimed | none found on the support pages |
| status | virtual agent GA 2023-11 (Premium/Enterprise); request-type suggestions announced 2024 (date unconfirmed) |
AI triage in JSM is a bulk action. An agent selects requests with no type, the model proposes a request type per issue, and on acceptance it rewrites the type and fills that type's fields from the body (Atlassian n.d.-c). The virtual service agent, GA in November 2023 on Slack, Teams, email and the portal, answers from linked knowledge and hands off what it cannot resolve (Atlassian 2023).
The request type matters for the clock: JSM SLA goals are usually keyed on request type and priority, so a reclassification can change which SLA applies. Atlassian does not document that interaction, and nothing found shows the triage model reading the SLA.
8.3.5. Jira Triage Agent and Rovo agents
| field | value |
|---|---|
| item | Jira work item (bug, request); JSM request for the service-request helper |
| metadata used | work item fields and similar items via the Teamwork Graph (component, team, links, assignee) |
| decisions | suggest field updates, detect duplicates, link related items; helper drafts replies, suggests next steps |
| time contract | not documented |
| agent shape | Rovo agent (LLM with tool access over the Teamwork Graph); can run from board-column automation |
| human in the loop | each suggestion accepted, rejected, or refined in chat; "Accept all" available |
| metrics claimed | none found |
| status | Jira Triage Agent documented 2026; GA status not stated on the page |
The Jira Triage Agent reviews a new work item, compares it with similar items, and proposes field updates and duplicate or related links, each with a stated reason (Atlassian n.d.-d). It can be bound to a board column so that moving an item into "Triage" runs it. Every suggestion needs explicit acceptance. Its quality depends on graph connectivity: practitioner posts note that an empty Component field leaves the agent unable to route.
Rovo's service-management agents (a service-request helper that summarizes, drafts replies, recommends experts and surfaces past tickets) sit on the same graph (Atlassian n.d.-a). No Rovo agent found reads SLA time remaining.
8.3.6. Opsgenie and JSM alert grouping (deprecated path)
| field | value |
|---|---|
| item | alert |
| metadata used | alert title, description, tags |
| decisions | dedupe/group semantically similar alerts; create incident from group; draft PIR summary |
| time contract | escalation and acknowledgement timeouts in the alerting rules; not an AI input |
| agent shape | semantic-similarity grouping (Atlassian Intelligence); LLM summary for PIR |
| human in the loop | responder reviews group; PIR draft edited by a person |
| metrics claimed | none found from Atlassian with a source |
| status | AI alert grouping and AI PIR GA mid-2024 in JSM; Opsgenie end of sale 2025-06-04, end of support 2027-04-05 |
Opsgenie itself never shipped an LLM triage agent. Atlassian folded its alerting into JSM (and Compass), where alert grouping by semantic similarity of title, description and tags, and an LLM-drafted post-incident review, went GA around June 2024 (Atlassian 2024). Opsgenie stopped selling on 2025-06-04 and shuts off on 2027-04-05, with data not yet migrated deleted (Atlassian n.d.-b). It belongs in the survey as the deprecated end of the line.
8.3.7. PagerDuty AIOps (Event Intelligence)
| field | value |
|---|---|
| item | alert / event before it becomes an incident |
| metadata used | alert content, service, timing, historical auto-resolve behavior |
| decisions | dedupe/group (intelligent, content-based, time-based); suppress; delay notification (Auto-Pause) |
| time contract | yes: Auto-Pause holds notifications for alerts that historically auto-resolve within a window; creates the incident if not |
| agent shape | ML grouping models plus rules; no LLM |
| human in the loop | per-service configuration; responders can trigger an incident from a paused alert |
| metrics claimed | "up to 98%" and "87%" alert or noise reduction (vendor-reported, no method stated) |
| status | GA; Event Intelligence launch date unconfirmed (before 2020) |
PagerDuty's noise reduction predates its LLM features. Alert grouping uses ML, content rules or time windows (PagerDuty n.d.-b). Auto-Pause is the one feature in this area where elapsed time is a direct decision input: it learns which alerts usually resolve on their own within a short period, pauses notifications for that period, and only pages if the alert is still open when it ends (PagerDuty n.d.-a). It trades a bounded delay in MTTA against pages that would not have mattered.
The 98% and 87% reduction figures appear on marketing pages without a stated baseline and are vendor-reported.
8.3.8. PagerDuty Advance and the AI agent suite (SRE Agent)
| field | value |
|---|---|
| item | incident |
| metadata used | incident urgency and service, past incidents and notes, logs (Grafana, Datadog, New Relic, CloudWatch), runbooks |
| decisions | summarize, draft status updates, surface likely root cause, recommend diagnostics and remediation, write playbooks |
| time contract | urgency scopes activation (docs advise high-urgency only); no SLA/SLO or ack-timeout input documented |
| agent shape | tool-using LLM agent ("Paige" in the docs) with connectors; separate Scribe, Shift and Insights agents |
| human in the loop | recommendations as buttons; responder approves each action; virtual-responder mode in Early Access |
| metrics claimed | Advance status updates save "up to 15 min per responder per incident"; suite resolves "up to 50% faster" (vendor-reported) |
| status | Advance announced 2024-07-30; agentic AI announced 2025-02-25; agent suite launched 2025-10-08; virtual responder EA as of 2026-10 |
PagerDuty Advance (July 2024) added generative summaries and audience-specific status-update drafts; the 15-minute saving comes from early-access customer interviews (PagerDuty 2024). The October 2025 suite added an SRE Agent, Scribe (call transcription), Shift (schedule conflicts) and Insights (PagerDuty 2025). The SRE Agent reads logs and runbooks through connectors, recalls similar incidents, prioritises its recommendations "by urgency and impact" and leaves execution to the responder (PagerDuty n.d.-c).
It can run as a virtual responder triggered from an incident workflow, and PagerDuty recommends scoping that trigger to high-urgency incidents. That is the time contract entering as a gate on whether the agent runs at all, not as a variable it reasons over. Escalation timeouts and acknowledgement are not mentioned in the agent docs.
8.3.9. incident.io AI SRE
| field | value |
|---|---|
| item | alert and incident (Slack-native) |
| metadata used | Slack channels, past incidents, pull requests, dashboards and logs (Grafana, Datadog), service topology |
| decisions | triage and investigate alerts, root-cause hypothesis, open fix PR, draft post-mortem |
| time contract | not an input; time appears as the outcome (MTTR) |
| agent shape | tool-using LLM agent with RAG plus a service relationship graph |
| human in the loop | engineers "make the final call"; fix PRs go through review |
| metrics claimed | "up to 80%" lower MTTR/downtime (vendor-reported) |
| status | announced 2025-07-02, live with "a handful" of customers at launch |
incident.io's launch post walks through a 2 a.m. payment outage: the agent names a memory leak from a specific PR at 2:01, opens a fix PR at 2:05, and service is restored at 2:15 (incident.io 2025). The example is narrated by the vendor. Its design point is citation: each hypothesis links the PRs, past incidents and dashboards it rests on. incident.io has also published a critique of a competitor's accuracy claims, which is vendor-against-vendor and not independent evidence.
8.3.10. Rootly AI SRE
| field | value |
|---|---|
| item | alert and incident |
| metadata used | alerts, telemetry, recent deploys and commits, config changes, past incidents, service catalog (owners, dependencies) |
| decisions | triage, ranked root-cause hypotheses with confidence scores, suggested fixes |
| time contract | not documented as an input |
| agent shape | tool-using LLM agents; Slack agent with permissioned actions |
| human in the loop | "every change requires explicit human sign-off before execution" |
| metrics claimed | "10x faster" resolution (vendor-reported) |
| status | GA; launch date unconfirmed (YC launch post exists; changelog active to 2026-10) |
Rootly investigates when an alert fires and returns ranked causes with confidence scores (Rootly n.d.). The service catalog gives it owners and dependencies, which is the routing metadata other tools lack. Severity or SLO does not appear as a decision input on the product page.
8.3.11. FireHydrant AI (now Freshworks)
| field | value |
|---|---|
| item | incident |
| metadata used | incident timeline, Slack and Zoom transcripts, responders, customer impact |
| decisions | summarize (whispered to new responders), draft status-page updates per audience, draft retrospectives |
| time contract | none |
| agent shape | LLM calls (OpenAI; Recall.ai for call capture) |
| human in the loop | responders edit drafts; feature can be switched off |
| metrics claimed | none on AI; 583,954 incidents run over five years (vendor-reported, not an AI metric) |
| status | released 2024-03-18; Freshworks acquisition closed 2026-01 ($88.7M) |
FireHydrant's March 2024 release covers summarization only: a summary pushed to each new responder, call transcription, status-page drafts and retrospective drafts (FireHydrant 2024). It is in the survey as the early, narrow form of "AI in incident response". Freshworks closed its acquisition in January 2026 to pair it with Freshservice (Constellation Research 2026).
8.3.12. Microsoft Copilot for Service and Dynamics 365 Case Management Agent
| field | value |
|---|---|
| item | customer service case (chat, email, voice) |
| metadata used | conversation, email body, case fields, customer history, intent |
| decisions | create and update case fields, draft and send replies, follow up, resolve or alert a representative; routing classifies and assigns |
| time contract | D365 SLA KPIs with timers and warnings before breach exist; Case Management Agent reading them is unconfirmed |
| agent shape | LLM agent over Dataverse; unified routing = rules plus ML classification |
| human in the loop | representative accepts chat; agent alerts a person when the customer reply needs it |
| metrics claimed | none found from Microsoft with a source |
| status | Copilot for Service GA 2024-02-01; Case Management Agent first release 2025-04, GA 2025-10-31 |
Copilot for Service (GA 2024-02-01) is an assist layer that also works over Salesforce, ServiceNow and Zendesk (Redmond Magazine 2024). The Case Management Agent goes further: it builds a case from a chat or an email, keeps fields current as mail arrives, drafts and sends replies from the detected intent, and closes the case or escalates to a person (Microsoft 2025b). D365 has SLA timers and breach warnings on the case entity, so the clock is one join away; Microsoft's material does not say the agent reads it. This overlaps area 01 (CRM).
8.3.13. Microsoft Research: incident triage and LLM root cause
| field | value |
|---|---|
| item | cloud service incident (Azure, Microsoft online services) |
| metadata used | title, body, discussion items, alert type, diagnostic data, owning-team history |
| decisions | route (responsible team), root-cause category, mitigation steps |
| time contract | used as motivation and outcome metric (time to mitigate, Time to Engage), never as a model input |
| agent shape | 2019-20: GBDT/clustering/DNN ensembles, GRU; 2023: fine-tuned GPT-3.x; 2024: LLM with handlers; 2025: multi-agent |
| human in the loop | recommendations to on-call engineers; human evaluation by incident owners (2023) |
| metrics claimed | DeepTriage F1 82.9%; DeepCT accuracy 0.641-0.729; RCACopilot accuracy up to 0.766; Triangle accuracy up to 97%, TTE -91% (author-reported on internal data) |
| status | papers 2019-2025; DeepTriage deployed in Azure since 2017-10; Triangle "Local Triage" in production since mid-2024 |
This is the longest-running line of work on the exact object of this survey. "An Empirical Investigation of Incident Triage" (ICSE SEIP 2019) measured reassignment on real systems (Chen, He, Lin, Xu, et al. 2019). DeepCT (ASE 2019) treats triage as continuous, re-predicting the owning team as discussion accumulates; across 8 systems 5.43% to 68.26% of incidents were reassigned (Chen, He, Lin, Zhang, et al. 2019). DeepTriage (KDD 2020) is an ensemble in Azure since October 2017, with F1 82.9%, motivated by the observation that a wrong assignment raises time to mitigate by 10x (Pham et al. 2020).
The LLM turn starts with Ahmed et al. (ICSE 2023), who evaluated GPT-3.x models on over 40,000 incidents for root cause and mitigation, with human evaluation by incident owners (Ahmed et al. 2023). RCACopilot (EuroSys 2024) routes each incident to a handler by alert type, collects diagnostics, and has an LLM predict the root-cause category with an explanation (Chen et al. 2024). Triangle (ASE 2025) uses multiple agents that negotiate ownership across teams, reporting accuracy up to 97% and Time to Engage cut by up to 91% (Yu, Fang, et al. 2025).
Across all six papers, time is the dependent variable. None feeds the remaining SLA or severity clock into the model as a feature; severity appears only as a slice for evaluation (DeepTriage reports F1 76.3% to 91.3% on high-impact incidents).
The newest Microsoft claim moves the model out of routing. Microsoft-Decision-1 was announced on 2026-10-09 as "better and faster than an LLM" for retrieving knowledge across logs, tickets, calls and messages during live incidents, with no number attached (8.8.3). That is a decision model used as a relevance scorer for the on-call engineer, not as the triage step these papers measure.
8.3.14. BMC Helix ITSM (HelixGPT agents)
| field | value |
|---|---|
| item | incident, work order, change, problem |
| metadata used | description, contextual signals, caller (VIP flag), knowledge articles, AIOps situations |
| decisions | categorize (cognitive service), predict urgency and impact, VIP prioritization, flag missing info, recommend and (26.3+) execute actions |
| time contract | priority derives from urgency x impact, which drives SLA selection; SLA itself not documented as an input |
| agent shape | trained categorization service; HelixGPT LLM agents (Ticket Resolver, Ops Swarmer, Service Collaborator) |
| human in the loop | autonomous execution in 26.3 gated by "confidence, risk, and context"; otherwise recommendations |
| metrics claimed | none found |
| status | cognitive categorization older (date unconfirmed); Ticket Resolver triage from 26.2; autonomous actions 26.3 |
BMC's Ticket Resolver predicts urgency and impact from incident data (BMC Helix n.d.). In ITIL-shaped tools priority is computed from urgency and impact, and the SLA is chosen by priority, so this agent sets the contract rather than reading it. That is a distinct position: the agent decides which clock starts. The docs do not say whether the predicted values write through or wait for approval below version 26.3.
8.3.15. Ivanti Neurons for ITSM
| field | value |
|---|---|
| item | incident, service request |
| metadata used | subject line (ticket classification); incident history (summaries) |
| decisions | classify service category at creation, summarize, generate KB articles; agentic routing |
| time contract | solution brief claims "SLA-aware" processing; mechanism unconfirmed |
| agent shape | ML classifier; generative summaries; agentic AI in customer preview |
| human in the loop | analyst reviews summaries; agentic tier described as analyst assistance |
| metrics claimed | 30-40% ticket volume reduction, up to 30% lower handle time (vendor-reported); agent preview resolving about 60% of low-tier tickets (vendor-reported, secondary source) |
| status | GenAI from Neurons for ITSM 2024.2 (Premium); summaries across objects Q1 2026; agentic preview early 2026 |
Ivanti Neurons classifies incidents at creation from the subject line and adds generative summaries and knowledge-article drafts (Ivanti 2025). Its agentic solution brief is the only vendor document in this area that names SLA awareness as an agent property (Ivanti n.d.), but it gives no mechanism, so the claim is unconfirmed.
8.3.16. Area synthesis
ITSM records carry the richest time contract of any area in this survey, and the agents mostly ignore it. Three positions appear.
- Clock as gate. PagerDuty's SRE Agent runs only on incidents that pass an urgency filter; Auto-Pause holds a page for a learned window. Time decides whether the AI acts; the conclusion ignores it. Auto-Pause is the only feature found where elapsed time is a model-adjacent input, and it predates the LLM features.
- Clock as subject. ServiceNow's Explain SLA reads target, elapsed time, pauses and breach likelihood, then explains them to a person on demand. It does not change priority or reroute.
- Clock as output. BMC's Ticket Resolver predicts urgency and impact, which set priority, which selects the SLA. JSM's request-type triage can change the applicable SLA as a side effect. Here the agent starts the clock, often without saying so.
Everywhere else (Predictive Intelligence, Moveworks, Jira Triage Agent, incident.io, Rootly, FireHydrant, the Microsoft papers) time is the metric reported afterwards: MTTR, TTE, time to mitigate, ticket lifespan. The research line is explicit that misrouting costs time (10x time to mitigate in DeepTriage), yet no model found takes "minutes until breach" as a feature.
The decision mix also splits by item type. ITSM tickets get classification and routing from cheap similarity models (short description in, assignment group out); LLMs arrive as summarizers and, from 2025, as agents that categorise by reading the caller's assets. Incident tools skip classification, since the alert source already names the service, and spend the LLM on investigation and root cause. Human approval is the default everywhere; BMC 26.3 is the only documented confidence-and-risk-gated autonomous execution.
Open questions:
- Does any shipped ServiceNow model predict SLA breach, or is it partner build-on-PI only? (unconfirmed)
- When a JSM or BMC agent changes request type or priority, is the SLA recalculated from creation time or from the change? Neither vendor documents it.
- Would a breach-aware router (route to the team with headroom, not the best-matching team) beat accuracy-optimised routing on SLA attainment? None of the surveyed papers tests it.
- Vendor MTTR claims (50%, 80%, 10x) have no stated baseline. Independent evaluation exists only for the Microsoft papers, on internal data.
8.4. Cloud platforms
The three hyperscalers triage two different items. The first is the customer's ticket: a contact-center case (Amazon Connect, Google Customer Engagement Suite) or a support case filed with the cloud provider itself. The second is the operational incident: an alarm or alert that an investigation agent (CloudWatch investigations, AWS DevOps Agent, Gemini Cloud Assist, Azure SRE Agent) picks up. Around both sit sample repositories and blog posts that show "ticket classification with an LLM" as a reference pattern. The question for this area is where the response-time contract lives in each, and whether the model ever sees it.
8.4.1. Amazon Connect Cases: SLAs, case rules and AI case summaries
| field | value |
|---|---|
| item | contact-center case spanning calls, chats, tasks and follow-ups |
| metadata used | case template, case field values, status; SLA related items; contact history for summaries |
| decisions | attach SLA, escalate or route on breach (rules); summarize case (LLM) |
| time contract | first-class SLA object: target time up to 90 days, target status, "time to breach", Due soon (<24 h), Overdue |
| agent shape | deterministic rules engine for SLA and routing; LLM summary with admin-configured prompts and guardrails |
| human in the loop | agent requests summary with one click; managers sort case list by "Next SLA Breach" |
| metrics claimed | none quantified; summaries "help resolve cases faster" (vendor-reported, unquantified) |
| status | Cases SLAs GA 2025-04-17; AI case summaries GA 2025-11; product renamed "Connect Customer Cases" in docs by 2026 |
Amazon Connect Cases has the most explicit time contract in this area. An SLA is a related item on the case with a name, a target date, a target field value and a computed status; "time to breach" keeps counting negative once overdue (Amazon Web Services n.d.-f). SLAs attach automatically through conversational-analytics rules keyed on case template and field values, and a "Case SLA Breach" rule trigger can escalate or reroute (Amazon Web Services 2025d). The announcement's example is a high-priority case reviewed within 4 hours and closed within 24 hours.
The generative layer sits beside this clock. AI case summaries (November 2025) condense a multi-contact case into background, actions and next steps for the human agent (Amazon Web Services 2025a). No documentation found says the summary, or any Connect AI agent, reads the SLA status. The breach decision is a rule; the model writes prose. Business-hours calendars for case SLAs are not mentioned on the SLA page: unconfirmed.
8.4.2. Amazon Connect Contact Lens: generative contact categorization
| field | value |
|---|---|
| item | voice or chat contact (transcript) |
| metadata used | transcript; rule conditions; contact attributes used by the rules engine |
| decisions | classify (natural-language category match), alert, create follow-up task |
| time contract | none on the category itself; generated tasks inherit Connect task and queue handling |
| agent shape | LLM semantic match of a natural-language criterion against the transcript, inside a rules engine |
| human in the loop | supervisors write the criteria; alerts and tasks go to people |
| metrics claimed | none quantified |
| status | GA 2024-12; regional expansions 2025-02 and 2025-03; included in conversational analytics pricing |
Contact Lens lets a supervisor write a category as a question ("did the customer try to make a payment on their balance?") and labels matching contacts with the supporting transcript points (Amazon Web Services 2024a). A match can raise an alert or create a task. That is post-contact triage: the item is already handled, and the category decides who looks at it next. Earlier Contact Lens categories were keyword and phrase rules; the generative match replaces the keyword list with a sentence.
8.4.3. Amazon Q in Connect (Connect AI agents)
| field | value |
|---|---|
| item | live customer contact (voice, chat) |
| metadata used | conversation, customer profile data, knowledge base, company policy |
| decisions | draft (agent assist answers), resolve (self-service), act via configured actions, hand off to human |
| time contract | contact-center queue metrics (handle time, wait); no SLA input documented |
| agent shape | LLM assistant on Bedrock; tool-using self-service agent from 2024-12 |
| human in the loop | agent assist suggestions to a human; self-service hands off to a queue |
| metrics claimed | none quantified in the sources read |
| status | announced 2023-11 (from Amazon Connect Wisdom); self-service 2024-12; later branded "Connect AI agents" |
Q in Connect is the conversation-time assistant: it reads the live contact and suggests answers, and from December 2024 it also runs self-service flows that take actions before a human is involved (Amazon Web Services 2024b; Amazon 2023). It triages by resolving or deflecting, not by classifying a queue. A November 2025 extension runs the same assistance inside Salesforce Contact Center with Amazon Connect (Amazon Web Services 2025b). Routing decisions in Connect remain the routing engine's (queues, routing criteria); whether the AI agent writes routing attributes beyond handoff is unconfirmed.
8.4.4. AWS Support: AI-enhanced troubleshooting and the support-plan clock
| field | value |
|---|---|
| item | support interaction that may become an AWS Support case |
| metadata used | natural-language description, imported Amazon Q conversations, account environment context |
| decisions | resolve without a case, ask follow-up questions, prefill case fields, invoke DevOps Agent for operational issues |
| time contract | plan-level first-response targets: Business Support+ <30 min, Enterprise <15 min, Unified Operations <5 min (critical) |
| agent shape | generative assistant in the Support Center Console; delegates investigations to AWS DevOps Agent |
| human in the loop | customer chooses to open a case; consent prompt before DevOps Agent spends paid credits |
| metrics claimed | none quantified |
| status | GA in us-east-1, us-east-2, eu-west-1; Developer/Business/Enterprise On-Ramp plans end 2027-01-01 |
AWS put the AI in front of its own case queue. A customer describes the issue in prose, the assistant asks follow-up questions and proposes fixes, and only then offers a case with fields already filled (Amazon Web Services n.d.-a). For operational issues it can hand the problem to AWS DevOps Agent, which streams its investigation into the support interaction.
The response-time contract here is commercial. The support plans page lists first-response targets per plan for critical cases: under 30 minutes for Business Support+, under 15 for Enterprise Support, under 5 for Unified Operations (Amazon Web Services n.d.-c). The plan restructure retires Developer, Business and Enterprise On-Ramp on 2027-01-01 (Amazon Web Services n.d.-d). Severity is still what the customer picks when the case opens; no source says the AI front door sets it. Whether AWS uses models internally to route or prioritize cases among its engineers is not publicly documented: unconfirmed.
8.4.5. AWS DevOps Agent
| field | value |
|---|---|
| item | operational incident (CloudWatch alarm, ServiceNow ticket, PagerDuty webhook, manual) |
| metadata used | metrics, logs, traces, deployments and code changes, learned application topology, past incidents |
| decisions | investigate, identify root cause, propose mitigation, create AWS Support case with findings |
| time contract | MTTR as the outcome; incident priority from the source tool; clock not documented as an input |
| agent shape | long-running tool-using agent ("frontier agent"); custom skills from GA |
| human in the loop | engineers steer investigations in chat; mitigations proposed; one-click support case |
| metrics claimed | "MTTR from hours to minutes" (vendor-reported); "3-5x faster incident resolution" in preview (vendor-reported, via secondary) |
| status | preview 2025-12-02; GA 2026-03-31; named migration target for DevOps Guru |
DevOps Agent is the incident-side counterpart to the support front door. The preview post lists three triggers: ServiceNow tickets through a built-in integration, CloudWatch alarm state changes, and PagerDuty through configurable webhooks (Stormacq 2025). GA added Azure and on-premises targets and custom skills (Amazon Web Services 2026c). Its pricing ties it to the support contract: Support customers get monthly DevOps Agent credits equal to 30% (Business Support+), 75% (Enterprise) or 100% (Unified Operations) of the prior month's support spend.
The agent triages by investigating. It does not reorder a queue; it shortens the time from page to root cause. The source ticket's priority arrives with the trigger, but no documentation says the agent changes depth or behaviour by priority or by time remaining.
8.4.6. Amazon CloudWatch investigations (formerly Amazon Q Developer operational investigations)
| field | value |
|---|---|
| item | operational issue raised from a CloudWatch widget, an alarm action, a console, or chat |
| metadata used | CloudWatch telemetry, CloudTrail, deployment events, resource configuration changes, AWS Health events |
| decisions | surface related signals, rank root-cause hypotheses, suggest remediation (runbooks, re:Post, docs) |
| time contract | none; MTTR named as the goal |
| agent shape | tool-using investigation agent |
| human in the loop | operator accepts or discards hypotheses; collaboration in Slack and Microsoft Teams |
| metrics claimed | none quantified at GA |
| status | preview 2024-12 as Amazon Q Developer operational investigations; GA 2025-06-24 renamed, no extra charge |
The preview launched at re:Invent 2024 under the Amazon Q Developer brand (Amazon Web Services 2024c) and reached GA in June 2025 as a CloudWatch feature at no additional cost, which can be started from more than 80 AWS consoles and from alarm actions (Amazon Web Services 2025e). The rename matters for a survey: the same capability appears under two product names a few months apart. The item is an alarm, not a ticket, and an alarm's time contract is implicit in whoever is paged.
8.4.7. Amazon DevOps Guru (sunset)
| field | value |
|---|---|
| item | operational insight (anomaly cluster) on AWS resources |
| metadata used | CloudWatch metrics, events, CloudFormation stack or tag boundary |
| decisions | detect, group into an insight, assign severity, recommend |
| time contract | none; insights notify through SNS to whatever on-call system exists |
| agent shape | ML anomaly detection; no LLM |
| human in the loop | operators read insights and act |
| metrics claimed | none in the end-of-support material |
| status | launched 2020-12 (re:Invent; date from memory, unconfirmed); sunset announced 2026-09-29; no new customers from 2026-10-29; end of support 2027-09-30 |
DevOps Guru is the earlier, non-LLM generation of AWS operational triage, and the first dead entry in this area. AWS's end-of-support page recommends CloudWatch for anomaly detection and alarms and Amazon DevOps Agent "for AI-assisted operational investigation and insight enrichment" (Amazon Web Services n.d.-b, 2026d). Teams that piped DevOps Guru insights into ticketing systems through its API lose that feed on 2027-09-30. The succession is direct: classifier-style anomaly grouping is replaced by an agent that investigates.
8.4.8. AWS samples: event-driven ticket resolution with Bedrock Agents
| field | value |
|---|---|
| item | internal IT ticket (environment access request) |
| metadata used | title, environment ID, business justification, access duration, access type, employee ID |
| decisions | resolve automatically within policy, or escalate to the environment owner |
| time contract | none; no priority or creation-time field in the documented schema |
| agent shape | DynamoDB stream triggers Lambda, which invokes a Bedrock Agent with action groups over DynamoDB tables |
| human in the loop | environment owner reviews requests the agent does not auto-resolve |
| metrics claimed | none |
| status | sample in aws-samples/amazon-bedrock-samples; three lab notebooks; date of first commit unconfirmed |
This sample is the minimal event-driven shape: a write to a ticket table fires a stream, a Lambda invokes the agent, and the agent either grants access through action groups or escalates (Amazon Web Services n.d.-e). "Access Duration" is the only time field, and it describes the grant, not the response.
8.4.9. Step Functions sample: triage support tickets with Bedrock AgentCore invokeHarness
| field | value |
|---|---|
| item | customer support ticket |
| metadata used | ticket text from execution input |
| decisions | classify category, sentiment, urgency; recommend an action |
| time contract | urgency is a model output; no clock input |
| agent shape | two LLM calls with role prompts (analyst, then "triage director") in a state machine |
| human in the loop | none in the sample; decision written to DynamoDB |
| metrics claimed | none |
| status | Step Functions starter template; publish date unconfirmed (AgentCore-era, 2025-2026) |
The state machine runs two harness invocations with different system prompts and records the result in DynamoDB (Amazon Web Services n.d.-g). It is the only official AWS template found with "triage support tickets" in its title. Urgency is inferred from text alone.
8.4.10. AWS samples: Jira ticket classification
| field | value |
|---|---|
| item | Jira issue (Server export or API pull) |
| metadata used | ticket content; project IDs to pull |
| decisions | classify; deduplicate exported records (Glue, not semantic dedupe) |
| time contract | none; batch over exports |
| agent shape | single LLM classification prompt per ticket from Lambda |
| human in the loop | results stored for analysis or write-back |
| metrics claimed | none |
| status | created 2024-10-04, last push 2024-10-28, 12 stars, MIT-0 (GitHub API, 2026-10-08) |
The repository targets Jira Server users who lack Jira Cloud's automation (Amazon Web Services 2024d). The purpose stated in the companion community post is reporting (how teams spend time), not live routing (AWS Community n.d.). Classification after the fact has no clock to respect.
8.4.11. AWS samples: Nemotron to Claude classification cascade (TicketTriage)
| field | value |
|---|---|
| item | B2B SaaS support ticket (synthetic) |
| metadata used | id, subject, body, optional customer tier ("context only") |
| decisions | category (8), priority P0-P3, confidence, needs_human |
| time contract | none; no timestamp in the ticket schema; P0/P1 always require a human |
| agent shape | model cascade: small model first, Claude Sonnet on low confidence, P0/P1, or needs_human; three-tier variant adds a gateway model |
| human in the loop | rubric forces needs_human for P0/P1, ambiguity, consequential actions, policy exceptions, suspected compromise |
| metrics claimed | on 150 synthetic tickets, one human reviewer: joint agreement 83.3% Sonnet only, 74.0% Nano only, 72.7% Nano to Sonnet (repo-reported) |
| status | created 2026-06-08, last push 2026-09-24, MIT-0; workshop sample |
This is the most careful sample in the area. The schema states its own taxonomy
version and human-review rubric, and the results document records its limits:
synthetic inputs, a single reviewer, a reused test split
(Amazon Web Services 2026b). One rule stands out for this survey:
"Customer tier is context only. Never change category, priority, confidence,
or needs_human solely because of customer tier." The entitlement that would
carry an SLA in a real desk is deliberately cut off from the priority
decision, and the ticket has no creation time at all.
8.4.12. AWS ML Blog: LLM-generated labels for support-case root-cause classification
| field | value |
|---|---|
| item | customer support case |
| metadata used | case text |
| decisions | classify root-cause category (six classes: billing, security awareness, feature request, defect, docs, education) |
| time contract | none; offline labelling |
| agent shape | Claude 3.5 Sonnet labels with XML-structured prompt; AutoGluon classifier trained on the labels |
| human in the loop | prompt iteration against examples |
| metrics claimed | over 90% accuracy on support-case root-cause categories (vendor-reported) |
| status | blog post 2025-03-27 |
The post uses the LLM as a labeller, not as the production classifier: Bedrock generates a balanced ground-truth set, and a cheaper model is trained on it (Amazon Web Services 2025c). The post sits in the AWS Support category of the ML blog, which suggests the data is AWS Support's own cases; the post does not say so outright: unconfirmed.
8.4.13. AWS Cloud Operations Blog: CloudWatch alarms to AWS Support cases via Bedrock
| field | value |
|---|---|
| item | CloudWatch alarm converted into an AWS Support case |
| metadata used | alarm event values, resource tags, AWS Support CTI (service and category) data |
| decisions | classify ServiceCode and CategoryCode; severity from the alarm's SeverityCode tag, default low |
| time contract | case severity sets AWS's response target; chosen by tag, never by the model |
| agent shape | semantic search plus Bedrock over the CTI catalogue |
| human in the loop | none in the flow |
| metrics claimed | none |
| status | blog 2023-12-11; repo aws-samples/cloudwatch-alarms-support-cases-automation, last push 2025-07-31 |
The sample makes the split explicit: the model picks where the case goes (service and category), and a human-set tag picks how fast AWS must answer (Al Okeh 2023). It is the clearest example found of a designer keeping the time contract out of the model's hands.
8.4.14. Google Contact Center AI, Agent Assist and Customer Engagement Suite
| field | value |
|---|---|
| item | live contact (voice, chat) and the conversation record afterwards |
| metadata used | transcript, knowledge base, conversation datasets for custom smart-reply models |
| decisions | draft (smart reply, knowledge assist), summarize, coach; topic categories and quality scoring in Conversational Insights |
| time contract | contact-center handle time; no SLA input documented |
| agent shape | Gemini-based assist models; virtual agents (Dialogflow) for self-service |
| human in the loop | representative accepts suggestions; QA teams review scores |
| metrics claimed | Best Buy up to 90 s faster with summarization; Bell Canada $20M savings (vendor-reported customer claims) |
| status | CCAI announced 2018-07-24 (alpha); renamed Customer Engagement Suite 2024-09-24; CCAI Insights renamed Conversational Insights |
Contact Center AI launched at Cloud Next '18 as Virtual Agent plus Agent Assist, in alpha (SiliconANGLE 2018). The September 2024 relaunch as Customer Engagement Suite added Gemini-based knowledge assist, coaching, summarization with sub-five-second latency, smart reply and translation, and renamed CCAI Insights to Conversational Insights (Lennox 2024). Insights produces "inquiry topic categories to prioritize": prioritization of topics for managers, not of individual items in a queue. Agent Assist triages the conversational turn; the ticket queue is out of its scope.
8.4.15. Gemini Cloud Assist investigations
| field | value |
|---|---|
| item | infrastructure or application issue (log error, monitoring alert, free-text description) |
| metadata used | logs, Cloud Asset Inventory, App Hub, errors, log themes, observation time, affected resources |
| decisions | rank observations, hypothesize root causes, recommend fixes, hand off to a support case |
| time contract | none in the agent; support case response targets come from the support package |
| agent shape | tool-using investigation agent; API for Slack and incident tooling |
| human in the loop | user starts and reviews; "Request support" converts the investigation to a case with context |
| metrics claimed | ZoomInfo "hours to minutes"; Waze root cause in 2 minutes (vendor-reported anecdotes) |
| status | preview (blog 2025-08-22); from 2026-04-10 limited to Premium Support customers or approved access; GA unconfirmed |
Investigations accept a description, an error, a log snippet or an observation time, and return ranked observations and root-cause hypotheses (Google Cloud 2025b). The handoff is the triage link: one click turns the investigation into a Google Cloud support case that carries the observations, and Cloud Assist fills case fields in advance from the chat (Google Cloud n.d.-b). From 2026-04-10, creating and running investigations requires a Premium Support contract or account-team approval (Google Cloud n.d.-a). The agent is gated by the same contract that sets support response times, but nothing documents the response target as an agent input.
8.4.16. Google internal: bugs, test failures and incident write-ups
| field | value |
|---|---|
| item | integration test failures (Auto-Diagnose); Google Issue Tracker bugs (Passerine); security and privacy incidents (summaries); fuzzer crashes (ClusterFuzz) |
| metadata used | failure logs; bug reports and code; incident records; crash stacks and regression ranges |
| decisions | diagnose root cause; attempt a fix; summarize for audiences; dedupe, bisect, file, assign component, close |
| time contract | none documented as a model input |
| agent shape | LLM summarizer in code review; agentic repair with Gemini 1.5 Pro; LLM summary drafts; rules and bisection (ClusterFuzz) |
| human in the loop | engineers in Critique; patch review; responders edit summaries; sheriffs triage ClusterFuzz bugs |
| metrics claimed | Auto-Diagnose 90.14% on 71 failures, 52,635 failing tests, "not helpful" 5.8%; Passerine plausible patches 73% machine / 25.6% human bugs; summaries 51% faster (author-reported) |
| status | Auto-Diagnose arXiv 2026-04 (ICSE SEIP 2026); Passerine arXiv 2025-01 (ICSE SEIP 2025); summaries blog 2024-04-26; ClusterFuzz open source since 2019 |
No Google paper was found on LLM classification or routing of Google Issue Tracker bugs: the brief's "Google internal bug-triage research" has no direct hit. The nearest work sits on either side of triage. Auto-Diagnose reads integration test failure logs and posts a diagnosis into Critique; it reports 90.14% accuracy on 71 failures and use on 52,635 distinct failing tests (Ziftci et al. 2026). Passerine samples 178 bugs from Google's issue tracker (78 human-reported, 100 machine-reported) and fixes them; the gap between machine-reported and human-reported bugs (73% versus 25.6% plausible patches) is a triage signal in itself (Rondon et al. 2025). Google's security team reports that LLM-drafted incident summaries were written 51% faster (Google Security Blog 2024). ClusterFuzz, the non-LLM baseline, automates filing, dedupe, bisection and closure of fuzzer bugs (Google n.d.-a).
8.4.17. Azure SRE Agent
| field | value |
|---|---|
| item | incident from Azure Monitor alerts, PagerDuty or ServiceNow |
| metadata used | severity (Azure Monitor); impacted service, incident type, priority (PagerDuty/ServiceNow); title keywords; telemetry; repos |
| decisions | route to a custom agent by response plan; investigate; propose or run mitigation; create or update ticket |
| time contract | priority and severity filter which plan handles an incident; no clock input documented |
| agent shape | tool-using agent with custom sub-agents, skills, Python tools, MCP servers and hooks |
| human in the loop | Review mode: an administrator approves write actions; Autonomous mode runs them; per-tool allow/ask/deny |
| metrics claimed | customer quotes: triage "hours down to minutes", incident investigation down 80% (vendor-reported via search snippets) |
| status | preview 2025 (month unconfirmed); GA 2026-03; docs updated 2026-08-26 |
Azure SRE Agent makes ticket priority a routing key. A response plan matches incoming PagerDuty or ServiceNow incidents by service, incident type, priority (for example P1 and P2) and title keywords, and Azure Monitor alerts by severity, then hands the incident to a chosen custom agent (Microsoft n.d.-b, 2026c). The run mode decides whether mitigations wait for approval. Priority thus selects the agent and, through the plan, its autonomy; the time remaining on the incident is not an input in any page read. Microsoft's internal predecessors are covered under 8.3.13.
8.4.18. Azure Copilot observability agent (formerly Azure Monitor investigate)
| field | value |
|---|---|
| item | Azure Monitor alert, grouped into an Azure Monitor issue |
| metadata used | alert, resource telemetry (AKS, VMs, application services) |
| decisions | correlate alerts into issues, investigate, summarize findings, suggest remediation |
| time contract | none documented |
| agent shape | investigation agent in Azure Copilot; autonomous-operations mode |
| human in the loop | on-call reviews the short list of explained issues |
| metrics claimed | none |
| status | preview (gated) in 2026; earlier name "Azure Monitor investigate"; start date unconfirmed |
The observability agent is alert-noise triage: it correlates related alerts into Azure Monitor issues and investigates each, so on-call starts "with a short list of explained issues instead of a flood of raw alerts" (Microsoft n.d.-f). It overlaps Azure SRE Agent; the boundary between the two products is not documented in the pages read.
8.4.19. Microsoft Learn: Foundry multi-agent ticket triage lab
| field | value |
|---|---|
| item | support ticket (free text) |
| metadata used | ticket text only |
| decisions | priority, team assignment, effort (2025 lab); category plus confidence, routed in code (2026 lab) |
| time contract | none; priority inferred from impact described in text |
| agent shape | 2025: triage agent with three connected agents (priority, team, effort); 2026: one agent with structured output |
| human in the loop | 2026 lab: low-confidence tickets sent back for detail; billing escalated by code rule |
| metrics claimed | none (training material) |
| status | connected-agents lab present by 2025-07-29, removed 2026-05-22 as deprecated; Agent Framework task marked draft |
Microsoft's official training used ticket triage as the multi-agent example. The 2025 lab built a triage agent that delegated to priority, team and effort agents through Foundry "connected agents" (Microsoft Learning n.d.). Connected agents is now a classic feature: the Foundry classic agents are deprecated with retirement on 2027-03-31, and the docs point to workflows (Microsoft 2025a). The replacement task inverts the design: one agent returns `category` and `confidence` as JSON and Python code owns the routing, so business rules stay testable outside the prompt. Neither version has a timestamp or an SLA.
8.4.20. IBM watsonx Orchestrate: incident triage solution
| field | value |
|---|---|
| item | IT incident (VPN, application access, password problems) |
| metadata used | free-text request; existing ticket for follow-ups |
| decisions | classify, create ticket, find runbook, run remediation, write resolution summary |
| time contract | none documented |
| agent shape | five agents (coordinator, first responder, runbook specialist, remediation engineer, summary specialist) in a fixed workflow |
| human in the loop | not documented in the tutorial |
| metrics claimed | none |
| status | ADK tutorial; publish date unconfirmed |
The tutorial pipeline runs classification, ticket creation, runbook lookup, remediation and summary in sequence; for an existing ticket the coordinator skips the pipeline and calls tools directly (IBM n.d.). It targets the repetitive low-complexity tail where resolution is a known runbook. The ticket is created by the agent, so no contract exists until the agent opens one.
8.4.21. Oracle Fusion Service: Service Request Triage Agent
| field | value |
|---|---|
| item | service request (SR) in Oracle Fusion Service |
| metadata used | SR content; product and category catalogue |
| decisions | classify product and category, detect severity and sentiment, request missing information |
| time contract | Fusion Service has milestones; whether the agent reads them is unconfirmed |
| agent shape | prebuilt AI agent on OCI, paired with a Service Request Resolution Agent |
| human in the loop | service representatives work the triaged SR; approval steps not documented |
| metrics claimed | none quantified ("lessens the resolution time") |
| status | documented in Fusion Service; announced with Oracle's AI agents release 2025-10-06 (press release not read) |
Oracle names the agent after the job: categorize, check that the SR has enough information, set severity and sentiment, then pass to a resolution agent (Oracle n.d.). Asking for missing information is the distinctive decision; most samples in this area classify whatever text arrives. Secondary coverage of Oracle's October 2025 agent release says the agents ship at no additional cost in Fusion Applications; the press release itself returned 403 to the fetcher (Oracle 2025).
8.4.22. Area synthesis
Three patterns hold across the cloud platforms.
First, the clock and the model live in different components. Amazon Connect Cases has a full SLA object with time to breach and breach-triggered rules, and its LLM writes summaries beside it. AWS support plans publish response targets (under 30, 15 or 5 minutes for critical cases) and put a generative front door before the case, but the customer still sets severity. The CloudWatch-to-Support sample routes with the model and sets severity from a human tag. The Nemotron cascade forbids customer tier from moving priority. Azure SRE Agent uses priority and severity as routing keys for response plans, the closest any product comes to letting the contract steer an agent, and it still routes on priority alone. No product or sample read here passes elapsed time, time to breach or business-hours state to a model.
Second, the operational side has converged on investigation agents, and the older ML generation is being retired. DevOps Guru (anomaly grouping, 2020) enters sunset with AWS DevOps Agent as its named successor; Amazon Q Developer operational investigations became CloudWatch investigations within six months; Azure Monitor investigate became the Azure Copilot observability agent. Each hands off to the vendor's own support queue: DevOps Agent opens an AWS Support case with findings, Gemini Cloud Assist converts an investigation into a case, and AWS bills DevOps Agent against support spend while Google gates investigations behind Premium Support. The support contract has become the agent's entitlement; it is still absent from the agent's input.
Third, the sample code teaches classification without a timestamp. Every reference sample found (the AWS sample repositories, the Step Functions template, the Microsoft Learn labs, the IBM tutorial) takes the body and at most a title or tier, and emits category, priority, team or urgency inferred from text. Microsoft's 2026 lab moves routing out of the prompt into code, and the Nemotron cascade publishes its rubric and a single-reviewer evaluation; both are steps toward auditable triage, neither touches the clock.
Microsoft's internal incident-triage work (DeepTriage, RCACopilot, Triangle, the ICSE 2026 Intelligent Triage paper (Liu et al. 2026)) is the exception on metrics: it measures time to engage and time to mitigate as outcomes; see 8.3.13. GitHub-issue triage on Foundry (IssueLens) is under 8.5.18.
Open questions: whether AWS, Google or Microsoft use models to route or prioritize their own customers' support cases among engineers (no public source found); whether Connect AI agents can read case SLA status as context; whether Oracle's triage agent reads Fusion milestones; and when, if ever, Gemini Cloud Assist investigations leave preview.
8.5. Code hosts and open-source issue triage
An issue on a code host is the purest instance of the triage item: a title, a body, a reporter, a creation time, labels, and almost never a written contract. Code hosts and open-source projects mostly express the time contract as inactivity: an item goes stale after N days without a comment, and closes N days later. A few projects state a real service level: Mozilla wants new bugs triaged within one week, GitLab attaches a fix SLO to each bug severity, and Linear sells SLAs as a feature. The question for this area is whether any LLM triager reads either kind of clock.
8.5.1. GitHub AI assessment comment labeler and AI moderator
| field | value |
|---|---|
| item | GitHub issue (labeler); issue or comment (moderator) |
| metadata used | issue body and title; trigger label; repository prompt files |
| decisions | classify (one label per prompt file, ai:<prompt-stem>:<assessment>); request info; flag spam or AI-generated text |
| time contract | none; runs on a label event |
| agent shape | single LLM call per prompt file via GitHub Models, prompts run in parallel; no tools |
| human in the loop | a maintainer applies the trigger label (default request ai review) and reviews the suggestions |
| metrics claimed | none |
| status | GA as a Marketplace action, announced 2025-09-05; repository created 2025-08-13 |
GitHub ships two Actions on top of the GitHub Models inference API:
github/ai-assessment-comment-labeler and github/ai-moderator
(GitHub 2025d). The labeler is opt-in per
issue: nothing happens until a person or a workflow applies the trigger
label. Output is a comment plus a structured label, and the comment can be
suppressed so that the label alone drives project automation. Authentication
is the workflow GITHUB_TOKEN with models: read, so no external key is
needed.
GitHub's documentation describes the tool as "AI-powered issue intake" and tells the maintainer to "review the suggestions provided by the tool and take appropriate action" (GitHub n.d.-b). The design is a labeler: one prompt, one verdict, no retrieval of other issues, and no notion of how long the issue has waited.
8.5.2. GitHub Models in Actions: actions/ai-inference and maintainer workflows
| field | value |
|---|---|
| item | GitHub issue |
| metadata used | title, body; for dedup, the repository's open issues; for the resolver, issue state and age |
| decisions | dedupe (link similar issues), request missing info, flag spam or "slop", close stale or resolved, welcome |
| time contract | the continuous resolver runs on a weekly cron (Sundays 00:00 UTC); no per-item clock |
| agent shape | workflow YAML calling actions/ai-inference (single LLM call) plus actions/github-script |
| human in the loop | per workflow; the examples comment or label rather than close |
| metrics claimed | survey of 500+ maintainers: 60% want triage help, 30% duplicate detection (vendor-reported) |
| status | actions/ai-inference created 2025-04-03; blog examples 2025-08-28 |
actions/ai-inference is the generic primitive: an Action that sends a
prompt to a GitHub Models endpoint and returns text
(GitHub 2025a). GitHub's open source programs office
published five recipes on top of it: duplicate detection
(pelikhan/action-genai-issue-dedup), completeness checks, spam and slop
detection, a "continuous resolver" for stale or already-fixed issues, and
first-contributor onboarding (Wolf 2025).
The resolver is the only recipe that touches time, and it does so the way the stale bot does: a scheduled sweep that asks the model whether an old issue is still live. Age is a selection criterion for the sweep, not a feature the model weighs against a deadline.
8.5.3. GitHub Agentic Workflows: issue triage
| field | value |
|---|---|
| item | GitHub issue (opened or reopened) |
| metadata used | title, body, existing labels and assignees (skips issues already labeled or assigned) |
| decisions | classify into an allowlisted label set (7 labels in the sample); comment with rationale; close duplicates in some deployments |
| time contract | none |
| agent shape | tool-using coding agent (Copilot CLI, Claude, or Codex engine) driven by a Markdown prompt, compiled to Actions YAML |
| human in the loop | none before labeling; "safe outputs" allowlist bounds what the agent can write |
| metrics claimed | none |
| status | technical preview 2026-02-13; sample workflow in githubnext/agentics |
GitHub Next's gh aw extension turns a Markdown file of instructions into a
GitHub Actions workflow that runs a coding agent with scoped tools
(GitHub 2026a). Issue triage is the introductory example and
is described by its authors as "triage labelling of issues and pull requests
and not much more" (GitHub Next 2026). The distinctive
part is the permission model: the agent reads with a read-only token and
writes only through declared safe outputs, for example "add one of these
seven labels". Argo CD runs a variant that also closes duplicates
(Argo CD project n.d.).
8.5.4. GitHub Copilot coding agent (assign issue to Copilot)
| field | value |
|---|---|
| item | GitHub issue (or a chat or VS Code task) |
| metadata used | issue title, body and comments; repository contents; copilot-setup-steps.yml environment |
| decisions | resolve: open a draft pull request; it does not classify or route |
| time contract | none |
| agent shape | autonomous tool-using coding agent running in a GitHub Actions sandbox |
| human in the loop | a person assigns the issue; the PR requests review from that person; Business/Enterprise admins enable by policy |
| metrics claimed | none in the GA changelog |
| status | public preview 2025-05-19; GA for paid Copilot plans 2025-09-25 |
The coding agent sits downstream of triage: assignment to Copilot is the
routing decision, made by a human (GitHub 2025c, 2025e).
It matters to this survey because it changes what "route" means on a code
host. The assignee field can now hold an agent, so a triager that routes can
route to a machine. Linear made the same move with delegation (below).
8.5.5. GitHub Issues structure: issue forms, issue types, duplicate detection, issue fields
| field | value |
|---|---|
| item | GitHub issue at creation time |
| metadata used | form fields (YAML schema), org-level issue type, issue fields (priority, area, dates) |
| decisions | structured intake (required fields, default labels and assignees); suggest up to 3 duplicates at filing |
| time contract | issue fields can hold dates; no SLA engine |
| agent shape | forms are rules; duplicate suggestion is a similarity model (implementation not disclosed) |
| human in the loop | the reporter sees the duplicate suggestions and decides whether to file |
| metrics claimed | none |
| status | issue forms beta 2021-06-23; issue types and sub-issues GA 2025-04-09; duplicate detection public preview 2026-06-18 |
Issue forms move triage work to the reporter: required fields, dropdowns and
default labels replace a first-pass classifier (GitHub 2021).
Organization-level issue types (GA April 2025) give every repository the same
type field to classify into (GitHub 2025b). In June 2026
GitHub added duplicate suggestions in the creation form and let MCP clients
read and write issue fields, so an external agent can file "fully triaged"
issues with priority and dates already set
(GitHub 2026b). The host supplies the schema; the
clock, if any, is a date field that nothing on GitHub enforces.
8.5.6. Stale bots: actions/stale and probot/stale (time-contract baseline)
| field | value |
|---|---|
| item | GitHub issue or pull request |
| metadata used | last-updated time, labels (exempt, only, any-of), milestone, assignee |
| decisions | label Stale, comment, close |
| time contract | the whole decision: defaults 60 days inactive → stale, 7 more days → close |
| agent shape | rules on a cron; no model |
| human in the loop | any comment resets the clock; exempt labels opt items out |
| metrics claimed | none |
| status | actions/stale active; probot/stale (created 2017-03-13) archived, recommends the Action |
The stale bot is the baseline that every LLM triager on GitHub should be compared to, because it is the only widely deployed GitHub automation whose decision is a function of elapsed time (GitHub n.d.-a). Its contract is inverted relative to a support SLA: the clock measures the reporter's and maintainers' silence, and expiry closes the item rather than escalating it. The Probot app that popularised the pattern is archived and points to the Action (Probot 2017).
8.5.7. GitLab Duo (issue summaries, description generation, Planner agent, flows)
| field | value |
|---|---|
| item | GitLab issue / work item |
| metadata used | issue comments (summary); short summary (description generation); hierarchy, milestones, labels, weights (Planner); team workload (assigner flow) |
| decisions | summarize; draft description; prioritize with RICE/MoSCoW/WSJF; flag stale or duplicate items; assign to least-loaded member |
| time contract | none read by the agents; the assigner flow's selling point is latency ("in seconds") |
| agent shape | single LLM calls (summary, description); chat agent with tools (Planner); two-agent event-triggered flow |
| human in the loop | summaries are private to the requester; Planner asks before edits; the assigner flow has no approval step |
| metrics claimed | none |
| status | discussion summary experiment 16.0, GA 17.6; description generation experiment 16.3; Planner beta 18.6, GA 18.8; work-item-created trigger 2026-07-20 |
GitLab's AI triage arrived in layers. Discussion summary and issue description generation are single prompts on one issue (GitLab n.d.-a). The Duo Agent Platform's Planner agent works over the backlog, applies prioritization frameworks, and edits labels and milestones after confirmation (GitLab n.d.-c). In July 2026 GitLab added a "work item created" trigger and published a two-agent flow that measures each team member's open-item load and assigns the new item to the least-loaded person with no human step (GitLab 2026). That flow routes on capacity, not on urgency or on the severity SLOs GitLab itself publishes (next entry).
8.5.8. GitLab triage-ops and severity SLOs
| field | value |
|---|---|
| item | GitLab issue and merge request (gitlab-org projects; the gem is reusable) |
| metadata used | labels (type, severity, priority, group), created/updated timestamps, milestone |
| decisions | label, comment, assign, close stale, generate per-group triage report issues, flag missed-SLO |
| time contract | explicit: bug severity SLOs S1 30 days, S2 60 days, S3 90 days, S4 120 days (handbook) |
| agent shape | rules (gitlab-triage gem, .triage-policies.yml with older_than date conditions) on scheduled pipelines |
| human in the loop | humans set severity and priority; the bot reports and nags |
| metrics claimed | triage reports carry severity × priority heatmaps, including a missed-SLO heatmap |
| status | in use since at least 2019 (dated triage report issues); gem blog post 2025-03-13 |
GitLab is the one code host whose own engineering process states a time
contract per severity and automates against it
(GitLab n.d.-b). The gitlab-triage gem expresses
policies as conditions plus actions, with date conditions such as "updated
more than 5 days ago and unlabeled" (GitLab 2025). Weekly
triage reports per product group, generated since 2019, include heatmaps of
bugs that have missed their SLO (GitLab Bot 2019). The
SLO is a rule input. As of this pass, no source connects it to Duo: the LLM
agents in the previous entry neither read nor set the severity clock
(unconfirmed for internal GitLab tooling).
8.5.9. Linear Triage Intelligence
| field | value |
|---|---|
| item | Linear issue in a team's Triage inbox (often created by integrations: Slack, support tools, Sentry) |
| metadata used | issue content, historical workspace issues, team/project/label/assignee history |
| decisions | suggest or auto-apply team, project, assignee, labels; flag duplicates; link related issues |
| time contract | none in the documentation; processing takes 1-4 minutes per issue |
| agent shape | agentic LLM with retrieval over the workspace (vector search); earlier version used small models with fixed prompts |
| human in the loop | per-property: show, hide, or auto-apply; accept or decline each suggestion; reasoning visible |
| metrics claimed | none |
| status | launched 2025-09-03 (technology preview), Business and Enterprise plans |
Linear's engineering write-up describes two generations: a first version on GPT-4o mini and Gemini 2.0 Flash with "tightly scoped prompts and rigid workflows", then an agentic version on GPT-5 and Gemini 2.5 Pro that fetches extra context itself (Linear 2025b). Each suggestion carries a natural-language rationale, and teams can steer it with written guidance (Linear n.d.-c).
Linear also ships SLAs: deadlines applied by rule (defaults Urgent 24 hours, High 1 week), risk bands from low to breached, alerts 24 hours before breach, and an option to exclude non-working days (Linear n.d.-b). SLA rules key on priority and labels, which Triage Intelligence can set. So the coupling runs one way: the model's label or priority choice can start a clock, but no documentation says the model reads an SLA or its risk band when it suggests.
8.5.10. Linear for Agents (delegation)
| field | value |
|---|---|
| item | Linear issue |
| metadata used | whatever the third-party agent reads through the Linear API and agent SDK |
| decisions | resolve or scope (draft PRs, write requirements, run root-cause analysis), comment |
| time contract | none at the platform level |
| agent shape | external agents (Devin, ChatPRD, Codegen, Sentry, others) as workspace members |
| human in the loop | delegation keeps a human assignee accountable; the agent is a separate "delegate" field |
| metrics claimed | none |
| status | announced 2025-05-20; agent interaction SDK public |
Linear treats agents as workspace members that can be mentioned, added to teams, and delegated issues (Linear 2025a). The delegation model is the distinctive part: the issue keeps a human assignee and gains an agent delegate, so accountability for the time contract stays with a person while the work moves to a machine (Linear n.d.-a).
8.5.11. Sentry Seer issue scan and Issue Fix
| field | value |
|---|---|
| item | Sentry issue (grouped error or performance event) |
| metadata used | stack trace, event volume, handled/unhandled, log level, spans, logs, profiles, commits, repository code |
| decisions | score actionability/fixability; prioritize (High/Medium/Low); root cause; propose fix; open PR; hand off to a coding agent |
| time contract | no SLA; priority escalates automatically on an event-volume surge |
| agent shape | scan model per issue (cost $0.003 per issue) feeding a tool-using debugging agent ($1 per fix run) |
| human in the loop | per-project threshold and stopping point (after root cause, after solution, after PR draft, or hand-off) |
| metrics claimed | 94.5% root-cause accuracy, 38,000+ beta issues (vendor-reported) |
| status | Seer GA 2025-06-17; Sentry agent for Linear in beta |
Seer turns Sentry's triage into a two-stage gate. A cheap scan scores every new issue for actionability, meaning how likely a code change fixes it, and projects set a minimum score above which Seer starts an Issue Fix automatically (Sentry 2025). The automation stops at the point the project picks (Sentry n.d.-b). Separately, Sentry's issue priority combines log level with, on Business plans, the error message, handledness and past actions on similar issues; a volume surge "escalates" the issue and bumps priority unless a human has set it by hand (Sentry n.d.-a). Time enters only as event rate, not as a response deadline. The Sentry agent for Linear lets a Linear user delegate an issue to Seer (Sentry n.d.-c).
8.5.12. Dosu
| field | value |
|---|---|
| item | GitHub issue, pull request, discussion |
| metadata used | title, body; prior issues; repository code and docs; label corrections from maintainers |
| decisions | classify (auto-label), dedupe/link related issues, draft or post a response, flag stale issues |
| time contract | none stated; marketing frames response latency ("people wait days, weeks, or months") |
| agent shape | LLM agent with retrieval over issues, code and docs; few-shot examples harvested from corrections |
| human in the loop | maintainers correct labels; each correction becomes a few-shot example |
| metrics claimed | auto-label accuracy +30% from correction-driven few-shot (vendor-reported, 2024-05-02); Apache Superset 4× faster resolution (vendor-quoted) |
| status | commercial GitHub App; CNCF adopted it for auto-labeling |
Dosu is the most widely deployed LLM triage bot in open source by its own count (16,000+ repositories, CNCF adoption) (Intel n.d.). Its documented technique is continual in-context learning: a webhook records every label a maintainer adds or removes after Dosu acts, and the correction becomes a retrievable few-shot example for later runs (LangChain 2024). The time contract appears only as a motivation (reporters waiting weeks for a reply) and as a monitored latency, never as a model input.
8.5.13. Probot apps and Issue Label Bot
| field | value |
|---|---|
| item | GitHub issue |
| metadata used | title and body |
| decisions | classify (bug / feature request / question); dedupe (probot duplicate detection); welcome, stale |
| time contract | only in probot/stale (see stale bots) |
| agent shape | text classifier trained on public GitHub issues, later a Kubeflow-specific Google AutoML model; abstains when uncertain |
| human in the loop | the bot applies no label when the prediction is uncertain |
| metrics claimed | none located |
| status | Issue Label Bot created 2019-03-22, taken down for infrastructure cost, repository archived 2022 |
Probot (Node.js framework for GitHub Apps, 2017) carried the first wave of triage bots: stale, first-timers, and duplicate detection apps (Attard n.d.). Issue Label Bot, by Hamel Husain and Jeremy Lewi, was the first ML labeler offered as a hosted GitHub App, and Kubeflow retrained it on its own issues (Husain and Lewi 2019). Its end is a data point for the cost question: the owners shut it down because hosting inference for free did not pay. The abstain-when-uncertain rule is the same design later used by BugBug and VS Code.
8.5.14. Kubernetes needs-triage and the triage robot (Prow lifecycle)
| field | value |
|---|---|
| item | issue or pull request in Kubernetes org repositories |
| metadata used | labels (needs-triage, triage/accepted, kind/*, sig/*, lifecycle/*), last-activity time |
| decisions | label needs-triage on open; SIG accepts via /triage accepted; stale → rotten → close |
| time contract | inactivity: 90 days → lifecycle/stale, +30 days → lifecycle/rotten, +30 days → closed |
| agent shape | Prow plugins and periodic jobs; rules only |
| human in the loop | /remove-lifecycle stale, /lifecycle frozen, /triage accepted slash commands |
| metrics claimed | KEP baseline (Sep 2020): ~3,004 open issues, ~13% stale |
| status | KEP-1553 implemented 2020-09-30; k8s-triage-robot succeeded fejta-bot |
KEP-1553 made "not yet triaged" an explicit state: every new issue gets
needs-triage, and only a SIG member's /triage accepted removes it
(Kubernetes SIG Contributor Experience 2020). The lifecycle robot then runs a 150-day inactivity
contract that ends in closure (Kubernetes n.d.). It is a
time-based SLA in practice, enforced on the project rather than on a vendor.
The contract measures neglect of the item, not urgency, and the classifier
(which SIG, which kind) is still a human with a slash command.
8.5.15. Rust triagebot
| field | value |
|---|---|
| item | issue or pull request in rust-lang repositories |
| metadata used | labels, file paths touched (autolabel), commands in comments, team membership |
| decisions | relabel, assign reviewers, ping teams, autolabel, request prioritization (I-prioritize), notify Zulip |
| time contract | none in the bot; prioritization is a weekly human working-group process |
| agent shape | rules and commands (@rustbot label, @rustbot prioritize), Rust service configured by triagebot.toml |
| human in the loop | every decision beyond autolabel is a human command; WG-prioritization assigns P-labels |
| metrics claimed | none |
| status | active; repository created 2019-02-10; no LLM features |
Triagebot is the counter-example: a large project that automated the
plumbing of triage (labels, pings, review assignment, Zulip topics) and kept
the judgement human (Rust project n.d.). Regression and
soundness labels auto-add I-prioritize, which posts to a Zulip stream for
the prioritization working group. The urgency signal is the label kind
(regression-from-stable-to-stable, I-unsound); elapsed time plays no part.
8.5.16. VS Code issue triage actions (deep classifier)
| field | value |
|---|---|
| item | issue in microsoft/vscode and related repositories |
| metadata used | title, body; per-category confidence thresholds and feature-area owners in .github/classifier.json |
| decisions | classify feature area, assign owner; detect non-English; close needs more info after 7 days; close feature requests after 60 days below 20 upvotes; lock old closed issues |
| time contract | inactivity and vote windows: 7 days (info), 60 days (backlog candidates); classifier batch every 30 minutes |
| agent shape | transformer classifier (simpletransformers / Hugging Face) trained on Azure GPU, applied in batches; rules for the time windows |
| human in the loop | weekly rotating "inbox tracker" fixes what the bot mis-triages |
| metrics claimed | none published |
| status | repository created 2020-04-07, archived 2025-11-14; successor not documented |
VS Code ran one of the longest-lived ML issue classifiers in open source.
The "deep classifier" assigned feature-area labels and owners when confidence
cleared a per-category threshold, and a human on weekly inbox rotation caught
the remainder (Microsoft 2020, n.d.-c).
The time rules sit beside the model rather than inside it: needs more info
closes after 7 days, and feature requests in Backlog Candidates close after
60 days unless they reach 20 upvotes (Microsoft n.d.-e).
The 60-day/20-vote rule is a time contract with a demand signal attached,
which no LLM system in this area reproduces. The repository was archived in
November 2025; this pass found no public description of what replaced it.
8.5.17. Mozilla BugBug and BugBot (autonag)
| field | value |
|---|---|
| item | Bugzilla bug (Firefox and related products) |
| metadata used | title, first comment, keywords, flags; severity, product, whiteboard, crash signature, STR for other models |
| decisions | route (product/component), classify (defect/enhancement/task, regression, spam, severity), nag triage owners |
| time contract | policy: daily assessment, full triage within 1 week, S1 assigned within 2 business days, needinfo nag after 10 days; fixtime model predicts fix time above/below median |
| agent shape | XGBoost classifiers per question; BugBot rules run hourly (autofix) or daily (nags) |
| human in the loop | component moved only above 60% confidence; triage owners act on nags; [no-nag] opt-out |
| metrics claimed | ~350 bugs auto-triaged Feb-Apr 2019, >80% precision on validation, manual routing took ~1 week (author-reported) |
| status | BugBug in production since February 2019, repository active; BugBot (formerly autonag) active |
BugBug answers one triage question per model: component, defect type, regression, spam, severity, and more (Mozilla n.d.-a). The component model moves a bug out of "Untriaged" only above 60% confidence; Mozilla reported more than 80% precision and contrasted it with roughly a week of manual routing (Castelluccio and Ledru 2019). BugBot, the rule engine formerly called autonag, applies the model and needinfos or emails triage owners when severity or priority is missing (Mozilla Release Management n.d.).
Mozilla is the strongest case in this area of a stated time contract sitting
next to a classifier. The triage policy says new bugs are "fully triaged, or
under active investigation, within one week", S1 and sec-critical bugs are
assigned within 2 business days, and needinfo requests unanswered for 10 days
need attention (Mozilla n.d.-b). The classifier shortens the
first leg of that contract; the rules enforce the remaining legs. BugBug also ships a
fixtime model that labels historical bugs by whether their time-to-fix fell
above or below the median and learns to predict that class. This pass found
no evidence that its output feeds routing or nags (unconfirmed).
8.5.18. Microsoft IssueLens
| field | value |
|---|---|
| item | GitHub issue |
| metadata used | issue text, repository source and tests, project wiki |
| decisions | flag critical/blocking/regression, dedupe, label, route to owner, write action plan, send triage report |
| time contract | none |
| agent shape | tool-using agent on the GitHub Copilot SDK, hostable on Microsoft Foundry |
| human in the loop | plans need maintainer approval; the agent does not open PRs |
| metrics claimed | none |
| status | open-source repository; release date unconfirmed |
IssueLens draws the line that Copilot coding agent does not: it triages and plans, and states that "planning approval does not implement changes" (Microsoft n.d.-d). Reports go to email or Teams, which moves the triage output to where maintainers already watch, rather than into labels alone.
8.5.19. Kilo Auto-Triage
| field | value |
|---|---|
| item | GitHub issue (newly opened) |
| metadata used | title and body; previously triaged issues (vector index); repository label set |
| decisions | dedupe (comment linking the original), classify (bug / feature / question / unclear with 0-1 confidence), label |
| time contract | none |
| agent shape | embedding similarity plus LLM classification |
| human in the loop | dashboard to review runs and reasoning; skip and required labels gate scope |
| metrics claimed | none |
| status | documented product feature; launch date unconfirmed |
Kilo's feature is representative of the 2025-2026 crop of hosted triage add-ons: retrieval for duplicates, a four-way classifier with a confidence score, and a review dashboard (Kilo n.d.). It is included for breadth; nothing distinguishes its handling of time.
8.5.20. Literature: developer assignment, from Anvik to instruction-tuned LLMs
| field | value |
|---|---|
| item | bug report in Bugzilla-style trackers (Eclipse, Mozilla, Chromium) |
| metadata used | title, description; some work adds component and tossing history |
| decisions | route (recommend developer), sometimes component |
| time contract | absent from task formulation |
| agent shape | SVM (2006); attention bi-RNN (DeepTriage); fine-tuned transformers; LoRA-tuned 8B LLM with constrained decoding (2025) |
| human in the loop | recommendation lists (top-k) for a human triager |
| metrics claimed | DeepTriage: higher rank-10 accuracy than bag-of-words baselines; 2025 LLM: Top-1/Hit@10 0.156/0.475 (Eclipse JDT), 0.013/0.753 (Mozilla) (author-reported) |
| status | papers, 2006-2025 |
Anvik, Hiew and Murphy framed triage as "who should fix this bug?" and learned a developer recommender from Eclipse and Firefox history (Anvik, Hiew, and Murphy 2006). DeepTriage replaced bag-of-words with an attention-based bidirectional RNN and released a benchmark of 383,104 Chromium, 314,388 Mozilla Core and 162,307 Firefox reports (Mani, Sankaran, and Aralikatte 2019). A 2023 comparison found DeBERTa the strongest fine-tuned transformer for developer and component assignment (Dipongkor and Moran 2023). The 2025 instruction-tuned LLM work reports a near-zero Top-1 on Mozilla next to 0.753 Hit@10, which restates the task as shortlist generation for a human (Kiashemshaki et al. 2025). Across two decades, the input is text plus history; creation time appears only as the split between training and test data.
8.5.21. Literature: duplicate bug report detection
| field | value |
|---|---|
| item | bug report at filing |
| metadata used | summary, description; REP adds product, component, version, priority |
| decisions | dedupe (rank candidate masters) |
| time contract | absent; some datasets restrict candidates to a time window (not a deadline) |
| agent shape | vector-space NLP (2007); BM25F-based REP (2011); ChatGPT zero-shot extraction feeding REP (Cupid) |
| human in the loop | ranked candidates for a triager or reporter |
| metrics claimed | Runeson: ~2/3 of duplicates findable (Sony Ericsson); REP: recall@k 37-71% on 209,058 Eclipse reports; Cupid: Recall Rate@10 0.602-0.654, +5-8% over prior best (author-reported) |
| status | papers, 2007-2024 |
Runeson, Alexandersson and Nyholm showed at Sony Ericsson that NLP retrieval finds about two thirds of duplicates (Runeson, Alexandersson, and Nyholm 2007). Sun et al.'s REP extended BM25F to weight both text and categorical fields, and it remains the baseline that LLM papers beat (Sun et al. 2011). Cupid keeps REP and uses ChatGPT only to extract the essential sentences of each report first, an early example of an LLM as a preprocessor rather than a decider (Zhang et al. 2024). GitHub's 2026 duplicate suggestions at filing time (GitHub Issues structure) are the product form of this line.
8.5.22. Literature: issue type classification and LLM replications
| field | value |
|---|---|
| item | GitHub issue |
| metadata used | title and body |
| decisions | classify (bug / enhancement / question) |
| time contract | absent |
| agent shape | fine-tuned transformers (NLBSE tool competition); prompted GPT-3.5, GPT-4o, Llama 3.1 8B |
| human in the loop | none in the benchmark setting |
| metrics claimed | NLBSE'22: 1.4M-issue benchmark, CatIss 87.2% micro-F1; ICPC 2025: GPT-4o 0.8639 F1, project-agnostic GPT-4o 0.8680, Llama 3.1 8B about 0.05 below GPT-3.5 (author-reported) |
| status | competition 2022-2024; paper 2025 |
The NLBSE tool competition fixed a shared benchmark for the bug/feature/question split on GitHub issues (Kallis et al. 2022). Heo and Lee's 2025 replication found prompted GPT-4o only about one F1 point above GPT-3.5 and roughly level with the fine-tuned transformers (Heo and Lee 2025). That plateau near 0.87 F1 is the accuracy the GitHub and Kilo labelers inherit for the same three-way task.
8.5.23. Literature: SWE-bench-adjacent issue localization
| field | value |
|---|---|
| item | GitHub issue paired with a repository snapshot |
| metadata used | issue text; repository code graph, files, functions |
| decisions | localize (file, function, line) and, downstream, resolve |
| time contract | absent; cost per issue is the only resource reported |
| agent shape | fixed pipeline (Agentless); graph-guided tool-using agent (LocAgent); scheduled search agent (OrcaLoca) |
| human in the loop | none in the benchmark |
| metrics claimed | SWE-bench: 2,294 tasks, best 2023 model 1.96% resolved; Agentless 27.33% on Lite at $0.34/issue; LocAgent up to 92.70% file-level on Lite; OrcaLoca 65.33% function match on Lite (author-reported) |
| status | papers, 2024-2025 |
SWE-bench turned "resolve this GitHub issue" into a benchmark of 2,294 tasks from 12 Python repositories (Jimenez et al. 2024). Localization, finding where the fix goes, became its own subtask: Agentless runs a fixed localize-repair-validate pipeline (Xia et al. 2024), LocAgent searches a heterogeneous code graph (Chen et al. 2025), and OrcaLoca adds priority scheduling and context pruning to the search (Yu, Zhang, et al. 2025). For triage, localization is a routing signal: the file a fix touches implies an owner. None of this work treats the issue's age or urgency as an input; the benchmark issues are already resolved.
8.5.24. Area synthesis
Three generations, one decision set. Rule bots (Probot, Prow, triagebot, gitlab-triage, BugBot) handle state and time. Classifiers (Issue Label Bot, VS Code deep classifier, BugBug) handle route and type, with an abstain threshold (BugBug 60%, VS Code per category). LLM systems (GitHub labeler, gh-aw, Dosu, Linear, Duo, Seer, Kilo, IssueLens) add duplicate linking, rationale text and, for Seer and Copilot, a fix attempt. The decisions barely changed. What changed is that the output now carries an explanation and can include a pull request.
Where the clock is. On code hosts the time contract is mostly an
inactivity rule, and it lives in the rule layer: stale at 60 days on GitHub's
default, a 150-day stale-rotten-close chain on Kubernetes, 7 and 60 days on VS
Code. Stated deadlines exist at Mozilla (triage within one week, S1 assigned
in 2 business days), GitLab (S1-S4 fix SLOs of 30-120 days, with a
missed-SLO label) and Linear (SLAs with risk bands and non-working-day
exclusion). In every case found, the clock is enforced by rules next to the
model, and no LLM triager reads it. The closest couplings are indirect:
Linear's model sets the priority that starts an SLA, Sentry escalates on
event rate, and BugBug trains a fix-time model whose deployment is unconfirmed.
Accountability for the clock stays human. Linear's delegation keeps a human assignee next to the agent; Copilot asks the assigning person for review; GitLab's assigner flow routes on workload alone. Agents get the work, people keep the deadline.
Open questions. Would a triager that saw "S2, 41 of 60 days elapsed" or "needs-more-info, day 6 of 7" make different decisions than one that sees only text? No surveyed product or paper tests this. The literature drops time entirely; the benchmarks (DeepTriage, NLBSE, SWE-bench) contain only resolved history. What replaced VS Code's classifier after November 2025, and whether GitLab's internal triage-ops consumes Duo output, are unconfirmed.
8.6. Model vendors and agent frameworks
Model vendors and framework maintainers publish the reference patterns that most custom triage agents copy. This area records what those references put into the prompt and the tools. One question runs through every entry: does the item's creation time, its age, or a response-time target reach the model? In most examples the item has a timestamp somewhere in its data model. In almost none does that timestamp, or any SLA, reach the decision.
8.6.1. Anthropic: Ticket routing guide (Claude docs)
| field | value |
|---|---|
| item | customer support ticket |
| metadata used | none; the prompt takes one string, ticket_contents |
| decisions | classify (intent, one of three in the example); hierarchical classifiers suggested past 20 intents |
| time contract | named as a routing factor ("urgency, customer type, SLAs") but not passed to the model |
| agent shape | single LLM call, XML-tagged <reasoning> then <intent>, regex-parsed |
| human in the loop | none in the code; the guide asks how humans route today before automating |
| metrics claimed | proposed targets, unmeasured: 95% accuracy, time-to-assignment under 5 minutes, rerouting under 10% |
| status | GA documentation; current text names Claude Haiku 5.5 at low effort |
The guide (Anthropic n.d.-f) is the clearest case of the
pattern this area keeps finding. Its "understand your current support
approach" section asks "What criteria are used to determine what SLA/service
offering is applied?", and the categories list includes "Emergency support:
time-sensitive problems". The prompt template that follows takes a single
f-string variable, ticket_contents, and returns intent and reasoning. No
creation time, customer tier or SLA field enters the call.
The success-criteria list is the most complete in the area: classification consistency (95% or higher), adaptation within 50–100 samples, edge-case accuracy of at least 80%, routing accuracy of 90–95%, average assignment under 5 minutes, rerouting below 10%, escalation below 20%. These are targets the guide proposes. They are not measured results. Time-to-assignment is the only time metric, and it measures the router's latency, not the ticket's remaining SLA budget. The integration section offers push (webhook from Zendesk) or pull (scheduled poll), which is where a clock would enter if one were used.
8.6.2. Anthropic: classification cookbook (insurance tickets)
| field | value |
|---|---|
| item | insurance support ticket text |
| metadata used | body text only (train.tsv / test.tsv) |
| decisions | classify into 10 categories |
| time contract | none |
| agent shape | LLM call; few-shot, then retrieval of similar labelled tickets, then chain of thought |
| human in the loop | offline evaluation only (Promptfoo) |
| metrics claimed | vendor-reported: ~70% (prompt), 94% (retrieval), 97% (chain of thought) on the test set |
| status | cookbook, maintained |
The ticket-routing guide links here and cites "71% accuracy to 93%". The notebook as fetched on 2026-10-08 reports ~70%, then 94% with retrieval, then 97% with chain of thought (Anthropic n.d.-b). The two Anthropic pages disagree on the numbers. Both are vendor-reported on a vendor test set. The dataset has no timestamp column.
8.6.3. Anthropic: cookbook customer-service agent and tool_use/utils
| field | value |
|---|---|
| item | mock support ticket (Ticket dataclass); separately, a chat turn in customer_service_agent.ipynb |
| metadata used | subject, description, customer name and email, created_at, category, priority, status |
| decisions | classify, set priority, route to team, search knowledge base, draft response, add note, mark complete |
| time contract | created_at set to now minus 0–48 h and serialised into get_next_ticket output; no SLA defined |
| agent shape | tool-using agent over 8 @beta_tool functions; reference determine_priority is a keyword rule |
| human in the loop | none; mark_complete refuses a ticket without a priority |
| metrics claimed | none (used as a fixture for compaction and programmatic tool calling demos) |
| status | cookbook fixture; chat notebook first commit 2024 (unconfirmed month) |
The parent note already treats this backend as its baseline; see
2. Two facts matter for
this survey and are not stated there. First, it is the one vendor fixture in
this area that hands the model the ticket's age: _serialize_ticket emits
created_at as ISO-8601, and the generator back-dates tickets by up to 48
hours (Anthropic n.d.-d). Second, nothing consumes
it. The reference determine_priority rule maps keywords ("locked out",
"urgent") to urgent and billing to high, and there is no response target
against which 48 hours would mean anything. The older
customer_service_agent.ipynb is a chat bot with three tools
(get_customer_info, get_order_details, cancel_order) and no queue.
8.6.4. Anthropic: claude-code repository's own issue triage and dedupe
| field | value |
|---|---|
| item | GitHub issue on anthropics/claude-code |
| metadata used | title, body, labels, comments, event type (issues vs issue_comment), repository label list; form fields read but body declared authoritative |
| decisions | classify (exactly one of bug, enhancement, question, invalid, duplicate), area labels, lifecycle labels, dedupe (up to 3 candidates) |
| time contract | lifecycle timeouts in deterministic code: invalid 3 d, needs-repro 7 d, needs-info 7 d, stale 14 d, autoclose 14 d; duplicates closed after 3 d |
| agent shape | Claude Code via claude-code-action; triage on Opus 4.6, dedupe on Sonnet 4.6; dedupe fans out to 5 parallel search subagents plus a filter |
| human in the loop | none at labelling; issue author can veto an auto-close with a thumbs-down; 10 upvotes exempt an issue from stale |
| metrics claimed | none published; dedupe comments logged to Statsig as github_duplicate_comment_added |
| status | production on a public repo; triage workflow first commit 2025-04-15, auto-close 2025-08-05, sweep 2026-02-12; last change 2026-09-28 |
This is the most complete vendor-operated triage system in the area, and it
splits the work cleanly by clock. The LLM part is event-driven and
clock-free: /triage-issue (Anthropic 2025c)
reads the issue and its comments through a restricted gh.sh wrapper and
may only add or remove labels from the existing list. It may not comment.
Write, Edit, WebFetch and WebSearch are disallowed, and
CLAUDE_CODE_SCRIPT_CAPS limits the label script to two calls per run. The prompt never mentions the issue's age.
The time contract lives in TypeScript that never calls a model.
scripts/issue-lifecycle.ts is the "single source of truth" for five
lifecycle labels and their timeouts (Anthropic 2026).
sweep.ts runs at 10:00 and 22:00 UTC, marks issues stale after 14 days
without update, and closes labelled issues once the label is older than its
timeout. lifecycle-comment.ts posts the nudge with the deadline the moment a
label lands. auto-close-duplicates.ts closes an issue three days after the
dedupe comment unless the author objects. The model's one time-relevant act
is choosing needs-repro or needs-info. That choice starts a seven-day clock
it never sees. The prompt warns that "false positives are worse than missing
labels" for exactly that reason.
The contract here is a closure deadline on the reporter, not a response deadline on the maintainer. No rule measures how long an issue waits for first maintainer contact. Compare the stale-bot baseline in the code-hosts area, 8.5.6.
8.6.5. Anthropic: claude-code-action triage and dedupe examples
| field | value |
|---|---|
| item | GitHub issue |
| metadata used | issue number, title, body, author login |
| decisions | classify (bug/feature/question), priority (critical/high/medium/low), labels, dedupe, comment |
| time contract | none; trigger is issues: opened only |
| agent shape | Claude Code in GitHub Actions, slash-command prompt (/label-issue) |
| human in the loop | maintainers edit labels afterward |
| metrics claimed | none |
| status | GA (v1.0.0 released 2025-08-26); examples/issue-triage.yml first commit 2025-08-25 |
The published examples (Anthropic 2025b) are the
portable subset of the claude-code repository's setup: issue-triage.yml,
issue-deduplication.yml, and an "Issue Auto-Triage and Labeling" recipe in
docs/solutions.md. The recipe asks the model for a priority but gives it no
timestamp and no schedule. The lifecycle sweep, which is what carries the
clock in Anthropic's own repository, is not part of the published example.
Adopters copy the classifier without the timer.
8.6.6. Anthropic: Claude Agent SDK email-agent demo
| field | value |
|---|---|
| item | email over IMAP |
| metadata used | sender, subject, body, date (listener email.date field) |
| decisions | notify (priority low/normal/high), star, summarise, draft, scheduled summaries |
| time contract | none; scheduled_time listeners run on cron, unrelated to item age |
| agent shape | Claude Agent SDK agent plus user-authored TypeScript "listeners" it can write |
| human in the loop | local single-user client; the README says not to deploy it |
| metrics claimed | none |
| status | demo; first commit 2025-09-17 |
No Agent SDK example targets support tickets. The nearest is this email
client (Anthropic 2025a). Triage lives in listeners such
as boss-urgent-watcher.ts, which notifies at high priority when the subject
contains "urgent" or "asap". The urgency signal is a keyword in the text, not
the age of the message.
8.6.7. Anthropic's own support desk (Intercom Fin)
| field | value |
|---|---|
| item | support conversation (help center and messenger) |
| metadata used | unconfirmed; free users are the stated target segment |
| decisions | resolve, hand off to human; refund automation "being tested" |
| time contract | none stated |
| agent shape | third-party agent (Fin) over Anthropic's help-center content |
| human in the loop | human agents take enterprise and power-user issues |
| metrics claimed | vendor-reported (Intercom): 50.8% resolution, 96% involvement, 1,700 hours saved in month one |
| status | production; case-study date unconfirmed |
Anthropic's own customer support runs on a vendor product built on Claude, not on its cookbook patterns (Intercom n.d.-a). Segmenting by customer type (free versus enterprise) is the only routing input the case study names. It is the customer-type factor the ticket-routing guide lists and its example leaves out. Intercom's SLA features are covered with the CRM area. Whether they apply to Anthropic's queue is unconfirmed.
8.6.8. OpenAI: Swarm and the "Orchestrating agents" cookbook
| field | value |
|---|---|
| item | customer chat turn |
| metadata used | conversation messages only |
| decisions | route (handoff from a "Triage Agent" to sales or refunds agents), refund, discount |
| time contract | none |
| agent shape | multi-agent with handoffs as functions returning an Agent |
| human in the loop | refund tool docstring asks the model to get user confirmation |
| metrics claimed | none |
| status | deprecated: README says "replaced by the OpenAI Agents SDK"; cookbook first commit 2024-10-11 |
Swarm (OpenAI 2024b) and the "Orchestrating agents: routines and
handoffs" notebook (OpenAI 2024a) fixed the meaning
of "triage agent" for a generation of examples: a router whose only job is
to pick the next agent. The triage_agent example has three agents and one
instruction line. Triage in this sense is conversational routing, with no
queue, no item metadata and no clock.
8.6.9. OpenAI: Agents SDK customer service example and openai-cs-agents-demo
| field | value |
|---|---|
| item | airline customer chat |
| metadata used | context object: passenger name, confirmation, seat, flight and account numbers, itinerary, baggage claim, compensation case, origin/destination |
| decisions | route (Triage Agent hands off to seat, flight-info, booking, refunds/compensation, FAQ agents), act through tools |
| time contract | none; flight times are domain data, not a response target |
| agent shape | multi-agent, Agents SDK handoffs, relevance and jailbreak input guardrails on every agent |
| human in the loop | demo UI shows agent trace; no escalation-to-human agent |
| metrics claimed | none |
| status | demo; repository created 2025-06-14, 6,582 stars on 2026-10-08 |
The demo (OpenAI 2025b) and the smaller
examples/customer_service in the Agents SDK
(OpenAI n.d.-b) carry Swarm's shape forward
with typed context and guardrails. The context model has 13 fields. None is
a timestamp for the conversation, a wait time, or a customer tier.
8.6.10. OpenAI: "Optimizing customer support agents for cost and quality" cookbook
| field | value |
|---|---|
| item | synthetic e-commerce support ticket (10 in the eval set) |
| metadata used | ticket id, customer id, message; customer record (tier, region, verified, lifetime value); order record |
| decisions | resolve via lookups, create refund case, escalate to human (ambiguous, policy-sensitive, high value, account security) |
| time contract | none on the ticket; latency is measured per agent run; follow-up work moved to Batch (24 h window) |
| agent shape | tool-using agent, five tools; model routing of simple steps to smaller models |
| human in the loop | escalate_to_human tool, always priority: normal; must_escalate labels in the eval set |
| metrics claimed | quality, latency, tool calls and cost per run on a deterministic simulation; no production numbers |
| status | cookbook; first commit 2026-09-18, "last verified September 14, 2026", GPT-5.4 models |
This is the most recent vendor support-agent cookbook in the area
(OpenAI 2026) and the only one that splits work by
latency tier: customer-facing work stays synchronous, follow-up work (QA,
analytics tagging, routing audits) moves to Batch or flex processing. The
deadline it optimises is the API's, not the customer's. The phrase "carrier
SLA tables" appears once, inside INTERNAL_POLICY_APPENDIX, a block of
filler repeated 45 times so the baseline prompt is oversized. In this
cookbook the SLA is context bloat to be trimmed.
8.6.11. OpenAI: classification cookbooks (embeddings, fine-tuning, tool_choice)
| field | value |
|---|---|
| item | short text (reviews, transactions); a customer-service chat in the tool_choice notebook |
| metadata used | text only |
| decisions | classify |
| time contract | none |
| agent shape | embeddings + classifier, fine-tuned model, or zero-shot; tool_choice: required for a CS bot |
| human in the loop | none |
| metrics claimed | per-notebook accuracy on public datasets |
| status | cookbooks; Using_tool_required_for_customer_service first commit 2024-05-02 |
The cookbook has no notebook dedicated to support-ticket classification. As
of 2026-10-08 the classification notebooks are
Classification_using_embeddings, Zero-shot_classification_with_embeddings,
Fine-tuned_classification, and two transaction-classification notebooks
(OpenAI n.d.-a). They are recorded for completeness; none
has an item with a creation time.
8.6.12. OpenAI's own support: AI operating model and Presence
| field | value |
|---|---|
| item | support interaction (chat, email, embedded help; English phone line under Presence) |
| metadata used | unconfirmed |
| decisions | resolve, escalate to human; classifiers flag patterns; reps turn interactions into eval cases |
| time contract | none stated in coverage seen |
| agent shape | agents with knowledge, approved actions, escalation rules; graders, simulations; Codex proposes behaviour changes from traffic |
| human in the loop | staff approve Codex-proposed changes before deployment; reps design classifiers |
| metrics claimed | vendor-reported: 75% of inbound phone issues resolved without a person; handoffs down 15 points in 10 days |
| status | operating-model post September 2025; Presence limited GA 2026-07-22, enterprise only, deployed by forward-deployed engineers |
OpenAI described its support organisation in September 2025 (OpenAI 2025a). The post is a feedback loop: reps flag interactions as test cases and build classifiers. Presence productises the loop (Help Net Security 2026). Coverage of the launch reports no SLA, wait-time or priority mechanism. The openai.com page returned 403 to the fetcher, so the September post's details here come from secondary summaries and are unconfirmed.
8.6.13. xAI: Grok cookbook refund agent and Grok Bot for support
| field | value |
|---|---|
| item | customer refund email shown as a helpdesk ticket |
| metadata used | customer email, order record (items, order date), refund policy |
| decisions | look up, propose refund amount, refuse out-of-policy, draft reply, flag threats in an internal note |
| time contract | 30-day refund window on the order, enforced by policy; nothing on the ticket |
| agent shape | Grok tool-using agent (grok-4.7); stop-and-resume on a stored response, call id as idempotency key |
| human in the loop | every refund waits in an approval queue; audit log of request, approval, rejection |
| metrics claimed | none |
| status | cookbook guide dated 2026-10-05; Grok Bot support guide date unconfirmed |
No xAI example classifies or prioritises a queue. The closest is the refund agent with human approval (Zakariasson 2026), published three days before this survey. Its contribution is durable approval: the run stops before an irreversible tool call, survives a server restart, and resumes exactly once. The only clock is the 30-day refund window. It is a property of the order, and policy applies it without asking anyone. Grok Bot's support guide (xAI n.d.) advertises "connectors, triage jobs, and routines that keep tickets moving" but, as fetched, gives no fields, rules or metrics. An xAI page on internal tools describes a ticket helper with "severity classification" in search snippets; the page returned 403 and the claim is unconfirmed.
8.6.14. Google: run-gemini-cli issue triage and gemini-cli's own workflows
| field | value |
|---|---|
| item | GitHub issue |
| metadata used | title, body, repository label list; scheduled mode selects issues with no labels or status/needs-triage |
| decisions | labels (kind, area, priority p0–p3, effort in gemini-cli), dedupe (gemini-cli) |
| time contract | none; priority from keywords ("security", "outage", "crash"); hourly sweep is a batch cadence, not a deadline |
| agent shape | Gemini CLI in GitHub Actions; real-time (CSV labels) and hourly batch (JSON array with per-issue explanation) |
| human in the loop | maintainers adjust labels; "precision over coverage" leaves uncertain issues unlabelled |
| metrics claimed | none |
| status | GA action; repository created 2025-07-11; gemini-cli runs automated and scheduled triage and dedupe on itself |
Google's equivalent of claude-code-action ships the triage pair as examples
(Google 2025). The scheduled prompt defines
priority/p1 as "a critical issue requiring immediate attention" and maps
priority to keywords. gemini-cli's own gemini-scheduled-issue-triage.yml
(Google n.d.-c) fetches open issues with
--json number,title,body,labels. The field list omits createdAt and
updatedAt. The model cannot tell a one-hour-old p1 from a one-month-old p1.
8.6.15. Google: Gemini cookbook and Vertex AI customer-support notebooks
| field | value |
|---|---|
| item | shopper question with an image (Vertex); generic text (Gemini cookbook classification) |
| metadata used | image, product catalogue, store location via function calling |
| decisions | answer, match products, call functions; generic classification elsewhere |
| time contract | none |
| agent shape | single model with structured output and function calling |
| human in the loop | none |
| metrics claimed | none |
| status | notebooks; Vertex notebook first commit 2025-04-10 |
Neither Google repository has a ticket-triage notebook as of 2026-10-08. The
Gemini cookbook offers Basic_Classification and Text_Classification
(Google n.d.-b). The Vertex "customer support use case"
(Google Cloud 2025a) is a furniture-shopping assistant.
It answers questions and does not triage. Recorded as an absence.
8.6.16. Mistral: classification prompting and Classifier Factory
| field | value |
|---|---|
| item | bank customer inquiry (prompting); user utterance (Classifier Factory intent cookbook) |
| metadata used | text only |
| decisions | classify (card arrival, change pin, exchange rate, country support, cancel transfer, charge dispute) |
| time contract | none |
| agent shape | prompted LLM; or fine-tuned classifier trained on JSONL (mteb/amazon_massive_intent subset) |
| human in the loop | none |
| metrics claimed | training and validation loss curves only |
| status | prompting cookbook first commit 2024-07-11; Classifier Factory cookbooks 2025-04-16, moved to deprecated_cookbooks/ 2026-09-07 |
Mistral's bank-inquiry prompt (Mistral AI 2024) is a
textbook support-routing prompt, also taught in the DeepLearning.AI Mistral
course. The Classifier Factory cookbooks (Mistral AI 2025)
fine-tune intent and moderation classifiers. On 2026-09-07 they moved into a
deprecated_cookbooks folder. The deprecation status of the Classifier
Factory product itself is unconfirmed. The Agents API cookbook
prd_linear_ticket creates Linear tickets from a PRD; it writes tickets
rather than triaging them.
8.6.17. Cohere: Classify endpoint customer-support tutorial
| field | value |
|---|---|
| item | customer email to an insurer |
| metadata used | text only |
| decisions | classify into 4 request types (policy details, account settings, claims, cancellation) |
| time contract | none |
| agent shape | few-shot classify endpoint over embeddings; later fine-tuned embed models only |
| human in the loop | none |
| metrics claimed | none |
| status | tutorial created 2022-09-08, now under -ARCHIVE-; default-model Classify deprecated 2025-01-31 |
Cohere's tutorial (Cohere 2022) is the oldest vendor ticket-routing example in this area. It dates from September 2022, before the chat-model wave, and it is the same insurance-email shape that Anthropic's classification cookbook later used. On 31 January 2025 Cohere deprecated Classify with default embed models. Only fine-tuned models remain supported (Cohere 2025). The tutorial now sits in the docs archive.
8.6.18. LangChain: agents-from-scratch email assistant (triage router, ambient agent, LangSmith evals)
| field | value |
|---|---|
| item | |
| metadata used | author, to, subject, thread body; EmailData also carries send_time, which the triage prompt does not use |
| decisions | triage (ignore / notify / respond) with reasoning; respond agent drafts, schedules meetings |
| time contract | none at triage; "Today's date" injected only into the response agent for meeting scheduling; ingest cron looks back 60 min |
| agent shape | LangGraph router (structured output RouterSchema) then tool-using response agent; HITL and memory variants |
| human in the loop | Agent Inbox interrupts: accept, edit, ignore, or respond on notify and on tool calls; feedback updates stored preferences |
| metrics claimed | LangSmith experiment: exact-match accuracy of classification_decision against labelled dataset; no published figure |
| status | course repository created 2025-03-31, last push 2026-08-11; executive-ai-assistant predecessor archived |
This is LangChain's reference ambient agent
(LangChain 2025a), built for the Interrupt 2025
workshop. The triage step is a three-way router whose output schema is
reasoning plus classification. The evaluation file
(LangChain 2025b) creates a LangSmith dataset
named "Interrupt Workshop: E-mail Triage Dataset" and scores exact match on
the triage label. A second set of LLM-judge evals grades the response agent's
tool calls.
The time handling is the sharpest example of the area pattern. The input
schema EmailData declares send_time. parse_email returns author, to,
subject and thread and drops it, and triage_user_prompt has four
variables, none temporal. The current date does appear: it is concatenated
into the response agent's system prompt so that schedule_meeting gets the
right day. The model knows today's date when it books a meeting, and does not
know how long the email has waited when it decides whether to answer it.
8.6.19. LangChain: executive-ai-assistant (archived)
| field | value |
|---|---|
| item | |
| metadata used | from, to, subject, page content; few-shot examples retrieved from a store |
| decisions | triage (no / email / notify); draft, calendar |
| time contract | none |
| agent shape | LangGraph; triage prompt with configurable triage_no, triage_email, triage_notify rules |
| human in the loop | Agent Inbox; "If unsure, opt to notify", corrections become few-shot examples |
| metrics claimed | none |
| status | created 2025-01-14; archived (read-only) on 2026-10-08 |
The predecessor of agents-from-scratch (LangChain 2025c). It introduced "ambient agent" and the Agent Inbox as LangChain terms. It learns triage from corrections stored as few-shot examples. It has the same four inputs, and again none is temporal.
8.6.20. LangGraph: customer support tutorial (Swiss Airlines)
| field | value |
|---|---|
| item | airline customer chat |
| metadata used | passenger id, user flight info, current time |
| decisions | route to specialised assistants (flights, cars, hotels, excursions), act through safe or sensitive tools |
| time contract | none; current time injected so the model can reason about flight times |
| agent shape | LangGraph, primary assistant delegating to sub-assistants; route_tools splits safe from sensitive tools |
| human in the loop | interrupt before sensitive (write) tools |
| metrics claimed | none |
| status | tutorial; examples/ copy archived 2026-01-26, content moved to docs.langchain.com |
Every assistant prompt in this tutorial ends with "Current time: {time}.",
bound with .partial(time=datetime.now) (LangChain n.d.).
It is the one framework example found where the wall clock reaches every
model call. It serves domain reasoning (is the flight in the past?), not a
response target. The interrupt before sensitive tools is the human-approval
pattern that the xAI refund agent later made durable.
8.6.21. LlamaIndex: finance triage agent (LlamaClassify + LlamaExtract)
| field | value |
|---|---|
| item | email with attachment (invoice or expense) |
| metadata used | sender, attachment; extracted fields: amount, description, payee, invoice due_date |
| decisions | classify (invoice vs expense, by rules), extract, accept or deny expense at a 1000 threshold, draft reply |
| time contract | invoice due_date extracted and passed into the reply text; not used to order or prioritise |
| agent shape | LlamaIndex Workflow: classify_email step, then extract_contents step; LlamaCloud classify and extract |
| human in the loop | none |
| metrics claimed | none |
| status | example; first commit 2025-11-06 |
Named "triage" by its authors (LlamaIndex 2025), this
is the only example in the area that extracts a deadline from the item. The
invoice schema has due_date. The workflow branches on document type and
on amount (under 1000 accepted), and the due date ends up only in the
LLM-drafted acknowledgement. GitHub code search over the run-llama
organisation found no support-ticket triage example; a web search was not
possible in this pass.
8.6.22. CrewAI: email auto-responder flow
| field | value |
|---|---|
| item | Gmail thread |
| metadata used | id, thread id, snippet, sender |
| decisions | filter, identify action-required, draft responses |
| time contract | none; Gmail query newer_than:1d, 180-second poll |
| agent shape | CrewAI Flow kicking off a three-agent crew (filter, action, writer) |
| human in the loop | drafts only, not sent |
| metrics claimed | none |
| status | example; first commit 2025-08-12; config mismatch (see below) |
The crew (CrewAI 2025) defines three tasks,
filter_emails, action_required_emails and draft_responses. The
tasks.yaml in the same directory, as fetched on 2026-10-08, defines only
send_followup_email, a recruiting task, so the example appears not to run
as checked in. That is unverified by execution. The email record keeps a
snippet, not the body, and no date. Time appears only as the one-day search
window and the poll interval.
8.6.23. AutoGen: Swarm customer support example
| field | value |
|---|---|
| item | customer chat (flight refund) |
| metadata used | messages only |
| decisions | route (travel agent to flights refunder), refund, hand off to user |
| time contract | none |
| agent shape | AgentChat Swarm team with handoffs; HandoffTermination pauses for the user |
| human in the loop | handoff to "user" stops the team until input arrives |
| metrics claimed | none |
| status | AutoGen in maintenance mode (README, 2026-10-08); successor is Microsoft Agent Framework; swarm team first commit 2024-10-25 |
AutoGen's version of the OpenAI Swarm pattern (Microsoft n.d.-a). It is included because it is the framework's documented customer-support example. It is deprecated in practice: the repository README declares maintenance mode and points new users to Microsoft Agent Framework.
8.6.24. DSPy: customer service agent tutorial
| field | value |
|---|---|
| item | airline customer request |
| metadata used | user profile, itinerary, flights with a custom Date type |
| decisions | fetch flights, pick, book, cancel; file a ticket for unresolved requests |
| time contract | none; dates are flight data |
| agent shape | dspy.ReAct with tools; optimiser compiles the prompt against a metric |
| human in the loop | none |
| metrics claimed | tutorial-level evaluation only |
| status | tutorial; first commit 2025-05-28 |
DSPy's tutorial (DSPy 2025) is a ReAct airline agent. It sits at the resolve end of the decision list. DSPy's distinctive offer, compiling a prompt against a labelled metric, would suit a triage classifier. A published DSPy triage example was not found.
8.6.25. Haystack: GitHub issue resolver agent and routers
| field | value |
|---|---|
| item | GitHub issue |
| metadata used | issue description and comments, repository files |
| decisions | locate relevant files, propose next steps as an issue comment |
| time contract | none |
| agent shape | Haystack Agent component (Claude Sonnet 4 in the recipe) with GitHub tools |
| human in the loop | comment only; maintainers act |
| metrics claimed | none |
| status | cookbook; first commit 2025-02-26 |
Haystack has no ticket-triage cookbook found by repository search. The
issue-resolver recipe (deepset 2025) works after triage:
given an issue URL, it proposes a fix plan. Haystack's routing components
(ConditionalRouter, LLMMessagesRouter, metadata routers) are building
blocks a triage pipeline would use. None ships with a support example.
8.6.26. Area synthesis
Twenty-five systems were surveyed. Three patterns hold across the area.
The timestamp is in the data model and out of the prompt. Anthropic's
cookbook Ticket has created_at. agents-from-scratch's EmailData has
send_time. The LlamaIndex invoice has due_date. The claude-code and
gemini-cli workflows run against GitHub issues that carry createdAt and
updatedAt. In every case the triage prompt omits the field, or receives it
with no rule that uses it. The Anthropic tool_use fixture does pass
created_at to the model, with no SLA to compare it against. The only
example where the clock reaches every call is the LangGraph airline tutorial,
and it uses the clock for flight logic. No example in this area computes
time remaining against a target, or breach risk.
Where a time contract exists, it is in deterministic code beside the
agent. Anthropic's own repository is the clearest case. The model chooses
needs-info. A five-entry TypeScript table and a twice-daily sweep turn that
choice into a seven-day deadline. The model never sees it. The xAI refund
agent's 30-day window and the CrewAI one-day search window are of the same
kind: a time rule, applied outside the model's judgement. A deterministic timer cannot
hallucinate a deadline, which favours the split. It also means the published
examples teach classification and leave out scheduling. claude-code-action's
public examples ship the classifier without the sweep.
Vendor guides name the SLA and then drop it. Anthropic's ticket-routing guide lists "urgency, customer type, SLAs" as routing factors and asks how SLAs are assigned today. Its prompt takes one variable. Its only time metric, time-to-assignment under 5 minutes, measures the router. OpenAI's optimisation cookbook measures latency per run and moves non-urgent work to a 24-hour batch window. Its one mention of SLA is filler in a prompt to be trimmed. Where frameworks use the word "triage" (Swarm, Agents SDK, AutoGen, LangGraph), it means conversational routing, the choice of the next agent, not queue prioritisation.
Open questions for the synthesis pass:
- Does a measurable gain follow from passing age and SLA remaining into the triage prompt, or is a deterministic re-sort after classification enough? No example in this area runs the comparison.
- Anthropic's and Google's own repositories label priority but publish no accuracy or time-to-label metrics. Statsig logging in claude-code suggests the numbers exist internally.
- The two Anthropic accuracy figures for the same classification recipe (71%→93% in the guide, ~70%→94%→97% in the notebook) need reconciling.
- Presence and Fin are the vendors' production answers. Neither public account says whether the agent reads queue time or SLA policy.
8.7. Our projects and prior work
This area asks what we have already built that has the shape of a triage
item: a body of text, metadata, and a clock. Twelve projects qualify. One of
them, cinque-ports, is a direct study of support-ticket triage. The others
triage something adjacent: backlog items for an agent loop, pull requests
for a change gate, research claims for a verifier, feed items for a reader.
The common finding is that time enters almost none of them as an input. Where
a field such as created_at exists, it orders the queue or bounds a lookup;
it never computes a deadline.
8.7.1. Beads ticket backend and compaction loop (www.wal.sh)
| field | value |
|---|---|
| item | beads issue (git-native JSONL record): work item for a coding agent |
| metadata used | priority (0-4), status, dependencies, issue_type, acceptance_criteria; created_at only as sort tiebreak |
| decisions | pick next (dependency-aware ready queue), set status; the demo agent writes a one-sentence resolution note and closes |
| time contract | none; ticket->tool-result drops created_at and updated_at before the agent sees the ticket |
| agent shape | Messages-API tool-use loop, two tools (get_next_ticket, update_ticket_status), client-side context compaction |
| human in the loop | dry-run backend by default (queue-backend, in-memory cursor); live writes go through set-status! as one-line diffs |
| metrics claimed | tokens: 57,929 to 36,762 total (-37%), peak context 5,785 to 3,224 (-44%), 15 tickets, one run each, Sonnet 5 |
| status | prototype; beads.clj and agent.clj last changed 2026-07-26 |
The backend is the subject of the parent note
(REPL-Driven Ticket Triage) and its empirical companion
Compaction Against a Beads Backend.
It wraps the beads tracker (Yegge n.d.) in two EDN tool definitions and a
dispatch function, and drives them with a Clojure port of Anthropic's
automatic-context-compaction cookbook
(Anthropic n.d.-a). The baseline it replaces is the
mock customer_service_api in the cookbook's tool_use/utils
(Anthropic n.d.-c).
The ordering is (sort-by (juxt :priority :created_at)) in ready-queue, so
age breaks ties within a priority bucket and does nothing else. The trimmed map
handed to the model (:id :title :description :status :priority :issue_type
:acceptance_criteria :dependencies) has no timestamp at all, so the agent
cannot reason about age even if prompted to. The token numbers are from one
run per arm and measure context management. No triage-quality or accuracy
metric exists for this backend.
8.7.2. cinque-ports: ten-option support-triage study
| field | value |
|---|---|
| item | support ticket (synthetic corpus: 250 tickets / 980 lifecycle events at seed 5; a v1 corpus of 300) |
| metadata used | required: id, created_at, channel (email web chat api), subject, body, lifecycle_stage; optional: requester, product_area, severity_hint |
| decisions | proposal verbs: route, assign, escalate, close_duplicate, request_info, no_action; plus priority, target, confidence |
| time contract | none in the schema or any build; created_at orders duplicate candidates only; build 03 forbids wall-clock reads |
| agent shape | ten builds: rules/SQL, classifier, similarity (JITIR), local-LLM worker, memory framework, OSS-helpdesk webhook, four vendor adapters |
| human in the loop | propose-only contract; a deterministic gate decides what leaves the pipeline; abstention is a typed outcome |
| metrics claimed | self-measured on the synthetic corpus: naive floor 0.68 action accuracy; classifier 0.956 holdout (n=68); similarity recall@5 0.81 |
| status | prototype, wave 1; ADR 001 is "Draft / RFC, no decision recorded"; 420 commits 2026-07-17 to 2026-07-20; private |
cinque-ports is the only project here built to answer the survey's question
directly. A coordinator Claude Code session ran four, later ten, worker
sessions, each building one triage option as a Ruby reference with a spec
extracted from it. All ten score against the same language-neutral vectors and
JSON schemas (ticket, proposal, verdict). The ADR scores the options on
four scenarios (visibility into reasoning, bad input data, misclassification,
offline grading) and seven concerns, and proposes a composite, explicitly "not a
decision": a classifier as baseline and eval harness, Jira Service Management as
system of record, Claude via MCP in propose-only mode as the reasoning layer.
Refutation condition R2 says the LLM layer is unjustified for classification if
the classifier alone meets the reassignment-rate target.
The time contract is the gap the project's own reviews found. The simulated CIO
review lists "No SLA, support-hours, or incident-response evaluation for any
option" as finding 5 and calls the gap uniform across all ten options. The
simulated CPO review answers that SLA timers come with the system of record
(options 6 and 9 have native timers), so the triage layer need not carry them.
The specs agree with the CPO by construction. Build 03 MUST be deterministic
and forbids any wall-clock read that is not already a field on the ticket.
Build 01 uses created_at only to require that a duplicate candidate predate
the subject. The only "clock" implemented anywhere is build 08's
SessionClock, which models the 8-hour session ceiling of the vendor runtime, not
a customer deadline. Build 04 ("local LLM worker") ran against a deterministic
stub; real-model behaviour is marked [H], so no live LLM result exists. All
accuracy numbers are on synthetic data the project generated, and two builds
flag their own scores as corpus-invertibility or sighted-label artifacts.
8.7.3. crowsnest ITIL4 workload simulator
| field | value |
|---|---|
| item | synthetic ITSM event ("sighting"): incident, service request, change, problem, event, deployment |
| metadata used | practice, priority (P1-P4), impact, urgency, assignment_group, origin (servicenow, jira-sd, pagerduty, …), ticket, ci |
| decisions | none; the simulator emits load, the dashboard renders it |
| time contract | sla_breached is a random flag (P1/P2 only, p=0.25), not derived from any clock; durations are sampled ranges |
| agent shape | none; seeded generator feeding a local observability receiver |
| human in the loop | a person watches the dashboard |
| metrics claimed | none |
| status | working test fixture; Python 2026-06-25, ClojureScript port in src/wal_sh/tools/crowsnest.cljs 2026-10-07; private |
The simulator (crowsnest-grind --mode itil4) exists to stress crowsnest's
status coercion: ITIL states such as breached and on_hold are outside the
receiver's {ok,error,unknown} enum and collapse to unknown. The
crowsnest PBT grinder note records
that loss. The crowsnest v3 spec
then sketches the generalization to ITIL monitoring: a standing-state fold per
monitored entity with sla : { window, breaches } as a derived aggregate. That
is the closest any of our designs comes to computing an SLA. It remains a
draft section marked for review against Datadog monitors, PagerDuty services and
ServiceNow configuration items, with no implementation.
8.7.4. standard-change: label-driven change classification
| field | value |
|---|---|
| item | pull request treated as an ITIL 4 change |
| metadata used | changed file paths (diff), existing labels, change-schedule calendar (freeze, window, emergency events) |
| decisions | classify itil:standard / itil:normal / itil:emergency; app:* routing labels; gate promotion |
| time contract | yes, as a guard: freeze and change-window events are read "at reliance", when the change is acted on |
| agent shape | rules (actions/labeler globs plus a shell step); TLA+ model of the promotion guards |
| human in the loop | itil:emergency and release:start are human-applied; emergency needs approval |
| metrics claimed | none on triage accuracy; "7 checks both directions" for the TLA+ model (repo badge) |
| status | working demo on mock storefront apps; 453 commits; last commit 2026-10-07; public |
This is triage without a model. The labeller classifies a PR from its diff
(standard is the complement of normal, which actions/labeler cannot
express, so a shell step derives it), and guards read a Google Calendar change
schedule at the moment of deploy. It is the one project here where time is an
input to the decision. The input is a window (may this act now?), not a
deadline (how long until this is late?). The scenarios document records the
failure that motivated "at reliance": a PR carried itil:emergency from an
earlier exercise, so classification must be re-confirmed when the change is
activated. A triage label goes stale as the world moves, which is the
SLA problem seen from the other side.
8.7.5. ChangeFlow: ITIL 4 change management over MCP
| field | value |
|---|---|
| item | change request (RFC) |
| metadata used | production impact, data modification, security, multi-system scope, historical success rate (weighted) |
| decisions | risk score 0-100 to standard (<30) / normal (30-70) / emergency (>70); freeze check; CAB approval state |
| time contract | freeze periods; a test asserts a 15-minute MTTR target over simulated approval times |
| agent shape | eight MCP tools over JSON-RPC for an external LLM client; rules-based scoring inside |
| human in the loop | CAB approval; emergency_override break-glass with audit |
| metrics claimed | get_metrics returns fixed literals (MTTR "22 minutes", change failure rate 0.042), not measurements |
| status | dormant (dsp-dr); last commit 2026-04-18; 221 commits; public |
ChangeFlow exposes change triage as tools for an LLM client rather than
embedding an LLM. The MTTR "SLA" in test/battle-test.scm is an output of a
simulation (approval time plus a fixed 8-minute implementation time against a
15-minute target), not an input to scoring. The metrics tool is a stub. Two
siblings in the same org are adjacent and also dormant: guile-pr2cr
(2025-09-28) turns a GitHub PR plus its Jira ticket into an ITIL change-request
document, and guile-deploy-ledger (2026-04-11) computes MTTR and lead time from
deployment events with a 1,800-second MTTR alert threshold. Neither performs triage.
8.7.6. Annotation review queue and research-editor fan-out (www.wal.sh)
| field | value |
|---|---|
| item | a research-note heading carrying a claim |
| metadata used | property drawer: CUSTOM_ID, VERDICT, VERIFIED_AT, VERIFIED_BY, FINDING, REVIEW; ledger hash |
| decisions | queue (needs-review, unverified-anchored), route to citation-auditor or claim-validator, record verdict |
| time contract | staleness ordering by VERIFIED_AT (full-second ISO-8601 required so the sort is correct); no deadline |
| agent shape | orchestrator subagent fanning out to two verifier subagents |
| human in the loop | author reviews verdict drawers; the hash-chain ledger makes rewrites detectable |
| metrics claimed | counts per verdict from summary; no accuracy figure |
| status | working; in daily use on this site |
The annotation-systems spec describes
the queue as "the queue the grind draws from". It is triage of claims, not
tickets, and the routing decision is which verifier type a heading needs. Time
appears as staleness: the spec requires second precision in VERIFIED_AT
because minute precision broke the staleness sort. That gives an age, and the
age orders the queue, but no rule says a verdict expires.
8.7.7. Agent work queues on beads (swarms, supervisors, skills)
| field | value |
|---|---|
| item | beads issue as the unit of work for a fleet of Claude Code sessions |
| metadata used | priority 0-4 (P0 "production down" to P4 "backlog"), status, blocked-by edges; updated_at via bd stale |
| decisions | claim next ready item, create blocker items, close; bd stale surfaces items untouched for N days |
| time contract | only bd stale --days N (default 30) on updated_at; a human or agent runs it, nothing acts on it |
| agent shape | tmux-driven Claude Code sessions prompted to run bd ready and claim work |
| human in the loop | operator reads supervisor logs and weekly audit reports |
| metrics claimed | none |
| status | in use: hydra-setup (2026-09-26, private), sage.el bin/dx-swarm (2026-05-18), CONTINUE beads channel (2025-12-25), skills bd and beads-workflow (2026-06-25) |
Several repos use beads as a shared work queue for agents.
hydra-setup/bin/session-supervisor sends each tmux session "Check open beads
with 'bd ready'". sage.el/bin/dx-swarm tells a swarm to close finished work,
claim from bd ready, and file blockers at priority 0. The bd skill in
aygp-dr/skills names bd stale, bd blocked and bd orphans as "Triage" and
forbids an agent from bypassing the ready queue. Priority is set by whoever
files the item. The only time signal is staleness, and it is a report, not a
trigger.
8.7.8. builder-workspace coordinator: LLM task triage
| field | value |
|---|---|
| item | project task submitted to a Redis-backed queue |
| metadata used | task type, payload, priority (CRITICAL..DEFERRED), dependencies, created_at, retry count |
| decisions | suggest priority and assignee; decompose tasks; the prompt rules out final assignment |
| time contract | prompt asks the model to weigh "task urgency and impact on project timeline"; no SLA field or timer |
| agent shape | single LLM coordinator persona (Claude and Gemini providers) over a priority queue of worker agents |
| human in the loop | coordinator suggests, a person decides |
| metrics claimed | none |
| status | abandoned prototype; first commit 2024-09-18, last 2026-03-15; public |
The coordinator prompt pins its own date to 14 September 2024 and lists "Task
Triage and Assignment" as one of six duties. Urgency is a word in the prompt,
not a field. The queue stores created_at, assigned_at and completed_at,
which would support time-in-queue measurement, but nothing reads them back into
the decision.
8.7.9. Research feed triage (tech-crawler, agentic-research-crawler)
| field | value |
|---|---|
| item | crawled feed or page item (paper, post, event listing) |
| metadata used | title, URL, date, source, tags; deduplication by canonical URL with first-seen timestamp |
| decisions | deduplicate; a human reader decides relevance |
| time contract | none; daily crawl cadence is conjecture C-002 |
| agent shape | deterministic Clojure crawler; agent used for parse-error diagnosis at the REPL, not for triage |
| human in the loop | the reader does the triage |
| metrics claimed | none; conjecture C-003 ("title+url+date+source+tags suffices for triage") open, user study deferred |
| status | working; tech-crawler main 2026-06-27 with weekly discovery worktrees to 2026-10-04; private |
The agentic research crawler note states C-003 as a falsifiable claim about triage metadata. The repo's conjecture-validation document argues against it from other tools: abstracts, venue and author are triage fields that the five-field schema omits. It is the only project here that treats "which metadata does triage need" as an experiment. Nothing has run the experiment yet.
8.7.10. Patient-message triage schema (event note design)
| field | value |
|---|---|
| item | patient-portal message (free text, PHI-bearing) |
| metadata used | receivedAt, triage route, draft with citations to approved knowledge-base chunks |
| decisions | route ADMINISTRATIVE / CLINICAL_QUESTION / URGENT, or abstain (ABSTAIN_NO_GROUNDING, ABSTAIN_AMBIGUOUS); draft reply |
| time contract | none stated; receivedAt is tagged as a quasi-identifier (weight 0.3), not a clock |
| agent shape | unspecified; GraphQL federation schema with an append-only audit subgraph |
| human in the loop | clinician reviews drafts; edits classed STYLE vs CORRECTION so the correction rate stays visible |
| metrics claimed | none for triage |
| status | design sketch in InterSystems AI meetup notes (2026-08-25, last edit 2026-09-18) |
The schema makes abstention a categorical route, not a low confidence score,
because a route is a labelled event the review loop can later confirm or
contradict. That is a design choice the survey's vendor systems rarely expose.
An URGENT route implies a response-time contract, but the schema stops short
of stating one. Its receivedAt field is modelled for re-identification risk,
not for deadline arithmetic.
8.7.11. defect-triage skill
| field | value |
|---|---|
| item | a found defect (failing test, implementation disagreement, wrong output, bug report) |
| metadata used | the defect description and which party disagreed |
| decisions | classify into one of three ledgers: thing, checker, or specification defect |
| time contract | none |
| agent shape | skill (procedure) loaded into a Claude Code session |
| human in the loop | the session's operator |
| metrics claimed | anecdotal: on one project three of four reference defects had a spec defect underneath |
| status | staging (not promoted); last change 2026-08-01 |
The skill is a classification procedure with no queue. It is the only triage-named procedure
in the skills repo. Its question ("what did our contract fail to say?") is the
same move cinque-ports makes with numbered clarifications. The other
triage-named skills (code-search-triage, emacs-config-triage) choose tools
and debug configs and are out of scope.
8.7.12. Coverage in this week's event notes
| field | value |
|---|---|
| item | conference sessions about other parties' triage agents (pointers, not systems we built) |
| metadata used | n/a |
| decisions | n/a |
| time contract | not stated in any session listing we recorded |
| agent shape | Rovo triage agent (Atlassian); Claude in PyTorch CI and issue triage (Meta) |
| human in the loop | n/a |
| metrics claimed | none recorded in our notes |
| status | session listings only; no talk attended or summarized for triage content |
Atlassian Team '26 Europe (6 to 8 October) lists a hands-on "build your first Rovo triage agent" session (Tuesday 15:00, repeated Thursday 10:00) and a service-desk session on the customer-service side of the Teamwork Graph. Our note records titles and slots, not content. PyTorch Conference 2026 lists Meta's "Fighting Agents with Agents: Bringing Claude to PyTorch CI, triage, and PR review" for 20 October. EmTech AI lists a ServiceNow keynote on agentic workforces. The Boston AI Week note's 27 research-relevant picks contain no triage-specific session (searched 2026-10-08). Areas 02 and 04 cover these vendors directly.
8.7.13. Area synthesis
Three patterns hold across our own work.
First, the clock is never an input to an agent's triage decision. The beads
backend strips timestamps before the model sees a ticket. cinque-ports puts
created_at on every ticket and uses it only to order duplicate candidates;
its determinism rule forbids wall-clock reads. The crowsnest simulator's
sla_breached is a coin flip. The annotation queue and bd stale compute age
but only report it. The one decision that does read time, standard-change's
freeze and window guard, is rule-based and asks "may this act now", not "how
long until this is late".
Second, where an SLA is discussed, it is assigned to the system of record. The cinque-ports CPO review states this outright: SLA timers come with Jira Service Management or the OSS helpdesk, so the triage layer does not carry them. The CIO review counts the same fact as a gap uniform across all ten options. That disagreement is the open design question for this survey. Is breach risk a feature the triage agent should see, or a constraint enforced around it?
Third, evaluation is offline and synthetic. cinque-ports is the only project with accuracy numbers, and they come from a seeded synthetic corpus with self-reported caveats. No project here measures MTTA, time-in-triage or breach rate on real items, though cinque-ports lists time-in-triage among the telemetry its options would emit, and builder-workspace stores the timestamps that would allow it.
Open questions:
- Would adding
created_atand an SLA target to the beads tool result change the agent's ordering, or only the explanation? The backend makes this a one-lineselect-keyschange and a rerun. - Can a cinque-ports vector express a time contract? The verdict schema has no field for "proposed escalation because breach is imminent". Adding one would be a G2 schema clarification.
- Does the standing-state SLA fold in crowsnest v3 belong to the triage agent's inputs or to the dashboard's outputs? The draft does not say.
8.8. Small, fine-tuned and reasoning models, and the demo genre
This area tests a claim: that small, fine-tuned or reasoning models with structured output cover the core triage cases, provided the data is curated. It also records the agent-collaboration demo genre, where nearly every framework since late 2024 has shipped a triage or support-queue example. The question for every entry is the same: is the item's creation time or a response-time target an input? Small-model pipelines are where a time column costs least to add, so the absence is more telling here than anywhere else.
The vendor cookbooks and the larger frameworks are in the model-vendors area: 8.6.1, 8.6.2, 8.6.11, 8.6.9, 8.6.16, 8.6.17, 8.6.18, 8.6.23 and 8.6.24.
8.8.1. Strands Decider 2B (Strands Agents, AWS)
| field | value |
|---|---|
| item | any text passed as state; the launch example is a support message |
| metadata used | whatever the caller puts into state (text, optionally images); no typed fields |
| decisions | choice (one of N, e.g. team), noul (yes/no, e.g. "Does this convey urgency?"), score (ordered rubric) |
| time contract | none; the only time in the example is prose in the body ("failing for 3 days") |
| agent shape | decision model: Qwen3.5-2B-Base torso, language head replaced by a ~1M-parameter pointer head, rank-16 LoRA; no generation |
| human in the loop | repo guidance: answers at confidence 0.9 or more are right ~95% of the time; "below that, confirm or ask a person" |
| metrics claimed | repo-reported: JevBench public set 0.762 (176 of 231); easy tier 1.000; median 115 ms on RTX 3090, 153 ms on M3 Pro |
| status | open weights (Apache-2.0); blog 2026-10-01; repository created 2026-09-29, 522 stars on 2026-10-08 |
Strands Decider (Brooker, Chambers, and Nonato de Paula 2026) is AWS's entry in a class
its authors call decision models or "system one" models. The model cannot
write text. It scores the options the request supplies, so label sets live in
the request, not the weights, and every answer carries a probability. The
launch example is triage. --state "Help! My payouts have been failing for 3
days!" with --choice "Which team should handle this?=billing,sales,retail"
returns billing at 0.835 confidence, and the noul question "Does this
convey urgency?" returns 0.875 (Strands Labs 2026).
The model fills a triage field; it has no notion of when. There is no
timestamp, SLA or business-hours input. The three days of failure reach the
model only because the customer typed them. Because several questions about
one state cost little more than one ("the text is read once and each question
adds only its own tokens"), a caller could add "Has this ticket waited past
its first-response target?" as one more noul. The model would then be
reading a computed fact as text. The repository warns that the README's v21
and v19 columns are "two single runs, not a measured gain", and that a
difference under about 10 of 231 tasks is unresolved. That is the most careful
self-description of any model in this area. No ticket-domain evaluation is
published.
8.8.2. TypeSafe Jev and decision models in Pydantic AI and Agent Framework
| field | value |
|---|---|
| item | support ticket (worked example: a project-management app's support desk) |
| metadata used | the prompt is the state; output-type fields become questions; tool calls and results join the state |
| decisions | route (Triage, Refund, Reply or a tool), area, urgent, app, impact rubric; escalate to an LLM for any str field |
| time contract | none as data: a datetime field is a UserError; urgent criterion is "losing money or has a deadline today", judged from text |
| agent shape | Pydantic AI DecisionModel (TypeSafeModel, SystemOneModel) behind FallbackModel; Agent Framework TypeSafeChatClient |
| human in the loop | none in the example; thresholds route unsure answers to a language model |
| metrics claimed | none in the docs; per-field confidence in provider_details |
| status | Pydantic AI docs first commit 2026-09-18 (typesafe.md), 2026-09-24 (decision.md); Ollama runs decision models from v0.35.0 |
The Pydantic AI pages (Pydantic 2026a) are the
fullest public account of putting a small decision model in front of an LLM
for triage. The support-desk example maps each field of a Triage model to
one question. A route the decision model cannot fill, such as Reply with
its free-text field, raises UnfillableRoute and hands the whole step to the
language model. Microsoft's Agent Framework ships a matching connector whose
sample asks for department, frustration and is_urgent
(Microsoft 2026b). The same interface runs local models
through Ollama's /v1/systemone endpoint. The local example labels "Our
checkout has returned 500 errors since 9am." as bug.
This entry states the time boundary most plainly of any source in the survey.
A datetime is on the list of field types that fail before a request is
sent. TypeSafe's published limits, as quoted in the Pydantic AI docs, say Jev
"reads dates as text rather than as ordered quantities. Compute these in
Python and ask Jev about the result" (Pydantic 2026b). The
urgent field's BoolCriteria defines yes as "The customer is losing money
or has a deadline today": a deadline the customer states, read from prose.
The ticket's own age is not modelled. The design implies the split the
model-vendors area found in Anthropic's lifecycle sweep. Code computes the
clock, and the model judges text.
Both independent tests of this model class on tickets are negative. On the Tobi-Bueck queues, zero-shot Jev scored 0.5458 against a fine-tuned ModernBERT's 0.7294 and did not beat the 0.5946 majority-class baseline; see 8.8.6. Liquid's d1-3B, run locally on the same tickets, scored 0.5752 and did not beat it either (8.8.4).
8.8.3. Microsoft-Decision-1 (Microsoft Foundry)
every figure below is quoted from the vendor's announcement; none has been reproduced, and the incident-response claim has no number or method attached
OpenRouter's model page, read 2026-10-11: 33K context; one provider (Azure); p50 latency 0.23-0.25 s end to end over three days; "Weights are updated continually while the API shape stays the same"
| field | value |
|---|---|
| item | any text with a fixed answer set; the named internal uses include incident response, Xbox labelling and Copilot quality control |
| metadata used | whatever the request carries as text; no typed fields |
| decisions | yes/no, multiple choice, rating, rubric grading; routing, classification, verification, workflow control, UI automation |
| time contract | none stated |
| agent shape | decision model: Qwen3.5-9B "post trained … for fast, single-pass decision scoring", one calibrated probability per option; no generation |
| human in the loop | not described |
| metrics claimed | vendor-reported: highest accuracy on a 36-benchmark comparison of nearly 150,000 questions held blind from training; P50 2.5x faster than H2O-Lightning-4B v1.1 and 35x faster than GPT-6 Sol; decision changes on 1.3% of perturbations, none under paraphrased, reversed or shuffled options |
| status | announced 2026-10-09; API only (Foundry, OpenRouter, Vercel AI Gateway with TypeSafe-compatible and decision APIs); $0.042 per million input tokens, output free; 33K context; served by Azure alone; no weights, and those behind the API change without notice |
Microsoft's entry in the decision-model class (Srivastava 2026) is the first of the decision models covered here to name incident response as a use. The claim in full: "Our on-call engineers use AI to retrieve relevant knowledge to respond to live incidents across logs, ticketing systems, calls, messages, and other data sources. Microsoft-Decision-1 performed better and faster than an LLM for knowledge retrieval." No number, baseline model or method is given. The shape is still readable. Retrieval with a decision model is scoring: each candidate passage from a log, ticket or call is asked "is this relevant to the incident?", and the probabilities rank them. That puts the model where a reranker sits, beside the incident triage line Microsoft Research has run since 2017 (8.3.13), and it says nothing about routing the incident itself.
The robustness figure answers the weakness this area found in Jev. Zero-shot Jev lost to the majority class on the Tobi-Bueck queues and was right 65% of the time at 0.98 confidence (8.8.6). Microsoft reports a 1.3% flip rate under perturbation, a measure of stability, not of calibration, and no calibration error. The comparison that would settle it, Decision-1 zero-shot against the fine-tuned ModernBERT on the same held-out tickets, has not been published.
Size bears on where it can run. The base is a 9B model; the open model nearest
to it, autotrust/JEV-9B on the same Qwen3.5-9B base, is 18.4 GB at bf16 on
the Hub (autotrust 2026), more than the whole memory of a 16 GB
laptop; it would run there only quantised. With no weights released, Decision-1 runs only as a hosted API.
Two facts from OpenRouter's listing (OpenRouter 2026) bear on using it as an evaluation target. The weights "are updated continually while the API shape stays the same", so a score measured against it is a measurement of one week's model, and a regression can arrive with no version to pin. And its measured p50 through OpenRouter is 0.23 to 0.25 s end to end, about twice Decider's local median and two orders of magnitude above GitHub's encoder; the vendor's 35x is against a frontier LLM, not against a local classifier. In its first three days the listing shows about 469M prompt tokens against 1.85M completion tokens, the shape of a model that reads and does not write; one of the five public apps sending it the most traffic is a loan-event triage tool.
8.8.4. Liquid AI d1 and d1-omni
sizes, base models, licence field and dates from the Hugging Face Hub API on 2026-10-11 (verified)
the d1-3B model card, read 2026-10-11: licence lfm1.0, 32,768-token context, Decision Index 0.2.1 score and latencies are vendor-reported, scored with the official scorer and not a leaderboard submission
reproduced 2026-10-11: zero-shot d1-3B (Q4_K_M, llama.cpp b11554, M1 Pro) on OpsPilot's val split scored 0.5752 against ModernBERT's 0.7294 and Jev's 0.5458, and 0.5790 against 0.7398 on test; script and predictions in experiments/d1-opspilot-queues (verified)
| field | value |
|---|---|
| item | "shared state" as text or images |
| metadata used | the state; typed questions |
| decisions | classification, routing, scoring; visual classification and inspection |
| time contract | none stated |
| agent shape | d1-3B on LFM2.5-VL-3B (3.12B parameters, 6.25 GB; 1.67 GB as Q4_K_M GGUF); d1-omni-600M on LFM2.5-Encoder-350M (2.35 GB) |
| human in the loop | not described |
| metrics claimed | card-reported: Decision Index 0.2.1 48.57, "ahead of every 4B and 9B model" and of Decider 35B-A3B (47.11); one question 30 ms on an Apple M5 Pro, 8 ms on an RTX 4090; a 3.4k-token state 640 ms on the M5 Pro |
| status | Hub repositories created 2026-10-05; on Vercel AI Gateway 2026-10-09 (Vercel 2026); licence lfm1.0; 32,768-token context |
Liquid's d1 (Liquid AI 2026) is the decision model that fits on a
laptop: the 4-bit file is under 2 GB, and a GGUF build means llama.cpp-family
runtimes can load it. Its card names "routing and triage" and "reranking"
first among its uses, the two jobs this area and Microsoft's incident claim
are about, and its interface is the same three question types as Decider and
Jev (noul, choice, score). The latency table carries the caveat that
matters for tickets: one question costs 30 ms on an M5 Pro, but a 3.4k-token
state costs 640 ms, so a long ticket thread, not the number of questions, sets
the price. Several questions over one state are read in one pass (three
questions, 41 ms). d1-omni is the more interesting of the two for this
area. Its base is a 350M encoder, so it sits between OpsPilot's fine-tuned
149M ModernBERT and the 2B-to-9B decoder torsos of Decider, Jev and
Decision-1: an encoder that takes its labels in the request rather than in its
weights. Nobody has published its accuracy on tickets.
We ran d1-3B on the tickets OpsPilot used to test Jev: the 3,562-ticket validation split, the same seven-queue question copied word for word, the 4-bit file served by llama.cpp on a 16 GB M1 Pro. It did not close the gap.
| model | accuracy (95% CI) | macro-F1 | ECE | auto-routed at >= 0.80 |
|---|---|---|---|---|
| d1-3B Q4, zero-shot | 0.5752 (0.558-0.592) | 0.3148 | 0.2073 | 55.5%, 65.7% right |
| Jev, zero-shot | 0.5458 (0.530-0.562) | 0.3289 | 0.3143 | 70.0%, 61.2% right |
| ModernBERT, fine-tuned | 0.7294 (0.715-0.743) | 0.5584 | 0.0841 | 61.8%, 86.5% right |
d1 beats Jev by three points (exact McNemar p = 1.8e-6) and is better
calibrated, but it does not clear the 0.5946 majority-class baseline, and it
trails ModernBERT by fifteen points (p = 7.5e-74). The 3,563-ticket test
split agrees: 0.5790 against ModernBERT's 0.7398 and a 0.5947 baseline,
ECE 0.2010 against 0.0803. It sends 70% of tickets to
the technical queue; recall on returns is 0.03, on human resources 0.09, and
on the merged customer/general queue 0.13. The small queues and the merged
one are where both zero-shot models fail, which is the curation effect
OpsPilot reported. The median ticket took 750 ms at 515 input tokens; the
card's 30 ms is one short question on newer hardware, and ticket length, not
the question, set the cost here. ECE is computed one way for all three rows,
on the top-1 probability, so Jev's figure differs from OpsPilot's 0.2896;
d1's own confidence field is the top-two margin, not a probability. This is one quantisation and one wording;
the card's figures are for bf16. The scripts and predictions are in
experiments/d1-opspilot-queues in this site's repository.
The class is now large enough to need a filter. On 2026-10-09 the Hub gave
decision models a decision-model tag, applied to GGUF uploads automatically
from their llama.cpp metadata (Hugging Face 2026). Two days
later the tag listed at least 1,000 repositories, 341 of them GGUF, most of
them quantisations by third parties. The most downloaded were TypeSafe's
JEV-27B-VL (1.54M) and GEV-26B-Decide, JEV-9B, a community decider-2b,
and GLiNER2.5-Decide, which, like d1-omni, puts the decision interface on an
encoder. No Microsoft repository carries the tag.
8.8.5. Strands Agents samples: triage-agent UX demo
| field | value |
|---|---|
| item | patient conversation (medical triage), not a ticket |
| metadata used | answers to decision-tree questions; calendar and weather through MCP servers |
| decisions | navigate a JSON decision tree, recommend, schedule an appointment |
| time contract | none; calendar slots are scheduling data |
| agent shape | single Strands agent on Bedrock (Claude or Nova) with MCP tools |
| human in the loop | proof-of-concept disclaimer; no clinician step |
| metrics claimed | none |
| status | demo; first commit 2026-03-27, last change 2026-09-01 |
The only sample in strands-agents/samples named "triage"
(Strands Agents 2026) is clinical. The
industry folder has an SRE incident-response agent and a Jira scrum-master
assistant, which belong with the incident and code-host areas. No Strands
sample triages a support queue. The Decider is the Strands project's actual
answer to ticket routing.
8.8.6. OpsPilot: ModernBERT routing on the Tobi-Bueck tickets
| field | value |
|---|---|
| item | support email (23,747 English tickets from Tobi-Bueck/customer-support-tickets, deduplicated) |
| metadata used | subject and body; reference_answer excluded as post-resolution leakage |
| decisions | queue (7 classes after merging), priority, escalation risk |
| time contract | none; README: priority "depends on SLA, customer tier and business impact, none of which are in the ticket text" |
| agent shape | fine-tuned ModernBERT-base plus a rules layer (escalation keywords, policy categories, confidence thresholds) |
| human in the loop | category confidence below 0.65 or escalation risk at or above 0.80 goes to human_review; legal terms override the model |
| metrics claimed | author-reported, held-out test n=3,563: accuracy 0.7398 (95% CI 0.7258–0.7544), macro-F1 0.5759, p50 56.7 ms; priority 0.5240 |
| status | portfolio project; created 2026-05-04, last push 2026-09-22 |
OpsPilot (Joshi 2026) is the strongest evidence found for and against the user's claim, from one author on one dataset. For: a 149M-parameter encoder routes queues at 74% with calibrated confidence. At a 0.80 threshold it auto-routes 62.5% of tickets at 85.0% accuracy, in 57 ms. Against: the largest gain in the table came from curation. Accuracy went from 0.5229 to 0.7313 when four overlapping queues were merged into two (10 to 7 classes). The author writes that "queue design mattered as much as architecture", and flags the 10-class split as unrecorded. Macro-F1 lags accuracy by 16 points because the majority queue carries the score.
The time contract is named as the missing input. Priority prediction scores
0.5240, and the README attributes it to the data: SLA and tier are
not in the text. The dataset has no timestamp column, so the fix the author
proposes (add ticket_type, tags, channel) cannot add age either. The same
repository holds the paired comparison against Jev described above, on the
3,562-ticket validation split (ModernBERT's 0.7294 is its validation score;
0.7398 is test): McNemar p = 5.5e-91, and Jev's expected calibration error is
0.2896 against 0.0841. Where Jev reported 0.98 confidence it was right 65% of
the time. Its saved predictions carry ticket ids, which is what let us score
d1-3B on the identical tickets.
8.8.7. GitHub secret protection: ModernBERT at push time
figures quoted from GitHub's post of 2026-10-07; no precision or recall is published
| field | value |
|---|---|
| item | candidate secrets in a push, with the code around them |
| metadata used | the candidate string and its code context |
| decisions | is this a secret: generic and unstructured credentials that pattern rules miss |
| time contract | the push is the deadline: the decision blocks it or not, before the secret enters history |
| agent shape | fine-tuned ModernBERT classifier, built with Microsoft Applied Sciences, in front of push protection |
| human in the loop | the developer whose push is blocked |
| metrics claimed | vendor-reported: candidate batches in under 2 ms; "more than double" the secrets prevented; no precision or recall |
| status | post 2026-10-07; private preview, then Enterprise Cloud and Teams later in October 2026; public preview in GHES 3.23 |
Not ticket triage, but the strongest production case for the fine-tuned encoder route in this area (GitHub 2026c). GitHub found LLM pipelines less precise for this decision. The labels are fixed, the decision runs on every push (a blocked secret "at least once every second"), and the budget is milliseconds. Those are the conditions under which OpsPilot's ModernBERT beat zero-shot Jev. A decision model asks for a different trade: labels per request, at Decider's median of 115 ms.
8.8.8. bilingual-ticket-triage: Qwen2.5-3B QLoRA against Llama 3.3 70B
| field | value |
|---|---|
| item | support email or web-form ticket in Roman Urdu, Urdu script, English or mixed |
| metadata used | ticket text only |
| decisions | category (8), urgency, sentiment, entities (order id, product, email); grounded reply draft |
| time contract | none; "bohot late ho raha hai" ("it is getting very late") is the urgency signal, in text |
| agent shape | QLoRA adapter (r=8) on Qwen2.5-3B-Instruct, one Pydantic schema shared by training, eval, API and UI |
| human in the loop | review-first inbox: a person approves, corrects or discards every draft |
| metrics claimed | author-reported, 148 held-out synthetic tickets: category 80.4% vs 63.5% (Llama 3.3 70B), urgency 66.2% vs 71.6% |
| status | repository created 2026-08-03; adapter and GGUF on the Hub; DPO "decision" adapter pushed 2026-10-06 with an empty card |
This is the one published three-arm comparison found between a fine-tuned small model and a large prompted model on ticket triage (Khan 2026). Fine-tuning on 1,414 synthetic tickets lifts category accuracy from 25.0% (zero-shot 3B) to 80.4%. That is 16.9 points above Llama 3.3 70B, with 100% valid JSON against 99.3%. The result splits by field. The 70B model is better on urgency (71.6% against 66.2%) and on sentiment (87.2% against 86.5%). Category is a filing convention the fine-tune learns from labels. Urgency is a judgement about consequences, where the larger model's general knowledge still wins.
The caveats are the author's own: the data is fully synthetic, the test set is drawn from the same generator, and the model card says ~155 test tickets where the README says 148. None of the 8 categories or the urgency field uses a timestamp.
8.8.9. Vietnamese ticket-triage LoRA cohort (AICB Day 21)
| field | value |
|---|---|
| item | Vietnamese customer-support ticket (synthetic) |
| metadata used | text only |
| decisions | intent, urgency, product, sentiment as a 4-field JSON record |
| time contract | none |
| agent shape | LoRA SFT (r=16) on Qwen3.5-4B; 225 training tickets, 30 steps on a T4 |
| human in the loop | none |
| metrics claimed | card-reported, 50 tickets: field accuracy 0.970 vs 0.765 for a hand-prompted base; general-knowledge regression 0.758 to 0.678 |
| status | course exercise; at least five student adapters pushed 2026-08-21, another 2026-10-07 |
A course assignment ("AICB-P2T3 Day 21, Track 3") produced a cluster of
*-vi-ticket-triage-lora adapters on the Hub in one day. The best-documented
one (licht666 2026) publishes the number most cards
omit. It gains 0.205 field accuracy over a properly prompted base and fails
its own regression gate. General knowledge drops 0.080 against a 0.020
tolerance, because all 225 training examples map ticket to JSON. The card
calls this "textbook catastrophic forgetting". It also reports latency of
1,331 ms for the adapter against 991 ms for the prompted base.
That a ticket-triage fine-tune is now a day-21 exercise says the technique is
commodity. None of the cohort's schemas include time.
8.8.10. Other Hub fine-tunes for ticket triage
| field | value |
|---|---|
| item | support ticket, IT ticket, tenant maintenance request |
| metadata used | text only in every card read |
| decisions | category, urgency, sentiment, action_required (auto_reply / escalate / close); maintenance category, vendor, entry needed |
| time contract | none; "leaking from yesterday morning" is prose in the example input |
| agent shape | QLoRA on Qwen2.5-1.5B and Qwen3-2B emitting JSON; ModernBERT-base prototype classifier; many DistilBERT and BERT heads |
| human in the loop | none in the cards |
| metrics claimed | card-reported: 91.3% exact match, 95.6% schema compliance (Qwen2.5-1.5B); none for most |
| status | individual uploads, 2024-05 to 2026-10; most-downloaded adaptive-classifier/support-ticket, 735 downloads |
The Hub search for "ticket" returns dozens of fine-tunes, nearly all
single-author and created in 2026. Three show the range. The
qwen2.5-1.5b-ticket-classifier card (Singh 2026)
reports 0% schema compliance for the base model and 95.6% after LoRA. Its
cost table ("<$25 / month" against "$750 - $1,500" for a GPT-4o class API) is
an estimate; nothing was measured. ticket-triage-qwen3-2b
(dvr76 2026) extracts maintenance fields from tenant
messages. Its example input carries its own clock ("from yesterday
morning"). The output schema has urgency but no time field.
adaptive-classifier/support-ticket (Sharma 2025) is a
ModernBERT prototype classifier trained on 20 examples (5 per class). It is
the smallest credible instance of the few-shot route, but publishes no
accuracy. No SetFit ticket model with a published evaluation was found; one
SetFit customer-support classifier exists with an empty card.
8.8.11. GPT-5 distilled into 8B students for ServiceNow ticket routing (CBS thesis)
| field | value |
|---|---|
| item | IT support ticket (26,331 tickets, Nuuday YouSee DAWN platform on ServiceNow) |
| metadata used | ticket text; three-level ServiceNow taxonomy |
| decisions | classify to assignment taxonomy (L1, L2, L3) |
| time contract | none |
| agent shape | GPT-5 teacher writes silver labels and rationales with confidence-informed self-consistency; QLoRA students Llama 3.1 8B, Ministral 8B |
| human in the loop | none at labelling (the point of the design) |
| metrics claimed | author-reported: 84.14% / 84.30% full-label agreement with the teacher; L1 97%, L2 95%; macro-F1 ~64–66%; cost per ticket $0.00015 vs $0.0018 |
| status | master's thesis, Copenhagen Business School, 2026; repository created 2026-02-02 |
This is the distillation route on real enterprise tickets (staerkjoe 2026). The problem it names is taxonomy cold start: a newly introduced ServiceNow taxonomy with no labelled history. A frontier model labels; small models learn the labels. The headline number is agreement with the teacher, not accuracy against people, so the students inherit any teacher error. The 84% figure is fidelity to GPT-5, and the thesis reports no human gold set in the summary read. Macro-F1 near 65% again shows the long tail of small queues. Zero-shot students score about 0% on the proprietary taxonomy, which is the clearest statement in the area that the labels, not the model, carry the knowledge.
8.8.12. TickIt: LLM ticket escalation at ByteDance Volcano Engine
| field | value |
|---|---|
| item | customer support ticket on a public cloud (over 20,000 tickets analysed) |
| metadata used | ticket messages, updated ticket state, related tickets |
| decisions | escalate or not, route to analyst group by topic, deduplicate escalations |
| time contract | SLAs and MTTR named as on-call engineers' goals; whether ticket age is a model input is unconfirmed |
| agent shape | Doubao-Pro with chain of thought, in-context examples and reflection; category-guided SFT; Qwen2.5 and Llama 3.1 compared |
| human in the loop | analyst feedback defines ground truth |
| metrics claimed | paper-reported F1: embedding + logistic regression 0.765, + MLP 0.801; LLM CoT 0.819; SFT + CoT 0.862 |
| status | arXiv 2025-04-11; deployed at Volcano Engine per the authors |
TickIt (Liu et al. 2025) is the industrial counterpoint to the small-model results. On a real escalation task, embedding classifiers trail an LLM with chain of thought by 2 to 5 F1 points, and fine-tuning the LLM on its own corrected reasoning adds 4 more. The decision here is escalation, which depends on the ticket's state as it evolves, not on its first message. That is closer to the time contract than queue routing is. The paper motivates escalation with SLAs and MTTR. The pages read do not show elapsed time as a feature, so its role is unconfirmed.
8.8.13. Hosted fine-tuning and distillation recipes
| field | value |
|---|---|
| item | generic; Slemify's worked example is Kubernetes support queries |
| metadata used | text only |
| decisions | classification, routing, scoring, extraction |
| time contract | none |
| agent shape | teacher-student distillation within a model family (Bedrock); encoder head trained on Bedrock-generated pairs (Slemify) |
| human in the loop | Slemify: "you verify them before training" |
| metrics claimed | vendor-reported (Bedrock): up to 5x faster, up to 75% cheaper, under 2% accuracy loss "for use cases like RAG"; Slemify: SLM handles "70-90% of requests" |
| status | Bedrock Model Distillation preview 2024-12-03, GA May 2025; Slemify sample created 2026-05-13 |
The hyperscalers sell distillation as a managed step, but none of the pages read uses support tickets as the worked example. Bedrock Model Distillation (Amazon Web Services 2024e) requires teacher and student from the same family. Its accuracy claim names RAG as the use case. AWS's Slemify sample (Amazon Web Services 2026a) is closer to triage. One YAML file declares the labels. Bedrock writes 1,200 synthetic pairs, which a person checks. A frozen encoder with a logistic head trains on CPU, and the report compares held-out accuracy against a majority-class baseline. Its diagram is the cascade pattern: confident answers from the small model, the rest to an LLM. AWS's own ticket versions are in the cloud area: 8.4.11 and 8.4.12. OpenAI's distillation and fine-tuning guides returned 403 to the fetcher and are unconfirmed here beyond the cookbook in the model-vendors area. No Vertex AI ticket-tuning recipe was found.
8.8.14. Literature: fine-tuned small models against prompted LLMs
| field | value |
|---|---|
| item | text classification generally; support and ITSM tickets in four papers |
| metadata used | text; COTA (2018) also used structured ticket features |
| decisions | classify, route, rank answers |
| time contract | none in any paper read; COTA's motivation is "speed" of support |
| agent shape | fine-tuned BERT-class encoders, SetFit, AutoML, centroid classifiers, against GPT-3.5/4 and Claude Opus zero-shot |
| human in the loop | none |
| metrics claimed | paper-reported: fine-tuned small models beat zero-shot LLMs "in all cases" (Bucher and Martini); ITSM hierarchical F1 0.731 vs SVM 0.727 |
| status | papers 2018–2025 |
The general result predates the agent wave and has not reversed. Bucher and Martini (2024) compared ChatGPT and Claude Opus zero-shot with fine-tuned encoders across sentiment, stance and topic tasks. Fine-tuning won in every case (Bucher and Martini 2024). None of their tasks are tickets. SetFit (2022) is the few-shot route: contrastive fine-tuning of a sentence encoder, then a classification head, with no prompts (Tunstall et al. 2022). ModernBERT (2024) is the encoder most 2026 ticket projects start from (Warner et al. 2024).
On tickets specifically, the ITSM papers are sober. A centroid classifier on 8,968 tickets across 123 categories matches an SVM (hierarchical F1 0.731 against 0.727) and updates 152 times faster (Mohanna and Ait-Bachir 2025). It cites real-world BERT and XLNet F1 of 38–55% on IT tickets, and a negative correlation between model complexity and F1 below 10,000 examples. AutoML gives usable support-ticket classifiers without specialists (Truss and Boehm 2024). Uber's COTA (2018) already ranked thousands of ticket classes and suggested answers to agents, with "speed" in its title as the goal rather than as an input (Molino, Zheng, and Wang 2018).
8.8.15. Constrained decoding for triage fields: Outlines, Instructor, Claude and OpenAI structured outputs
| field | value |
|---|---|
| item | customer email (Outlines); meeting transcript to tickets (Instructor); generic feedback (Claude docs) |
| metadata used | text only |
| decisions | priority enum (low/medium/high/urgent), category, requires_manager, summary, action items; assignees (Instructor) |
| time contract | none in any schema; Outlines' example email says "paid … 3 hours ago" and "a client presentation in an hour" |
| agent shape | grammar-constrained generation over a Pydantic model; Outlines runs Phi-3-mini locally |
| human in the loop | none; Outlines routes urgent or requires_manager to an alert_manager function |
| metrics claimed | none |
| status | Outlines README example (date unconfirmed); Instructor auto-ticketer first commit 2023-08-28; Claude structured outputs GA |
Every structured-output library uses a support ticket as a showcase, and
every one models priority as an enum inferred from text. The Outlines
"Customer Support Triage" example (dottxt n.d.) is typical. A
3.8B-parameter Phi-3-mini fills ServiceTicket(priority, category,
requires_manager, summary, action_items) under a grammar, so the output always
parses. The deadline in the email (a presentation in an hour) is exactly the
time contract. It reaches the model as prose and leaves as the token urgent.
Instructor's auto-ticketer (567 Labs 2023) turns a
meeting transcript into tickets with PriorityEnum and assignees. It
creates tickets rather than triaging them.
The hosted APIs now constrain decoding natively. Claude's structured outputs
use "constrained sampling with compiled grammar artifacts"
(Anthropic n.d.-e). The supported JSON Schema features
include enum and the date-time string format, but not minimum or
maximum. A schema can therefore require a timestamp field, while a
confidence or hours-remaining number cannot be range-checked by the grammar.
Constrained decoding guarantees a well-formed priority. It says nothing about
whether the priority accounts for time. Guidance has no ticket example;
OpenAI's structured outputs and Agents SDK handoffs are in the model-vendors
area.
8.8.16. Reasoning models and chain of thought for classification
| field | value |
|---|---|
| item | classification tasks generally; tickets in TickIt and the Anthropic cookbook |
| metadata used | text |
| decisions | classify |
| time contract | none |
| agent shape | chain of thought or reasoning models against direct answers; format-restricted against free generation |
| human in the loop | none |
| metrics claimed | paper-reported: CoT gains are concentrated in math and symbolic tasks; CoT cut o1-preview accuracy by up to 36.3 points on some tasks |
| status | papers 2024; replication blog 2024 |
The evidence on whether reasoning helps triage is mixed, and none of it measures the time contract. A meta-analysis of over 100 papers and 20 datasets found chain of thought helps mainly on math and logic, with "much smaller gains on other types of tasks" (Sprague et al. 2024). Deliberation can hurt: on tasks where thinking makes humans worse, o1-preview lost up to 36.3 points against GPT-4o (Liu et al. 2024). On tickets, two results point the other way. Anthropic's classification cookbook reports chain of thought as its best step, and TickIt's best configuration is SFT plus chain of thought (F1 0.862 against 0.819 for chain of thought alone).
On format, "Let Me Speak Freely?" reported that JSON constraints degrade reasoning (Tam et al. 2024). dottxt's replication attributes the drop to prompt differences, and notes the paper itself found structured generation better on several classification tasks (Kurt 2024). Latency is the cost side. The Vietnamese LoRA card measures 991–2,946 ms per ticket for a 4B generator. The Decider and OpsPilot report 57–153 ms for non-generating models. A reasoning model adds thinking tokens on top of the generator's figure. For a first-response clock measured in hours, either latency is negligible. For an interactive router it is not.
8.8.17. Public ticket datasets: which ones carry a clock
| field | value |
|---|---|
| item | support email, chatbot utterance, IT helpdesk ticket, Jira-style help-desk issue |
| metadata used | varies; see the paragraphs below |
| decisions | labels offered: queue, priority, type, tags (Tobi-Bueck); intent and category (Bitext); priority and category (Console-AI) |
| time contract | absent from the most-used sets; present in Mendeley help-desk tickets (created, resolved, workflow time, holidays) and two synthetic sets |
| agent shape | training and evaluation data |
| human in the loop | n/a |
| metrics claimed | n/a |
| status | Tobi-Bueck 2025-02-06 (5,212 downloads); Bitext 2023-08-24 (10,697); Mendeley v3 mirror 2026-09-23 |
The datasets most small-model projects train on have no time column. That is
the strongest finding in this area for the time contract.
Tobi-Bueck/customer-support-tickets (Bück and Softoft 2025), the
most-downloaded ticket set on the Hub and the source for OpsPilot, has 61,765
rows. Its columns are subject, body, answer, queue, priority, type, language,
version and eight tags, with no timestamp. Its card promotes a synthetic
ticket generator; whether these rows are synthetic is not stated, so
unconfirmed. Bitext's 26,872 utterances (Bitext 2023) are "hybrid
synthetic" and mark dates as slot placeholders ({{Date}},
{{Shipping Cut-off Time}}): time is a template variable with no value.
The clock existed and was dropped in one lineage. Kaggle's "Customer Support
Ticket Dataset" (2023) lists First Response Time, Time to Resolution,
Ticket Channel and Date of Purchase among 17 fields
(suraj520 2023). A Hub derivative with the same 8,469 rows keeps
six columns: email, product, type, subject, combined text and priority
(gorkemsevinc 2023). The derivation is inferred from row count and
column names. Every time field went in the conversion to a classification
set.
Three sets keep time. The Mendeley "Help Desk Tickets" data
(Abdellatif 2023) covers January 2007 to March 2023 at one
software company: 66,691 issues with creation and resolution timestamps,
total workflow time, and processing steps. Version 3 adds
holidays_by_country.csv, the only business-calendar input found in any
public ticket set. It defines no classification target. The synthetic
KameronB/synthetic-it-callcenter-tickets (27,602 rows) has date,
resolved_at, resolution_time and reassigned_count
(KameronB 2025). Console-AI/IT-helpdesk-synthetic-tickets
has createdAt, but its 500 rows were stamped within seconds of each other
on 2024-10-28 (Console AI 2024). A generation timestamp that looks
like a creation time is a trap for any model that learns from it.
8.8.18. Label noise, taxonomy drift and correction loops
| field | value |
|---|---|
| item | historical support tickets and their human-assigned labels |
| metadata used | SIA: text, channel, priority, resolution time; ModelForge: synthetic IAM tickets |
| decisions | detect priority mislabels ("Hidden Crisis", "False Alarm"); measure sensitivity to label noise; add categories without retraining |
| time contract | SIA uses normalised resolution time as one of three pseudo-label signals (weight 0.2); offline, after resolution |
| agent shape | keyword scores + resolution time + MiniLM embeddings make pseudo-labels; DistilBERT learns them (SIA); TF-IDF noise ablation (ModelForge) |
| human in the loop | 2508.00957: users revise category definitions in natural language; Dosu: maintainer corrections become few-shot examples |
| metrics claimed | author-reported: 28.9% of 20,000 tickets flagged as priority mismatches (SIA); 30% injected label noise cut test F1 from 1.000 to 0.960 (ModelForge) |
| status | SIA created 2026-06-09; ModelForge 2026-08-20; arXiv 2508.00957 2025-08-01 |
The Support Integrity Auditor (arshit-0101 2026) is the one small-model pipeline found that uses time as a feature. It uses it to audit labels after resolution. Normalised resolution time is one of three signals that decide whether a ticket's recorded priority was wrong. Across 20,000 tickets (source unconfirmed), 28.9% were flagged. If that rate holds anywhere near real data, priority is the noisiest label a triage model learns from. A model trained on it learns the desk's habits, including its mistakes.
The other curation evidence points the same way. OpsPilot's 10-to-7 queue merge outweighed its architecture change, and its roadmap's first item is hand-labelling 50 disagreements to test whether "the 73% partly measures memorized convention". ModelForge (rubiatExe 2026) measures label noise directly but only on template-generated tickets, where 30% noise costs 4 points. Its frontier comparison was never run ("Requires paid API credentials"). For drift, the description-refinement method in arXiv 2508.00957 lets users add or redefine ticket categories in prose and reports small accuracy shifts on unseen classes, measured on AGNews and DBpedia rather than tickets (Rajeev et al. 2025). The centroid classifier's 152x faster incremental update is the same need met statistically. The production correction loop is Dosu's, in the code-host area: 8.5.12.
8.8.19. Synthetic ticket generation for evals
| field | value |
|---|---|
| item | generated support ticket |
| metadata used | sampled label spec (company, style, language), then text |
| decisions | category, priority, sentiment, needs_human, suggested action, reason |
| time contract | none generated |
| agent shape | gpt-4o-mini writes 1,680 tickets from labels sampled first; Qwen2.5-0.5B LoRA to be compared with the base and gpt-4o-mini |
| human in the loop | a reviewer checked 50 test labels and confirmed 41 (82%) |
| metrics claimed | dataset only; training and evaluation not yet run |
| status | work in progress; repository created 2026-10-05 |
Synthetic tickets are the default eval set in this area. Almost every fine-tune above trains and tests on generated data. This repository (Mehmood 2026) is worth recording for two disclosures. First, labels are sampled before the ticket is written, and a human still rejected 9 of 50, an 18% label error in a generator designed to avoid it. Second, the generator is also one of the compared models, so "its phrasing is built into the data". The AWS cascade sample, the Bilingual adapter, the Vietnamese cohort and cinque-ports in our own projects all evaluate on corpora they generated. None of the generators emit a creation time, a business-hours window or a customer tier. The Nemotron cascade's tier field is the exception, and that sample forbids it from changing priority. A synthetic corpus could cheaply include the clock. None found does.
8.8.20. Board and work-management agents: Asana, monday.com, Notion, Trello, GitHub Projects
| field | value |
|---|---|
| item | work request, task card, service ticket |
| metadata used | unconfirmed beyond marketing pages; monday service: "knowledge base …, past ticket history, the business rules" |
| decisions | intake, categorise, route to team or owner, prioritise; monday: resolve or escalate with summary and suggested reply |
| time contract | Asana AI Studio lists "apply SLA rules" next to categorise and score; mechanism unconfirmed; none stated for monday or Notion |
| agent shape | no-code AI workflow builders and agent "teammates" inside the board |
| human in the loop | escalation to "the right person" (monday); approval steps configurable (Asana); details unconfirmed |
| metrics claimed | vendor-reported: 18,000 annual assignments automated (Asana case); "98% of tickets auto-triaged" (monday, ZopaBank) |
| status | Asana AI Studio GA, launch date unconfirmed; Notion Custom Agents 2026-02-24; Trello unconfirmed |
The board vendors put triage at the front of their agent pitches. Asana AI Studio's page lists "Prioritize and triage every request" and "Categorize, normalize, score, and apply SLA rules" (Asana n.d.). It is the only work-management product found that puts SLA rules in the same sentence as AI scoring. The page does not say whether the rule is an input to the model or a step after it. monday service advertises "Smart intake & triage" with "automatic categorization, summarization, and routing" (monday.com n.d.). Notion's Custom Agents (Notion 2026) name "Task Routing Agents" and a co-founder's agent that routes product feedback. Neither page mentions a deadline or an SLA.
Trello's AI pages did not load, so its triage features are unconfirmed. GitHub Projects has no triage agent of its own. Issue fields and Copilot assignment are in the code-host area. This is the genre the user describes: staff engineers demonstrating agents on a Kanban board. The board supplies the visual queue and due dates. The demos found route by content. The due date is a column on the board. None of these pages says the agent reads it.
8.8.21. Framework triage samples: Agent Framework, Google ADK, Pydantic AI, Mastra, smolagents
| field | value |
|---|---|
| item | IT support chat (Agent Framework); GitHub issue (ADK samples repo); refund request and security alert (Mastra) |
| metadata used | Agent Framework TicketItem: id, subject, description, notes, status; ADK triage: number, title, body; Mastra: order and policy records, alert JSON |
| decisions | self-serve, create ticket, route to one of 7 teams, resolve, escalate by email (AF); label issues (ADK); refund proposal, containment plan (Mastra) |
| time contract | none; no sample's ticket model has a timestamp; Mastra's refund example states "five days ago" in the customer's message |
| agent shape | AF: declarative YAML workflow over six agents with Pydantic response_format; ADK: one model call returning JSON, validated in CI; Mastra: durable workflow with approval suspend |
| human in the loop | AF: escalation agent gathers an email for human follow-up; Mastra: refunds and containment wait for approval |
| metrics claimed | none |
| status | AF sample first commit 2026-02-12; ADK triage workflow 2026-08-13; Mastra templates in repository since 2026-09-17 |
Microsoft's Agent Framework, the successor that AutoGen's README points to,
ships a six-agent CustomerSupport workflow
(Microsoft 2026a). A TicketRoutingAgent picks
one of seven teams through a RoutingResponse(TeamName: str) schema. The
mock TicketItem has five fields and no time; send_notification is the
escalation. Semantic Kernel's repository has no triage sample. The
Foundry training lab built on these pieces is in the cloud area:
8.4.19.
Google's adk-samples repository triages its own issues with
gemini-3.5-flash through the Antigravity CLI. The design note says "the
model only DECIDES which labels apply" and has no GitHub token; CI validates
the JSON against the real label set (Google 2026). The
input is {number, title, body}, the same field list without createdAt
that the model-vendors area found in gemini-cli. Mastra's
customer-refund and security-incident-triage templates
(Mastra 2026) are the most finished demos in this group.
Both have a support dashboard, policy retrieval, and durable suspension at
an approval step. Neither carries a response clock. smolagents has no triage or
support sample. Pydantic AI's support desk is under
8.8.2.
8.8.22. Zello
| field | value |
|---|---|
| item | push-to-talk voice message |
| metadata used | n/a |
| decisions | none triage-related; AI answers questions, transcribes, translates |
| time contract | none |
| agent shape | "Zello AI (Ella)" knowledge assistant; "operational intelligence" from conversations |
| human in the loop | n/a |
| metrics claimed | none |
| status | product live 2026-10-08 |
Zello is a push-to-talk walkie-talkie app for frontline teams. Its AI features are a knowledge assistant, transcription, live translation, and "operational intelligence" mined from radio chatter (Zello n.d.). It has dispatch tooling but no ticket or queue triage. The name was probably misremembered. Given the Kanban framing, Trello is the likeliest intended product. Zendesk and Zoho Desk are covered in the CRM area.
8.8.23. Area synthesis
The claim under test was that small, fine-tuned or reasoning models with structured output cover the core triage cases when the data is curated. The evidence splits it in two.
Category and queue routing: supported, with curation doing most of the work. A fine-tuned 3B model beat a prompted 70B model by 16.9 points on category. A 4B LoRA beat a hand-prompted base by 20.5 points. Two 8B students reached 84% agreement with GPT-5 on a real ServiceNow taxonomy at a twelfth of the cost. ModernBERT routes 62.5% of tickets at 85% accuracy in 57 ms. In each case the gain came from labels the model could not infer. OpsPilot's 10-to-7 queue merge moved accuracy by 21 points. Zero-shot students scored about 0% on a proprietary taxonomy. A zero-shot decision model lost to the majority-class baseline; the vendors' answer so far is stability figures (Microsoft-Decision-1: 1.3% of perturbations flip), not a ticket benchmark. The caveat is the evidence base. Nearly every result is single-author, single-run, and on synthetic data scored against labels from the same generator. One human check of such labels found 18% wrong. The only industrial paper, TickIt, found the LLM ahead of embedding classifiers by 2 to 5 F1 points, and further ahead after fine-tuning.
Priority and urgency: not supported, and the cause is missing inputs. OpsPilot's priority accuracy is 0.524, and its author names SLA and customer tier as the absent features. In the three-arm study the 70B model beat the fine-tune on urgency while losing on category. The Support Integrity Auditor flagged 28.9% of recorded priorities as inconsistent with how long tickets actually took. Priority is the field the time contract governs. It is the noisiest label and the one text alone cannot predict.
Where the clock is. In this area time reaches the model only as prose the
customer wrote: "failing for 3 days", "since 9am", "3 hours ago", "a client
presentation in an hour", "from yesterday morning", "it is getting very
late". No small-model pipeline found passes ticket age, time to SLA breach, or
business hours as a column. The decision-model interface makes the boundary
explicit. A datetime field is refused, and the vendor advice is to compute
dates in code and ask about the result. The datasets enforce the same
boundary from below. The most-used sets have no timestamp, and one apparent
derivative lacks the first-response and resolution times its Kaggle source
lists. Only the Mendeley help-desk data carries creation,
resolution and holiday calendars, and it defines no label. The one pipeline
that does use time, the Support Integrity Auditor, uses resolution time after
the fact to audit labels.
The demo genre. Board vendors (Asana, monday.com, Notion) and frameworks (Agent Framework, ADK, Mastra, Strands, Pydantic AI) all ship triage as the first example. Asana alone puts "apply SLA rules" next to AI scoring. The mechanism is unconfirmed. Every framework sample's ticket model lacks a timestamp, and the human step in each is approval of an action (a refund, a containment), not a check on the response deadline.
Open questions for the synthesis pass:
- Does adding
hours_open,hours_to_breachandcustomer_tieras columns lift priority accuracy for a small classifier? The Mendeley data has the timestamps and holiday calendar to compute them; nobody has run it. - Does a decision model asked "Has this ticket passed its first-response target?" over a computed fact add anything over the same comparison in code? The vendor guidance implies not.
- How much of the small-model category advantage survives a human-labelled test set? Only the CBS thesis and TickIt use real tickets, and neither compares against a frontier model on human gold.
- Microsoft-Decision-1 was "better and faster than an LLM" for incident knowledge retrieval. Against which LLM, on what measure, and does the same hold for routing the incident rather than retrieving for it?
- Asana AI Studio's "apply SLA rules" needs a primary source: is the SLA a model input, a post-classification rule, or a field the workflow sets?
9. Open conjectures
- Data-first tools generalize. The
tool-defs+handle-toolshape is domain-agnostic; the same two-function backend should wrap logs, the search index, or the verdict ledger with no new machinery. Refutation: a domain whose tool surface can't be expressed as EDN + acase. - Line-preserving writes keep an agent's mutations reviewable. If every
agent-driven
set-status!is a one-line diff, a day of autonomous triage is a readable patch series, not an opaque state jump. Refutation: a mutation that necessarily rewrites unrelated lines. - The REPL beats a visualize module. For a store small enough to hold in a
process,
group-byat the prompt is faster to author and adapt than any rendering layer. Refutation: an analysis whose ergonomics genuinely need a dedicated view (large-scale, streaming, or shared dashboards).
10. Claims to test  refutation
- The tool backend is fully data (no code regen to change tools)
- a required
tool capability that can't be added by editing
tool-defs+handle-tool. - Every tool call returns a value, never throws
- an input that escapes the
case/ error-object contract and propagates an exception into the loop. - Configuration and analysis are one surface
- a triage task that forces a separate read-model and write-model rather than one store.
11. Resources
all 8 beads fns referenced exist; 5/5 internal + 2/2 external links resolve (citation-auditor).
- Store + tools
src/wal_sh/site/beads.clj(read-issues,ready,ready-queue,next-ticket,set-status!,queue-backend,tool-defs,handle-tool).- Harness
src/wal_sh/site/agent.clj· run:gmake ticket-agent/gmake tickets.- Empirical companion
- Compaction Against a Beads Backend.
- The series
- Feed Crawling · Flight Tracking · Chat Mutation.
- Upstream baseline
- claude-cookbooks/tool_use/utils.
- Tracker
- beads (git-native issues).