wal.sh
home > research > llm-cost-monitoring · Jun 19, 2026

LLM Cost Monitoring and Local-First Harness

llmcostmonitoringollamaclaude-codegcloudlocal-firsttoken-economics

LLM Cost Monitoring and Local-First Harness

Table of Contents

1. Overview

Every tool call in an agentic workflow costs tokens.

2. GCloud AI Dev Tools pricing (June 2026)

needs-citation · pilot-annotation-pass · 2026-10-08

Per-token pricing for Claude models via Google Cloud:

Model Input Output Cache Read Cache Write Long Input Long Output
Opus 4.6 $5.00/MTok $25.00/MTok $0.50/MTok $6.25/MTok $10.00/MTok $37.50/MTok
Sonnet 4.6 $3.00/MTok $15.00/MTok $0.30/MTok $3.75/MTok $6.00/MTok $22.50/MTok
Opus 4.5 $5.00/MTok $25.00/MTok $0.50/MTok $6.25/MTok - -

SKU IDs for billing reconciliation: 

Opus 4.6 Input:       BFE9-94F6-685E
Opus 4.6 Output:      2E1D-DD2E-E744
Opus 4.6 Cache Read:  507A-6150-C2B2
Opus 4.6 Cache Write: C0C8-E5DB-D710
Sonnet 4.6 Input:     D684-B80F-A826
Sonnet 4.6 Output:    C24D-A71A-D031

3. Cost by use case

Use case Tokens (in/out) Opus 4.6 Sonnet 4.6 Ollama 7B
This session (~1M ctx) ~500K/200K ~$10 ~$6 n/a
Daily audit (style check) ~10K/2K $0.10/day $0.06/day $0.00
Brief generation ~50K/10K $0.50/day $0.30/day $0.00*
Monthly audit 300K/60K $3.00 $1.80 $0.00
Monthly brief gen 1.5M/300K $15.00 $9.00 $0.00*

* Ollama brief generation requires a model that can synthesize 5000+ items into a coherent narrative. 7B models struggle here.

4. Local inference (Ollama)

Available on the LAN at 192.168.86.22:11434 (Mac): 

curl -s http://192.168.86.22:11434/api/tags | jq '.models[].name'

Models used in this project:

  • flux2-klein – banner image generation (728x90 grayscale)
  • qwen2.5-coder – code generation (candidate for daily audit)

5. Breakeven analysis

The daily audit is the first candidate for local-first: a style check against 5 rules (em-dash, contrastive antithesis, overused words, title format, day-of-week) doesn't need Opus. A 7B model on Ollama could do it for free.

At $0.10/day for Opus audit, the annual cost is $36.50. A used Mac Mini with M1 (running Ollama) costs ~$400 and handles unlimited audits. Breakeven: ~11 years for audit alone. But the Mac also generates banners, runs local models for research, and serves as a second evaluation context.

The brief generation ($0.50/day, $182.50/year) changes the math: if a local model can generate comparable briefs, breakeven drops to ~2 years.

6. Monitoring

Token usage is tracked in:

  • .beads/interactions.jsonl – per-session tool calls and token counts
  • GCloud billing export – per-SKU costs
  • OTEL metrics at 192.168.86.100:4317 – session telemetry

TODO: Build a REPL namespace (wal-sh.site.token-cost) that reads interactions.jsonl and computes per-session and per-day costs using the pricing table above.

7. Provider model listings

Cached at site/research/ai_model_outputs/:

Provider File Status
Anthropic anthropic_models.json Stale (empty)
OpenAI openai_models.json Stale (empty)
Google Gemini google_gemini_models.json Stale (empty)
HuggingFace huggingface_models.json 100 models
Ollama ollama_models.json Stale (empty)
AI21 ai21_models.json 2 models
Cohere cohere_models.json Stale (empty)

TODO: Refresh via gmake list-foundation-models.

8. LiteLLM proxy spend (live)

verified · literate-exec · 2026-06-22

Live spend from the LiteLLM proxy at 192.168.86.22:4000 (queried 2026-06-22):

Model Spend Notes
gemini/gemini-2.5-flash $0.070 Only model with traffic
9 others (ollama, claude, gpt-4o-mini) $0.000 Configured but unused

Total: $0.07 across 10 models. The Ollama models (qwen2.5-coder, llama3.2, deepseek-coder, mistral) are free and show zero spend because LiteLLM routes them locally.

9. LiteLLM proxy and sandbox isolation

corrected · pilot-annotation-pass · 2026-10-08

LiteLLM (pinned near v1.83.3 – no v1.83.4 tag exists upstream) runs on the Mac at http://192.168.86.22:4000/. It routes requests to Ollama (local models) or cloud APIs (Anthropic, OpenAI) through a unified OpenAI-compatible interface. The OpenAPI spec is saved at spec/rfcs/litellm-openapi.json.

On FreeBSD 15.0 (nexus), isolation uses Bastille jails rather than Docker/podman. The credential flow: 

Agent (Claude Code / cron)
  → LiteLLM proxy (Mac :4000, holds API keys)
    → Ollama (Mac :11434, local models) — free
    → Anthropic API (cloud) — per-token cost
    → OpenAI API (cloud) — per-token cost

The proxy holds the API keys, not the agent. An agent in a Bastille jail has no direct access to ~/.authinfo or pass. It calls LiteLLM with a proxy key; LiteLLM resolves the model and routes to the right backend. This resembles Docker Sandboxes, Docker's own isolated-environment product for AI agents with credential handling – though Docker's experimental Sandbox CLI plugin was removed in Desktop 4.80.0 in favor of sbx, not a stable feature pinned to a specific Desktop version.

9.1. Sandbox architecture

Layer FreeBSD (nexus) Linux/macOS
Isolation Bastille jails Docker Sandboxes / podman
Credential store LiteLLM proxy (external) Docker credential helpers
Model routing LiteLLM 192.168.86.22:4000 Same (LAN accessible)
Local inference Ollama via LiteLLM Ollama direct
Monitoring OTEL 192.168.86.100:4317 Same

9.2. Cost routing

LiteLLM can route by cost: for a style audit (simple, 5 rules), route to the cheapest model that passes a quality gate. For brief generation (complex, 5000 items), route to Opus. The routing decision is in the LiteLLM config, not in the agent code. 

model_list:
  - model_name: audit-style
    litellm_params:
      model: ollama/qwen2.5-coder:7b
      api_base: http://localhost:11434
  - model_name: brief-generate
    litellm_params:
      model: claude-opus-4-6
      api_key: os.environ/ANTHROPIC_API_KEY

10. Related