wal.sh
home > research > llm-cost-monitoring · Jun 19, 2026
llmcostmonitoringollamaclaude-codegcloudlocal-firsttoken-economics

LLM Cost Monitoring and Local-First Harness

Table of Contents

1. Overview

Every tool call in an agentic workflow costs tokens. This note tracks the pricing, measures actual usage, and identifies where local inference (Ollama on nexus or the Mac) can replace API calls.

2. GCloud AI Dev Tools pricing (June 2026)

Per-token pricing for Claude models via Google Cloud:

Model Input Output Cache Read Cache Write Long Input Long Output
Opus 4.6 $5.00/MTok $25.00/MTok $0.50/MTok $6.25/MTok $10.00/MTok $37.50/MTok
Sonnet 4.6 $3.00/MTok $15.00/MTok $0.30/MTok $3.75/MTok $6.00/MTok $22.50/MTok
Opus 4.5 $5.00/MTok $25.00/MTok $0.50/MTok $6.25/MTok - -

SKU IDs for billing reconciliation: 

Opus 4.6 Input:       BFE9-94F6-685E
Opus 4.6 Output:      2E1D-DD2E-E744
Opus 4.6 Cache Read:  507A-6150-C2B2
Opus 4.6 Cache Write: C0C8-E5DB-D710
Sonnet 4.6 Input:     D684-B80F-A826
Sonnet 4.6 Output:    C24D-A71A-D031

3. Cost by use case

Use case Tokens (in/out) Opus 4.6 Sonnet 4.6 Ollama 7B
This session (~1M ctx) ~500K/200K ~$10 ~$6 n/a
Daily audit (style check) ~10K/2K $0.10/day $0.06/day $0.00
Brief generation ~50K/10K $0.50/day $0.30/day $0.00*
Monthly audit 300K/60K $3.00 $1.80 $0.00
Monthly brief gen 1.5M/300K $15.00 $9.00 $0.00*

* Ollama brief generation requires a model that can synthesize 5000+ items into a coherent narrative. 7B models struggle here.

4. Local inference (Ollama)

Available on the LAN at 192.168.86.22:11434 (Mac): 

curl -s http://192.168.86.22:11434/api/tags | jq '.models[].name'

Models used in this project:

  • flux2-klein – banner image generation (728x90 grayscale)
  • qwen2.5-coder – code generation (candidate for daily audit)

5. Breakeven analysis

The daily audit is the first candidate for local-first: a style check against 5 rules (em-dash, contrastive antithesis, overused words, title format, day-of-week) doesn't need Opus. A 7B model on Ollama could do it for free.

At $0.10/day for Opus audit, the annual cost is $36.50. A used Mac Mini with M1 (running Ollama) costs ~$400 and handles unlimited audits. Breakeven: ~11 years for audit alone. But the Mac also generates banners, runs local models for research, and serves as a second evaluation context.

The brief generation ($0.50/day, $182.50/year) changes the math: if a local model can generate comparable briefs, breakeven drops to ~2 years.

6. Monitoring

Token usage is tracked in:

  • .beads/interactions.jsonl – per-session tool calls and token counts
  • GCloud billing export – per-SKU costs
  • OTEL metrics at 192.168.86.100:4317 – session telemetry

TODO: Build a REPL namespace (wal-sh.site.token-cost) that reads interactions.jsonl and computes per-session and per-day costs using the pricing table above.

7. Provider model listings

Cached at site/research/ai_model_outputs/:

Provider File Status
Anthropic anthropic_models.json Stale (empty)
OpenAI openai_models.json Stale (empty)
Google Gemini google_gemini_models.json Stale (empty)
HuggingFace huggingface_models.json 100 models
Ollama ollama_models.json Stale (empty)
AI21 ai21_models.json 2 models
Cohere cohere_models.json Stale (empty)

TODO: Refresh via gmake list-foundation-models.

8. LiteLLM proxy spend (live)

Live spend from the LiteLLM proxy at 192.168.86.22:4000 (queried 2026-06-22):

Model Spend Notes
gemini/gemini-2.5-flash $0.070 Only model with traffic
9 others (ollama, claude, gpt-4o-mini) $0.000 Configured but unused

Total: $0.07 across 10 models. The Ollama models (qwen2.5-coder, llama3.2, deepseek-coder, mistral) are free and show zero spend because LiteLLM routes them locally.

9. LiteLLM proxy and sandbox isolation

LiteLLM (v1.83.4) runs on the Mac at http://192.168.86.22:4000/. It routes requests to Ollama (local models) or cloud APIs (Anthropic, OpenAI) through a unified OpenAI-compatible interface. The OpenAPI spec is saved at spec/rfcs/litellm-openapi.json.

On FreeBSD 15.0 (nexus), isolation uses Bastille jails rather than Docker/podman. The credential flow: 

Agent (Claude Code / cron)
  → LiteLLM proxy (Mac :4000, holds API keys)
    → Ollama (Mac :11434, local models) — free
    → Anthropic API (cloud) — per-token cost
    → OpenAI API (cloud) — per-token cost

The proxy holds the API keys, not the agent. An agent in a Bastille jail has no direct access to ~/.authinfo or pass. It calls LiteLLM with a proxy key; LiteLLM resolves the model and routes to the right backend. This is the same pattern Docker Sandboxes (Docker Desktop 4.45+) uses for container-scoped credentials, but via FreeBSD jails.

9.1. Sandbox architecture

Layer FreeBSD (nexus) Linux/macOS
Isolation Bastille jails Docker Sandboxes / podman
Credential store LiteLLM proxy (external) Docker credential helpers
Model routing LiteLLM 192.168.86.22:4000 Same (LAN accessible)
Local inference Ollama via LiteLLM Ollama direct
Monitoring OTEL 192.168.86.100:4317 Same

9.2. Cost routing

LiteLLM can route by cost: for a style audit (simple, 5 rules), route to the cheapest model that passes a quality gate. For brief generation (complex, 5000 items), route to Opus. The routing decision is in the LiteLLM config, not in the agent code. 

model_list:
  - model_name: audit-style
    litellm_params:
      model: ollama/qwen2.5-coder:7b
      api_base: http://localhost:11434
  - model_name: brief-generate
    litellm_params:
      model: claude-opus-4-6
      api_key: os.environ/ANTHROPIC_API_KEY

10. Related