# Agents - Context and Cost Economics An agent session pays for context twice. It pays at turn zero, when every connected tool injects its schema into the prompt. It pays during the session, when cache traffic dominates the bill. Tool schemas can be deferred without losing reachability. Cache economics, not raw token counts, set the real cost. Measure the burden per tool and the spend per model before you tune. ## Key points - MCP gateway search mode defers tool schemas, not tool access. Moving three heavy servers to search mode cut the first request from 55,138 to 40,209 tokens. Every server stayed reachable. - A per-tool defer flag drops an eager schema to a loader. `{"defer": true}` in a pi package config cut one tool from 4,752 eager tokens to 82. No capability was lost. - One headless run attributes the entire turn-zero burden to each package, because every tool reports its source. The numbers matched leave-one-out A/B runs within 1%. The burden is concentrated: 18 of 36 packages contributed zero context tokens. - The client keeps a per-server MCP tool inventory cache, for example `~/.pi/agent/mcp-cache.json` in pi. It holds complete name, description, and schema inventories, refreshed by the client. Generate plan-mode allowlists from it instead of hand-listing tools. - Cache reads dominate daily token spend. One setup logged about 171M cache-read tokens per day at roughly 10% of the input rate, against about 397K fresh input and 920K output. Raw token count is a misleading cost proxy. - A client-side context window is a compaction budget. The client never sends it to the provider. Above 272K input tokens, one provider charges 2x on input and cache and 1.5x on output for the entire request. A blanket 1M setting triggered the surcharge and was reverted. - Prompt-cache economics are measurable on real sessions. One Opus session had 15 full cache rewrites, about 5.7M write tokens, about $28. Most rewrites came from the 5-minute idle expiry. A 1h TTL at 2x write price saved about $17. - The same retention variable sends different fields per provider. `PI_CACHE_RETENTION=long` makes pi send `ttl: "1h"` on Anthropic-style routes and `prompt_cache_retention: "24h"` on OpenAI-style routes. Test per provider first. Gateways may reject the field. - Context-cost estimates must verify the load mechanism, not the file's existence. A rules file mentioned only by path in prose costs nothing per session. Only `@import`ed files yield savings when migrated. - Per-model agent spend is measurable from LLM telemetry spans. Take the estimated-cost metric from each span and group by the matched model name. ## How to apply 1. Run a headless burden dump before you tune. Boot the agent, dump the assembled prompt plus every tool's source, and exit before the provider request. The run takes about 8 seconds, makes no model call, and bills nothing. 2. Move heavy MCP servers to gateway search mode. Set `{"defer": true}` on tools with large schemas. 3. Generate plan-mode allowlists from the per-server inventory cache. Do not hand-list tools. 4. Read cache-read, cache-write, fresh-input, and output tokens separately. Total token count is a bad cost proxy. 5. Keep the client-side context window near the provider's surcharge line, 272K input tokens here. Do not set the maximum. 6. Set the cache retention variable per provider and test it. Watch for gateway rejection. 7. Verify that a rules file is `@import`ed before you count saved context. A file only mentioned by path saves nothing. 8. Group the estimated-cost metric by matched model name in LLM telemetry to get per-model agent spend. ## Observations - [mcp] Gateway search mode defers tool schemas without losing reachability. The first request fell from 55,138 to 40,209 tokens. #mcp (source: agent sessions, 2026-10) - [mcp] `{"defer": true}` drops one tool's eager schema from 4,752 tokens to 82. #mcp (source: agent sessions, 2026-09) - [measurement] A headless run attributes the turn-zero burden per package within 1% of leave-one-out runs. 18 of 36 packages contributed zero context tokens. #agents (source: agent sessions, 2026-09) - [mcp] The per-server tool inventory cache holds complete name, description, and schema inventories. Generate allowlists from it. #mcp (source: agent sessions, 2026-09) - [cost] Cache reads dominate daily token spend, about 171M per day at about 10% of input rate. #llm-cost (source: agent sessions, 2026-06) - [cost] A client-side context window is a compaction budget never sent to the provider. Above 272K input, one provider charges 2x input and cache and 1.5x output. #llm-cost (source: agent sessions, 2026-09) - [cost] Fifteen cache rewrites from the 5-minute idle expiry cost one session about $28. A 1h TTL at 2x write price saved about $17. #llm-cost (source: agent sessions, 2026-09) - [cost] The same retention variable sends different fields per provider. Test per provider first. #llm-cost (source: agent sessions, 2026-09) - [agents] A rules file mentioned only by path in prose costs nothing per session. Only `@import`ed files yield savings. #agents (source: agent sessions, 2026-09) - [observability] Per-model agent spend comes from LLM telemetry spans. Group the estimated-cost metric by matched model name. #observability (source: agent sessions, 2026-10) ## Relations - relates_to [[resources/concepts/MCP Tool Optimization Framework]] (turn-zero schema cost and deferral tactics) - relates_to [[resources/documentation/Agents/Agents - MCP Server Onboarding and Credentials]] (onboarding decides which servers load schemas eagerly) - relates_to [[resources/documentation/Agents/Agents - Model Selection and Catalogs]] (model choice sets the price of cached and fresh tokens) ## Sources Personal agent-session notes, June 2026 to October 2026