Writing ·
How to monitor OpenAI prompt caching for an AI agent in 2026
Baseline cache reads and writes by model and service tier, then change one request-shape or routing variable at a time; retain a change only when cache evidence and accepted task outcome support it.
By Youssef Hemimy · OpenAI · agent observability · cost controls · AgentOps
Monitor OpenAI prompt caching as a request-shape diagnostic: baseline cache reads and writes by model and service tier, then change one variable at a time and keep it only when cache evidence and accepted task outcome improve together. A stronger cache signal alone is not evidence that an agent became better, safer, or cheaper overall.
- 1,024
- automatic-caching threshold in OpenAI’s guide
- 2 signals
- cache reads and newly written cache tokens
- 1 change
- request-shape or routing variable per comparison
Start with a baseline that can explain a change
As of August 20, 2026, OpenAI says its Prompt Caching dashboard tracks cache hit rate over time, cache reads per write, and the breakdown of cache-read, cache-write, and uncached tokens. It also supports filters for model and service tier. Record a stable baseline before changing prompt structure, cache-key assignment, or model routing.
For Bonfire's AgentOps Hardening approach, pair that provider signal with accepted completion, retry or fallback behavior, latency, and failure evidence. The dashboard can describe cache activity; the agent harness has to establish whether the changed behavior was useful.
Keep reusable context ahead of changing context
OpenAI says prompt caching is enabled automatically for prompts of at least 1,024 tokens. For GPT-5.6 and later, that is a strict minimum. Those model families match exact prompt prefixes at eligible cache breakpoints: changing content before a breakpoint prevents a cache hit, while changing content after it does not invalidate the earlier cached prefix.
The operational implication is structural, not cosmetic. Place content intended for reuse—stable instructions, shared schemas, and shared context—before request-specific content. That makes a reusable boundary explicit; it is not a reason to freeze content that needs to change for correctness or safety.
Separate cache reads from repeated cache writes
For GPT-5.6 and later, OpenAI documents cached_tokens as input tokens read from the cache and cache_write_tokens as newly written cache tokens. In the Responses API, both appear inusage.input_tokens_details. A response can reuse an earlier prefix and still write newly appended content, so treat the two observations separately.
| Observed pattern | Bounded next check |
|---|---|
| Cache reads fall after a prompt edit | Compare the exact reusable prefix and breakpoint boundary before and after the change. |
| Cache writes remain high while reads remain low | Check for changing timestamps, tool history, or user-specific data before the reusable boundary. |
| Cache signals move with a model or service-tier change | Split the comparison with the dashboard’s model and service-tier filters before attributing movement to prompt shape. |
| Cache reads improve but accepted outcomes fall | Revert or revise the change; cache telemetry is not an outcome-quality gate. |
Run one controlled request-shape experiment
- Choose a representative request cohort and record model, service tier, cache-read/write signals, accepted outcome, latency, and retry or fallback rate.
- Change one variable: move changing data after a stable prefix, add an eligible breakpoint, or keep cache-key assignment consistent for requests with a long common prefix.
- Keep the relevant policy and evaluation inputs fixed for that comparison.
- Compare cache evidence and accepted task outcome with the baseline, then retain, revise, or roll back the change with that record attached.
The request shape is part of the agent harness. Preserve the before-and-after structure and decision record so a later operator can distinguish a cache effect from a routing, traffic, or workload shift.
Use caching evidence in the broader cost decision
Caching telemetry tells you what happened to eligible input-token reuse. It does not identify the accountable workload, show whether a retry was useful, or establish that an accepted output met the system's standard. Join it to OpenAI API-key cost attribution for a provider-side workload ledger, an agent observability dashboard for run-level signals, and the agent cost runbook when the evidence establishes a cost incident.
Caveats
- The dashboard capability is an OpenAI-announced feature as of August 20, 2026; this article does not assert a refresh interval, role requirement, or universal availability guarantee.
- Repeated prompts do not guarantee cache reuse; OpenAI documents matching-prefix, availability, and routing conditions.
- Cache efficiency is not agent quality. Keep acceptance, safety, latency, and retry evidence in the release or operating decision.
- The 1,024-token strict minimum and exact-breakpoint discussion are bounded to GPT-5.6 and later, where OpenAI documents them.
Sources
FAQ
Does a higher prompt-cache hit rate prove an AI agent is better?
No. Prompt-cache telemetry describes eligible input-token reuse. Keep accepted outcome, safety, latency, and retry evidence in the operating decision before retaining a request-shape or routing change.
What should change first when cache reads fall?
Compare the exact reusable prompt prefix and its cache breakpoint before and after the change. Then isolate one request-shape or routing variable rather than attributing the movement to several changes at once.
What is the difference between cache reads and cache writes?
For GPT-5.6 and later, OpenAI documents cached_tokens as input tokens read from cache and cache_write_tokens as newly written cache tokens. Reads and writes should be reviewed as separate signals.
Can a prompt_cache_key guarantee cache reuse?
No. OpenAI says a prompt_cache_key can improve matching for requests with long common prefixes, but reuse still depends on matching content, cache availability, and routing conditions.
Building something that has to hold up?
We do this work for teams — agent reliability hardening, custom MCP servers, and full-stack AI systems built to survive production.