Prompt cache misses
Cached-input tokens stay at 0 even though the system prompt never changes.
Why it happens
Provider caches match an exact prefix: any variation before the static part (timestamps, user ids, reordered tools), a prefix under the minimum size (1,024 tokens on most models, 512 on Opus 5 / Haiku 5.5), or on Anthropic a missing cache_control breakpoint, defeats it.
Fixes
- Read
X-TokenZero-Cache-Format: it names what the formatter did and why a prefix is too small. - Put tools first, then the system prompt, then dynamic content; keep the static part byte-identical. The CAPC step minifies tool schemas deterministically so identical tools always produce identical prefixes.
- On OpenAI pre-5.6 models the gateway sets
prompt_cache_keyfrom the static prefix; on GPT-5.6+ Responses,X-TokenZero-Prompt-Cache-TTL: 12hkeeps the cache warm longer. - Run
npx -y @tokenzero/cli audit: uncached static prompts are the first finding it reports, with the line.
Start here
curl -s -X POST https://tokenzero.sh/v1/signup -d '{}' # workspace + key, no form
npx -y @tokenzero/cli init # or: audits the repo, writes .envOPENAI_BASE_URL=https://tokenzero.sh/w//v1 ANTHROPIC_BASE_URL=https://tokenzero.sh/w/ # provider keys stay in your code exactly as they are; nothing else changes
Questions
How do I know the cache worked?
The receipt's usage.cached count and X-TokenZero-Savings-USD; both come from the provider's own usage numbers, not an estimate.