# TokenZero

> Drop-in BYOK gateway that cuts OpenAI, Anthropic and Gemini API bills. Change one line (base_url) and keep your own API key; TokenZero formats prompts for provider caching, serves deterministic repeats from an edge cache, optionally routes simple requests to cheaper models, and signs a savings receipt for every request. Free up to $500/month routed spend, then 15% of verified savings.

Base URL: https://tokenzero.sh
OpenAI-compatible: https://tokenzero.sh/v1/chat/completions, https://tokenzero.sh/v1/responses (set base_url=https://tokenzero.sh/v1)
Anthropic-compatible: https://tokenzero.sh/v1/messages (set base_url=https://tokenzero.sh)
Gemini (native API): https://tokenzero.sh/v1beta/models/{model}:generateContent and :streamGenerateContent?alt=sse (google-genai SDKs: set the base URL to https://tokenzero.sh, keep x-goog-api-key or ?key=). Other /v1beta/* paths (models, countTokens, cachedContents) pass through.
Gemini (OpenAI-compatible): the OpenAI routes above with a Gemini key (AIza…).
Auth: send your provider key exactly as you would to the provider. Optional X-TokenZero-Key (from POST /v1/workspaces) groups requests into a named workspace; without it an anonymous workspace is derived from an HMAC of the key (the key itself is never stored).

## Integration (for coding agents)
- Python OpenAI: OpenAI(base_url="https://tokenzero.sh/v1", api_key=OPENAI_API_KEY)
- Python Anthropic: Anthropic(base_url="https://tokenzero.sh", api_key=ANTHROPIC_API_KEY)
- Any SDK: OPENAI_BASE_URL=https://tokenzero.sh/v1  ANTHROPIC_BASE_URL=https://tokenzero.sh
- Model lifecycle: 17 ids with provider-announced retirement dates (OpenAI and Anthropic deprecation pages); audits flag them, the gateway warns in X-TokenZero-Warning, /models/<id> shows the date and replacement.
- CLI (one command, no install, no code changes): npx -y @tokenzero/cli init (or npx -y https://tokenzero.sh/cli/tokenzero.tgz init) — audits locally (source never uploaded), creates a workspace, writes OPENAI_BASE_URL / ANTHROPIC_BASE_URL to .env. Also: audit [--fail-over PCT] as a CI gate, usage, verify <request_id>.
- Convenience form when you cannot add headers: https://tokenzero.sh/w/{tz_key}/v1 (OpenAI) and https://tokenzero.sh/w/{tz_key} (Anthropic). Prefer the X-TokenZero-Key header; treat key-bearing URLs as secrets and rotate (POST /v1/workspaces/me/rotate-key) if one leaks.
- Local, open-source (Apache-2.0, https://github.com/tokenzero-ai/tokenzero): npx -y @tokenzero/cli audit . | MCP over stdio: npx -y @tokenzero/mcp | GitHub Action: uses: tokenzero-ai/tokenzero/action@v0
- Overhead benchmarks: https://tokenzero.sh/benchmarks (raw: /benchmarks.json)
- MCP server (Streamable HTTP, spec 2026-07-28 with initialize-era compatibility): https://tokenzero.sh/mcp — tools: audit_repo_token_waste, optimize_prompt_payload, calculate_model_arbitrage, provision_workspace, workspace_metrics. Registry name sh.tokenzero/mcp; server card https://tokenzero.sh/.well-known/mcp/server-card.json
- Claude Code: claude mcp add --transport http tokenzero https://tokenzero.sh/mcp

## Request headers (all optional)
- X-TokenZero-Key: workspace key (recommended way to attribute traffic; without it an anonymous workspace is derived from your provider key)
- X-TokenZero-Provider: openai | anthropic | gemini | openrouter (default: inferred from key prefix, then model name)
- X-TokenZero-Upstream: https://… custom upstream base (Azure, self-hosted)
- X-TokenZero-Cache: auto | off | always (exact cache; auto = only temperature:0/seed requests)
- X-TokenZero-Cache-TTL: seconds for exact-cache entries (default 3600)
- Plans (self-serve, POST /v1/workspaces/me/plan {plan, billing}): free ($0, up to $500/mo routed spend) | performance (15% of verified savings, 3x-ROI-or-$0 guarantee) | pro_cap (15% capped at $199/mo, $1,790/yr prepaid) | scale_cap (15% capped at $999/mo, $8,990/yr prepaid). Annual prepay takes 25% off the rate and the cap. Nothing is billed until billing goes live; GET /v1/usage shows the fee preview under the workspace's plan.
- Workspace setting monthlyBudgetUsd (default none): hard cap on real spend through the gateway per calendar month (UTC); past it, requests that would reach a provider get 402 budget_exceeded while cache hits still serve. Headers X-TokenZero-Budget-Usd / -Spent-Usd / -Remaining-Usd / -Resets-At.
- Workspace setting shadowEval (default false): re-runs ~1% of auto-routed requests on the model you asked for, with your key, and reports output parity per route pair on your dashboard; a pair below 99.8% rolling parity stops being auto-routed (advisory only) for your workspace. Shadow spend is yours and is never counted as a saving.
- X-TokenZero-Prompt-Cache-TTL: 5m | 1h | 30m | 12h. Short (5m/30m) is the provider default; long maps to Anthropic's 1h tier (2x write) or OpenAI GPT-5.6+ prompt_cache_options.ttl=12h (2.5x write, Responses API).
- Processing tiers are priced, not optimised away: OpenAI service_tier flex (0.5x) / fast (2x) / ultrafast (6x) and Anthropic speed: fast (2x) are read from the provider's response and signed into the receipt as price_tier; the baseline uses the same tier, so a tier is never counted as a saving.
- X-TokenZero-Semantic-Cache: true (near-duplicate matching, opt-in)
- X-TokenZero-Cascade: off | advise | auto (default advise)
- X-TokenZero-Cache-Strategy: static | conversation | off (Anthropic breakpoint placement; default static)
- X-TokenZero-Prompt-Cache-TTL: 5m | 1h (Anthropic cache TTL)
- X-TokenZero-Bypass: true (pure pass-through, ledger only)
- X-TokenZero-Fallback: off | auto (default off; or workspace setting fallback). auto retries 429/500/502/503/504/529 and connection failures with exponential jitter (honouring short Retry-After), then re-issues once to X-TokenZero-Fallback-Model with X-TokenZero-Fallback-Key when that provider speaks the same request shape. The response header X-TokenZero-Fallback reports attempts (retried=N;from=S;final=S, cross-provider;model=…) and the receipt records engine "failover".
- X-TokenZero-ZDR: true (or workspace setting zdr): zero data retention for the request — no response-cache lookup or store, no shadow evaluation, metadata-only ledger (token counts, costs, hashes; never prompt or completion text). Reported as X-TokenZero-ZDR: on;no-cache;no-shadow;metadata-only-ledger.
- Streaming responses end with an SSE comment ": tokenzero-receipt {request_id, signature, baseline_cost_usd, actual_cost_usd, savings_usd, engines}" after the provider's final event, so streaming callers get the signed accounting without buffering or a second call. The gateway injects stream_options.include_usage for OpenAI-style streams so usage is exact.

## Response headers
Every x-tokenzero-* header is exposed to browsers (Access-Control-Expose-Headers: *). Receipts are hash-chained per workspace: each carries seq and prev_receipt_hash; GET /v1/ledger/chain walks the chain and POST /v1/ledger/verify {items:[…]} verifies a chain segment (CLI: tokenzero verify --chain).
X-TokenZero-Request-Id, X-TokenZero-Workspace, X-TokenZero-Plan, X-TokenZero-Free-Tier-Used-USD, X-TokenZero-Free-Tier-Limit-USD, X-TokenZero-Notice, X-TokenZero-Baseline-Cost-USD, X-TokenZero-Actual-Cost-USD, X-TokenZero-Savings-USD, X-TokenZero-Savings-Pct, X-TokenZero-Engines, X-TokenZero-Cache (hit-exact | hit-semantic | miss | bypass;reason=…), X-TokenZero-Cascade-Recommendation, X-TokenZero-Receipt-URL, X-TokenZero-Receipt-Signature (Ed25519, key at /.well-known/tokenzero.json).
Ledger durability: rows are committed in batches every 25 ms. A hard crash can drop at most that window, which can only under-count savings (never over-count); graceful shutdown flushes everything. Receipts are signed before the response and do not depend on the write.

## Other endpoints
- POST /v1/signup {name?} → 201 {workspace_id, api_key, dashboard_url, …} (one call, no account; CORS *). Also: tokenzero signup / tokenzero dashboard in the CLI, provision_workspace in the MCP server.
- Billing (Stripe, BYOK plans): GET /v1/billing/status, POST /v1/billing/checkout {plan, interval} → {url}, POST /v1/billing/portal → {url}; POST /v1/billing/webhook is Stripe's. Plan changes to paid plans go through checkout.
- POST /v1/workspaces {name?} → {workspace, key}   GET /v1/workspaces/me (X-TokenZero-Key)   PATCH /v1/workspaces/me {settings}   POST /v1/workspaces/me/rotate-key (old key dies immediately)
- Privacy: stats, usage and badges are private by default. The owner sees them with X-TokenZero-Key (anonymous workspaces: the same provider key the traffic used). Setting publicStats: true makes aggregates and the badge readable by anyone with the workspace id; per-request detail is always owner-only. Unknown and private ids return identical 404s.
- GET /v1/usage?workspace=ws_… (or X-TokenZero-Key) — this month's routed spend vs the $500 free tier, history, fee preview. Routed spend = list-price baseline of your traffic. Over the free tier, requests keep flowing as unoptimised pass-through (engines pause, nothing breaks) until the month resets or you upgrade.
- GET /v1/ledger/stats?workspace=ws_… (or X-TokenZero-Key)   GET /v1/ledger/receipt/{request_id}   POST /v1/ledger/verify {receipt, signature} | {items:[{receipt, signature}…]}   GET /v1/ledger/chain?limit=&before= (X-TokenZero-Key)
- GET /v1/ledger/leaderboard   GET /v1/ledger/global
- GET /v1/badge/{workspace_id}.svg   GET /v1/badge/audit/{audit_id}.svg
- POST /v1/simulate {model, system, prompt, requests_per_month, repeat_fraction} or {body}
- POST /v1/audit {repo:"owner/name"} | {files:[{path,content}]}   GET /audit?repo=owner/name   GET /audit/{id}
- POST /v1/github/pr-audit {diff} | {repo, pr} → {report, comment_markdown}
- GET /v1/pricing — the versioned list-price table (2026-10-26)
- Any other /v1/* path is forwarded verbatim to the provider inferred from your key (models, embeddings, count_tokens, batches).

## Status and support
- GET https://tokenzero.sh/v1/status (JSON) and https://tokenzero.sh/status (HTML): gateway overhead, Ed25519 signer self-test, pricing freshness, per-provider retries and rescued failovers.
- GET/POST https://tokenzero.sh/v1/doctor (optional X-TokenZero-Key / Authorization: Bearer / {key}): six checks (gateway, signer with a sample receipt + signature, pricing, workspace key, quota and billing, settings and ZDR) plus copy-paste base URLs. CLI: npx -y @tokenzero/cli doctor.
- POST https://tokenzero.sh/v1/support {subject, message, email?, workspace_id?, category?, receipt_id?} → 201 {ticket}; https://tokenzero.sh/support has the doctor, troubleshooting and the ticket form. Engineering first-response target: 15 minutes.
- GET /v1/workspaces/me/metrics?window=24h|7d|30d|all (X-TokenZero-Key; alias /v1/workspaces/metrics; ?demo=1 returns a labelled sample without a key): the value scorecard — cost_and_roi per window, by_engine (prompt cache + CAPC, edge cache, cascade + reasoning, signed ledger), speed_and_latency, quality_and_safety, reliability_and_privacy, by_provider, by_model, daily_series, actionable_recommendations with setting_patch for POST /v1/workspaces/settings, system_improvements. CLI: tokenzero metrics; MCP tool workspace_metrics.
- Account controls: POST /v1/workspaces/rotate-key (alias of /v1/workspaces/me/rotate-key, returns a one-click dashboard_url) and POST /v1/workspaces/settings (alias of PATCH /v1/workspaces/me; accepts zdr_default, shadow_eval_opt_in, public_stats, semantic_cache, monthly_budget_usd aliases).

## Guides
- Switching from another gateway: https://tokenzero.sh/compare (/switch/from-openrouter, /switch/from-helicone, /switch/from-portkey, /switch/from-litellm, /switch/from-cloudflare-ai-gateway, /switch/from-langfuse, /switch/from-braintrust, /switch/from-kong-ai-gateway, /switch/from-martian)
- MCP setup per client: https://tokenzero.sh/mcp (/mcp/cursor, /mcp/claude-code, /mcp/windsurf, /mcp/cline, /mcp/codex, /mcp/github-actions, /mcp/langchain, /mcp/vercel-ai-sdk)
- Error playbooks: https://tokenzero.sh/errors (/errors/openai-429-rate-limit, /errors/anthropic-529-overloaded, /errors/prompt-cache-miss-debugging, /errors/deepseek-r1-reasoning-token-bloat, /errors/gpt-5-tool-schema-token-bloat)
- Model comparisons: https://tokenzero.sh/compare/{a}-vs-{b} (list prices, cache tiers, the same workload priced on both); model pages: https://tokenzero.sh/models
- Legal: https://tokenzero.sh/terms, https://tokenzero.sh/privacy

## Docs
- https://tokenzero.sh/openapi.json
- https://tokenzero.sh/llms-full.txt (this file, long form) · https://tokenzero.sh/ai.txt (crawler and agent usage policy)
- CLI install: curl -fsSL https://tokenzero.sh/install.sh | sh  (or npx -y @tokenzero/cli); usage: https://tokenzero.sh/cli
- Agent discovery: https://tokenzero.sh/.well-known/mcp.json, https://tokenzero.sh/.well-known/agent.json, https://tokenzero.sh/.well-known/ai-plugin.json
- https://tokenzero.sh/.well-known/tokenzero.json (receipt verification key)
Support: support@tokenzero.sh