explodinggradients/ragas
120 source files scanned · 13 LLM call site(s) · ~2% of LLM spend looks recoverable (static heuristics · repo looks like a library)
Library/SDK layout detected (installable Python package with src/ layout and no app entry point): call sites are most likely pass-through wrappers where the caller chooses the model and prompt, so findings are downgraded one level and the estimate is discounted.
[](https://tokenzero.sh/audit/explodinggradients/ragas)
Models referenced
gpt-3.5-turbo ×3gpt-4o ×2 (cheaper sibling: gpt-4o-mini)gpt-5-mini-2025-08-07 ×1
Findings
| Severity | Where | Finding | Est. |
|---|---|---|---|
| low | examples/ragas_examples/prompt_evals/prompt.py:15 | gpt-4o used for a task whose prompt reads like extraction/classification Full-size pricing ($2.5/M in, $10/M out) on a constrained task. gpt-4o-mini costs ~94% less per call at this shape. Fix: Opt into TokenZero cascade routing (`X-TokenZero-Cascade: auto`) so this call is routed to gpt-4o-mini only when the classifier is confident (an opt-in shadow parity monitor is coming; nothing is promised), or switch the model here behind an eval. | 94% |
| low | src/ragas/prompt/metrics/answer_correctness.py:33 | 9 static prompt template(s), ~5,729 tokens, and no caching directive anywhere in the repo Most call sites use providers with automatic prefix caching, which only pays off when these templates are sent as a byte-identical prefix: static template first, nothing volatile (dates, ids, user data) interpolated into it. Fix: Keep templates first and static; TokenZero adds a stable `prompt_cache_key` so related requests land on the same cache. | 12% |
| info | examples/ragas_examples/prompt_evals/prompt.py:15 | Repeat-prone task with no response-cache opt-in Classification and extraction calls often see identical inputs. Without a determinism signal no proxy can safely serve repeats from cache. Fix: If identical inputs should get identical answers, send `X-TokenZero-Cache: always` on these calls (or `temperature: 0` / a `seed` on models that still accept them). Repeats are then served in <10 ms at $0. | 10% |
| info | examples/ragas_examples/workflow_eval/workflow.py:167 | Unrecognised model id "gpt-3.5-turbo" Not in the current pricing table: it may be retired or renamed. Retired models return errors or are silently remapped by some providers. Fix: Check the provider model list; TokenZero flags unpriced models in `X-TokenZero-Warning`. | 0% |
Fix all of it with one line
OPENAI_BASE_URL=https://tokenzero.sh/v1 ANTHROPIC_BASE_URL=https://tokenzero.sh
Keys pass through untouched. Cache formatting and exact caching are on by default; cascade routing stays opt-in. How it works