Diagnostic Resolution: What Agent Diagnoses Preserve for Downstream Use
Abstract
Failure diagnoses are often evaluated by whether they identify the right failure, but later use depends on what information the diagnostic record preserves. We show that task outcomes, failure labels, recurring patterns, and unchanged citations can each collapse distinctions that matter for subsequent decisions: a correct label may lack the evidence for the next check, similar failures may require different procedures, and previously valid judgments may require reassessment as evidence changes. We introduce Diagnostic Resolution (DiagRes), a consumer-conditional diagnotic protocol that evaluates whether a frozen diagnostic record supports correct later judgments under declared source access and reading budgets. DiagRes tests judgment support, check applicability, and reassessment, requiring answers to be correct, sufficiently supported, and bound to the appropriate requirement, execution, and version. Across 22 public tasks, DiagRes-S selection improves follow-up correctness over failure-label selection. Controlled studies across DiagRes-S/A/T distinguish material information loss from harmless record changes; with equal fact budgets and equal individual-decision accuracy, complete versus split evidence yields 50% versus 0% joint success. DiagRes-guided revision further raises correct coverage from 59.6% to 78.9% across 114 questions from 19 sources. These results suggest that reusable diagnosis depends not simply on retaining more information, but on preserving the complete grounds needed for later decisions.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.