acceptodds
Withdrawn from ICLR 2027

Can the Agent Know? Certifying Answerability in Power-Grid Fault Diagnosis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predictive confidence does not establish whether the available evidence uniquely determines an answer. In fault diagnosis, distinct physical events can remain indistinguishable under sparse sensing, making a confident prediction unjustified even when it matches the recorded label. We study this distinction through power-grid line-outage diagnosis and introduce ObsGuard, a training-free answerability layer based on physical feasibility. Given network topology, sparse phasor measurements, and bounded injection and measurement uncertainties, ObsGuard uses linear programming to construct the set of compatible contingencies. It returns a unique answer only when this set is a singleton, otherwise reporting feasible alternatives or model mismatch. Under the specified DC observation model and valid uncertainty bounds, we establish true-contingency inclusion, singleton correctness, and monotonic set refinement as consistent measurements are added. We also introduce GridID, a multimodal benchmark with matched sensor views that hold the underlying event fixed while varying information availability. On 3,072 simulated scenarios from held-out IEEE 57- and 118-bus networks, ObsGuard achieves 100% true-contingency coverage. Increasing sensor coverage from 10% to 100% raises singleton coverage from 24.7% to 93.8% and reduces mean feasible-set size from 40.3 to 1.15. A learned confidence rejector issues singleton predictions on 44.4–50.0% of physically ambiguous cases, whereas ObsGuard avoids such commitments by construction. These results establish a concrete distinction between statistical confidence and evidence-based identifiability, supporting diagnostic systems that determine whether an answer is warranted before committing to it.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
answerability, set-valued prediction, uncertainty quantification, physics-informed reasoning, selective prediction, fault diagnosis, multimodal benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_UtxhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ObsGuard, a training-free method for checking whether power-grid measurements uniquely identify a line outage. It uses linear programs to retain all contingencies consistent with a DC model and bounded uncertainties. It returns a unique answer only when one candidate remains. GridID provides matched sensor views, and experiments on IEEE 57/118 networks report full true-contingency coverage and increasing singleton coverage as sensing improves.

Critical strengths
  1. The decision rule has explicit, auditable guarantees. Under the stated assumptions, the true contingency remains in the feasible set, and a singleton must be correct. Feasible perturbations and residuals provide inspectable evidence.
  2. The sensor-view design separates information availability from the underlying event. The experiments report both safety and utility: singleton coverage increases from 24.7% to 93.8% as sensing increases. The analysis also accounts for dependence among matched views through cluster bootstrap.
Critical weaknesses
  1. Agent-facing answerability is unclear. The certificate uses numerical parameters and uncertainty bounds that may not be available to the agent. Its unique answer may therefore not be identifiable from the rendered input alone. No end-to-end multimodal-agent evaluation is reported.
  2. Novelty and validation are limited. The method follows established set-membership diagnosis principles (Puig, 2010). Experiments use the same DC model for generation and certification, without testing realistic model error or comparing against existing physical diagnosis methods.
Questions for authors
  1. Can the agent access the numerical network parameters, injection estimate, and uncertainty model used by ObsGuard? Please clarify what information defines each answerability label.
  2. What is the specific advance over existing set-membership diagnosis methods? Can you provide a relevant physical-method comparison or an agent-with/without-guard evaluation?
  3. How are the uncertainty bounds justified beyond the simulator? Could model error produce a wrong, nonempty singleton rather than an empty-set warning?
  4. Does singleton coverage remain similar across more operating points and sensor placements?
Reviewer_8VonOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ObsGuard, a training-free method that identifies line outages by simulating every possible line outage with a DC power-flow model, given assumptions on sensor noises. The papers also introduces the GridID benchmark (IEEE 57/118, generated from the same model) for this specific task.

Critical strengths
  • The method is simple, doesn't require training and is auditable.
  • The authors provide a solid theorem, proving that we can always know the exact solution if it is among a set of one candidate.
  • The paper contains an ablation that shows that both uncertainty terms are needed, explaining why mismatch grows with more sensors when ϵ=0\epsilon=0.
Critical weaknesses
  • I am not sure about how this paper is contributing to ML research. The novelty is limited for ICLR: set-membership diagnosis [1] and interval state estimation [2] already handle bounded uncertainty. Apart from the logistic regression training 9 hard-coded features, the paper doesn't contain any ML experiment or theory.
  • The main result is a tautology and experiments only confirm Proposition 1. I would suggest generating observations with an AC power flow (or realistic PMU noise / parameter errors) and reporting false exclusions. (the "PMU" term is by the way never defined, which is even more important in a ML venue).
  • Simulation-based line-outage localization is standard and largely uncited: testing each candidate outage against PMU angles under a DC model is exactly [3], extended to multiple outages via sparse recovery [4], quickest/transient detection [5] and identifiability-driven PMU placement [6]. ObsGuard's actual additions should be positioned against this literature.
  • The uncertainty values look optimistic: ϵ=0.0005\epsilon = 0.0005 rad is far below the IEEE standard error for PMUs (IEEE C37.118.1, ≈0.57°\approx 0.57°), and ρ=0.03\rho=0.03 is not justified.
  • No agent, VLM or MLLM is tested, although Appendix C.2 mentions a "frozen 25-pair MLLM subset". Even a small evaluation (do VLMs answer UNANSWERABLE on sparse views and commit on full ones?) would justify the title and the benchmark.

Minor:

  • Figure 1, Figure 3, Table 3, Table 4 and Table 6 are never referenced in the text.
  • There are about twenty-five "it's X, not Y" / "rather than" constructions; this is perhaps too much. Examples: "an information statement, not a lower bound" (l. 158), "The comparison is not 'physics versus learning'" (l. 350), "These are not violations of conformal theory" (l. 356), "rather than a defect to tune away" (l. 372).

References: [1] V. Puig, "Fault diagnosis and fault tolerant control using set-membership approaches: Application to real case studies", International Journal of Applied Mathematics and Computer Science, 2010. [2] A. K. Al-Othman, M. R. Irving, "Uncertainty modelling in power system state estimation", IEE Proceedings - Generation, Transmission and Distribution, 2005. [3] J. E. Tate, T. J. Overbye, "Line outage detection using phasor angle measurements", IEEE Transactions on Power Systems, 2008. [4] H. Zhu, G. B. Giannakis, "Lassoing line outages in the smart power grid", IEEE SmartGridComm, 2011. [5] Y. C. Chen, T. Banerjee, A. D. Domínguez-García, V. V. Veeravalli, "Quickest line outage detection and identification", IEEE Transactions on Power Systems, 2015. [6] Y. Zhao, A. Goldsmith, H. V. Poor, "On PMU location selection for line outage detection in wide-area transmission networks", IEEE PES General Meeting, 2012.

Questions for authors
  • Why is no MLLM/VLM result reported, given the frozen 25-pair MLLM subset in Appendix C.2? Was this experiment run?

  • How are zz and η\eta sampled during data generation, and what are the dimension and structure of GG and the units of ρ\rho?

  • Why does l. 71 say "certify the language model", when no language model has been introduced at that point? Which model is this?

  • Why does "MLLM" suddenly appear at l. 687 ("The frozen local MLLM subset contains 25 pairs")? The acronym is never defined, this subset is never mentioned in the main text, and no result is reported for it. The paper repeats three times that no end-to-end MLLM result is reported (l. 462-464, 471-473, 508-509).

  • Why printing a truncated SHA-256 digest of the protocol file in the main text (l. 305)? What does this specific hash establish?

  • Why so much emphasis on process assurances ("frozen" appears 10 times, "parameters and selection rules were hashed" l. 276, "four predeclared questions" l. 309)? What are the protocol "amendments" stored in audit/ (l. 739)?

  • I am not sure to see the point of reporting the number of unit tests (l. 307) in the main content as this is a technical detail. Same for the laptop model and memory (l. 306).

  • Not sure as well stating "reviewers are not required to inspect this document" (l. 597) is necessary here. And why does Appendix H state that it "is not a separate conference-mandated checklist" (l. 748-749)?

  • What do the terms "anonymous MIT license" (l. 762), "warm-start-free deterministic calls" (l. 233), "code-to-paper annotations" (l. 757) and "upload archive" (l. 415) mean?

Reviewer_27qtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ObsGuard, a physics-based verification layer that determines whether sparse, uncertain measurements uniquely identify a transmission-line outage or leave multiple feasible alternatives. It also presents GridID, a benchmark that varies sensor availability for the same underlying event to distinguish predictive confidence from evidence-based identifiability.

Critical strengths
  1. The paper clearly distinguishes predictive confidence from physical identifiability and provides conditional correctness guarantees for unique diagnoses under explicit model and uncertainty assumptions.

  2. The matched sensor-view benchmark holds the underlying outage fixed while varying sensor availability, enabling controlled analysis of how additional measurements resolve diagnostic ambiguity.

Critical weaknesses
  1. Checking which fault hypotheses remain compatible with bounded uncertainty follows similar idea of established set-membership diagnosis principles. The paper should isolate what is technically new in ObsGuard beyond its agent-facing interface and benchmark.

  2. The comparison with probabilistic approaches is limited to a lightweight logistic-regression baseline, leaving the practical advantage over stronger calibrated models insufficiently established. Although statistical confidence and physical feasibility are distinct, a well-trained model could assign substantial probability to multiple compatible outages and appropriately represent ambiguity. Instance-level comparisons of predicted probabilities and feasible sets, together with stronger uncertainty-aware baselines, would also clarify the additional value of ObsGuard.

  3. Although the motivation emphasizes diagnostic agents interpreting images, alarms, and telemetry, the reported results use trusted numerical inputs. They do not establish whether the framework remains useful when an actual agent makes parsing or channel-alignment errors.

  4. The evaluation focuses on exact line-level identification and does not assess whether ambiguous cases still permit useful regional localization. When several feasible lines lie within the same predefined region, the observations may support a reliable regional diagnosis despite failing to identify a unique line. Evaluating localization at multiple spatial resolutions would better characterize the practical utility of the returned candidate sets.

Questions for authors
  1. What technical advances does ObsGuard offer beyond established set-membership diagnosis methods, apart from its agent-facing interface and benchmark?

  2. How does ObsGuard compare with stronger calibrated, uncertainty-aware models? Can the authors provide instance-level comparisons showing whether predicted probabilities appropriately capture ambiguity among physically feasible outages?

  3. How does the framework perform when integrated with an actual multimodal agent, particularly when image parsing, measurement extraction, or channel alignment introduces errors? How do these errors affect the validity of its certificates?

  4. In cases where exact line identification is ambiguous, can the feasible set still support reliable regional localization? Could the authors evaluate localization at multiple spatial resolutions to quantify this additional utility?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.