acceptodds
Withdrawn from ICLR 2027

Auditing Node-Importance Measures on Learned Dynamical Models with Embedded Ground-Truth Probes

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Node-importance measures derived from learned dynamical models, including controllability energies, trajectory-level leverage, and attribution through time, are increasingly reported as substantive findings, even though the true influence of any node is unknown in real data. We introduce an audit for such measures based on an embedded ground-truth probe: a small synthetic subsystem added to the analyst's panel and estimated jointly with it using the same model. Because the probe's true multi-step importance is known exactly, the fitted measure can be tested against ground truth under the same estimation conditions and computational pipeline as the real data. A simple diagnostic, the coupling-to-persistence ratio, indicates when the main failure mode is likely to arise, and a theoretical decomposition explains its source. We apply the audit to an existing research pipeline across five empirical panels. The results reveal both a successful validation and three distinct failure modes: the measure succeeds when the estimator can represent the relevant dynamics, inherits the estimator's structural blind spots, becomes dominated by node persistence when coupling is weak, and can be distorted by pooled linearizations evaluated away from the observed trajectory. The audit requires only a few additional channels and no specialized tooling.

Primary area:
interpretability and explainable AI
Keywords:
node importance, learned dynamical models, neural Granger causality, interpretability auditing, semi-synthetic ground truth, sanity checks, controllability

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_xWAHOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a six-variable "probe" to help monitor the reliability of node importance measures in dynamical systems.

Critical strengths

The idea of adding "test" variables to the system is a sensible idea.

Critical weaknesses

I found the paper extremely poorly positioned in the literature, a point perhaps best exhibited in the introduction which has literally no citations. The terse "Related work" section added very little.

There is no explanation where the datasets come from, or even what they're about.

Too much crucial information is missing here for me to give a meaningful assessment of the work. The basic idea is sensible, but even if it was perfectly executed the contribution would be modest. With the flaws above, I do not think it is close to acceptable.

Questions for authors

No questions.

Reviewer_GXcGOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes embedded ground-truth probes, which integrate synthetic subsystems into real-world data to facilitate joint fitting, thereby enabling the auditing of importance metrics against known ground-truth values. Through five distinct panels and two types of estimators, the authors analyze how representation constraints, persistence, and off-trajectory linearization affect the interpretation of these metrics.

Critical strengths

The probe-based approach compares fitted results against ground truth within a consistent standardization and computational pipeline, mitigating issues arising from inconsistent measurement conventions. Furthermore, experiments involving the substitution of different estimators help disentangle model representation constraints from failures originating within the downstream metrics themselves.

Critical weaknesses

In Section 4.3, fixing single-step coupling is not equivalent to fixing multi-step intervention effects; therefore, manipulations of persistence alone are insufficient to prove that a metric has failed. The target quantity should be clearly defined to distinguish between actual cumulative effects and distortions in the metric.

The proposition in Section 2.1 employs a first-order, diagonal decomposition of persistence, whereas Appendix A utilizes an adjoint system with K=3K=3. The derivation for higher-order self-lags and shifted blocks must be provided; simply stating that the approach is "applicable term-by-term" is inadequate.

Questions for authors

In Section 2, energy is normalized as a share of the total system; however, the energy of the actual channels is unknown. Regarding the "calibration" and ground-truth values ​​mentioned in Table 1: do these refer to the raw energy, the share within the probe, or the share relative to the entire system?

In Section 3.2, the diagnostic is constructed using only the lag-1 Jacobian—could this lead to missing higher-order lagged couplings? Please specify the time horizon TT used, and evaluate the sensitivity of the diagnostic to both the lag order and the chosen time horizon.

Given that the average leakage slope (as per Section 3.1) is ≤ 0.006, can this guarantee that the long-term energy deviation remains sufficiently small? Please provide sensitivity analysis results concerning two-way leakage and the impact of zeroing out the cross-blocks.

Reviewer_pHMcOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposed an audit for node-important measures for learned dynamical systems. Experiments were performed on a 16-component democracy panel with an additive neural Granger VAR.

Critical strengths

None

Critical weaknesses

There are no proper citations until Section 7, and the bibliography includes two anonymous companion submissions (2026a, 2026b), both listed as under review, with no clear way to access or verify them. This seems unusual. It made it very hard to fairly assess this work.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.