- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper distinguishes visual attribution from functional reliance and introduces Reliance Uncertainty Fields (RUF), an energy-based model over layer-wise representation graphs. Its RIFT framework combines attribution preferences, finite-intervention responses, representation flow, stability, and optional semantic references to infer reliance scores and marginal entropy. These quantities support diagnostics of hidden reliance, highlighted-but-unused evidence, attenuation, and shortcut candidates. A non-identifiability theorem motivates supplementing finite-order local attribution with intervention evidence. Experiments across classification, corruption, dense prediction, medical imaging, and tracking report improved prediction of held-out intervention responses and mismatch detection, while sparse probing reduces the cost of collecting evidence.
Critical strengths
1、The distinction between attribution preference and intervention-defined reliance is well motivated, and the local non-identifiability result provides a clear theoretical rationale for collecting additional evidence.
2、RIFT integrates complementary evidence across representation layers and produces explicit diagnostic scores, offering a useful framework for investigating hidden reliance and highlighted-but-unused evidence beyond conventional heatmaps.
3、The evaluation spans multiple vision domains and backbone families, includes component ablations, and separates semantic-free results from reference-assisted diagnostics, helping assess the framework’s breadth and contributions.
Critical weaknesses
1、In Section 3, marginal entropy depends only on the aggregated posterior mean; conflicting and agreeing evidence can yield identical entropy, so its interpretation as evidence conflict is insufficiently justified.
2、Flow uses feature-change magnitude without class-specific relevance, while outgoing normalization largely removes absolute propagation strength. These choices weaken the interpretation of coupling and attenuation as prediction-supporting evidence flow.
3、The projection ablation changes node-level functional predictiveness from 0.86 to 0.78 (Table 1), but the manuscript does not clarify whether input evidence or the evaluation domain also changes.
4、Equation (6) rewards stability independently of response magnitude, allowing consistently zero intervention effects to increase inferred reliance. This conflates evidence reliability with positive evidence of functional support.
Questions for authors
1、How should entropy distinguish conflicting from agreeing evidence when both produce the same unary score and graph messages? Please clarify the intended interpretation and provide a controlled comparison separating evidence conflict from reliance magnitude.
2、How does the flow potential exclude feature changes irrelevant to the target prediction? If two nodes have proportional outgoing disturbances with very different total magnitudes, what enables the attenuation diagnostic to distinguish their propagation strength?
3、What exactly changes in “RIFT w/o RF-aware projection”? Please specify whether preference aggregation, posterior inference, node selection, or the evaluation domain changes.
2 comments
Official Comment (2/2)
CW3 / Q3 (The projection ablation changes ...)
We distinguish input-side preference aggregation from output-side projection. In D.2, Eq. (48), we aggregate an image-space preference map into node evidence using . When this input route is used, the kernel can change the unary evidence. In D.9, Eq. (87), we instead map inferred means to image coordinates using . We infer the posterior on the representation graph, as stated in E.7, and define primary functional predictiveness over held-out representation nodes in Section 4. At fixed node means and evaluation nodes, changing only the output rendering cannot change that Spearman correlation.
Our direct projection experiment is the image-space comparison in J.10, Table 12. Bilinear resizing gives pointing-game accuracy 0.62, IoU@10% 0.31, and image-calibration loss 0.184; analytic RF gives 0.69, 0.37, and 0.142; effective RF gives 0.72, 0.40, and 0.128. These results support the localization and image-calibration benefit. For the Table 1 row, we have not specified the run-level aggregation and selection settings needed to identify whether the comparison also changed an input-side operation. The existence of Eq. (48) does not establish that it was toggled in that run. We therefore do not attribute the 0.86-to-0.78 node-level difference to an output-only rendering change, or assert an undocumented change in evaluation domain. We use Table 12 as the direct evidence for the projection contribution and keep that result separate from the configuration question concerning the Table 1 row.
CW4 (Equation (6) rewards stability ...)
We use stability as an auxiliary predictor of reproducibility across baselines, separately from functional magnitude. In Eq. (6), the two contributions are additive, so a consistently zero response can receive positive stability support. We do not identify invariance itself with a measured nonzero functional effect. The empirical rationale for including the channel is our component comparison: in Table 1, full RIFT improves functional predictiveness from 0.82 without stability to 0.86, RCE from 0.101 to 0.083, and uncertainty–error correlation from 0.43 to 0.56. We select the coefficients on validation data as described in D.6. These results support the auxiliary feature's predictive value in the evaluated setting.
The direct factor effect can be stated exactly. Holding other factor inputs and neighboring states fixed, a normalized stability increase multiplies conditional odds by , at most under our defaults. A normalized functional increment gives . This is a conditional comparison, not a global bound after graph feedback, and the smaller stability coefficient does not eliminate the stable-zero effect. Our functional-variance penalty in D.5 is zero at zero functional variance; the preference-variance penalty concerns a different source. Neither acts as a zero-response detector. We consequently distinguish the component's measured predictive benefit from a guarantee that every stable-zero node is correctly classified. The ablation supports including reproducibility in the joint model, without changing what the observed functional response means.
Official Comment (1/2)
CW1 / Q1 (Conflicting and agreeing evidence ...)
We use separate readouts for marginal uncertainty and source disagreement. Our entropy is a function of the fused Bernoulli mean, so equal means necessarily give equal entropy. We do not use entropy to reconstruct which sources produced that mean. In D.7, Eq. (81), we separately define preference–function disagreement as DJS(ppref∥pfunc), and Eq. (82) compares preference and RUF views. These source-conditioned distributions retain a distinction that is absent from the fused mean alone. When referring to uncertainty, we mean model-relative marginal uncertainty; source conflict itself is assessed through the separate disagreement readout.
For the controlled comparison, consider the unary-only case with graph messages and other contributions set to zero, using βpref=1, βfunc=2. The views are ppref(R=1)=σ(zpref) and pfunc(R=1)=σ(2zfunc). With (zpref,zfunc)=(1,0.5), both views equal 0.731059 and JS divergence is zero. With (zpref,zfunc)=(0,1), the views are 0.5 and 0.880797 and JS divergence is 0.089541 nats. Both cases have η=2, fused mean 0.880797, and entropy 0.365334 nats. Thus our existing disagreement statistic distinguishes the weighted source views while reliance magnitude and marginal uncertainty are fixed. This is an algebraic comparison, not a new experiment or an interpretation of zero as negative binary evidence. We also evaluate uncertainty against prediction error: Table 1 reports correlation 0.56 for RIFT versus 0.42 for RP+FI. Our framework therefore reports the fused belief, its uncertainty, and source-view disagreement as distinct quantities.
CW2 / Q2 (Flow uses feature-change magnitude ...)
We use flow edges to represent feature propagation, and functional evidence and class-conditioned beliefs to represent target information. Eq. (7) combines these through a soft compatibility factor. As stated in K.3, a feature disturbance can propagate without changing the target score; we do not treat an edge's existence as a class-specific causal certificate. Our model also does not guarantee that every target-irrelevant edge is excluded. We evaluate the value of combining these signals through the graph comparisons in Table 7 and the attenuation diagnosis in Table 10. Empirical-flow graphs reach functional predictiveness 0.87 for CNNs and 0.86 for ViTs, and attenuation AUROC is 0.84 for RIFT versus 0.68 for RP+FI and 0.60 for FI-only.
We retain propagation magnitude separately from the normalized pattern. D.4 defines Ai→j in Eq. (63), outgoing mass Mflow(i)=∑jΦi→jc in Eq. (64), and flow entropy in Eq. (65); we also record mass in E.5. For disturbances (1,3) and (10,30), neglecting the small normalization constant, both patterns are A=(0.25,0.75) but the masses are 4 and 40. At local belief 0.8 and downstream beliefs (0.6,0.8), both have downstream support 0.75 and attenuation 0.05. This follows from our definition of attenuation as the posterior-support gap in Appendix G. It does not measure absolute feature-amplitude decay. The separate mass readout distinguishes the total defined disturbance; we do not multiply it into Eq. (7) or claim that it makes attenuation scale-sensitive. The proportional example therefore yields equal support attenuation and different flow mass, exactly the two quantities our definitions separate.