acceptodds
Under review as a conference paper at ICLR 2027

Not Every Node Benefits from Expensive Recovery: Risk-Aware Routing for Incomplete Multimodal Graph Learning

Abstract

Nodes with missing modalities have limited information for downstream prediction in multimodal graphs, prompting the development of increasingly sophisticated and computationally expensive recovery methods, which are often applied uniformly to affected nodes. However, our analysis shows that expensive recovery can turn correct predictions into errors even when it improves overall performance, while it can still correct cheap-path errors when it degrades overall performance. These motivate selecting nodes for expensive recovery by balancing potential correction against the risk of introducing errors. We propose Risk-aware Selective Modality Recovery (R-SMR), which formulates this selection as budget-constrained routing between cheap and expensive recovery endpoints. Using only cheap-path information at inference, R-SMR estimates the probabilities of four joint correctness states and scores each switch by balancing predicted correction against harm. It selects the highest-scoring nodes with positive utility within a budget for expensive recovery calls, leaving budget unused when further switches are predicted to be unfavorable. Across six datasets and five recovery methods, R-SMR consistently outperforms the stronger endpoint up to 4.10%, with improvements persisting in both globally beneficial and harmful recovery regimes even at a 5% budget. Comparisons with random routing and ablations demonstrate the importance of balancing correction against harm and rejecting unfavorable switches.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Incomplete Multimodal Graph Learning, Modality Recovery, Node Classification, Graph Neural Networks

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_nLQCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper asks which nodes actually benefit from expensive missing-modality recovery. R-SMR predicts correction and harm probabilities using cheap-path information, then selects positive-utility switches under a call budget. It is tested on six multimodal graph datasets, five recovery methods, and multiple budgets. Endpoint comparisons, matched-call random routing, and ablations support the benefit of selective switching.

Critical strengths

The experiments show why average recovery performance is insufficient for deciding which nodes to recover: correction and harm coexist even within the same node groups. Tests across six datasets and five recovery methods support selective switching, including cases where the expensive endpoint is worse overall. Matched-call random comparisons help establish that the selection itself matters.

Critical weaknesses

The comparisons with fixed endpoints and random routing do not establish an advantage over a simpler learned router. At λ = 1, correction minus harm is just the conditional accuracy difference between the two endpoints, so the necessity of four-state modeling remains unclear. The paper also reports selected-node counts rather than demonstrated end-to-end savings. These gaps limit both the methodological novelty and the evidence for computational efficiency.

Questions for authors
  1. In the reported experiments, are expensive predictions computed only for selected nodes, or obtained from a full-graph expensive pass before switching? What computational savings are actually demonstrated after including the cheap path and router?
  2. At λ = 1, correction minus harm equals the conditional accuracy difference between endpoints. Which reported comparison isolates the benefit of four-state modeling over directly learning this difference?
3 comments
AuthorsOpenReview ↗

Rebuttal response for Reviewer nLQC

Thank you for the careful review and constructive feedback. We have added experiments and analyses and uploaded the revised manuscript, with new text marked in blue.

Weaknesses-1: Fixed endpoints and random routing do not establish an advantage over a simpler learned router

Change. Section 6.5 ("Alternative routers", Figure 7, p.10) and Appendix M (pp.32–35; results in §M.3, p.33) add three simpler routers. All share frozen endpoints, budget caps and the positive-score, at-most-KbK_b selection rule of R-SMR. Learned alternatives share its routing features; Conf-Thr uses cheap predictive entropy only. Hyperparameters are chosen on validation data:

  • Conf-Thr routes on cheap predictive entropy with a validated threshold.
  • Direct-Δ learns the correctness change directly: ridge regression of Δv=I[y^vE=yv]−I[y^vC=yv]∈{−1,0,1}\Delta_v=\mathbb I[\hat y^E_v=y_v]-\mathbb I[\hat y^C_v=y_v]\in\{-1,0,1\}.
  • Sep-Correct fits two separately calibrated correctness predictors and scores q^vE−q^vC\hat q^E_v-\hat q^C_v.

Table R1. Mean Accuracy difference over 30 operator–dataset pairs (points). Positive values favor the first router.

ComparisonWhat it isolates20% cap50% cap
Learned routers − Conf-Thrlearning the switch benefit vs. cheap uncertainty+0.96 to +1.61+0.69 to +1.47
R-SMR (λ=1\lambda=1) − Direct-Δfour-state probabilities vs. direct regression+0.20+0.29
R-SMR (λ=1\lambda=1) − Sep-Correctjoint vs. marginal estimation, same target+0.13+0.22

The Conf-Thr range spans the five operators. Conf-Thr is the lowest of the four alternatives in 29/30 pairs at 20% and 28/30 at 50%.

Conclusion. Learning the switch benefit clearly beats cheap uncertainty. No alternative exceeds Full R-SMR on average at either cap: Direct-Δ trails it by 1.19 points at 20% and 1.57 points at 50% (Appendix M.3, p.33). Because Full also selects λ\lambda per setting, this gap combines estimation and harm weighting, and we do not attribute it to four-state estimation alone (see Weaknesses-2 / Questions-2).

Weaknesses-2 / Questions-2: At λ=1\lambda=1, correction minus harm is the conditional accuracy difference. Which comparison isolates four-state modeling from directly learning this difference?

In the original submission, no comparison did. The correction-only and λ=0\lambda=0 ablations test whether harm must be modeled, not how the routing score should be estimated. We agree that for true probabilities p01,v−p10,v=qvE−qvC,p_{01,v}-p_{10,v}=q^E_v-q^C_v, so at λ=1\lambda=1 the four-state score targets the conditional accuracy difference. Section 5.1.3 (Step 3: Risk-Aware Routing, p.6) now states this identity.

The isolating comparison. Sep-Correct has the same target as R-SMR (λ=1\lambda=1) and differs only in estimating it from two marginals instead of one joint four-state model. Direct-Δ learns the difference directly. In Table R1, R-SMR (λ=1\lambda=1) leads by only 0.13–0.22 points over Sep-Correct and 0.20–0.29 points over Direct-Δ. At λ=1\lambda=1, four-state modeling is therefore not necessary, and its empirical advantage is small.

Why keep correction and harm separate. For a general harm weight, uλ=p01−λp10=(qE−qC)−(λ−1) p10.u_\lambda=p_{01}-\lambda p_{10}=(q^E-q^C)-(\lambda-1)\,p_{10}. The marginals do not identify p10p_{10}; they only bound it, max⁡(0,qC−qE)≤p10≤min⁡(qC,1−qE)\max(0,q^C-q^E)\le p_{10}\le\min(q^C,1-q^E) (Section 5.1.3). Any rule with λ≠1\lambda\neq1 needs the joint distribution or a direct estimate of harm, which a difference-only router such as Direct-Δ or Sep-Correct cannot express.

Evidence that λ≠1\lambda\neq1 matters. In the harm-weight sweep at a 50% cap (Appendix N, Figure 30, p.36), validation selects λ>1\lambda>1 in 28 of 30 operator–dataset pairs (λ\lambda = 1 / 1.5 / 2 / 3 in 2 / 12 / 6 / 10 pairs). The penalty can help or hurt:

  • RITR–RedditS: 63.50% at λ=0\lambda=0, 90.25% at λ=1\lambda=1, 92.49% at λ=3\lambda=3.
  • NTSFormer–Grocery: 82.70% at λ=2\lambda=2, 74.50% at λ=10\lambda=10.

Conclusion. The four-state formulation is what makes a separate harm penalty available, and validation uses it in almost every setting. Its benefit at λ=1\lambda=1 alone is small.

AuthorsOpenReview ↗

Rebuttal response for Reviewer nLQC(continued)

Weaknesses-3 / Question-1. Selected-node counts versus end-to-end savings. Are expensive predictions computed only for selected nodes, and what savings remain after the cheap path and router?

How the accuracy tables were computed. Each frozen endpoint was run once on the test graph and its logits were cached. R-SMR fixes the selected set SbS_b from cheap-path information before any expensive output is accessed, then replaces only the selected nodes' logits. The accuracy tables therefore measure selection quality, not compute. Section 6.1 (p.8) now states this.

Change. New Appendix I (Figures 10–13, pp.25–27) measures inference latency, GFLOPs and peak GPU memory for NTSFormer, GRE-MC, MAGNET and MIGDiff on six datasets at six caps. Each R-SMR run executes the cheap path, routing-feature construction and selection, and expensive inference only on the selected nodes with their required graph context.

Table R2. R-SMR at a 50% cap relative to All-expensive, range over six datasets. All R-SMR costs include the cheap path and router.

OperatorLatency (R-SMR / All-exp.)GFLOPs (R-SMR / All-exp.)Peak memory vs. All-exp.
NTSFormer0.57–0.660.35–0.44lower on 6/6 datasets
GRE-MC0.003–0.440.05–0.36higher on 6/6
MAGNET0.56–0.730.44–0.97lower on 1/6 (Ele-Fashion)
MIGDiff0.014–0.160.10–0.72higher on 6/6

For example, on Ele-Fashion GRE-MC drops from 1,969.9 s to 135.0 s, and MIGDiff from 8.02 s to 0.42 s. Across all four operators, R-SMR's measured latency and GFLOPs are below All-expensive at every cap and dataset. Section 6.3 now separates call reduction from realized cost.

Where savings do not hold.

  • RITR runs graph-wide recovery for any non-empty selected set. R-SMR improves its accuracy but does not reduce its dominant cost, so we omit it from the cost sweep and say this in Appendix I and the Limitations paragraph.
  • Memory. Keeping the cheap path and its inputs resident raises peak memory above All-expensive for GRE-MC and MIGDiff (e.g., MIGDiff–Ele-Fashion: 0.94 → 2.84 GiB).

Conclusion. For the four node-selective operators, R-SMR reduces measured latency and computation after including the cheap path and router. Memory savings and savings for graph-wide operators such as RITR do not follow from fewer calls.

Please let us know if any further experiments or clarification are required during the discussion period or before the camera-ready version. We will be happy to discuss and perform further experiments.

AuthorsOpenReview ↗

Follow-up on the Revised Manuscript

Dear Reviewer nLQC,

Thank you very much for your time and for the thoughtful feedback you provided during the previous round of review. We greatly appreciate the care you have taken in evaluating our manuscript, and your comments have helped us improve the revised version.

We understand that you may have a busy schedule. When convenient, could you please let us know if you have any further questions or concerns about the revised manuscript? We would be happy to provide any additional information, clarification, or modifications.

Thank you again for your valuable review and consideration.

With kind regards,

The Authors

Reviewer_ZA8SOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers selective modality recovery for incomplete multimodal graphs. A Risk-aware Selective Modality Recovery (R-SMR) framework is proposed to decide which nodes should use an expensive recovery method. The router predicts correction and harm probabilities from cheap-path information and selects nodes with positive estimated utility under a call budget.

Critical strengths

-The motivation is clear: expensive recovery can improve overall accuracy while harming individual nodes. The node-level analysis provides useful evidence for selective recovery. -The proposed router is simple and uses only cheap-path information at inference. -The evaluation covers six datasets, five recovery methods, and multiple budgets, with matched-call random baselines and component ablations.

Critical weaknesses

-The computational benefit is not sufficiently established. The paper measures the number of selected nodes, but graph-based recovery may require shared computation across nodes. Although routing complexity is analyzed, end-to-end runtime and memory comparisons are missing. Fewer selected predictions therefore do not necessarily imply proportional computational savings. -The routing baselines are relatively limited. Comparisons mainly involve random selection and variants of the proposed router. Confidence-based routing, direct prediction of switching benefit, and separate cheap/expensive correctness predictors would provide stronger evidence for the four-state formulation. -Several expensive endpoints perform substantially worse than the cheap endpoint. For example, on RedditS, RITR achieves 37.04% accuracy compared with 90.82% for the cheap path. Such large gaps raise concerns about endpoint adaptation or classifier compatibility and make the evaluation less representative of choosing between competitive recovery methods. -The missingness setting is restricted to random masking at a fixed rate, with each node missing at most one modality. It remains unclear whether the router remains effective under different missing rates, structured missing, or a change in missing patterns between development and testing.

Questions for authors

-Can each expensive endpoint actually be executed only for selected nodes? What are the measured runtime and memory savings after including cheap recovery, routing, and any shared graph computation? -At λ=1\lambda=1, the routing score equals the difference between the endpoints’ predicted correctness probabilities. Does modeling four joint states outperform directly estimating this difference? -Several expensive recovery methods perform substantially worse than the cheap baseline. Could the authors explain this performance gap and verify whether the routing gains remain when both endpoints are well-tuned and competitive? -How sensitive are the calibrated probabilities and selected harm penalty to changes in the missing rates or patterns?

4 comments
AuthorsOpenReview ↗

Rebuttal response for Reviewer ZA8S

Thank you for the careful review and constructive feedback. We have added experiments and analyses and uploaded the revised manuscript, with new text marked in blue.

Weaknesses-1 / Question-1: Can each expensive endpoint run only on the selected nodes, and what are the measured savings?

We fully agree that counting selected nodes does not establish a computational saving; this is the point we had not measured.

Change. New Appendix I, “Inference Cost across Budgets” (pp. 25–27; Figures 10–13), reports measured end-to-end latency, GFLOPs, and peak allocated GPU memory for NTSFormer, GRE-MC, MAGNET, and MIGDiff on all six datasets at caps of 5% to 50%. Each measurement covers cheap recovery, routing, and shared graph computation. The main accuracy tables remain replay-based, switching precomputed logits, as §6.1 now states.

Execution granularity. For NTSFormer, GRE-MC, MAGNET, and MIGDiff, measured cost falls as fewer nodes are routed, so the expensive computation scales with the selected set. RITR is excluded: any non-empty routed set triggers its graph-wide recovery pass, so routing fewer nodes does not reduce its dominant cost. We no longer claim a saving for RITR.

Table R1. R-SMR inference cost as a percentage of All-expensive, at the 50% cap (single measured run per cell).

OperatorLatency, 6 datasetsGFLOPs, 6 datasetsPeak GPU memory
NTSFormer57–66%35–44%below All-expensive on all 6
GRE-MC0.3–44%5–36%above All-expensive on all 6
MAGNET56–73%44–97%below on 1, above on 5
MIGDiff1–16%10–72%above All-expensive on all 6

Conclusion. For all four operators, on all six datasets and at every plotted budget, fewer selected nodes give lower latency and GFLOPs than All-expensive. The saving is not proportional to the call reduction, because shared graph computation remains. Memory does not follow from fewer calls, because the routed pipeline must keep the cheap path, the expensive endpoint, and the router in GPU memory at the same time. GRE-MC and MIGDiff therefore use more GPU memory than All-expensive on every dataset, and MAGNET on five of six, so we claim no memory saving.

Weaknesses-2 / Question-2: The routing baselines are relatively limited, and does the four-state model beat directly estimating the difference?

Change. Appendix M, “Alternative Router Baselines” (pp. 32–35; Figures 25–29), adds three alternatives across all 30 operator–dataset pairs at 20% and 50% caps: Conf-Thr uses cheap-path entropy; Direct-Δ regresses the correctness change; Sep-Correct subtracts two separately calibrated correctness probabilities. All share frozen endpoints, candidates, and the positive-score selection rule; learned alternatives use the same features and preprocessing (§M.1, p. 32).

You are right about λ=1\lambda=1: marginal correctness difference equals joint correction probability minus joint harm probability, so Sep-Correct and R-SMR (λ=1\lambda=1) target the same risk-neutral benefit (§5.1.3, p. 6, lines 305–309; derivation in §M.2, p. 33, lines 1743–1752).

Table R2. Mean Accuracy change from Full R-SMR (pp) across 30 operator–dataset pairs, from Figures 25–29 (pp. 34–35). “Below” counts negative plotted differences at 20% / 50%.

Router20% cap50% capBelow
Conf-Thr−2.38−2.6430 / 30
Direct-Δ−1.19−1.5730 / 30
Sep-Correct−1.12−1.4929 / 30
R-SMR (λ=1\lambda=1)−0.99−1.2722 / 26
Full R-SMR0.000.00—

Full R-SMR scores joint correction minus validation-weighted harm. The figures report Accuracy only; means and counts use rounded bar labels, and 0.00 is not counted as lower. At λ=1\lambda=1, R-SMR leads Sep-Correct by 0.13 / 0.22 points (§M.3, p. 33, lines 1766–1770).

Conclusion. Learning task benefit matters more than the choice of estimator: Conf-Thr is about 1.1–1.4 points below the learned alternatives on average (§M.3, p. 33, lines 1756–1759), reflecting the difference between cheap-path uncertainty and correctability by the expensive endpoint. No alternative exceeds Full R-SMR in mean Accuracy at either cap.

At λ=1\lambda=1, joint and separate correctness models perform similarly, with no uniform ordering across panels, so we do not claim joint estimation is necessary for this target. For λ≠1\lambda\ne1, marginal probabilities alone do not identify the harm probability; estimating harm enables the weighted policy, though the gains should not be attributed to joint estimation alone. The results support learning task benefit and accounting for harm, not universal superiority of four-state estimation.

AuthorsOpenReview ↗

Rebuttal response for Reviewer ZA8S(continued)

Weaknesses-3 / Question-3: Why are several expensive endpoints far below the cheap path, and do gains survive when both are competitive?

The gap. Appendix G documents operator-specific tuning (Table 8, p. 21; procedures pp. 21–22): validation accuracy (NTSFormer, MAGNET), classification cross-entropy (RITR), interface-specific accuracy (MIGDiff), reconstruction MSE (GRE-MC). This neither shows every endpoint is optimally adapted nor explains the causal source of the gaps, so we do not claim tuning rules out adaptation problems. “Expensive” denotes computational cost, not necessarily higher accuracy (§3).

This regime is central to the claim: does a costly recovery method benefit every node? A globally weaker endpoint is the clearest case where it does not, and a lightweight router can still find the nodes it helps.

Table R3. Cross-dataset means at the 50% cap (Table 9, Appendix H.1, p. 22). Accuracy (%).

OperatorAll-cheapAll-expensiveR-SMRbeffb_{\rm eff} (%)Gain (pp)
NTSFormer72.7077.3878.3350.01+0.95
GRE-MC72.8751.8774.817.00+1.94
MAGNET73.6249.2575.668.45+2.04
RITR73.0147.6074.926.04+1.91
MIGDiff73.1760.4774.325.70+1.15

For the four weaker endpoints, All-expensive lies 12.7–25.4 pp below All-cheap, yet R-SMR improves on All-cheap by 1.15–2.04 pp while routing only 5.70%–8.45% of candidates to expensive recovery. On RedditS, RITR alone reaches 37.04% against 90.82% for the cheap path, while R-SMR reaches 92.49% by calling RITR on 3.05% of candidates. Most of the budget stays unused: the positive-score filter keeps nodes with non-positive predicted utility on the cheap path.

A competitive endpoint. NTSFormer beats the cheap path on all six datasets, and R-SMR still improves on it: 78.33% vs. 77.38% for All-expensive at the 50% cap, with about half the expensive calls and +1.08 pp Macro-F1. At caps of 5%–30% it stays below All-expensive (74.10% vs. 77.38% at 5%): the cap, not the router, limits how many nodes can switch (§6.3).

Conclusion. We agree the evaluation has only one competitive pair, so choosing among several strong, well-adapted recovery methods is a setting not yet well represented. The aim is narrower: a complex, costly recovery method does not benefit every node, and a lightweight router can recover the benefit selectively, whether the expensive endpoint is weaker or stronger. Better endpoint adaptation and routing among more competitive methods remain future work.

Weaknesses-4 / Question-4: Does the router hold up under other missing rates, structured missingness, and a shift between development and test?

We agree the original evaluation was limited; we have widened it for the first two.

Change. New Appendix P (pp. 38–43; Tables 12–18) re-runs RITR (weak) and NTSFormer (strong) on Grocery, Toys, and RedditS under seven missingness settings, re-fitting the router, temperature, and λ within each: random (0.2, 0.4, 0.6), asymmetric text/image (0.6/0.2), degree-biased, and clustered (patch sizes 10 and 50) missingness.

Table R4. R-SMR Accuracy gain (pp) over the stronger endpoint at a 50% cap: mean over three datasets [min, max], from the Overall rows of Tables 12–18 (pp. 39–43).

SettingRITR (vs. All-cheap)NTSFormer (vs. All-expensive)
Random 0.2+1.26 [0.91, 1.45]+1.08 [0.38, 2.18]
Random 0.4+1.60 [1.09, 2.10]+1.10 [0.41, 2.25]
Random 0.6+0.62 [0.61, 0.64]+0.87 [0.50, 1.50]
Asymmetric 0.6/0.2+0.86 [0.71, 1.05]+1.08 [0.60, 2.01]
Degree-biased 0.4+1.05 [0.95, 1.11]+1.20 [0.49, 1.91]
Clustered, patch 10+1.43 [1.18, 1.81]+1.02 [0.49, 1.70]
Clustered, patch 50+1.30 [1.07, 1.54]+1.07 [0.67, 1.84]

On calibration and λ. The admission rule is analytic. For fixed logits a node is admitted iff ℓ₀₁ − ℓ₁₀ > T log λ: at λ = 1 the eligible set is independent of temperature, while for λ ≠ 1 a miscalibrated temperature moves the boundary and λ amplifies the harm term (Appendix O.2, p. 37). λ is selected on validation data per operator–dataset setting: across the six datasets no single value in {1, 1.5, 2, 3} is chosen everywhere, and the selected λ is 1 for none of the NTSFormer datasets (Appendix N, pp. 36–37).

Conclusion. R-SMR beats the stronger endpoint in all 42 cells, with the smallest gains at a 0.6 missing rate, where fewer cheap-path errors are correctable. We have not tested transfer between missingness patterns: the router is fitted within each setting, and the Limitations paragraph states that T and λ should be re-validated when the pattern changes.

Please let us know if any further experiments or clarification are required during the discussion period or before the camera-ready version. We will be happy to discuss and perform further experiments.

AuthorsOpenReview ↗

Follow-up on the Revised Manuscript

Dear Reviewer ZA8S,

We would like to express our sincere gratitude for your time and thoughtful comments during the review. Your questions and suggestions have been very helpful in improving our manuscript.

We understand that you may have many commitments, and we fully respect your schedule. We just wanted to kindly ask whether you have any further questions or concerns regarding our revised manuscript. We would be pleased to provide any additional clarification if needed.

Thank you again for your careful consideration and valuable feedback.

With sincere appreciation, The Authors

Thank you for the response. The added results on end-to-end runtime/memory comparisons, more routing baselines, more experimental analyses, and different missingness settings have addressed most of my concerns. I therefore raised my rating from 2 (Weak rejection) to 3 (Weak acceptance).

Reviewer_WAkAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the problem of selective recovery in incomplete multimodal graph learning. The core motivation is that costly modality recovery is not always beneficial. It might correct erroneous predictions at certain nodes, but it could simultaneously have a negative impact on predictions at other nodes.The authors propose a method called Risk-aware Selective Modality Recovery (R-SMR). This approach utilizes features extracted via low-cost recovery paths to predict four joint correctness states, based on which a risk-aware utility is estimated. Given a budget constraint on high-cost recovery operations, this method prioritizes nodes with a positive expected utility. We evaluated R-SMR across six datasets using five different recovery operators, comparing it against baselines such as all-low-cost recovery, all-high-cost recovery and random routing. Furthermore, we conducted ablation studies and analyzed factors including routing budget, calibration issues, and computational complexity. Our experimental results demonstrate that, in the scenarios evaluated, selective routing enhances the predictive performance of downstream tasks compared to fixed recovery strategies.

Critical strengths

This paper identifies a significant limitation of unified recovery strategies: while costly recovery operators may improve predictions for certain nodes, they can simultaneously degrade predictions for others. By evaluating both "corrections" and "damages," R-SMR aligns its objectives more closely with the requirements of practical decision-making. The motivation behind this approach is clear and intuitive.

Critical weaknesses

W1: The paper does not provide a sufficiently systematic discussion of the limitations of R-SMR or the conditions under which its benefits may not generalize. The method calculates utility using predicted error-correction and damage probabilities. Consequently, routing quality depends not only on the accuracy of the four-state classification but also on the reliability of the probability estimates. It is also worth discussing whether a router—if overconfident or poorly calibrated—might erroneously route high-risk nodes into costly recovery processes, as well as the sensitivity of the weight parameter settings to different datasets, recovery operators, and budget constraints.

W2: The goal of R-SMR is to select between existing "inexpensive" and "expensive" recovery endpoints, rather than to directly improve the recovery endpoints themselves. This raises a question: when an expensive recovery endpoint does not strictly outperform an inexpensive one, could the proposed approach be limited in its effectiveness—or even prove detrimental? Addressing this helps to precisely define the paper’s contribution: R-SMR is a selection strategy for a fixed set of candidate recovery endpoints, rather than a general method for enhancing the quality of modal recovery. Consequently, the scope of the paper’s conclusions can be more accurately delimited.

Questions for authors

Please refer to W1 and W2.

3 comments
AuthorsOpenReview ↗

Rebuttal response for Reviewer WAkA

Thank you for the careful review and constructive feedback. We have added experiments and analyses and uploaded the revised manuscript, with new text marked in blue.

Weaknesses1-1: Routing depends on probability reliability, not only on four-state accuracy

We fully agree; this is an important distinction the original version did not make.

Change. New Appendix O, “Temperature Sensitivity and Probability Reliability,” specifically §O.2, “How Temperature Changes Routing Utility” (p. 37, lines 1970–1988; Eqs. (1)–(2)), analyzes how probability errors propagate into routing. Temperature scaling leaves the four-state argmax unchanged but changes the utility u^v=p^01,v−λp^10,v\hat u_v=\hat p_{01,v}-\lambda\hat p_{10,v}. The utility error satisfies ∣u^v−uv∣≤∣p^01,v−p01,v∣+λ ∣p^10,v−p10,v∣,|\hat u_v-u_v|\le|\hat p_{01,v}-p_{01,v}|+\lambda\,|\hat p_{10,v}-p_{10,v}|, so harm-probability errors are amplified by λ\lambda.

Conclusion. Accurate state classification does not guarantee good routing. Probability quality matters, and more so for larger λ\lambda.

Weaknesses1-2: Can an overconfident or miscalibrated router send high-risk nodes to expensive recovery?

This is a valid concern, and the answer is yes. For fixed logits, a node is admitted iff ℓ01−ℓ10>Tlog⁡λ\ell_{01}-\ell_{10}>T\log\lambda (Appendix O.2, “How Temperature Changes Routing Utility,” p. 37, lines 1971–1981; Eq. (1)). At λ=1\lambda=1, the eligible set does not depend on TT. For λ>1\lambda>1, a sharper (lower-TT) router lowers the bar and can admit more high-harm nodes.

Change. We stress-tested all 30 operator–dataset pairs by rescaling the calibrated temperature, T/T⋆∈{0.25,0.5,1,2,4}T/T^\star\in\{0.25,0.5,1,2,4\}, at a 50% cap (Appendix O.1, “Sweep Setup and Reading Guide,” p. 37, lines 1961–1967; Figure 31, “Test Accuracy versus temperature ratio T/T⋆T/T^\star at a 50% expensive-call cap,” p. 38, lines 2005–2033).

Table R1. Mean Accuracy gain of R-SMR over All-cheap (points), and number of datasets (of 6) where R-SMR falls below All-cheap.

Source: aggregate calculations from the data underlying Figure 31 (p. 38, lines 2005–2033); see also Appendix O.3, “Sensitivity Results and Practical Implications” (pp. 37–38, lines 1990–2003).

OperatorT⋆/4T^\star/4T⋆T^\star4T⋆4T^\starBelow All-cheap (T⋆/4T^\star/4 / T⋆T^\star / 4T⋆4T^\star)
NTSFormer+4.02+5.45+1.820 / 0 / 0
GRE-MC+0.81+1.93+0.792 / 0 / 2
MAGNET+0.97+2.03+0.813 / 0 / 0
RITR+0.60+2.21+0.500 / 0 / 1
MIGDiff+0.70+1.75+0.580 / 0 / 1
All 30+1.42+2.68+0.905 / 0 / 4

Conclusion. Miscalibration in either direction roughly halves the mean gain (2.68 → 1.42 / 0.90 points) and can push R-SMR below All-cheap in 9 of the 60 cells at T⋆/4T^\star/4 and 4T⋆4T^\star, by at most 0.20 points (e.g., GRE-MC on Toys: 75.50 vs. 75.64). The harm penalty and positive-score filter limit the damage: removing them costs 3.72 and 2.23 points on average (§6.5, “Which Router Design Choices Matter?”, paragraph “Risk-aware components,” p. 9, lines 465–475; Figure 5, p. 9, and the complete results in Figure 19, p. 30). These are safeguards, not guarantees, and the “Limitations and future work” paragraph now says so (§7, “Conclusion,” p. 10, lines 527–533).

Weaknesses1-3: Systematic limitations

We agree that the original version lacked this discussion.

Change. A new "Limitations and future work" paragraph (§7, “Conclusion,” p. 10, lines 527–533) states that:

  1. Positive predicted utility does not guarantee a beneficial switch, and temperature scaling does not yield calibrated probabilities or harm-free routing (Appendix O, “Temperature Sensitivity and Probability Reliability,” pp. 37–38, lines 1971–2003; Figure 31, p. 38; Table R1 in this response).
  2. λ\lambda and TT are selected per operator–dataset setting at a single cap and should be re-validated when the budget, endpoints, or missingness pattern change (Appendix N, “Sensitivity to the Harm Weight,” p. 36, lines 1901–1906, and p. 37, lines 1949–1953; Figure 30, p. 36; Table R2 in this response).
  3. The router is fitted within each missingness setting; transfer from one pattern to another remains unstudied (Appendix P, “RITR and NTSFormer under Varying Missingness: Node-Subgroup Results,” p. 38, lines 2039–2042).
AuthorsOpenReview ↗

Rebuttal response for Reviewer WAkA(continued)

Weaknesses1-4: Sensitivity of the harm weight to datasets, operators and budgets

We agree that one fixed weight need not transfer.

Change (datasets, operators). New App. N (pp. 36–37, lines 1899–1953; Figure 30) sweeps λ∈{0,0.5,1,1.5,2,3,5,10}\lambda\in\{0,0.5,1,1.5,2,3,5,10\} for all 30 pairs at a 50% cap.

Table R2. Accuracy change vs. λ=1\lambda=1 (points), mean over six datasets (Figure 30 data).

Operatorλ=0\lambda=0λ=10\lambda=10Selected λ\lambdaSpread in {1,1.5,2,3}Selected λ\lambda=1/1.5/2/3
NTSFormer−0.22−3.65+1.271.15–5.130/3/2/1
GRE-MC−9.25−0.48+1.140.16–2.140/2/2/2
MAGNET−9.16−0.45+1.160.17–2.460/4/0/2
RITR−11.40−0.40+1.630.62–3.101/2/1/2
MIGDiff−6.83−0.76+1.080.05–2.641/1/1/3

(i) Too little penalty hurts the four weaker endpoints (λ=0\lambda=0: −6.8 to −11.4; RITR–RedditS 90.25→63.50). (ii) Too much hurts NTSFormer (λ=10\lambda=10: −3.65; Grocery 82.70→74.50). (iii) No single λ\lambda wins everywhere; validation-selected λ\lambda beats λ=1\lambda=1 by 1.26 points on average, so λ\lambda is selected per setting.

Budgets. The sweep uses one cap, now a stated limitation. Across five caps (Table 9, p. 22), the four weaker endpoints use ≤8.45% of candidates even at 50%, so the cap rarely binds; for NTSFormer it always binds (Table R4).

Change (missingness). New App. P (Tables 12–18, pp. 38–43) re-runs RITR and NTSFormer on Grocery/Toys/RedditS under seven missingness settings, re-fitting the router, TT and λ\lambda in each.

Table R3. R-SMR gain over the better endpoint (50% cap), mean [min, max] over three datasets (points).

SettingRITR (vs. All-cheap)NTSFormer (vs. All-expensive)
Random 0.2+1.26 [0.91, 1.45]+1.08 [0.38, 2.18]
Random 0.4+1.60 [1.09, 2.10]+1.10 [0.41, 2.25]
Random 0.6+0.62 [0.61, 0.64]+0.87 [0.50, 1.50]
Asymmetric 0.6/0.2+0.86 [0.71, 1.05]+1.08 [0.60, 2.01]
Degree-biased 0.4+1.05 [0.95, 1.11]+1.20 [0.49, 1.91]
Clustered, patch 10+1.43 [1.18, 1.81]+1.02 [0.49, 1.70]
Clustered, patch 50+1.30 [1.07, 1.54]+1.07 [0.67, 1.84]

R-SMR beats the better endpoint in all 42 cells; gains are smallest at missing rate 0.6. Transfer across patterns is untested and listed as a limitation (§7, p. 10, lines 527–532).

Weaknesses2: Is R-SMR limited, or even harmful, when the expensive endpoint does not dominate?

Most of our experiments are in this regime: four of the five expensive endpoints are globally worse than the cheap path. Change. §6.3 (pp. 8–9, lines 427–446) now discusses the two regimes separately.

Table R4. Cross-dataset Accuracy (%) from Table 9 (p. 22); beffb_{\rm eff} = realized expensive-call rate at the 50% cap.

OperatorAll-cheapAll-expensiveR-SMR @5%R-SMR @50%beffb_{\rm eff} @50%
NTSFormer72.7077.3874.1078.3350.01%
GRE-MC72.8751.8774.5274.817.00%
MAGNET73.6249.2575.3475.668.45%
RITR73.0147.6074.7274.926.04%
MIGDiff73.1760.4774.0974.325.70%
  • Weak endpoint (four operators). All-expensive is 12.7–25.4 points below All-cheap, yet R-SMR beats All-cheap at every cap and in all 24 pairs at 50% (+1.15 to +2.04), calling expensive recovery on only 2.12–18.93% of candidates (Table 4, p. 8). The positive-score filter leaves most of the budget unused.
  • Strong endpoint (NTSFormer). R-SMR beats All-expensive at 50% (78.33 vs. 77.38) but trails it at 5–30% caps, where the cap limits how many nodes can switch (§6.3, p. 9, lines 434–436).
  • Limits. Gains shrink when few cheap errors are correctable (+0.62 at missing rate 0.6, Table R3), and miscalibration can push R-SMR slightly below All-cheap (Table R1).

In our experiments, a non-dominant expensive endpoint did not make R-SMR harmful; R-SMR simply spent less budget.

Please let us know if any further experiments or clarification are required during the discussion period or before the camera-ready version. We will be happy to discuss and perform further experiments.

Reviewer_WAkAOpenReview ↗

Response to the authors

Thank you for the response; the authors have addressed my concerns, so I am changing my score to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Meta response to AC and reviewers

Dear Area Chair and reviewers,

We thank all reviewers for their constructive feedback. We are glad the reviewers found the motivation clear, namely that costly recovery can help some nodes while harming others (WAkA, ZA8S, nLQC), and valued the node-level analysis and the evaluation across six datasets, five operators and multiple budgets (ZA8S, nLQC). We addressed the shared concerns as follows (new text in blue):

  • End-to-end cost (ZA8S, nLQC). New Appendix I (pp. 25–27) measures latency, GFLOPs and peak memory, with the cheap path and router included. At a 50% cap, R-SMR needs 0.3–73% of All-expensive latency and 5–97% of its GFLOPs. It does not save memory for GRE-MC and MIGDiff, and it gives no saving for RITR, whose recovery is graph-wide.

  • Stronger routers and four-state modeling (ZA8S, nLQC). §6.5 and Appendix M (pp. 32–35) add Conf-Thr, Direct-Δ and Sep-Correct. None beats Full R-SMR on average. At λ = 1 the four-state score equals the accuracy difference (now stated in §5.1.3), and the gain over Sep-Correct is small (0.13 / 0.22 points). The four-state model is needed for λ ≠ 1, and validation selects λ > 1 in 28 of 30 settings.

  • Calibration and harm-weight sensitivity (WAkA, ZA8S). Appendices N and O (pp. 36–38) sweep λ and temperature on all 30 settings. Miscalibration roughly halves the mean gain over All-cheap (2.68 → 1.42 / 0.90 points), so T and λ are validated per setting.

  • Missingness patterns (WAkA, ZA8S). Appendix P (pp. 39–43) adds seven missingness settings, covering different rates and asymmetric, degree-biased and clustered missingness. R-SMR beats the better endpoint in all 42 cells.

  • Weak endpoints and limitations (WAkA, ZA8S). With weak expensive endpoints, R-SMR still improves on All-cheap by 1.15–2.04 points. With NTSFormer it beats All-expensive at a 50% cap but not at 5–30% (§6.3). A new Limitations paragraph (§7, p. 10) states the remaining gaps.

We thank the reviewers again and welcome further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.