- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper asks which nodes actually benefit from expensive missing-modality recovery. R-SMR predicts correction and harm probabilities using cheap-path information, then selects positive-utility switches under a call budget. It is tested on six multimodal graph datasets, five recovery methods, and multiple budgets. Endpoint comparisons, matched-call random routing, and ablations support the benefit of selective switching.
Critical strengths
The experiments show why average recovery performance is insufficient for deciding which nodes to recover: correction and harm coexist even within the same node groups. Tests across six datasets and five recovery methods support selective switching, including cases where the expensive endpoint is worse overall. Matched-call random comparisons help establish that the selection itself matters.
Critical weaknesses
The comparisons with fixed endpoints and random routing do not establish an advantage over a simpler learned router. At λ = 1, correction minus harm is just the conditional accuracy difference between the two endpoints, so the necessity of four-state modeling remains unclear. The paper also reports selected-node counts rather than demonstrated end-to-end savings. These gaps limit both the methodological novelty and the evidence for computational efficiency.
Questions for authors
- In the reported experiments, are expensive predictions computed only for selected nodes, or obtained from a full-graph expensive pass before switching? What computational savings are actually demonstrated after including the cheap path and router?
- At λ = 1, correction minus harm equals the conditional accuracy difference between endpoints. Which reported comparison isolates the benefit of four-state modeling over directly learning this difference?
3 comments
Rebuttal response for Reviewer nLQC(continued)
Weaknesses-3 / Question-1. Selected-node counts versus end-to-end savings. Are expensive predictions computed only for selected nodes, and what savings remain after the cheap path and router?
How the accuracy tables were computed. Each frozen endpoint was run once on the test graph and its logits were cached. R-SMR fixes the selected set from cheap-path information before any expensive output is accessed, then replaces only the selected nodes' logits. The accuracy tables therefore measure selection quality, not compute. Section 6.1 (p.8) now states this.
Change. New Appendix I (Figures 10–13, pp.25–27) measures inference latency, GFLOPs and peak GPU memory for NTSFormer, GRE-MC, MAGNET and MIGDiff on six datasets at six caps. Each R-SMR run executes the cheap path, routing-feature construction and selection, and expensive inference only on the selected nodes with their required graph context.
Table R2. R-SMR at a 50% cap relative to All-expensive, range over six datasets. All R-SMR costs include the cheap path and router.
| Operator | Latency (R-SMR / All-exp.) | GFLOPs (R-SMR / All-exp.) | Peak memory vs. All-exp. |
|---|---|---|---|
| NTSFormer | 0.57–0.66 | 0.35–0.44 | lower on 6/6 datasets |
| GRE-MC | 0.003–0.44 | 0.05–0.36 | higher on 6/6 |
| MAGNET | 0.56–0.73 | 0.44–0.97 | lower on 1/6 (Ele-Fashion) |
| MIGDiff | 0.014–0.16 | 0.10–0.72 | higher on 6/6 |
For example, on Ele-Fashion GRE-MC drops from 1,969.9 s to 135.0 s, and MIGDiff from 8.02 s to 0.42 s. Across all four operators, R-SMR's measured latency and GFLOPs are below All-expensive at every cap and dataset. Section 6.3 now separates call reduction from realized cost.
Where savings do not hold.
- RITR runs graph-wide recovery for any non-empty selected set. R-SMR improves its accuracy but does not reduce its dominant cost, so we omit it from the cost sweep and say this in Appendix I and the Limitations paragraph.
- Memory. Keeping the cheap path and its inputs resident raises peak memory above All-expensive for GRE-MC and MIGDiff (e.g., MIGDiff–Ele-Fashion: 0.94 → 2.84 GiB).
Conclusion. For the four node-selective operators, R-SMR reduces measured latency and computation after including the cheap path and router. Memory savings and savings for graph-wide operators such as RITR do not follow from fewer calls.
Please let us know if any further experiments or clarification are required during the discussion period or before the camera-ready version. We will be happy to discuss and perform further experiments.
Follow-up on the Revised Manuscript
Dear Reviewer nLQC,
Thank you very much for your time and for the thoughtful feedback you provided during the previous round of review. We greatly appreciate the care you have taken in evaluating our manuscript, and your comments have helped us improve the revised version.
We understand that you may have a busy schedule. When convenient, could you please let us know if you have any further questions or concerns about the revised manuscript? We would be happy to provide any additional information, clarification, or modifications.
Thank you again for your valuable review and consideration.
With kind regards,
The Authors
Rebuttal response for Reviewer nLQC
Thank you for the careful review and constructive feedback. We have added experiments and analyses and uploaded the revised manuscript, with new text marked in blue.
Weaknesses-1: Fixed endpoints and random routing do not establish an advantage over a simpler learned router
Change. Section 6.5 ("Alternative routers", Figure 7, p.10) and Appendix M (pp.32–35; results in §M.3, p.33) add three simpler routers. All share frozen endpoints, budget caps and the positive-score, at-most-Kb selection rule of R-SMR. Learned alternatives share its routing features; Conf-Thr uses cheap predictive entropy only. Hyperparameters are chosen on validation data:
Table R1. Mean Accuracy difference over 30 operator–dataset pairs (points). Positive values favor the first router.
The Conf-Thr range spans the five operators. Conf-Thr is the lowest of the four alternatives in 29/30 pairs at 20% and 28/30 at 50%.
Conclusion. Learning the switch benefit clearly beats cheap uncertainty. No alternative exceeds Full R-SMR on average at either cap: Direct-Δ trails it by 1.19 points at 20% and 1.57 points at 50% (Appendix M.3, p.33). Because Full also selects λ per setting, this gap combines estimation and harm weighting, and we do not attribute it to four-state estimation alone (see Weaknesses-2 / Questions-2).
Weaknesses-2 / Questions-2: At λ=1, correction minus harm is the conditional accuracy difference. Which comparison isolates four-state modeling from directly learning this difference?
In the original submission, no comparison did. The correction-only and λ=0 ablations test whether harm must be modeled, not how the routing score should be estimated. We agree that for true probabilities p01,v−p10,v=qvE−qvC, so at λ=1 the four-state score targets the conditional accuracy difference. Section 5.1.3 (Step 3: Risk-Aware Routing, p.6) now states this identity.
The isolating comparison. Sep-Correct has the same target as R-SMR (λ=1) and differs only in estimating it from two marginals instead of one joint four-state model. Direct-Δ learns the difference directly. In Table R1, R-SMR (λ=1) leads by only 0.13–0.22 points over Sep-Correct and 0.20–0.29 points over Direct-Δ. At λ=1, four-state modeling is therefore not necessary, and its empirical advantage is small.
Why keep correction and harm separate. For a general harm weight, uλ=p01−λp10=(qE−qC)−(λ−1)p10. The marginals do not identify p10; they only bound it, max(0,qC−qE)≤p10≤min(qC,1−qE) (Section 5.1.3). Any rule with λ=1 needs the joint distribution or a direct estimate of harm, which a difference-only router such as Direct-Δ or Sep-Correct cannot express.
Evidence that λ=1 matters. In the harm-weight sweep at a 50% cap (Appendix N, Figure 30, p.36), validation selects λ>1 in 28 of 30 operator–dataset pairs (λ = 1 / 1.5 / 2 / 3 in 2 / 12 / 6 / 10 pairs). The penalty can help or hurt:
Conclusion. The four-state formulation is what makes a separate harm penalty available, and validation uses it in almost every setting. Its benefit at λ=1 alone is small.