- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Message Effect Intervention (MEI) for frozen multimodal graph models. An estimator predicts the task-loss change from deleting one modality-specific message. Its predictions guide candidate filtering and attenuation of at most one incoming edge bundle per receiver, with shared coefficients across modalities and receiver-local normalisation. The paper specifies CM-MGNN as a reference host, compatibility conditions for other hosts, and structural guarantees for the replay operator. Experiments cover seven datasets spanning node classification, link prediction, retrieval, and graph-to-text and graph-to-image generation.
Critical strengths
-
A useful adaptation setting and task-grounded diagnostic.
Improving a frozen multimodal model without updating its parameters is a meaningful setting. Equation 2 defines a concrete model-specific loss response, and the estimator uses training supervision while deployment uses available representations and context. This supports selective intervention in existing models.
-
An explicit intervention boundary and operator contract.
Equations 12–13 and Appendix C specify what changes during replay and establish identity recovery, a local selection budget, shared coefficients across modalities, and preservation of their sum. Appendix E.2 states the conditions required of another host. These properties make the intervention inspectable and distinguish structural guarantees from predictive benefits.
Critical weaknesses
-
The diagnostic's utility and MEI's independent contribution need validation. Equations 2 and 8 supervise singleton message deletion, while Equations 9 and 12 deploy compensated bundle attenuation. Let count all exposed incoming messages, including the self-message. For one modality, write and . The respective perturbations are
\delta S_{\mathrm{MEI}}=\frac{\eta d}{d-\eta}(\bar q-q_e).$$ Within-receiver rankings can agree under a local linear approximation, but this need not hold for finite interventions through a nonlinear readout. The paper acknowledges this distinction; even an accurate singleton estimator still needs validation against deployed action utility. Appendix F.6 defines the relevant audit with matched random selection and an offline oracle, but reports no outcomes. The component ablations do not resolve this, and the replacement policy in “No message effect” is unspecified. Learned sparsification is established in [NeuralSparse](https://proceedings.mlr.press/v119/zheng20d/zheng20d.pdf) and [PTDNet](https://arxiv.org/pdf/2011.07057), which jointly train the host; [GraphMask](https://arxiv.org/html/2010.00577v3) learns masks for a frozen model to preserve predictions; [TMTE](https://arxiv.org/html/2603.27723v2) learns task-aware multimodal topology. MEI's distinct contribution appears to be predicting singleton task-loss contrasts to guide constrained inference-time bundle attenuation. Its advantage over simpler frozen-host policies remains unestablished. Tables 1 and 4 do not isolate MEI from the identical open CM-MGNN host, whose architecture and auxiliary losses also differ from the baselines (Appendix E.1). The sensitivity maps provide partial no-intervention evidence: displayed maxima exceed zero-attenuation values by only 0.06 points on DY and 0.04 on Grocery (Figures 4 and 6). Their graph conditions and paired variability are needed to relate these gains to the main results. -
The cross-task evidence requires numerical substantiation and clearer protocols. All six Table 6 gains are exactly 2.80 points for GAT, 2.70 for MMGCN, 2.60 for MGAT and 2.40 for DGF, at displayed precision. Each spans Macro F1 and ranking metrics across four datasets. MRR/Hits@3 can share equal gains, but their correlation does not explain equality with Macro F1 on other datasets; MEI's budget imposes no such constraint. All 24 corresponding uncertainty terms also increase by 0.01.
Across all 25 combinations with both parameters nonzero, the DY and Grocery heatmaps satisfy , with residuals at most 0.005 points. Baseline uncertainty terms also match scaled and rounded Table 1 values in 54/54 cases in Table 2 (factors 1.05/1.10/1.16) and 53/54 in Table 3 (1.06/1.12/1.18). The run count, uncertainty definition and explanations of these relationships are missing. The maps are presented as hyperparameter evaluations without identifying any fitted surfaces. These observations do not identify their cause, but they leave the empirical evidence unresolved.
The cross-task protocols are also incomplete. Equations 10–11 define classification prototypes, yet Figure 6 ablates a prototype filter on Bili Dance without specifying its link-prediction construction. Table 9 supplies no node-class count for these link-prediction tasks. Appendix B's label-guided rewiring describes the motivating study; the labels, proxies or alternative construction used for DY and Bili Dance are unidentified. Moreover, increased cross-class connectivity alone does not establish a high absolute heterophily level. The original and achieved levels, together with the class or proxy-label distributions used in the construction, are needed to characterise the evaluated graphs and the scope of the heterophily claims.
Questions for authors
-
Could the authors report held-out sign accuracy for the singleton target and within-receiver rank correlations for predicted versus true singleton effects and for versus realised utility ? The Appendix F.6 audit results and clarification of the “No message effect” replacement would also be helpful.
-
Please confirm that the CM-MGNN rows in Tables 1–5 include MEI and label them explicitly. Paired results for the identical frozen host with and without MEI under matched graph conditions, alongside a comparison with a gate trained directly on task loss under the same constraints, would help isolate MEI's contribution.
-
Could the authors share paired Table 6 runs, seeds, uncertainty definitions and underlying prediction changes? It would be helpful to understand the uncertainty scaling and heatmap relationship, including whether entries are measured or fitted. The evaluation and table/figure-generation scripts would also aid verification.
-
Could the authors describe the prototypes and candidate rules used outside classification, the pressure-graph construction, and the original/achieved heterophily levels and any class or proxy labels used? For link prediction, could they also clarify how rewiring relates to held-out target edges and ranking candidates?