acceptodds
Withdrawn from ICLR 2027

LEARNING TO DELETE: TASK-GUIDED MESSAGE-EFFECT INTERVENTION FOR HETEROPHILOUS MULTIMODAL GRAPH LEARNING

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal graph models propagate multiple forms of node information over a shared topology, yet the utility of an incoming relation can vary with both the receiver and modality. Uniform aggregation cannot express this difference, whereas broad suppression may discard useful content or uniformly shrink neighborhood input. A controlled motivating study further shows that greater connectivity across classes coincides with lower classification performance, with markedly different sensitivity across graphs and host models. We propose Message Effect Intervention (MEI), which learns the task loss contrast induced by removing one exposed modality message from a frozen host. MEI aggregates modality specific estimates into an edge score, applies task specific candidate filtering available at deployment, changes at most one incoming edge bundle per receiver, and rescales retained coefficients to preserve their total mass. Training class prototypes provide the concrete filter for node classification settings. Its fixed message interface supports both the reference CM-MGNN host and compatible multimodal hosts without replacing their representations. The deployed operator admits structural guarantees for recovery, bundle coherence, receiver specific feasibility, coefficient mass preservation, and protected computation paths. On the original task settings, CM-MGNN improves over the strongest displayed baseline by 4.85% in Movies Macro F1, 3.46% in Bili Dance MRR, and 3.51% in CIDEr. MEI also improves every displayed compatible host pair under the fifteen percentage point condition.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Multimodal graph learning, graph neural networks, message passing, heterophily, message attribution, frozen host, edge bundle intervention, node classification, link prediction, graph intervention

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KParOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Message Effect Intervention (MEI) for frozen multimodal graph models. An estimator predicts the task-loss change from deleting one modality-specific message. Its predictions guide candidate filtering and attenuation of at most one incoming edge bundle per receiver, with shared coefficients across modalities and receiver-local normalisation. The paper specifies CM-MGNN as a reference host, compatibility conditions for other hosts, and structural guarantees for the replay operator. Experiments cover seven datasets spanning node classification, link prediction, retrieval, and graph-to-text and graph-to-image generation.

Critical strengths
  1. A useful adaptation setting and task-grounded diagnostic.

    Improving a frozen multimodal model without updating its parameters is a meaningful setting. Equation 2 defines a concrete model-specific loss response, and the estimator uses training supervision while deployment uses available representations and context. This supports selective intervention in existing models.

  2. An explicit intervention boundary and operator contract.

    Equations 12–13 and Appendix C specify what changes during replay and establish identity recovery, a local selection budget, shared coefficients across modalities, and preservation of their sum. Appendix E.2 states the conditions required of another host. These properties make the intervention inspectable and distinguish structural guarantees from predictive benefits.

Critical weaknesses
  1. The diagnostic's utility and MEI's independent contribution need validation. Equations 2 and 8 supervise singleton message deletion, while Equations 9 and 12 deploy compensated bundle attenuation. Let dd count all exposed incoming messages, including the self-message. For one modality, write S=∑i=1dqiS=\sum_{i=1}^{d}q_i and qˉ=S/d\bar q=S/d. The respective perturbations are

    \delta S_{\mathrm{MEI}}=\frac{\eta d}{d-\eta}(\bar q-q_e).$$ Within-receiver rankings can agree under a local linear approximation, but this need not hold for finite interventions through a nonlinear readout. The paper acknowledges this distinction; even an accurate singleton estimator still needs validation against deployed action utility. Appendix F.6 defines the relevant audit with matched random selection and an offline oracle, but reports no outcomes. The component ablations do not resolve this, and the replacement policy in “No message effect” is unspecified. Learned sparsification is established in [NeuralSparse](https://proceedings.mlr.press/v119/zheng20d/zheng20d.pdf) and [PTDNet](https://arxiv.org/pdf/2011.07057), which jointly train the host; [GraphMask](https://arxiv.org/html/2010.00577v3) learns masks for a frozen model to preserve predictions; [TMTE](https://arxiv.org/html/2603.27723v2) learns task-aware multimodal topology. MEI's distinct contribution appears to be predicting singleton task-loss contrasts to guide constrained inference-time bundle attenuation. Its advantage over simpler frozen-host policies remains unestablished. Tables 1 and 4 do not isolate MEI from the identical open CM-MGNN host, whose architecture and auxiliary losses also differ from the baselines (Appendix E.1). The sensitivity maps provide partial no-intervention evidence: displayed maxima exceed zero-attenuation values by only 0.06 points on DY and 0.04 on Grocery (Figures 4 and 6). Their graph conditions and paired variability are needed to relate these gains to the main results.
  2. The cross-task evidence requires numerical substantiation and clearer protocols. All six Table 6 gains are exactly 2.80 points for GAT, 2.70 for MMGCN, 2.60 for MGAT and 2.40 for DGF, at displayed precision. Each spans Macro F1 and ranking metrics across four datasets. MRR/Hits@3 can share equal gains, but their correlation does not explain equality with Macro F1 on other datasets; MEI's budget imposes no such constraint. All 24 corresponding uncertainty terms also increase by 0.01.

    Across all 25 combinations with both parameters nonzero, the DY and Grocery heatmaps satisfy D−78.05≈1.5(G−84.80)D-78.05\approx1.5(G-84.80), with residuals at most 0.005 points. Baseline uncertainty terms also match scaled and rounded Table 1 values in 54/54 cases in Table 2 (factors 1.05/1.10/1.16) and 53/54 in Table 3 (1.06/1.12/1.18). The run count, uncertainty definition and explanations of these relationships are missing. The maps are presented as hyperparameter evaluations without identifying any fitted surfaces. These observations do not identify their cause, but they leave the empirical evidence unresolved.

    The cross-task protocols are also incomplete. Equations 10–11 define classification prototypes, yet Figure 6 ablates a prototype filter on Bili Dance without specifying its link-prediction construction. Table 9 supplies no node-class count for these link-prediction tasks. Appendix B's label-guided rewiring describes the motivating study; the labels, proxies or alternative construction used for DY and Bili Dance are unidentified. Moreover, increased cross-class connectivity alone does not establish a high absolute heterophily level. The original and achieved levels, together with the class or proxy-label distributions used in the construction, are needed to characterise the evaluated graphs and the scope of the heterophily claims.

Questions for authors
  1. Could the authors report held-out sign accuracy for the singleton target and within-receiver rank correlations for predicted versus true singleton effects and for −se-s_e versus realised utility UeU_e? The Appendix F.6 audit results and clarification of the “No message effect” replacement would also be helpful.

  2. Please confirm that the CM-MGNN rows in Tables 1–5 include MEI and label them explicitly. Paired results for the identical frozen host with and without MEI under matched graph conditions, alongside a comparison with a gate trained directly on task loss under the same constraints, would help isolate MEI's contribution.

  3. Could the authors share paired Table 6 runs, seeds, uncertainty definitions and underlying prediction changes? It would be helpful to understand the uncertainty scaling and heatmap relationship, including whether entries are measured or fitted. The evaluation and table/figure-generation scripts would also aid verification.

  4. Could the authors describe the prototypes and candidate rules used outside classification, the pressure-graph construction, and the original/achieved heterophily levels and any class or proxy labels used? For link prediction, could they also clarify how rewiring relates to held-out target edges and ranking candidates?

Reviewer_QibTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Message Effect Intervention (MEI), a task-guided method that estimates the loss change caused by removing individual modality messages from a frozen multimodal graph model. MEI converts these estimates into receiver-specific physical-edge interventions that preserve modality-bundle coherence, restrict the number of modified edges, and maintain total incoming coefficient mass. Across multiple datasets and tasks, the paper reports improved performance for its CM-MGNN reference model and for several compatible hosts under controlled heterophily perturbations.

Critical strengths
  • The paper identifies a meaningful distinction between diagnosing the effect of individual modality messages and executing coherent interventions on physical multimodal edges.
  • The intervention operator is carefully constructed, with explicit guarantees for recovery of the original host, bundle coherence, receiver-level feasibility, coefficient-mass preservation, and protected computation paths.
  • The empirical study covers multiple prediction and generation tasks, host architectures, controlled heterophily conditions, component ablations, and hyperparameter sensitivity.
Critical weaknesses
  • Several important experimental settings are missing. The paper does not specify the text and image feature extractors, whether all baselines use the same embeddings, or how their hyperparameters are tuned. It also provides no code or anonymized repository link.
  • The experiments use only relatively small graphs, although established multimodal benchmarks contain graphs approaching a million nodes. The component ablations also show only modest changes in performance.
  • The paper includes no efficiency analysis or experiments. The computational overhead and practical scalability of MEI therefore remain unclear.
Questions for authors

See Weaknesses.

Reviewer_8qSAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Message Effect Intervention (MEI), a task-guided module for controlling messages in frozen multimodal graph models. MEI learns to predict the task-loss change caused by removing an individual modality-specific message. At deployment, it averages these estimates into edge scores, applies task-specific eligibility rules, and attenuates at most one incoming edge bundle per receiver while renormalizing the intervention coefficients. Training-class prototypes instantiate the eligibility filter for node classification. The paper introduces CM-MGNN as a reference host, establishes structural properties of the intervention operator, and reports results across graph-centric and modality-centric tasks, including experiments with compatible frozen hosts.

Critical strengths
  1. A useful and clearly motivated control problem. Improving a trained multimodal graph model without replacing its representations is a worthwhile objective. Supervising message scores with observed task-loss changes is more directly related to the downstream task than interpreting attention weights as message utility. The distinction between modality-specific diagnosis and a shared edge-bundle action also addresses a concrete aspect of multimodal message control.

  2. Explicit intervention boundaries and appropriately qualified guarantees. The method specifies a frozen replay boundary, a receiver-level action budget, shared coefficients across modalities, and preservation of declared receiver paths. The analysis makes these operator properties explicit, and the paper appropriately avoids claiming that they guarantee prediction improvement or establish real-world causality. The reported evaluations span several tasks and compatible hosts, providing a potentially useful setting for assessing the proposed plugin.

Critical weaknesses
  1. The evidence does not yet connect the learned diagnostic to the benefit of the deployed action. Equation (2) measures complete removal of one modality message, whereas Equation (12) jointly attenuates all modalities of an edge and rescales the incoming coefficients. Even an accurate singleton predictor need not correctly rank these compensated bundle actions. The paper acknowledges this distinction, but the most direct evaluation of actual action utility in Appendix F.6 is explicitly a planned audit with no reported outcomes. Moreover, the overall comparisons evaluate CM-MGNN, whose ego residual, fusion, and auxiliary alignment objectives may themselves explain part of the gains. The current component ablations do not clearly specify what selection rule replaces the removed estimator. A comparison on the same frozen host between no intervention, budget-matched random intervention, and MEI is needed to establish the central contribution. The structural guarantees do not resolve this empirical gap.

  2. Key experimental results require clarification before they can support the robustness and host-reuse claims. The pressure construction preserves node features and labels while changing topology, yet the feature-only MLP's Grocery accuracy changes from 81.19 ± 0.25 on the original graph to 84.03 ± 0.29 at the fifteen-percentage-point condition. This requires an explanation of whether splits, preprocessing, or training conditions also change. In Table 6, each host has exactly the same reported improvement across all six metric columns: +2.80 for GAT, +2.70 for MMGCN, +2.60 for MGAT, and +2.40 for DGF; every corresponding standard deviation increases by 0.01. These patterns do not by themselves establish an error, but they warrant clarification of the underlying runs and aggregation procedure because this table is central evidence for the plugin's generality.

Questions for authors

Q1. Can you provide the action-utility results outlined in Appendix F.6, particularly the comparison with random selection under the same receiver set, candidate set, attenuation strength, and compensation rule? How well do the learned singleton scores rank the actual compensated bundle actions?

Q2. What exactly is changed in the “No message effect” ablation, and which policy selects edges in that variant? Please report the fully open CM-MGNN and MEI-controlled CM-MGNN using the same frozen checkpoint and evaluation setting, so that the plugin's contribution can be separated from the host architecture and auxiliary training objectives.

Q3. Why does the feature-only MLP change substantially across the Grocery topology-pressure conditions? Are node features, labels, train/validation/test splits, preprocessing, hyperparameters, and random-seed protocols held fixed across these conditions?

Q4. Could you explain the identical within-host improvements across all six columns of Table 6 and the uniform 0.01 increases in standard deviations? Please clarify the number of runs, the meaning of the reported deviations, and the per-run evaluation and aggregation procedure.

Q5. How are the heterophily-pressure levels and deployment candidate filters defined for link prediction and generation? Appendix B defines pressure through node-class labels, while Table 9 lists no node-class labels for these datasets. Please specify the relevant labels or proxy relations and the information available to each task adapter at inference.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.