acceptodds
Under review as a conference paper at ICLR 2027

No-Free-Graph: Learning When Multimodal Data Should Be Graphified

Abstract

Multimodal graph learning has recently emerged as an effective paradigm for in corporating inter-entity relationships into multimodal representations. Existing studies have made substantial progress on how to construct and optimize graphs, but rarely consider a more fundamental question: whether additional relational structures should be introduced for a given dataset and task. Through empirical studies across diverse datasets, tasks, and graph constructors, we reveal that graphification is not consistently beneficial: introducing relational structures can provide substantial improvements in some cases, while offering limited or even negative gains. This observation motivates a new perspective that graph construction should be treated as a selective decision based on its expected utility rather than a default preprocessing step. To address this issue, we propose MAG-SCOUT, a pre-construction graph assessment framework that estimates whether introducing graph structures is beneficial before generating the complete topology. MAG-SCOUT collects limited relational evidence, analyzes its potential taskspecific contribution, and estimates the expected utility of graphification together with construction cost to make a build-or-skip decision. Extensive experiments across six multimodal datasets, three downstream tasks, and diverse graph constructors demonstrate that MAG-SCOUT effectively identifies when graph structures should be introduced, saving 33.1% of task-macro graph work while retaining 96.7% of held-out positive-gain mass under the pre-registered floor.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Learning, Graph Construction, Multimodal Graph Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aMQ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether a specified graph constructor should be invoked for a multimodal dataset and task. MAG-SCOUT collects a bounded relational sketch, combines necessity, constructability and utility diagnostics, and predicts the performance gain over a graph-free pipeline. Predicted cost and calibrated uncertainty enter a Build/Skip score. Evaluation includes nine constructors on six benchmarks, prediction and policy evaluation on a 13-parent-source registry, and component ablations on a separate rebuilt 11-source bank. At a validation-selected 95% retention target, the paper reports 96.7% task-macro retention of positive gain and 23.6% avoided graph work.

Critical strengths
  1. Equations 2–3 distinguish the incremental benefit over raw multimodal learning from absolute graph-model performance and account for construction cost. Table 1 reports both positive and negative gains under the evaluated pipelines. This motivates a practical assessment stage before committing to graph construction.

  2. Appendix B.2 establishes the greedy approximation for regularised log-det selection with fixed features and a cardinality budget, while explicitly limiting the guarantee to sketch volume. Appendix B.4 separates fitting, scale calibration, policy validation and outer testing by parent source. These are useful foundations for assessing the method without conflating the sketch guarantee with downstream predictive reliability.

Critical weaknesses
  1. The central decision and uncertainty results need reconciliation. At a 95% retention floor, Figure 4(b) places MAG-SCOUT's avoided graph work at approximately 33%, whereas Section 6.4 and Table 9 report 23.6%. At the 90% floor, the corresponding values are approximately 35% and 24.2%. These differences change the apparent comparison with competing policies and require corrected numerical inputs and plotting scripts.

    Table 6 also needs a more precise calibration account. Every reported mean width falls to approximately 78% of its raw value, while every coverage increases; for NC at the 80% nominal level, width falls from 4.69 to 3.66 and coverage rises from 72.5% to 77.4%. With fixed means and one common multiplier below one, each calibrated interval is contained in its raw interval, so coverage cannot increase on the same observations. Aggregation over outer folds with different multipliers can produce narrower mean widths and higher mean coverage, however. The aggregate table alone therefore does not establish a contradiction; fold-specific multipliers and paired predictions are needed to assess compatibility with Equation 41.

  2. The released implementation does not substantiate the proposed training procedure or benchmark comparisons. In the repository snapshot archived on 19 October 2026, experiments/evaluate_q1.py generates synthetic targets and defines MAG-SCOUT predictions as those targets plus noise. evaluate_q3_policy.py likewise generates gains and costs randomly, and evaluate_q4_q5.py prints a stability conclusion without running a sweep. These demonstrations do not reproduce the reported source-held-out results.

    There are also consequential method mismatches. MAGScout.forward does not call its relation encoder, and the supplied training loop takes an independently generated relation context. The loss omits the Gaussian uncertainty term in Equation 39, leaving the scale head without supervision through that objective. The NumPy selector omits the identity regularisation used by Algorithm 1; a direct check with duplicate candidates selects one relation despite a strictly larger regularised objective at budget two. Its projection update also raises a broadcasting error with three candidates in two dimensions. The one-time refinement ablated in Tables 3 and 8 is not specified as an executable procedure. A connected implementation and the actual experimental records are needed to determine which method produced the results. These observations establish gaps in the released reproduction package; the manuscript's experimental generation process remains unresolved.

Questions for authors
  1. Which values in Figure 4(b) or Table 9 are correct? Could the authors provide the corresponding plotting inputs and held-out episode records, including source, task, constructor, realised gain, measured cost, prediction and action? This would clarify the reported policy comparison.

  2. Appendix B.4 specifies calibration within each outer-training split. Could the authors provide the multiplier for each fold and seed, paired interval endpoints, and the coverage/width aggregation used for Table 6? What is the scope of "one global scale multiplier", and which uncertainty-training implementation produced these results? This would resolve whether the reported changes follow the stated procedure.

  3. Could the authors identify the implementation that connects relation encoding, discrete sketch selection, diagnostic extraction and the four-term loss, and specify one-time refinement? How does the selector used in the experiments implement the regularised marginal in Equation 19? This would distinguish an incomplete release from a mismatch in the evaluated algorithm.

  4. Could the authors report realised downstream utility or regret and net computation after assessment overhead, with a paired comparison against NetInfoF? Section 6.4 gives a saved-work difference of 0.5 points with a 95% source-bootstrap interval of [−6.7, 7.8]. Retained positive-gain mass does not account for performance losses on negative-gain Build cases, and Equation 50 does not subtract assessment overhead. These results would clarify the policy's incremental net benefit over NetInfoF.

Reviewer_cFX5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that graph construction is not always beneficial for multimodal learning. Therefore, it proposes MAG-Scout, a framework that samples a limited set of potential relationships and predicts whether building a full graph is worth the expected performance gain and computational cost. The main contribution is treating graph construction as a selective, cost-aware decision rather than a default preprocessing step.

Critical strengths
  • The paper addresses an important inefficiency in graph-based learning

  • The framework is a reusable decision-making layer for expensive graph ML operations.

  • The framework considers predicted performance alongside estimated graphification cost and uncertainty in the predicted gain.

  • Metrics other than performance are reported

Critical weaknesses
  • How is the graphification cost C(D,C) operationally defined in your experiments? Is it measured in computational operations, memory consumption, or another unit? How is it normalized to make it consistent with GraphGain when selecting λ ? How robust is MAG-SCOUT to different computational-cost definitions and cost preferences? For example, would the build-or-skip decisions remain effective changing scenarios ?

  • How are Y_graph and Y_raw obtained experimentally? Are both models evaluated on identical data splits using the same metrics and comparable hyperparameter-tuning budgets? Given that graph-based and raw-feature models can have different architectures, how do you ensure that the measured GraphGain isolates the benefit of graphification rather than differences in model capacity or training procedures?

  • How sensitive is MAG-SCOUT's graphification prediction accuracy to the choice of relational sketch construction algorithm? Does the proposed selector significantly improve GraphGain prediction and build-or-skip decisions, rather than merely optimizing the internal sketch objective?

  • Does the reported computational saving account for the full overhead of MAG-SCOUT for example : candidate-relation sampling, relational sketch construction, and utility prediction?

  • Do you also check for duplicate or near-duplicate entities and highly related graph examples across the training, validation, and test sets ? Do you use any graph similarity metric to verify this (meaning can you say they are statistically independent) ?

Questions for authors

Please see "weaknesses"

Reviewer_Srg7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates when graph construction is beneficial for multimodal learning. The authors show that graphification can improve or degrade performance depending on the dataset, task, and graph constructor. Based on this observation, they propose MAG-SCOUT, a framework that predicts graphification gains from limited relational evidence before constructing the full graph. The method combines relation sampling, three diagnostic branches, and a cost-aware Build/Skip policy. Experiments evaluate its prediction accuracy and ability to avoid unnecessary graph construction.

Critical strengths

The paper addresses a meaningful practical problem: deciding whether graph construction is worthwhile. The empirical results provide reasonable motivation, and the proposed framework follows a clear design. The evaluation covers multiple tasks and graph constructors, with source-held-out testing, component ablations, and statistical analysis of the main results.

Critical weaknesses

W1. The benefit of the proposed architecture is not sufficiently demonstrated on the final Build/Skip task. MAG-SCOUT improves GraphGain prediction on several metrics, but its final decision performance is very close to NetInfoF. At the 95% retained-gain target, MAG-SCOUT retains 96.7% of positive-gain mass and saves 23.6% of graph work, compared with 96.4% and 23.2% for NetInfoF. The reported paired bootstrap difference in saved work is only 0.5 percentage points (95% CI: [-6.7, 7.8]). The ablations in Table 3 mainly examine prediction errors and rank correlations rather than final policy performance. Evaluating these variants on the Build/Skip task, along with a simpler predictor using the same diagnostic features and decision procedure, would help clarify the contribution of the proposed architecture. The current results do not establish that its additional complexity leads to meaningfully better decisions.

W2. The policy evaluation does not fully capture the practical benefits of Build/Skip decisions. The main evaluation focuses on retained positive-gain mass and saved graph work. However, the former excludes negative GraphGain values and therefore does not directly capture the performance losses from building harmful graphs, which occur in several evaluated settings. Saved graph work also does not explicitly account for MAG-SCOUT's own assessment overhead, including candidate discovery, sketch selection, and diagnostic computation. Although decision latency is reported, it is unclear whether the full assessment pipeline is included. Reporting the signed downstream performance of the selected decisions and the end-to-end computational cost would provide a more complete assessment of the policy's practical value.

W3. The adaptation of different graph constructors to the common evaluation setting is not sufficiently explained. The nine graph constructors were originally designed for different settings. For example, AutoG focuses on graph schema generation from tabular data, GraphMaster synthesizes graph data, and LAGA uses heterogeneous graphs with auxiliary nodes. The paper evaluates these methods using a shared two-layer GCN, but Appendix C.3 provides limited details on how they are adapted to the multimodal benchmarks. It is also unclear which parts of the original methods are retained and how construction costs are measured consistently. Since these implementations determine both the GraphGain supervision and the cost-aware decisions, clearer details are needed to assess the reliability and reproducibility of the experiments.

Questions for authors

Please refer to the above comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.