acceptodds
Under review as a conference paper at ICLR 2027

GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective

Abstract

Multimodal retrieval uses images, text, and object relationships to answer different questions about the same collection. A query may seek an object’s paired description, another object in the same category, or an object connected by an observed relationship. These goals rely on different notions of relevance. Paired matching requires object-specific distinctions, whereas cross-object retrieval benefits from relational agreement. Existing methods learn strong cross-modal correspondence or propagate information over a graph, but a shared output regularized toward neighbors can weaken identity distinctions. Moreover, gains from graph regularization can diminish after uniform graph propagation. We introduce Graph-Optimized Multimodal Alignment (GOMA), which assigns these roles to two connected embeddings. Each modality produces a content embedding directly supervised for paired identity and a semantic embedding jointly trained with cross-modal pairs and observed relationships. For complete records, both embeddings form an initial fused representation, semantic agreement sets positive weights on observed edges, and restart graph propagation reinjects this initial signal. This design lets a jointly trained model support single-modality and dual-attribute retrieval through a task-specific readout. Across six datasets and four tasks, GOMA achieves state-of-the-art performance on all 14 primary measures against 14 external methods. Controlled comparisons further show how separate supervision, graph regularization, and semantic-guided graph propagation shape the final representation and align the learned signal with each retrieval target.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
multimodal retrieval, multimodal graphs, cross-modal alignment, graph signal processing, graph propagation, representation learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_84BWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

GOMA addresses different relevance targets in multimodal retrieval: matching an object's own image and text, retrieving another object in the same category, and ranking related objects. It learns content and semantic embeddings with different supervision, uses semantic similarity to weight observed graph edges, and applies restart propagation. A task-specific readout supports four tasks. Across six datasets, the paper reports leading results on 14 primary measures and includes component and representation analyses.

Critical strengths
  1. The paper starts from an interesting distinction: graph neighbors need not be uniformly close for every retrieval target. Separating identity from cross-object semantics makes the task formulation and readout convincing.

  2. The empirical evaluation is extensive. Results are consistent on the primary measures, and the ablations examine supervision placement, graph regularization, and propagation.

Critical weaknesses

1.The novelity is limited. The method combines established representation learning, graph smoothing, and restart propagation techniques, so its advance over a closely matched simpler model remains unclear.

2.Several graph baselines require substantial adaptation for single-modality retrieval, making the improvements over those methods harder to interpret.

Questions for authors
  1. If you retrain a matched model with the same embeddings, losses, and restart propagation but uniform edge weights, how much improvement remains on each task?

  2. How did you verify that the single-modality adaptations preserve the graph baselines’ capabilities, and can you provide a comparison using a baseline designed for the same inference interface?

5 comments

Thank you for recognizing the relevance distinction and empirical evaluation. We address the matched simpler model and adaptation questions below.

Q1. GOMA's fully retrained uniform-weight ablation leads all 14 primary means against the external comparators. It retains GOMA's content-to-semantic architecture, objectives, graph regularization, fusion, and restart propagation. We trained it on all six datasets and three seeds with the same optimizer, sampling, budget, and validation rule. Only observed-edge weights become uniform before degree normalization. Each model selects its own Joint validation checkpoint (MRR for relation datasets). Both use learning rate 0.001 and α=0.1,K=8\alpha=0.1,K=8.

Sections 5.3 and 5.5 distinguish the retrained control from fixed-checkpoint interventions. The table shows its gain over the strongest external comparator, in percentage points, for every task. Full means and three-seed sample SDs are in Appendix H.1, Table 17.

DatasetIdentity R@1Single mAP@10Dual mAP@10Relation MRR
Grocery+0.27+2.61+3.43—
Reddit-S+0.14+3.43+9.63—
Toys+0.67+7.08+11.03—
Books+0.96+2.70+3.29—
Sports———+0.10
Cloth———+0.82

These gains are retained by GOMA's learned content and semantics. Its single-modality semantic gains are already present when each query and candidate uses only its own attribute. Reusing zz to guide neighborhood aggregation then adds 0.75/0.05/0.39/0.21 dual-semantic points on Grocery/Reddit-S/Toys/Books over the uniform ablation. Uniform weighting gives 0.51/0.66 higher Sports/Cloth MRR. Differences use unrounded means.

The tasks reward different matches: category retrieval seeks similar objects, whereas relation ranking can require a complementary item. GOMA's semantic guide emphasizes close semantic neighbors and improves every dual-attribute category task. With one jointly selected model per dataset/seed, GOMA thus provides the paired matching results above alongside substantial cross-object semantic gains, and reuses the same learned semantics to improve complete-record retrieval. This full-training comparison complements the fixed-checkpoint intervention in Table 12.

W1. GOMA's design connects graph-assisted alignment to the representation used for each retrieval goal. GOMA learns zz from identity-supervised cc using paired correspondence and graph agreement, retains both outputs for their retrieval targets, and reuses zz to guide complete-record propagation. At equal capacity, separating identity supervision and scoring improves identity by 0.53–2.04 points and semantics by 3.05–7.49 over shared-output supervision (Table 4). Against a parameter-matched shared backbone with parallel task heads, GOMA leads 11/14 means under each of two loss-coefficient controls (largest gain 1.48 points; Table 17). The matched controls show the value of learning relational semantics from identity-supervised content. Reusing these semantics to guide propagation further improves every dual-attribute category task over the uniform counterpart.

W2 and Q2(a). We corrected a training–inference mismatch and replaced the affected single-modality results in Tables 1 and 5–6. Ten graph adapters optimized paired retrieval on graph outputs but were evaluated on independently encoded attributes. The corrected adapters retain graph computations and method-specific auxiliary objectives, and average paired retrieval losses on graph outputs and independent outputs. The independent training computation is the one used at inference. DGF directly supervises its independent modality outputs.

The single-modality input rule is the same for every method, including GOMA: each query and candidate uses only its own visible attribute, without the missing attribute or evaluation-time neighbors. Graph information is available during training. We checked the adaptations against their sources: GCN/GAT/GraphSAGE/LGMRec retain the retrieval cores supplied by OpenMAG [1], with our independent-input changes identified separately; CoMAG retains its full multihop matcher; RoleMAG uses its original multihop recurrence; and DMGC uses its prescribed category counts. Appendix F.3 describes the current computations and supervision.

The corrected retrieval results provide direct performance evidence alongside these source checks. GCN/Grocery semantic mAP@10 increases from 8.86 to 32.04, and DMGC/Reddit-S from 50.60 to 56.18 (three-seed means). The corrected adapters receive up to 1,000 updates and task-specific validation selection; GOMA retains its 500-update cap and one jointly selected checkpoint. Against the corrected comparisons, GOMA leads the four single-modality semantic tasks by 2.61/3.56/7.62/2.84 points on Grocery/Reddit-S/Toys/Books.

Q2(b). VSE++, CLIP-Refine, and SigLIP directly match the independent-attribute inference interface. Their objectives cover hard-negative matching [2], contrastive modality-alignment distillation [3], and pairwise sigmoid matching [4]. Like GOMA, each encodes a query or candidate from its own visible attribute without an inference graph. They use the same frozen features, partitions, candidates, and metrics. Validation selects their task-specific learning rates and checkpoints; GOMA retains one model per dataset/seed for all tasks. Appendix H.3 specifies the baseline selection and reporting rules, and H.4 the joint GOMA selection.

Tables 1 and 5–6 report both bidirectional identity R@1 and single-modality semantic mAP@10 for these independent encoders. Against the strongest corresponding semantic comparator, GOMA gains 3.43/7.07/7.62/3.41 points on Grocery/Reddit-S/Toys/Books. Against these independent encoders, GOMA also leads identity R@1 by 0.27/0.23/0.09/1.06 points on the same four datasets. Across the full main comparison, GOMA leads all 14 primary means (Tables 1–2).

References

[1] Chenxi Wan et al. OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graph. ICML, 2026.

[2] Fartash Faghri et al. VSE++: Improving Visual-Semantic Embeddings with Hard Negatives. BMVC, 2018. [3] Shin'ya Yamaguchi et al. Post-pre-training for Modality Alignment in Vision-Language Foundation Models. CVPR, 2025. [4] Xiaohua Zhai et al. Sigmoid Loss for Language Image Pre-Training. ICCV, 2023.

Dear Reviewer 84BW,

With the rebuttal deadline approaching, we wanted to ask whether you have had a chance to read our response.

Could you let us know if the retrained uniform-weight controls and clarified baseline adaptations address your concerns?

We are happy to provide any further clarification. Thank you for your time.

Reviewer_5Nc5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Graph-Optimized Multimodal Alignment (GOMA), which learns a content embedding directly supervised for paired identity and a semantic embedding jointly trained with cross-modal pairs and observed relationships. This design lets a jointly trained model support four retrieval tasks through a task-specific readout. The authors evaluate the method against 14 external methods on 6 datasets.

Critical strengths
  1. The design of the model structure is well-motivated and supported by targeted empirical analysis. Identity retrieval and cross-object semantic retrieval tasks are distinguished, with different learned embeddings. The content embedding and the semantic embedding are supervised by dedicated losses. Extensive experiments are conducted. For example, Table 4 separates supervision placement from capacity and shows improvements in both identity and semantic retrieval.
Critical weaknesses
  1. The methodological contribution is limited. Supervising representations from different depths of a serial network with task-specific objectives has been adopted in NLP [1], representation learning [2] and recommendation systems [3] before. The methodological contribution of GOMA appears primarily to lie in applying this architectural separation to identity matching and cross-object retrieval, combined with graph regularization and semantic-guided graph propagation. Its semantic edge-weight function matches, up to a global scaling factor, a weighting rule listed in Table 1 of TWIRLS [4]. The fixed-model comparisons in Table 12 show only modest, task-dependent benefits of semantic edge weighting over uniform weights, and lower MRR on both relation-ranking datasets.

[1] Hashimoto, Kazuma, et al. "A joint many-task model: Growing a neural network for multiple NLP tasks." Proceedings of the 2017 conference on empirical methods in natural language processing. 2017.

[2] Yu, Jiahui, et al. "Coca: Contrastive captioners are image-text foundation models." Transactions on Machine Learning Research, 2022.

[3] Yan, Mingshi, et al. "Cascading residual graph convolutional network for multi-behavior recommendation." ACM Transactions on Information Systems 42.1 (2023): 1-26.

[4] Yang, Yongyi, et al. "Graph neural networks inspired by classical iterative algorithms." International Conference on Machine Learning. PMLR, 2021.

Questions for authors
  1. Task-specific representations could be obtained either at different depths of a serial network or through parallel heads over a shared backbone. How does GOMA compare with a parameter-matched shared-backbone baseline with two parallel task-specific heads, using the same supervision objectives, graph regularization, and task-specific readouts?
  2. The paper sets η and λ once from gradient norms on ten training batches at initialization, then retains them throughout learning; α=0.1\alpha=0.1 and K=8K=8 are fixed across all datasets. How sensitive are the reported improvements to the choices of hyperparameters, particularly the loss-balancing coefficients (η,λ)(\eta,\lambda) and propagation parameters (α,K)(\alpha,K)? Could the authors provide sensitivity results showing whether the gains over the strongest baselines remain consistent when these hyperparameters are varied around their prescribed values?
5 comments

Thank you for highlighting the supervision design and suggesting matched-head and sensitivity comparisons.

W1. GOMA learns relational semantics from identity-supervised content and uses them to guide retrieval. Image-text pairs identify the same object; graph links connect different objects. GOMA learns from both through a serial transformation. Identity loss directly supervises the content embedding cc. Paired correspondence and graph agreement train a second mapping from cc to the semantic embedding zz, with semantic gradients updating both mappings. The semantic mapping thus learns relations from content that simultaneously receives direct paired-identity supervision. Identity retrieval reads cc, and semantic retrieval reads zz. For complete records, zz also sets observed-edge weights and joins content in the initial representation for restart propagation.

This design follows our empirical distinction between the retrieval targets. At equal capacity, placing identity supervision and scoring at cc improves identity by 0.53–2.04 points and semantics by 3.05–7.49 over shared-output supervision (Table 4). Removing graph regularization lowers single-modality semantic mAP@10 by 2.13–6.80 points (Table 10).

The cited methods provide precedents for supervision at different depths: Hashimoto et al. [1] organize linguistic tasks hierarchically, CoCa [2] combines contrastive and captioning objectives, and CRGCN [3] successively models behavioral graphs. GOMA's technical advance connects paired and relational alignment: the semantic mapping transforms identity-supervised content, and the resulting semantics determine both cross-object scores and neighborhood influence. This makes the learned relationships serve representation learning and complete-record propagation within one model. Q1 isolates the serial transformation through a matched parallel-head comparison.

The edge kernel follows TWIRLS's logarithmic-penalty rule [4], up to a global factor that cancels in normalized propagation; Section 4.3, Eq. (4), and Appendix A.3 credit this explicitly. GOMA computes weights once from its learned semantic embeddings, whereas TWIRLS updates weights with optimization iterates. In the full-model comparison, semantic weighting improves all four dual-attribute category tasks by 0.05–0.75 points, while uniform weighting gives higher MRR on both relation datasets (Table 17). These tasks reward different matches: category retrieval seeks similar objects, while a co-purchase target can be a complementary item. GOMA's guide gives greater influence to semantically close neighbors, matching the category-retrieval goal. Its four category gains build on the graph-assisted alignment learned by the content-to-semantic model; Q1 tests that model against parallel heads.

Q1. GOMA exceeds each parameter-matched parallel model on 11/14 primary means. We trained a learned shared backbone per modality followed by two parallel task heads. Both heads receive the backbone output; the semantic head does not read the identity head. The architecture exactly matches GOMA's 8d2+8d8d^2+8d trainable parameters. The objectives, graph regularization, task-specific readouts, fusion, restart, optimizer, sampling, budget, and validation rule match, at learning rate 0.001 and α=0.1,K=8\alpha=0.1,K=8. We tested both GOMA's loss coefficients and an independently calibrated parallel model.

Across the two comparisons, the largest gain is 1.48 points and the largest deficit is 0.22, using unrounded means. Under both controls, GOMA improves identity R@1 on Grocery, Reddit-S, and Toys and improves all four dual-attribute category measures. With the same capacity and objectives, this comparison supports learning relational semantics from identity-supervised content and retaining both outputs for retrieval. All 14 means, three-seed sample SDs, and architecture details are in Appendix H.1, Table 17.

References

[1] Kazuma Hashimoto et al. A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks. EMNLP, 2017. [2] Jiahui Yu et al. CoCa: Contrastive Captioners are Image-Text Foundation Models. TMLR, 2022. [3] Mingshi Yan et al. Cascading Residual Graph Convolutional Network for Multi-Behavior Recommendation. ACM TOIS 42(1), online 2023. [4] Yongyi Yang et al. Graph Neural Networks Inspired by Classical Iterative Algorithms. ICML, 2021.

Q2. All eight category-semantic gains survive eight of nine tested settings; all four dual-attribute gains survive every setting. At learning rate 0.001, we tested the reference (α=0.1,K=8\alpha=0.1,K=8), multipliers {0.5, 2} on either η\eta or λ\lambda, α∈{0.05,0.2}\alpha\in\{0.05,0.2\}, and K∈{4,16}K\in\{4,16\}. Each variation holds the other settings fixed and uses the same three seeds and validation rule. Appendix H.2, Tables 18–19 give all 14 primary means and sample SDs; comparisons use the strongest external means in Tables 1–2.

In L=Lid+η(Lpair+λEG)L=L_{id}+\eta(L_{pair}+\lambda E_G), η\eta balances identity and semantic learning; λ\lambda balances graph agreement and paired correspondence within semantic learning. The same initialization rule applies to every dataset: ten training-only batches give η=aˉ/bˉ\eta=\bar a/\bar b and λ=bˉ/cˉ\lambda=\bar b/\bar c, where aa is the identity-gradient norm on the content mapping and b,cb,c are paired/graph-gradient norms on the semantic mapping. The numerical ratios depend on the dataset and seed, then remain fixed throughout that run. Section 4.2, Eq. (3), and Appendix D.1 specify this training-only calibration. Gradient balancing has precedent in GradNorm [1], which instead updates weights during training.

Changing η\eta by a factor of two changes single-semantic means by at most 0.55 points; changing KK to 4 or 16 changes any primary mean by at most 0.47. Restart balances the original representation and neighbor information, following APPNP [2]. Increasing α\alpha to 0.2 lowers Sports/Cloth MRR by 2.13/1.88 points. At α=0.05\alpha=0.05, MRR reaches 88.53/81.94, above the strongest external means of 88.31/81.16. The results favor retaining more neighbor information for these relation-ranking tasks.

For 2λ2\lambda, Reddit-S single-semantic mAP@10 changes from 59.74±0.2659.74\pm0.26 to 32.23±2.5132.23\pm2.51, compared with DMGC's 56.18; Grocery reaches 40.10 versus DMGC's 40.21. Doubling λ\lambda increases graph agreement relative to paired correspondence. GOMA's initialization calibration instead scales the graph term to the paired term's gradient norm. At that calibrated balance, GOMA leads all eight category-semantic measures; removing the graph term lowers single-semantic mAP@10 by 2.13–6.80 points on every category dataset (Table 10). These interventions support both using the extra relational information and balancing its influence against paired alignment.

References

[1] Zhao Chen et al. GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks. ICML, 2018. [2] Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. ICLR, 2019.

Dear Reviewer 5Nc5,

The rebuttal period will end soon, so we would appreciate your feedback on our response.

We hope the matched parallel-head comparisons and sensitivity analysis have addressed your concerns. Please let us know what you think.

If you have any further questions, we would be happy to discuss them.

Reviewer_YpDSOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies retrieval over multimodal graphs under four tasks. GOMA learns the content embedding and semantic embedding for each modality. The fused content+semantic representation is propagated with an APPNP-style restart filter over the observed graph. The paper reports best results on 14 primary measures across six datasets against 14 adapted baselines.

Critical strengths
  • Targeting different retrieval tasks, the proposed GOMA aligns different components into a unified solution. Personally, I do not think it hurts the overall novelty very much.
  • The reported performance shows gains over baseline methods across different datasets.
Critical weaknesses
  • The motivation of the paper is not very strong. Especially, the entire section 2, problem formulation, is abstract, not illustrative, and not easy to follow. The authors may want to add more real-world examples to motivate and explain each task, and distinguish the relationships and uniqueness of different tasks.

  • In section 4, the methodology, each proposed technique comes without sufficient motivation and context. It is not easy to evaluate the necessity of each component.

  • According to Table 3, the major gains seem to come from the propagation. However, in Table 12, the uniform edge weights perform quite well across all dataset benchmarks.

Questions for authors

How exactly are VSE++/CLIP-Refine/SigLIP evaluated in the dual-attribute setting?

Were baseline hyperparameters tuned on validation data?

4 comments

Thank you for recognizing GOMA's unified design and empirical gains. GOMA uses links between objects as additional evidence for image-text alignment while retaining the distinctions needed to identify an object's own pair.

W1. Why distinguish four tasks? Fig. 1 follows searches over the same product collection. A backpack image can seek its own description (identity) or a different backpack's description (single-modality semantic retrieval). A complete image-text record can seek other backpacks using both attributes and the observed graph (dual-attribute semantic retrieval). Recovering a missing co-purchase link instead seeks a connected item, possibly a complementary accessory (relation ranking).

The distinction is both practical and operational: identity requires the same object; semantic retrieval requires another object in the same category; relation ranking requires the linked object. The two semantic tasks share relevance but differ in available inputs. Single-modality queries and candidates each use only their own visible attribute. Semantic query/gallery objects are disjoint, and relation targets and reverse edges are removed before propagation. Sections 2.2–2.3 connect these rules to the example and apply them equally to every method.

W2. How the empirical analysis motivates the method. Section 3 and Appendix B.4, Tables 13–14, connect the three findings in Fig. 2 to the following design choices.

  1. Graph links add information beyond paired attributes. Same-category edge agreement exceeds degree-preserving relabelings by 22.67–62.97 percentage points across the four category datasets (Fig. 2a). GOMA therefore learns from both image-text pairs and observed cross-object links: paired correspondence trains semantic alignment, while graph regularization encourages agreement along training relations.
  2. Identity and semantic retrieval favor different representations. The shared-output probe evaluates intermediate and final embeddings at the same learned weights (Fig. 2b). GOMA directly supervises the content embedding cc for identity, then maps cc to the semantic embedding zz, supervised by paired correspondence and graph agreement. Semantic gradients update both mappings. Retrieval reads cc for identity and zz for semantics. At equal capacity, this placement improves identity R@1 by 0.53–2.04 points and semantic mAP@10 by 3.05–7.49 over shared-output supervision (Table 4).
  3. Learned semantics guide how neighbors contribute to retrieval. Uniform propagation reduces part of the gain from graph regularization in the matched probe (Fig. 2c). GOMA therefore uses its learned semantics to weight observed edges: neighbors closer in semantic space receive greater weight. For complete records, content and semantics also form the initial representation, which restart reinjects at every propagation step, following APPNP [1]. The learned semantics therefore contribute to the propagated representation and determine the weights with which observed neighbors contribute.

Removing graph regularization lowers single-modality semantic mAP@10 by 2.13–6.80 points on all four datasets (Table 10), so graph-assisted alignment already benefits retrieval before inference-time propagation. Sections 4.1–4.3 explain how each operation follows this motivation; W3 below separates propagation's contribution from the additional edge-weight benefit.

References

[1] Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. ICLR, 2019.

W3. GOMA's alignment design retains strong gains when its semantic edge weights are replaced by uniform weights. This is an ablation of GOMA: it retains our identity-supervised content and graph-regularized semantic mappings, objectives, fusion, and restart propagation, and changes only the observed-edge weights. At the common reference setting (learning rate 0.001, α=0.1,K=8\alpha=0.1,K=8), the fully retrained ablation uses its own validation selection and leads all 14 primary means against the external comparators. All eight category-semantic measures exceed the strongest external comparisons by 2.61–11.03 points (Appendix H.1, Table 17).

GOMA further uses the learned semantics to guide propagation: semantically closer neighbors receive greater edge weights. This adds 0.75/0.05/0.39/0.21 points in dual-attribute semantic mAP@10 on Grocery/Reddit-S/Toys/Books over the uniform ablation. Differences use unrounded means. The same semantic representation therefore supports cross-object scoring and determines how neighbors contribute to complete-record retrieval. The uniform ablation shows that GOMA's alignment design works with a standard propagation filter; semantic guidance further improves all four dual-attribute category tasks. The complete comparison in Table 17 tests full-model training and selection, complementing the fixed-checkpoint intervention in Table 12.

Q1. How VSE++/CLIP-Refine/SigLIP are evaluated with both attributes. For each complete query or candidate record, we obtain its image and text embeddings, normalize each, sum them, and normalize again: hi=N(N(vi)+N(ti))h_i=\mathcal N(\mathcal N(v_i)+\mathcal N(t_i)). We rank by hi⊤hjh_i^\top h_j using the same candidates and relevance rules as GOMA. These methods rank the fused embeddings directly, without graph propagation. Appendix F.2 gives this fusion rule; Section 4.4, Eq. (6), specifies retrieval scoring.

Q2. Validation-based hyperparameter selection. Yes. In the revised comparisons, learning rates for VSE++, CLIP-Refine, SigLIP, GCN, and LGMRec are selected from {0.0003, 0.001, 0.003} by the three-seed mean validation primary score. Baseline checkpoints and applicable output representations also use validation selection. Sports CoMAG selects among four validation configurations. Appendix H.3 documents these selections and the reporting rule: across completed comparison recipes, we retain the higher reported primary result for each external method, with secondary metrics from that same state. This reporting rule is separate from validation-based selection within each recipe.

GOMA uses one jointly selected learning-rate/restart pair per dataset and one checkpoint per seed for all tasks (Appendix H.4). The uniform and parallel comparisons keep the common reference setting to isolate the requested design changes.

Dear Reviewer YpDS,

As the rebuttal period is coming to an end, we wanted to follow up on our response.

Could you let us know whether our task clarifications and uniform-weight comparisons address your concerns?

We would be happy to answer any further questions. Thank you for your time.

Dear Reviewer YpDS,

As the discussion period draws to a close, we would like to briefly follow up on your concern about whether GOMA’s gains mainly come from propagation.

The representation design contributes before propagation. The uniform-weight ablation retains GOMA’s learned content and semantic representations and graph-regularized training. Removing graph regularization lowers single-modality semantic mAP@10 by 2.13–6.80 percentage points, without inference-time propagation (Table 10). At equal capacity, using the content representation for identity supervision and scoring also improves both identity and semantic retrieval (Table 4).

Semantic edge weights further guide propagation. For category retrieval, observed neighbors need not be equally relevant. GOMA reuses its learned semantics to give semantically closer neighbors greater weight. The fully retrained matched comparison shows improvements on all four dual-attribute category tasks (Table 17). Thus, the same learned semantics support retrieval and guide how neighbors contribute.

Could you let us know whether this addresses your concern about component necessity, or whether any point remains unresolved? If your concerns have been addressed, we would appreciate an updated assessment. Thank you for your time and consideration.

Comments on the paper

1 comment

We sincerely thank Reviewers YpDS, 5Nc5, and 84BW for their thoughtful and constructive reviews of our paper introducing GOMA.

We appreciate that the reviewers recognized several strengths of our work, including:

(1) a unified framework for four multimodal graph retrieval tasks, distinguishing exact identity matching, cross-object semantic retrieval, and relation ranking (YpDS, 5Nc5, 84BW);

(2) a task-motivated separation of content and semantic representations, supported by targeted analyses of supervision placement and graph regularization (5Nc5, 84BW); and

(3) extensive evaluation across six datasets, with improvements over external baselines and ablations examining representation learning, graph regularization, and propagation (YpDS, 5Nc5, 84BW).

During the rebuttal, we addressed the reviewers’ concerns with additional experiments and clarifications:

  • Methodological contribution and matched architectures (5Nc5 W1&Q1; 84BW W1) — We added parameter-matched parallel-head controls. GOMA outperforms the controls on 11/14 primary means. We further clarified that its semantic embedding supports both cross-object retrieval from individual modalities and semantic-guided weighting of observed edges.

  • Graph contribution and propagation (YpDS W3; 84BW Q1) — We fully retrained uniform-weight variants across all six datasets. They retain GOMA’s architecture, objectives, graph regularization, fusion, and restart propagation, changing only edge weights, and outperform the strongest external comparators on all 14 primary means. Semantic weighting further improves all four dual-attribute category tasks by 0.05–0.75 points, while uniform weights perform better on the two relation-ranking datasets.

  • Baseline fairness and evaluation (YpDS Q1&Q2; 84BW W2&Q2) — We corrected the training–inference mismatch in ten graph-baseline adaptations and added independent-modality comparisons with VSE++, CLIP-Refine, and SigLIP. GOMA improves single-modality semantic mAP@10 over the strongest corresponding comparisons by 2.61/3.56/7.62/2.84 points on Grocery/Reddit-S/Toys/Books.

  • Hyperparameter robustness (5Nc5 Q2) — All four dual-attribute category gains remain positive across every tested setting, and eight of nine settings preserve all eight category-semantic gains.

The discussion also led to important reviewer feedback. Reviewer 84BW explicitly confirmed that the retrained uniform-weight controls and clarified baseline adaptations addressed their main concerns and indicated that they would raise their score. Reviewer 5Nc5 acknowledged our additional experiments but maintained their recommendation because they continued to view the methodological contribution as incremental. We provided a further clarification of GOMA’s distinctive use of the learned semantic representation. We also addressed Reviewer YpDS’s questions regarding task formulation, baseline evaluation, and validation-based model selection.

We respectfully ask the Area Chair to consider the additional controlled evidence and the reviewer feedback when making the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.