acceptodds
Under review as a conference paper at ICLR 2027

OmniSpecGen: Where to Fuse Multimodal Spectra for Molecular Structure Prediction

Abstract

*De novo* molecular structure prediction from spectral measurements is a fundamental task in chemistry. Although existing models have primarily focused on single modalities or predefined modality sets, the measurements available in practice vary across molecules, calling for flexible use of complementary information from arbitrary subsets of modalities. The heterogeneous formats of these spectra motivate modality-specific processing, but the generation pipeline must transition to joint processing to generate a single molecule. This raises a central architectural question: *To best exploit complementarity, how much processing should remain modality-specific before joint processing begins?* To address this question, we introduce *OmniSpecGen*, a unified framework for comparing four fusion stages within a common diffusion architecture. Our comparison identifies fusion of modality-specific representations before pooling as the most effective strategy, with the resulting model outperforming the current state-of-the-art model for flexible spectrum-to-structure generation across the evaluated multimodal configurations. These findings clarify when to begin joint processing and provide an effective framework for molecular structure elucidation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Molecular structure elucidation, Spectroscopy, Molecules, Diffusion models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pBrvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates the impact of multimodal fusion strategies on molecular structure elucidation from spectral data. The authors present OmniSpecGen, a unified framework for systematically evaluating the integration of different spectral modalities at various stages of the molecular generation pipeline. Experimental results indicate that fusing modality-specific representations before pooling generally yields the best performance across the evaluated metrics.

Critical strengths
  1. The paper investigates a relevant and practical research question by systematically evaluating multimodal fusion at different stages of the molecular structure elucidation pipeline.
  2. The experimental results are promising, with representation-level fusion generally achieving better performance across multiple molecular similarity metrics and outperforming the SpectraLLM baseline.
Critical weaknesses
  1. While the systematic evaluation of multimodal fusion strategies is valuable, the contribution is primarily empirical, with relatively limited methodological novelty.
  2. The paper primarily compares against SpectraLLM, overlooking relevant methods such as MM-Spectrum (ICML 2026), which addresses the same task on the same benchmark.
Questions for authors
  1. Could the authors include comparisons with more recent baselines, such as MM-Spectrum (ICML 2026), to provide a more comprehensive evaluation?
  2. Could the authors additionally report Top-K exact structure recovery accuracy to better assess exact molecular structure prediction performance?
5 comments
AuthorsOpenReview ↗

Point-by-point responses to Reviewer pBrv (1/2)

Dear Reviewer pBrv,

Thank you very much for your careful review and constructive feedback. We appreciate your recognition of the value of systematically studying multimodal fusion strategies for molecular structure elucidation. In response to your comments, we have conducted additional evaluations and revised the manuscript accordingly. The revised manuscript uses the following highlight colors:

  • Red: Major changes introducing new experimental results.
  • Blue: Other revisions, including new discussions, clarifications, and corrections.

Comparison with MM-Spectrum

[Weakness 2] The paper primarily compares against SpectraLLM, overlooking relevant methods such as MM-Spectrum (ICML 2026), which addresses the same task on the same benchmark.

[Question 1] Could the authors include comparisons with more recent baselines, such as MM-Spectrum (ICML 2026), to provide a more comprehensive evaluation?

Thank you for pointing out this recent baseline. MM-Spectrum [1] evaluates on the Multimodal Spectroscopic Dataset introduced by Alberts et al. [2]. In their structure-elucidation benchmark, the molecular formula is explicitly provided as additional input alongside the spectra. This differs from our fully de novo setting, where no molecular formula is available (Sec. 5 highlighted in blue).

To align the input information available to the two methods, we retrained MM-Spectrum without molecular-formula information, following the data preprocessing and training procedure described in Appendix C.1 of their paper while keeping all other settings unchanged. We then evaluated both methods on the test set of the Multimodal Spectroscopic Dataset using the full multimodal input (NMR + IR + MS). For our model, we used the autoregressive variant with representation fusion to match the autoregressive generation setting of MM-Spectrum. Details of our autoregressive model are provided in Appendix I.

MethodExact@1 (%)Formula match (%)Tanimoto ECFP4
MM-Spectrum42.2154.260.698
OmniSpecGen (ours)46.3160.560.722

Under this full-multimodal setting, our model achieves higher performance than the retrained MM-Spectrum on all three reported metrics: Exact@1, formula match, and ECFP4 similarity. Notably, this improvement is achieved with a substantially smaller model: our autoregressive model has approximately 30M parameters, compared with 450M total parameters (135M active parameters) for MM-Spectrum. Thus, despite its considerably smaller capacity, our model achieves higher structure-recovery and similarity performance.

More broadly, the novelty of the two works lies in different design dimensions: our study uses a simple transformer-based generative backbone to enable a controlled, systematic comparison of when multimodal information should be fused, whereas MM-Spectrum introduces an MoE routing within the generative backbone for multimodal molecular generation. These contributions are therefore largely orthogonal, and combining the MoE architecture with the fusion strategies studied here would be an interesting direction for future work.

[1] https://openreview.net/forum?id=YFva63G3zA

[2] https://arxiv.org/pdf/2407.17492


Exact structure recovery and Top-K evaluation

[Question 2] Could the authors additionally report Top-K exact structure recovery accuracy to better assess exact molecular structure prediction performance?

Thank you for this suggestion. We have added Exact@1 evaluation for all four fusion strategies across three independent training runs in Appendix D. Exact@1 directly measures whether the generated molecular structure exactly matches the reference structure and therefore provides a stricter measure of successful structure elucidation than structural similarity alone.

We have additionally added a Top-K exact-match evaluation in Appendix E. For selected multimodal configurations from QM9S and the Multimodal Spectroscopic Dataset, we generate multiple candidates for each test example and report Top-K accuracy for K=1,2, ...,5, where a prediction is counted as successful if the reference structure is recovered among the first K generated candidates. As expected, exact structure recovery improves as K increases. Importantly, the relative ordering of the fusion strategies remains unchanged across K: representation fusion performs best, followed by latent and molecular-interaction fusion, while late fusion remains substantially weaker. This result indicates that the advantage of earlier fusion persists when multiple candidate structures are considered.


CONTINUE

AuthorsOpenReview ↗

Point-by-point responses to Reviewer pBrv (2/2)

Empirical contribution and methodological novelty

[Weakness 1] While the systematic evaluation of multimodal fusion strategies is valuable, the contribution is primarily empirical, with relatively limited methodological novelty.

We agree that the individual architectural components are largely based on established techniques. The novelty of our work lies primarily in the empirical design insight obtained from a controlled comparison of fusion stages: specifically, when information from heterogeneous spectral modalities should be integrated within a common molecular generation framework. In the original submission, we compared four fusion stages within the same basic architecture and further examined whether the observed trend persists under two conditioning formulations, cross-attention and in-context concatenation (Appendix H).

In the revision, we have substantially strengthened the evidence supporting this conclusion. We now evaluate the main comparison across three independent training runs and provide paired sample-level analyses with 95% confidence intervals (Appendices D and G). In addition, we newly extend the same four-way fusion comparison to an autoregressive generative backbone in Appendix I, where the overall advantage of earlier fusion persists. Together, these analyses show that the observed fusion-stage trend is robust across independent training runs, is supported at the individual-sample level, and persists across different conditioning formulations and generative backbones.

We believe that providing controlled and statistically supported evidence for this non-trivial architectural design choice offers useful empirical guidance and is valuable to the ICLR community.


We thank Reviewer pBrv again for the constructive suggestions. We believe that conducting an additional comparison with a recent baseline and adding an exact structure recovery evaluation provide a more comprehensive assessment of our method and further strengthen the empirical conclusions of the paper.

Sincerely,

The Authors

AuthorsOpenReview ↗

Follow-up on our response and revisions

Dear Reviewer pBrv,

Thank you again for your constructive feedback. We have responded to the main concerns raised in your review through the additional comparison with MM-Spectrum, Top-K exact structure recovery evaluation, and clarification of the scope of our empirical contribution. If you have any remaining concerns or questions, we would greatly appreciate your feedback.

Sincerely,

The Authors

Reviewer_xUkbOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

The paper studies molecular structure generation from variable subsets of spectral modalities, using a single model without a molecular formula or a candidate structure database. OmniSpecGen compares four fusion stages within a shared SAFE-based masked discrete diffusion framework: encoded spectral representations, independently pooled embeddings, conditions entering molecular denoising, and modality-specific output logits. Its main finding is that jointly processing modality representations before attention pooling generally yields higher structural similarity. The approximately 30M-parameter model is trained with modality-subset sampling.

Critical strengths
  1. A focused research question with useful controls for major alternative explanations. Rather than only assembling multiple components and reporting an end-to-end score, the paper compares four clearly defined fusion points within a shared generative backbone.
  2. A single model handles different spectral subsets, with an actual test of partial generalization to unseen combinations. Modality-subset sampling and dedicated spectral encoders support flexible use of heterogeneous inputs.
  3. The model can benefit from additional information in these unseen combinations, although it remains below unrestricted training. This is practically relevant evidence compared with training a separate model for each configuration, and it directly addresses the stated task.
Critical weaknesses
  1. The comparisons do not fully separate earlier interaction, more flexible allocation of compression capacity, and a potentially weak output-combination rule. Figure 3 usefully weakens a query-count explanation, but Representation still uses queries shared across modalities, whereas Latent assigns a fixed number of queries to each modality before fusion.
  2. Since the encoders, pooling, and conditioning mechanisms largely build on established components, the paper’s methodological novelty depends on these comparisons supporting a precise design conclusion.
  3. The evaluation does not yet establish whether similarity improvements translate into reliable exact elucidation or whether the closest internal advantage is statistically stable. Tables 2/3 emphasize structural similarities without full-test Exact@1 or candidate-coverage results.
  4. Fraggle in Table 3 does not consistently favor earlier fusion: with all modalities, Latent obtains 0.6972 versus 0.6881 for Representation. Most SpectraLLM numbers are quoted rather than obtained under a common evaluation implementation, and representation, generation, and training choices differ.
Questions for authors
  1. On the same test samples and with the same single-candidate budget, what are canonical non-stereochemical Exact@1 and molecular-formula accuracy for the four strategies? How are invalid outputs treated in the similarity averages?
  2. Beyond Figure 3, is there a control separating shared query-capacity allocation from cross-modal interaction before compression? If not, could the authors explicitly distinguish measured findings from mechanistic hypotheses?
  3. Does the Late-fusion gap persist after learning modality weights or calibrating temperatures on validation data? What motivates equally weighting modality logits?
  4. Is the Representation-Latent advantage stable across independent training runs and paired sample-level analyses for the key complete-input configurations? How do the authors explain the different ECFP4 and Fraggle orderings, and define “most effective” accordingly?
6 comments
AuthorsOpenReview ↗

Point-by-point responses to Reviewer xUkb (1/3)

Dear Reviewer xUkb,

Thank you very much for your careful review and constructive feedback. We appreciate your thoughtful comments on our controlled comparison of fusion strategies and on how its conclusions can be more precisely supported and interpreted. In response to your comments, we have conducted additional evaluations and revised the manuscript accordingly. The revised manuscript uses the following highlight colors:

  • Red: Major changes introducing new experimental results.
  • Blue: Other revisions, including new discussions, clarifications, and corrections.

Exact structure recovery, formula accuracy, and invalid outputs

[Weakness 3] The evaluation does not yet establish whether similarity improvements translate into reliable exact elucidation or whether the closest internal advantage is statistically stable. Tables 2/3 emphasize structural similarities without full-test Exact@1 or candidate-coverage results.

[Question 1] On the same test samples and with the same single-candidate budget, what are canonical non-stereochemical Exact@1 and molecular-formula accuracy for the four strategies? How are invalid outputs treated in the similarity averages?

We have added Exact@1 and molecular-formula match evaluations for all four fusion strategies across three independent training runs under the same single-candidate setting in Appendix D.

Across these runs, Exact@1 consistently favors earlier fusion. For the complete-input settings, representation/latent/molecular-interaction/late fusion achieve 61.90/57.38/54.95/32.29% on QM9S and 19.64/16.12/13.74/0.27% on the Multimodal Spectroscopic Dataset. Thus, Exact@1 provides a clearer ordering of the four fusion stages that is stable across three independent training runs. Paired sample-level analysis in Appendix G further shows positive differences for all adjacent fusion-stage comparisons, with all 95% confidence intervals excluding zero (see our response to Question 4). Molecular-formula match accuracy follows the same overall ordering, with 92.24/92.15/90.71/79.85% on QM9S and 44.57/41.11/36.34/0.96% on the Multimodal Spectroscopic Dataset.

To further assess candidate coverage, we additionally report Top-K exact-match accuracy for selected multimodal configurations in Appendix E, where the same fusion-stage ordering is preserved for K=1, 2, ..., 5.

We have also clarified that invalid generations are counted as failures for Exact@1 and formula match, and assigned a score of zero for all similarity metrics. All metrics are therefore computed over the full test set, including invalid generations (Evaluation metrics paragraph of Sec. 4 highlighted in blue).


Empirical comparison and architectural interpretation

[Weakness 1] The comparisons do not fully separate earlier interaction, more flexible allocation of compression capacity, and a potentially weak output-combination rule. Figure 3 usefully weakens a query-count explanation, but Representation still uses queries shared across modalities, whereas Latent assigns a fixed number of queries to each modality before fusion.

[Question 2] Beyond Figure 3, is there a control separating shared query-capacity allocation from cross-modal interaction before compression? If not, could the authors explicitly distinguish measured findings from mechanistic hypotheses?

We agree that Figure 3 establishes that the advantage of representation fusion over latent fusion cannot be explained solely by the number of active pooling queries, but it does not disentangle cross-modal interaction during compression from the flexible allocation of shared pooling capacity across modalities.

We have therefore revised Sec. 4.2 (highlighted in blue) to clearly separate this measured finding from the proposed mechanisms, presenting the latter explicitly as hypotheses rather than established explanations. We now state that our current comparison does not isolate their individual contributions. We have also added this limitation to the Conclusion and Limitations section (Sec. 6, highlighted in blue) and identify isolating these mechanisms as an important direction for future research.


CONTINUE

AuthorsOpenReview ↗

Point-by-point responses to Reviewer xUkb (2/3)

Late fusion and equal logit weighting

[Question 3] Does the Late-fusion gap persist after learning modality weights or calibrating temperatures on validation data? What motivates equally weighting modality logits?

Thank you for pointing this out. We use uniform logit averaging as a simple parameter-free instantiation of prediction-level fusion, avoiding an additional trainable aggregation mechanism.

To assess whether this choice explains the late-fusion gap, we have added an oracle weighted-logit analysis in Appendix F. Specifically, we replace uniform averaging with a convex weighted average,

Ot(w)=∑m∈SwmOtm,wm≥0,∑m∈Swm=1.O_t(\boldsymbol{w})=\sum_{m\in S} w_m O_t^m,\qquad w_m\ge 0,\quad \sum_{m\in S}w_m=1.

Rather than tuning the weights on a validation set, we deliberately select them to minimize the diffusion denoising loss directly on the QM9S test set, providing an optimistic oracle analysis of how much static modality reweighting alone can improve late fusion.

Oracle weighting consistently improves over uniform averaging, but generally remains below molecular-interaction fusion; for example, with all QM9S modalities, Exact@1 improves from 33.59% to 37.52%, compared with 55.37% for molecular-interaction fusion. Thus, the gap cannot be explained by equal logit weighting alone, suggesting that the fusion point itself also contributes to the observed difference.


Stability across runs and paired sample-level analysis

[Question 4] Is the Representation-Latent advantage stable across independent training runs and paired sample-level analyses for the key complete-input configurations?

Thank you for this suggestion. We evaluated the stability of the Representation–Latent advantage in two complementary ways.

First, Appendix D reports results across three independent training runs. For the complete-input settings, Exact@1 is 61.90 ± 1.09% vs. 57.38 ± 0.34% on QM9S and 19.64 ± 0.26% vs. 16.12 ± 0.44% on the Multimodal Spectroscopic Dataset for representation and latent fusion, respectively.

Second, Appendix G reports paired sample-level differences. The Representation–Latent Exact@1 differences are +5.36 pp [95% CI: 4.44, 6.28] on QM9S and +2.74 pp [2.48, 3.00] on the Multimodal Spectroscopic Dataset; the corresponding ECFP4 similarity differences are also positive with 95% confidence intervals excluding zero.

These results show that the Representation–Latent advantage is stable across independent runs and is also supported by paired sample-level analysis.


Different ECFP4 and Fraggle orderings

[Weakness 4] Fraggle in Table 3 does not consistently favor earlier fusion: with all modalities, Latent obtains 0.6972 versus 0.6881 for Representation.

[Question 4] How do the authors explain the different ECFP4 and Fraggle orderings, and define “most effective” accordingly?

We agree that different similarity metrics can induce different orderings because they capture different aspects of partial structural agreement. We therefore revised Sec. 4.1 (highlighted in blue) to use Exact@1 as the primary criterion for comparing fusion stages, since exact structure recovery is the most direct measure of success in the molecular structure elucidation task, while retaining the similarity metrics as complementary continuous measures of partial agreement.

Under Exact@1, the ordering is consistent across all multimodal configurations and independent runs: representation > latent > molecular-interaction > late fusion. We also explicitly note that the Fraggle ordering is less stable: although the single-run result highlighted by the reviewer favors latent fusion, the Fraggle ordering between representation and latent fusion is unstable across independent runs (0.6890 ± 0.0010 vs. 0.6757 ± 0.0187).

Accordingly, we have revised the Abstract, Introduction, and Sec. 4.1 (highlighted in blue) to avoid ambiguous claims such as “most effective” that could depend on the choice of similarity metric, and instead state our conclusions explicitly in terms of exact structure recovery.


CONTINUE

AuthorsOpenReview ↗

Point-by-point responses to Reviewer xUkb (3/3)

SpectraLLM comparison

[Weakness 4] Most SpectraLLM numbers are quoted rather than obtained under a common evaluation implementation, and representation, generation, and training choices differ.

We agree that it is important to clarify the evaluation protocol used for the comparison with SpectraLLM. OmniSpecGen and SpectraLLM are evaluated on the same benchmark, using the same released train/validation/test splits and the same evaluation metrics. We therefore compare against the SpectraLLM scores reported in the original paper, rather than reproducing them with our own evaluation implementation. We have clarified in Sec. 4 (highlighted in blue) that the comparison is conducted on the same dataset splits and evaluation metrics.

Although the two methods are evaluated on the same benchmark task, they naturally differ in their representations, architectures, training procedures, and generation strategies. We therefore use the SpectraLLM results as an external benchmark comparison, while our conclusions regarding the effect of fusion stage are based on the controlled comparison among the four OmniSpecGen variants under a shared architecture and training setup.


Scope and methodological contribution

[Weakness 2] Since the encoders, pooling, and conditioning mechanisms largely build on established components, the paper’s methodological novelty depends on these comparisons supporting a precise design conclusion.

We agree that the individual encoders, pooling mechanisms, and conditioning components are largely based on established techniques. The novelty of our work lies primarily in the design insight obtained from a controlled comparison of fusion stages, rather than in introducing a new individual architectural component. Specifically, we investigate the non-trivial question of when information from heterogeneous spectral modalities should be fused within a common generative framework.

We have further strengthened the evidence supporting this finding in the revision. For the main diffusion-based comparison, we now report results across three independent training runs and paired sample-level analyses with 95% confidence intervals (Appendices D and G), showing that the observed advantage is stable across independent runs and persists at the paired sample level. We also evaluate the same four fusion stages under two commonly used conditioning formulations, cross-attention and in-context concatenation (Appendix H). In addition, we newly added an autoregressive-generation comparison in Appendix I to examine whether the finding extends beyond the discrete diffusion backbone.

Across these settings, earlier fusion, particularly representation fusion before pooling, consistently yields stronger exact structure recovery. We believe that providing such controlled evidence for a non-trivial architectural design choice is valuable to the ICLR community.


We thank Reviewer xUkb again for the detailed feedback. We believe the new exact-recovery results, robustness and paired analyses, weighted late-fusion experiment, and revised discussion of the Representation–Latent mechanism make the empirical conclusions substantially more precise and better supported.

Sincerely,

The Authors

Reviewer_XGSqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OmniSpecGen compares four fusion stages within a shared diffusion architecture for molecular generation from variable spectral subsets. Representation fusion generally performs best on structural similarity.

Critical strengths

The controlled fusion comparison is useful. Figure 3 directly tests the pooling-budget explanation, and the alternative conditioning experiment adds support. A single model also handles multiple spectral subsets.

Critical weaknesses
  1. The finer fusion ranking lacks uncertainty estimates. Decoding is stochastic, only one candidate is generated per input, and representation–latent ECFP4 differences are as small as 0.0135. The consistent direction across configurations is encouraging, but targeted repeats are needed to establish that these smaller margins survive decoding and training variation.

  2. The ranking is metric-dependent: latent fusion wins on Fraggle throughout Table 3, while late fusion has the best validity among the four variants.

  3. The main tables do not report exact-structure accuracy. Fingerprint and fragment similarity cannot tell us how often the reference molecule is recovered. Top-1 exact match from the existing predictions would make the practical significance much clearer; invalid-output handling also needs to be specified.

  4. SpectraLLM's published numbers provide context, but do not establish a comparison through the same evaluation pipeline.

Questions for authors

What are the top-1 exact-structure rates, and how are invalid generations handled in the similarity averages?

4 comments
AuthorsOpenReview ↗

Point-by-point responses to Reviewer XGSq (1/2)

Dear Reviewer XGSq,

Thank you very much for your careful review and constructive feedback. We appreciate your recognition of the value of our controlled comparison of fusion strategies. In response to your comments, we have conducted additional evaluations and revised the manuscript accordingly. The revised manuscript uses the following highlight colors:

  • Red: Major changes introducing new experimental results.
  • Blue: Other revisions, including new discussions, clarifications, and corrections.

Exact structure recovery and invalid generations

[Weakness 3] The main tables do not report exact-structure accuracy. Fingerprint and fragment similarity cannot tell us how often the reference molecule is recovered. Top-1 exact match from the existing predictions would make the practical significance much clearer; invalid-output handling also needs to be specified.

[Question 1] What are the top-1 exact-structure rates, and how are invalid generations handled in the similarity averages?

Thank you for this important suggestion. We now report Exact@1 (Exact Match) for all multimodal fusion configurations in the three-run evaluation in Appendix D and summarize the resulting trend in Sec. 4.1 (highlighted in red). Exact@1 (%) results are shown below as mean ± standard deviation over three independent runs.

DatasetInputsRepresentationLatentMol. interactionLate
QM9SIR + Raman61.01 ± 0.9555.70 ± 0.4053.91 ± 0.4035.72 ± 1.69
Raman + UV–Vis57.04 ± 0.8648.78 ± 0.7246.21 ± 0.9528.29 ± 1.39
UV–Vis + IR36.45 ± 1.0726.11 ± 1.5124.61 ± 1.3718.68 ± 1.62
IR + Raman + UV–Vis61.90 ± 1.0957.38 ± 0.3454.95 ± 0.3932.29 ± 1.71
Multimodal SpectroscopicIR + MS/MS2.26 ± 0.071.72 ± 0.281.62 ± 0.060.44 ± 0.09
NMR10.09 ± 0.178.43 ± 0.417.06 ± 0.340.14 ± 0.21
NMR + IR16.74 ± 0.0912.92 ± 0.4111.07 ± 0.380.21 ± 0.24
NMR + MS/MS14.43 ± 0.3212.03 ± 0.499.96 ± 0.380.16 ± 0.20
NMR + IR + MS/MS19.64 ± 0.2616.12 ± 0.4413.74 ± 0.420.27 ± 0.23

We have also clarified in the Evaluation metrics paragraph of Sec. 4 (highlighted in blue) how invalid generations are handled: they are treated as failures for Exact@1 and assigned a score of zero for all structural similarity metrics, without excluding invalid generations from the evaluation, while validity is reported separately.


Robustness across stochastic generation and independent runs

[Weakness 1] The finer fusion ranking lacks uncertainty estimates. Decoding is stochastic, only one candidate is generated per input, and representation–latent ECFP4 differences are as small as 0.0135. The consistent direction across configurations is encouraging, but targeted repeats are needed to establish that these smaller margins survive decoding and training variation.

To address this concern, we independently trained each fusion variant with three random seeds and generated predictions with each resulting model. Representation fusion achieves the highest mean Exact@1 in every multimodal configuration. For the complete-input settings, Exact@1 is 61.90 ± 1.09% vs. 57.38 ± 0.34% for Representation vs. Latent on QM9S, and 19.64 ± 0.26% vs. 16.12 ± 0.44% on the Multimodal Spectroscopic Dataset. The same ordering is observed for Tanimoto similarity (ECFP4), with 0.7545 ± 0.0073 vs. 0.7249 ± 0.0029 on QM9S and 0.5370 ± 0.0017 vs. 0.5033 ± 0.0046 on the Multimodal Spectroscopic Dataset. The Representation–Latent ordering is therefore preserved across independent training and stochastic generation runs for both Exact@1 and Tanimoto similarity. Full results are reported in Appendix D.


CONTINUE

AuthorsOpenReview ↗

Point-by-point responses to Reviewer XGSq (2/2)

Metric-dependent ordering

[Weakness 2] The ranking is metric-dependent: latent fusion wins on Fraggle throughout Table 3, while late fusion has the best validity among the four variants.

We agree that individual similarity metrics can yield different orderings because they capture different aspects of partial structural agreement. As structure elucidation ultimately aims to recover the correct molecular structure, we use Exact@1 as the primary evaluation metric, while treating the structural similarity metrics as complementary continuous measures of partial structural agreement. Exact@1 consistently favors earlier fusion across all multimodal configurations in the three-run evaluation, while the structural similarity metrics broadly support the same trend. In particular, although Fraggle favors Latent over Representation in the single-run results of Table 3, this ordering is not stable across independent runs in the three-run evaluation. Taken together, these results show that the overall fusion-stage ordering remains consistent when evaluated by exact structure recovery, despite some variation across similarity metrics.

Although earlier-fusion variants have slightly lower validity in some configurations, invalid generations count as incorrect for Exact@1 and are assigned a score of zero for all structural similarity metrics. Thus, the higher Exact@1 and similarity scores of earlier fusion already account for these validity differences and indicate better target-structure recovery performance.

We have revised the discussion in Sec. 4.1 (highlighted in blue) to use Exact@1 as the primary measure of task performance and to interpret the structural similarity metrics as complementary measures of partial structural agreement.


SpectraLLM comparison

[Weakness 4] SpectraLLM's published numbers provide context, but do not establish a comparison through the same evaluation pipeline.

Thank you for pointing this out. We agree that the SpectraLLM scores were not recomputed using our evaluation implementation. However, our models are trained and evaluated using the same released train/validation/test splits and the same evaluation metrics as SpectraLLM. We therefore view this as a comparison against the published SpectraLLM benchmark, while noting that the SpectraLLM scores were taken directly from the published results.

We have clarified these points at the beginning of Sec. 4 (highlighted in blue).


We thank Reviewer XGSq again for the constructive comments. We believe that the additional exact-structure recovery evaluation, robustness analyses, and clarifications substantially strengthen the empirical support for our conclusions and make the scope of our claims more precise.

Sincerely,

The Authors

AuthorsOpenReview ↗

Follow-up on our response and revisions

Dear Reviewer XGSq,

Thank you again for your constructive feedback. We have responded to the main concerns raised in your review through additional exact-recovery and robustness analyses, as well as clarification of the SpectraLLM comparison. If you have any remaining concerns or questions, we would greatly appreciate your feedback.

Sincerely,

The Authors

Thanks for the solid rebuttal. I keep my rating at 3 with stronger support for acceptance.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Responses and Manuscript Revisions

Dear Reviewers and Area Chair,

We sincerely thank all reviewers for their careful and constructive feedback. We greatly appreciate their recognition of the value of our focused research question on the non-trivial architectural choice of when multimodal information should be fused, as well as the practical relevance of supporting flexible spectral inputs within a single model.

As the discussion period is approaching its end, we would greatly appreciate any further feedback from the reviewers, particularly if there are any remaining concerns or questions regarding our responses and revisions. To facilitate this final stage of the discussion, we provide below a concise summary of the main concerns, our responses, and key updates to the manuscript.


Main Concerns and Our Responses

  • Exact Structure Recovery and Robustness (Reviewers XGSq, xUkb & pBrv):

    • We added three-run Exact@1 and molecular-formula match evaluations, Top-K exact-match evaluation, and paired sample-level analyses with 95% confidence intervals.
    • These analyses consistently support Representation > Latent > Molecular Interaction > Late fusion in exact structure recovery.
    • We also clarified that invalid generations are counted as failures and assigned zero similarity, so all metrics are computed over the full test set.
  • Metric Dependence and Interpretation of the Fusion-Stage Trend (Reviewers XGSq & xUkb):

    • We now use Exact@1 as the primary measure of task performance because it most directly measures successful molecular structure elucidation, while treating similarity metrics as complementary measures of partial structural agreement.
    • We revised the manuscript to avoid metric-dependent claims such as “most effective” and instead state the conclusion directly in terms of exact structure recovery.
  • What Explains the Representation–Latent Difference? (Reviewer xUkb):

    • Our pooling-budget experiment shows that the Representation–Latent gap cannot be explained solely by the number of active pooling queries, but does not isolate cross-modal interaction during compression from flexible allocation of shared pooling capacity.
    • We therefore revised the discussion to clearly distinguish the measured finding from these mechanistic hypotheses and added this limitation to the Conclusion.
  • Late-Fusion Aggregation (Reviewer xUkb):

    • We added an oracle weighted-logit analysis to test whether uniform averaging explains the weak performance of late fusion.
    • Oracle reweighting improves late fusion, but it remains below molecular-interaction fusion, indicating that equal weighting alone does not explain the late-fusion gap.
  • Baseline Comparisons (Reviewers XGSq, xUkb & pBrv):

    • We clarified that our comparison with SpectraLLM is conducted under the same benchmark conditions, using the same released train/validation/test splits and the same evaluation metrics, and therefore its reported results can be directly compared with ours.
    • We retrained MM-Spectrum without molecular-formula information to match our fully de novo setting. Under the full multimodal input, our autoregressive model achieves higher Exact@1 (46.31% vs. 42.21%) with substantially fewer parameters.
  • Scope of the Contribution (Reviewers xUkb & pBrv):

    • As also recognized in the reviews, a key strength of our work is the controlled comparison of clearly defined fusion stages within a shared generative framework, addressing the non-trivial question of when information from heterogeneous modalities should be fused, rather than simply combining multiple components and reporting end-to-end performance.
    • We further strengthened this controlled study by showing that the same fusion-stage trend persists across multiple independent training runs, different conditioning formulations, and both diffusion and autoregressive generation.
    • We believe that such controlled evidence for this multimodal design choice is valuable to the ICLR community.

Summary of Major Updates

  • Three-run Exact@1 and formula-match evaluation (Appendix D).
  • Top-K exact-match evaluation (Appendix E).
  • Oracle weighted late-fusion analysis (Appendix F).
  • Paired sample-level analysis with 95% confidence intervals (Appendix G).
  • Four-way fusion comparison with an autoregressive backbone (Appendix I).
  • Revised discussion of fusion-stage interpretation, evaluation metrics, baseline comparisons, and methodological scope throughout the main text (highlighted in blue).

We hope these additional experiments and revisions address the reviewers’ concerns and strengthen the empirical support for our conclusions. We welcome any further comments or questions during the discussion period.

We sincerely appreciate your time and thoughtful feedback.

Sincerely,

The Authors

AuthorsOpenReview ↗

Closing Remarks

Dear Reviewers and Area Chair,

Thank you for the continued discussion and for the feedback provided on our revisions. As the discussion period draws to a close, we would like to highlight several points for consideration during the remaining reviewer–AC discussion, particularly regarding the scope of the paper’s methodological contribution.

The revisions have substantially addressed the reviewers’ concerns about the reliability of the empirical evidence. The Exact@1 advantage of representation fusion is consistent across three independent training runs and is supported by paired sample-level analyses, with 95% confidence intervals for the performance differences excluding zero (Appendices D and G). The central remaining question therefore concerns how this robust empirical conclusion should be assessed as a methodological contribution: whether identifying when to fuse spectral modalities through controlled empirical comparisons constitutes a sufficient methodological contribution.

In considering this question, we hope that the following three points will be carefully weighed:

  1. The finding is supported by a broad and controlled evaluation. The comparison covers two datasets, input configurations spanning two to five modalities, varied pooling budgets, two conditioning formulations, and both diffusion and autoregressive backbones. The consistent advantage of representation fusion across these settings indicates that the finding is not specific to a single configuration or modeling choice.

  2. The finding has practical significance. Using the same released dataset splits and evaluation metrics, our representation-fusion model outperforms the reported SpectraLLM (ICLR 2026) baseline across the evaluated multimodal configurations, despite being substantially smaller (approximately 30M parameters versus SpectraLLM’s 32B-parameter backbone). Under matched fully de novo conditions, our autoregressive representation-fusion model also outperforms the retrained MM-Spectrum (ICML 2026) baseline (46.31% versus 42.21% Exact@1), while using substantially fewer parameters (approximately 30M, compared with MM-Spectrum’s 450M total and 135M active parameters).

  3. Mechanistic uncertainty does not preclude empirical evidence of robustness. Our experiments do not fully disentangle why representation fusion performs better, but identifying the underlying mechanism is distinct from testing whether its advantage persists as experimental conditions change. The evaluations summarized above directly address the latter question and provide evidence of robustness across the conditions evaluated here.

While the individual architectural components are well established, determining how to combine established components effectively is itself an important methodological question. Our study addresses this question through a unified, controlled comparison, provides actionable empirical guidance on when to integrate multimodal information, and demonstrates that the resulting design achieves strong performance relative to existing state-of-the-art methods. We believe that this combination of controlled design insight and demonstrated practical effectiveness constitutes a valuable contribution to the ICLR community.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.