- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper investigates the impact of multimodal fusion strategies on molecular structure elucidation from spectral data. The authors present OmniSpecGen, a unified framework for systematically evaluating the integration of different spectral modalities at various stages of the molecular generation pipeline. Experimental results indicate that fusing modality-specific representations before pooling generally yields the best performance across the evaluated metrics.
Critical strengths
- The paper investigates a relevant and practical research question by systematically evaluating multimodal fusion at different stages of the molecular structure elucidation pipeline.
- The experimental results are promising, with representation-level fusion generally achieving better performance across multiple molecular similarity metrics and outperforming the SpectraLLM baseline.
Critical weaknesses
- While the systematic evaluation of multimodal fusion strategies is valuable, the contribution is primarily empirical, with relatively limited methodological novelty.
- The paper primarily compares against SpectraLLM, overlooking relevant methods such as MM-Spectrum (ICML 2026), which addresses the same task on the same benchmark.
Questions for authors
- Could the authors include comparisons with more recent baselines, such as MM-Spectrum (ICML 2026), to provide a more comprehensive evaluation?
- Could the authors additionally report Top-K exact structure recovery accuracy to better assess exact molecular structure prediction performance?
5 comments
Point-by-point responses to Reviewer pBrv (2/2)
Empirical contribution and methodological novelty
[Weakness 1]While the systematic evaluation of multimodal fusion strategies is valuable, the contribution is primarily empirical, with relatively limited methodological novelty.
We agree that the individual architectural components are largely based on established techniques. The novelty of our work lies primarily in the empirical design insight obtained from a controlled comparison of fusion stages: specifically, when information from heterogeneous spectral modalities should be integrated within a common molecular generation framework. In the original submission, we compared four fusion stages within the same basic architecture and further examined whether the observed trend persists under two conditioning formulations, cross-attention and in-context concatenation (Appendix H).
In the revision, we have substantially strengthened the evidence supporting this conclusion. We now evaluate the main comparison across three independent training runs and provide paired sample-level analyses with 95% confidence intervals (Appendices D and G). In addition, we newly extend the same four-way fusion comparison to an autoregressive generative backbone in Appendix I, where the overall advantage of earlier fusion persists. Together, these analyses show that the observed fusion-stage trend is robust across independent training runs, is supported at the individual-sample level, and persists across different conditioning formulations and generative backbones.
We believe that providing controlled and statistically supported evidence for this non-trivial architectural design choice offers useful empirical guidance and is valuable to the ICLR community.
We thank Reviewer pBrv again for the constructive suggestions. We believe that conducting an additional comparison with a recent baseline and adding an exact structure recovery evaluation provide a more comprehensive assessment of our method and further strengthen the empirical conclusions of the paper.
Sincerely,
The Authors
Follow-up on our response and revisions
Dear Reviewer pBrv,
Thank you again for your constructive feedback. We have responded to the main concerns raised in your review through the additional comparison with MM-Spectrum, Top-K exact structure recovery evaluation, and clarification of the scope of our empirical contribution. If you have any remaining concerns or questions, we would greatly appreciate your feedback.
Sincerely,
The Authors
Point-by-point responses to Reviewer pBrv (1/2)
Dear Reviewer pBrv,
Thank you very much for your careful review and constructive feedback. We appreciate your recognition of the value of systematically studying multimodal fusion strategies for molecular structure elucidation. In response to your comments, we have conducted additional evaluations and revised the manuscript accordingly. The revised manuscript uses the following highlight colors:
Red: Major changes introducing new experimental results.Blue: Other revisions, including new discussions, clarifications, and corrections.Comparison with MM-Spectrum
Thank you for pointing out this recent baseline. MM-Spectrum [1] evaluates on the Multimodal Spectroscopic Dataset introduced by Alberts et al. [2]. In their structure-elucidation benchmark, the molecular formula is explicitly provided as additional input alongside the spectra. This differs from our fully de novo setting, where no molecular formula is available (
Sec. 5 highlighted in blue).To align the input information available to the two methods, we retrained MM-Spectrum without molecular-formula information, following the data preprocessing and training procedure described in Appendix C.1 of their paper while keeping all other settings unchanged. We then evaluated both methods on the test set of the Multimodal Spectroscopic Dataset using the full multimodal input (NMR + IR + MS). For our model, we used the autoregressive variant with representation fusion to match the autoregressive generation setting of MM-Spectrum. Details of our autoregressive model are provided in
Appendix I.Under this full-multimodal setting, our model achieves higher performance than the retrained MM-Spectrum on all three reported metrics: Exact@1, formula match, and ECFP4 similarity. Notably, this improvement is achieved with a substantially smaller model: our autoregressive model has approximately 30M parameters, compared with 450M total parameters (135M active parameters) for MM-Spectrum. Thus, despite its considerably smaller capacity, our model achieves higher structure-recovery and similarity performance.
More broadly, the novelty of the two works lies in different design dimensions: our study uses a simple transformer-based generative backbone to enable a controlled, systematic comparison of when multimodal information should be fused, whereas MM-Spectrum introduces an MoE routing within the generative backbone for multimodal molecular generation. These contributions are therefore largely orthogonal, and combining the MoE architecture with the fusion strategies studied here would be an interesting direction for future work.
[1] https://openreview.net/forum?id=YFva63G3zA
[2] https://arxiv.org/pdf/2407.17492
Exact structure recovery and Top-K evaluation
Thank you for this suggestion. We have added Exact@1 evaluation for all four fusion strategies across three independent training runs in
Appendix D. Exact@1 directly measures whether the generated molecular structure exactly matches the reference structure and therefore provides a stricter measure of successful structure elucidation than structural similarity alone.We have additionally added a Top-K exact-match evaluation in
Appendix E. For selected multimodal configurations from QM9S and the Multimodal Spectroscopic Dataset, we generate multiple candidates for each test example and report Top-K accuracy for K=1,2, ...,5, where a prediction is counted as successful if the reference structure is recovered among the first K generated candidates. As expected, exact structure recovery improves as K increases. Importantly, the relative ordering of the fusion strategies remains unchanged across K: representation fusion performs best, followed by latent and molecular-interaction fusion, while late fusion remains substantially weaker. This result indicates that the advantage of earlier fusion persists when multiple candidate structures are considered.CONTINUE