- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes SLeuth, a lightweight Siamese network that uses a shared encoder to predict synthetic lethality via an inner product in a latent space over six categories of off-the-shelf multimodal pretrained representations. Under three settings of progressively harder generalization, the authors compare it against three complex baselines and find broadly comparable performance despite having orders of magnitude fewer trainable parameters.
Critical strengths
(1) It applies multimodal pretrained representations to synthetic lethality prediction and performs fusion with a minimal Siamese network, achieving performance competitive with complex graph models while using orders of magnitude fewer trainable parameters. This is a valuable direction with clear research significance.
(2) It systematically compares the performance differences of different modalities and different pretrained models on synthetic lethality prediction, providing empirical reference for subsequent representation selection.
Critical weaknesses
(1) The integration strategy of the method is relatively simple, and the architectural innovation is limited. Its core lies in integrating the pre-trained embeddings from six sources. The main integration methods include concatenation, standardization, and PCA dimensionality reduction, followed by learning through a lightweight encoder. There is no explicit modeling of the differentiated contributions of each data source to the synthetic lethal prediction, nor is there any innovation regarding the fusion mechanism itself. Table 2 shows that single-modal models such as go2vec and text_embed outperform multi-modal models in each Tier. This indicates that the necessity of multi-source fusion has not been fully demonstrated, and it also suggests that the gains may mainly come from the pre-trained representations themselves.
(2) There is a lack of analysis of potential issues that may exist in the integration of multi-source pre-trained embeddings. The embeddings from different models are not consistent in the feature space, semantic scale, and data distribution. Even with direct concatenation supplemented by matrix normalization and PCA, there may still be redundant information, semantic conflicts, alignment difficulties, and negative transfer. The paper merely mentions "PCA reduces redundancy" in a passing manner, without discussing semantic scale alignment and the risk of negative transfer, nor providing evidence to show that this fusion strategy has indeed alleviated these problems; and the multi-modal models perform worse than the single optimal modal in multiple Tiers, which may precisely be the manifestation of these problems.
Questions for authors
Table 5 reports only the mean and standard deviation of Spearman correlations across five fold models, without confidence intervals or significance tests, and most correlations are close to 0 or even negative. I suggest adding confidence intervals for the correlations and testing whether the differences between models are statistically significant, to establish the reliability of these results.
2 comments
Thank you for the clarification from the author. If the author positions this paper as an application and benchmark test of pre-trained embeddings in synthetic lethal prediction, and intends to keep the architecture simple, then the current title, abstract, and contribution statement do not fully align with this positioning:
The current writing makes it more likely for readers to think that the main contribution of this paper is the proposal of a new method or architecture. Considering the limited novelty at the method level of this paper and the main value lies in the problem setting, the systematic evaluation of existing pre-trained embeddings/foundation models representations, and the examination of the gains brought by multimodal information under various settings through simple fusion experiments.
I suggest the author adjust the presentation framework of the paper: Weaken the emphasis on methodological innovation, and strengthen the contributions in the aspects of benchmark testing and problem definition. Specifically, in the title, abstract, introduction, and contribution list, it should be clearly stated that this paper is an application/benchmark study, explaining the scope of evaluation, evaluation protocol, and main findings, and presenting the simple fusion architecture as an empirical component rather than a core method innovation. This will enable the paper's contributions to be consistent with the writing style and present the value of this paper more accurately.
In addition, the author needs to consider redesigning Figure 1 and Figure 2: The current two figures contain insufficient information and fail to fully support the positioning of benchmark testing/application research; if the paper positioning is adjusted as per the above suggestions, the content and narrative of these two figures will no longer be applicable. In conclusion, I will not modify my rating.
Revision Changes
W1. Fusion is not justified, and the novelty is limited. Without PCA, the multimodal model is significantly better than every single embedding at Tier I and on par with the best ones at Tiers II and III. No single embedding is significantly better at any tier. On novelty, we see the paper as an application and benchmark of pretrained embeddings for SL prediction, and we kept the architecture simple on purpose, also due to the data size limitations as the effective sample size (i.e. the number of positive SL pairs) is not that large.
W2. Scale, redundancy and negative transfer. Our PCA standardized every feature, which discards the relative scale of the features. Without PCA, AUPRG is higher in most model–tier combinations, but significantly so only at Tier I for SLeuth and SLeuth-all. Since PCA also changes the model size, this does not isolate the effect of magnitude. Using all 16 embeddings gives results close to the six selected ones, lower only for SLeuth at Tier III, and removing any one category has no significant effect.
Q. Table 5 has no confidence intervals or tests. The table now shows, under each model, the Holm-adjusted bootstrap p-value against SLeuth, and marks correlations whose 95% CI excludes zero. The full CIs are in Appendix Tables. Because screen pairs share genes, this bootstrap is optimistic. A gene-level bootstrap gives larger standard errors, and some of SLeuth's correlations on unseen genes then no longer exclude zero (Appendix). We agree that most correlations are near zero or negative. Outside Adamson's growth phenotype, all are below 0.11 in absolute value. SLeuth is negative on that phenotype, which mostly reflects single-gene fitness, and positive on Adamson's GI score.