- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes an evidence-filtering approach, based on Shapley values and ModernBERT, that selects knowledge-graph paths and experimental measurements for predicting the direction of a target gene's response (up, down, or no change) to a genetic perturbation. The selected evidence is used by two downstream predictors: a fixed LLM-based predictor and a voting-based predictor. The approach is evaluated on a held-out cell line under both in-distribution and out-of-distribution settings, and it shows improvements over baselines in macro-F1 for both predictors.
Critical strengths
The paper explores an important problem, perturbation response prediction, and how to efficiently leverage prior knowledge and experimental measurements.
Critical weaknesses
- Limited novelty: The contribution appears limited; the approach seems to be a straightforward application of previously proposed ModernBERT-based evidence filtering, with Shapley values used as the training signal.
The experimental evaluation and setup are underwhelming:
- Computational cost: Shapley values seem computationally expensive relative to the baselines. Without a report of computational efficiency, it is difficult to evaluate the performance improvements.
- Clarity and writing: The paper should more clearly explain the prior work it leverages and how it is adapted, e.g., the measurement voting predictor used for evaluation. It should also clarify how the experimental measurements and KG paths are processed and passed to both fixed predictors. The writing needs substantial improvement to highlight the motivation, the specific contributions, and why the proposed approach improves over the baselines. The appendix appears LLM-generated, with several references carrying TODO and verified local PDF annotations.
- Statistical significance and scope: It is unclear why the results do not report statistical significance and why only one cell line is evaluated.
- Missing baselines: It is unclear why standard perturbation-prediction baselines are not included, both deep learning and simple linear baselines.
Questions for authors
See weaknesses above