- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CellVELA studies whether cell foundation models and language models distinguish context-specific effects from general gene-level priors. It shows that a fixed target ranking can score well without identifying which contexts depend on a gene. The paper then compares parameter adaptation with retrieval of experimental measurements. On held-out cancer lines and perturbation-response tasks, the tested fine-tuning recipes transfer poorly, while relevant records improve discrimination. Controls vary retrieval similarity, record count, and reader model. The strongest contribution is the evaluation diagnosis and the importance of relevant measurements, rather than evidence that an LLM is necessary.
Critical strengths
-
The fixed-ranking control in Table 1 exposes a real evaluation shortcut. The within-gene context metric separates broadly essential genes from context-specific dependencies. This helps determine whether a model learns the distinction required by the biological task, rather than merely exploiting a strong gene prior.
-
Comparisons with the same retrieved records make the retrieval result informative. Similarity voting and learned readers often perform comparably, random retrieval removes the benefit, and quality and quantity of records affect performance. Stronger ridge references are also reported. Together, these controls support the view that access to relevant measurements drives much of the gain.
Critical weaknesses
-
The adaptation-versus-retrieval comparison leaves supervision coverage as an alternative explanation. The LLM sees a relatively sparse sample of line–gene labels during training, whereas retrieval supplies relevant measurements directly at inference. Appendix D includes useful sampling controls (PDF p. 15), but coverage and optimization are still not fully separated from where information is stored. The evidence is stronger against the tested adaptation recipes than against parametric learning in general.
-
Transfer without close screened neighbors remains uncertain. Figure 3a already shows increasing performance with retrieved-line similarity, and the limitations acknowledge the need for related screened contexts. What remains unclear is how much of the held-out gain depends on the test lineage being represented in the bank or on its closest neighbors. This limits the demonstrated range of use without invalidating the held-out results.
-
The near-perfect biological case scores add limited evidence beyond the aggregate evaluation. Appendix G states that genes were selected after inspecting similarity-vote results, before scoring CellVELA's twenty-record predictions (PDF p. 16). Some cases have too few dependent test lines to satisfy the main class-size criterion. The paper labels them as illustrations and reports results on 1,444 eligible genes. Claims about rare dependencies should therefore rely primarily on broader per-gene results.
Questions for authors
-
How many distinct eligible line–gene pairs are seen over the full fine-tuning run, rather than in one epoch? Does the current evidence distinguish limited supervision coverage from a limitation of parameter adaptation itself?
-
Beyond the similarity stratification in Figure 3a, does retrieval still help after excluding each test line's closest training neighbors? Can results be separated by whether the bank contains the test lineage? These analyses would clarify the transfer boundary using the existing data.
-
For the broader gene set from which the cases were selected, how does context AUC vary with the number of dependent test lines? Please distinguish genes meeting the main evaluation criterion from rarer dependencies, so the highlighted examples can be placed in context.
Thank you for your response and the additional experiments. These additions have alleviated my main concerns. I am therefore raising my rating to 3.