- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents ADAPT, an LLM-guided search framework for cross-species cell-type annotation using frozen scGPT embeddings. Two agents propose and implement adaptation models within a predefined experimental interface, using target-validation feedback to guide architecture, objective, and optimizer search. Experiments report mean shared-class Macro-F1 scores of 0.81 and 0.73 on human-to-mouse and human-to-macaque transfer.
Critical strengths
-
Structured agentic search framework. ADAPT provides a reasonably systematic closed-loop workflow that connects hypothesis generation, candidate implementation, experimental evaluation, and feedback-driven planning. This goes beyond using an LLM solely for conventional hyper-parameter tuning.
-
Careful evaluation protocol. The separation of validation, monitor, and sealed test sets, together with controlled per-round comparisons, makes the reported results relatively credible and reduces concerns about direct test-set overfitting.
Critical weaknesses
-
Limited connection to cross-species transfer. The current results mainly demonstrate task-specific improvements from agent-driven model search, but provide limited evidence that the discovered strategies capture insights specific to cross-species prediction. It is also unclear whether the search experience can transfer across settings. For example, whether findings from human-to-mouse transfer can provide a better starting point or reduce the search cost for human-to-macaque transfer.
-
The methodological advance over existing agentic AutoML is insufficiently established. Section 3 describes iterative generation and evaluation of architectures, objectives, and optimizer configurations. Although candidates can contain generated code, Appendix A.3 already supplies substantial design guidance. The paper does not clearly establish what its search procedure contributes beyond existing LLM-based AutoML methods.
Questions for authors
-
Why is scGPT kept frozen throughout the search? Since the paper aims to show that agents can discover effective strategies for cross-species adaptation, I would expect the search to potentially uncover mechanisms with clearer biological structure. However, the agent only operates on frozen 512-dim scGPT embeddings and cannot modify the representation itself. Does this constrain the search to adapting the embedding distributions and, consequently, to the biological structure already encoded by scGPT?
-
Can the search discover an efficient evolutionary trajectory? Beyond finding a strong final configuration, does the feedback from previous rounds lead to a progressively more effective search strategy? For example, compared with restarting the search, can intermediate discoveries provide better starting points or reduce the number of experiments needed to reach strong solutions?