acceptodds
Under review as a conference paper at ICLR 2027

CellScientist: From Execution Feedback to Auditable Model-Revision Trajectories for Cellular Perturbation Prediction

Abstract

Cellular perturbation-response modeling requires coordinated choices of representations, fusion mechanisms, objectives, and training procedures. Large language models (LLMs) can propose executable candidates, but unconstrained revision can produce invalid implementations, change task semantics, or discard useful components. We present CellScientist, a protocol-constrained workflow that converts execution and validation feedback into auditable model-revision trajectories. It records design states and outcomes, routes discrepancies to specific components, and applies local revisions under a fixed task contract. A matched-budget study fixes the candidate language, predictor, fitting, and evaluator: structured revision finds better held-out predictors at small budgets under two LLM backbones. Operational audits link history to fewer repeated proposals, contract checks to contained violations, and discrepancy routing to targeted repairs. Refits of two frozen designs on an independently acquired cohort evaluate external predictive utility. Open-workflow trajectories retain improvements, regressions, and failures, while transcriptomic and single-cell searches extend application to additional response spaces. CellScientist produces both a selected predictor and an inspectable record of its development.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for Science, LLM agents, automated model development, cellular perturbation prediction, feedback-guided search, auditable trajectories

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mDjmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CellScientist, a feedback-driven workflow for revising cellular perturbation predictors. It combines revision history, task constraints, and discrepancy-based routing, and evaluates both revision behavior and the predictive utility of discovered models.

Critical strengths
  1. Practical problem formulation. The paper addresses how to translate execution and validation feedback into targeted revisions of interacting model components while respecting task constraints.

  2. Diverse evaluations. Experiments cover morphology, transcriptomic, and single-cell prediction, alongside controlled routing comparisons and external refits of frozen architectures. The discovery results on CPG0016 show substantial improvements over the listed baselines.

Critical weaknesses
  1. Uneven gains and limited external benefits. In Table 28, PCC gains on BBBC036/047 over the strongest competing baselines range from 0.0072 to 0.0214, smaller than those on CPG0016. In the separate LKCP evaluation (Table 4), the BBBC036-derived architecture has lower PCC than RealMLP (0.1449 vs. 0.1496) and only a 0.25% MSE reduction. The BBBC047-derived architecture improves PCC to 0.1766 but has worse MSE and R².

  2. Unclear incremental value of LLM selection. The controlled routing experiments jointly evaluate predefined diagnostic ranking, candidate preferences, and LLM selection. Similar results from Gemini 3 Pro and Qwen2.5-0.5B-Instruct within this restricted setting do not establish the LLM’s added value beyond the predefined heuristics.

Questions for authors
  1. How should the mixed LKCP results be interpreted in terms of the practical predictive value of the discovered architectures?

  2. How much of the controlled routing improvement is attributable to predefined heuristics, and what additional benefit does LLM selection provide?

Reviewer_hufhOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces CellScientist, a framework that leverages execution and validation feedbacks for cellular perturbation-response prediction model-revision. Experiments include a matched-budget study with a fixed candidate language, predictor, fitting procedure, and evaluator under two LLM backbones, operational audits of the workflow components, and refits of two frozen designs on an independently acquired cohort.

Critical strengths

According to the experiments, the proposed CellScientist achieves higher mean held-out PCC than the compared policies at a budget, which shows the proposed revision strategy can find useful predictors with fewer candidate evaluations.

Critical weaknesses
  1. The motivation of the automated model revision for the cellular perturbation-response prediction task is not clearly stated. I am curious why automated is worth exploring in such specific task and how it would influence such field. Besides, from the experiment perspective, results in Table 4 do not show clear advantage over previous developed RealMLP baseline.

  2. CellForge and HarmonyCell are closely related paper, which are mentioned in this paper but not compared. Therefore, the effectiveness of the proposed method is also not well-supported. f

  3. Table 3 reads like just some functional tests, which does not reflect the effect on final performance. There's no component ablation study in this work.

Questions for authors
  1. Why not start with expert designed model such as RealMLP? If it is sufficient for the task, no search is needed. If it not, can the framework improves it?
  2. Why CellForge and HarmonyCell are not compared in experiments?
Reviewer_dMRMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CellScientist, a workflow for revising cellular perturbation-response models using execution and validation feedback. It keeps a history of attempted designs, checks proposed changes against a fixed task contract, and uses diagnostic rules and an LLM to choose what to edit next. The main experiment compares search policies within a shared space of 768 predictor configurations. The paper also reports checks of the history, validation and routing components, records several open-code search trajectories, and tests two discovered architectures by refitting them on an external cohort.

Critical strengths

The matched-budget comparison is a strong part of the paper. The policies share the initial model, candidate space, fitting seeds and evaluation protocol, so their results are easier to interpret than comparisons between unrelated final models. CellScientist has the best mean PCC in all four settings at budgets 3 and 5, and the early-budget pattern appears with both controller backbones (Tables 1-2, pp. 6-7). The paper also reports seed-level uncertainty separately from the descriptive intervals across settings (Appendix D.1).

Keeping crashes and unsuccessful revisions in the record is useful. Figures 2 and 4 show why the retained model differs from the last attempted model, rather than presenting only a successful final design. Freezing the two architectures before refitting them on LKCP is also a sensible test of whether the discovered designs remain useful on another cohort.

Critical weaknesses

The main comparison is missing a simple baseline: follow the first-ranked suggestion without an LLM. The rules rank edit addresses, generate one candidate per address, and give the LLM at most two choices, with an instruction to prefer the first. The released controller already has a deterministic first-choice path. The paper explicitly evaluates the combined policy, but without this baseline it is hard to tell what the LLM adds to the diagnostic and history rules. Random routing does not answer that question because it changes which addresses are offered.

The open-code results need clearer evidence of repeatability. Table 30 reports improvements from M0 to M4, but does not clearly state how many independent search trajectories were run or give paired uncertainty. The 100% PDR result concerns constructed configuration faults, checked without refitting, so it does not show that the system can reliably diagnose and repair naturally occurring modeling failures. The external refits are useful, but cover one 16-compound test partition, and all paired differences from RealMLP have intervals that cross zero.

Questions for authors

How does CellScientist compare with always taking the first-ranked pair, keeping all other rules and budgets fixed? How often does the LLM choose the second pair, and do those choices help?

How many independent open-code searches underlie each M0-M4 result in Table 30? Please report their variation and give examples where diagnosing a naturally occurring failure led to better held-out predictions.

Where are the run manifests, controller traces and paired-run CSVs for the reported results? Appendix D.1 refers to accompanying CSVs, but the downloaded 40-member archive appears to contain reproduction code and documentation rather than those records.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.