acceptodds
Under review as a conference paper at ICLR 2027

HICELLS: HIERARCHICAL CELL-TYPE ANNOTATION WITH SELF-EVOLVING SKILL AGENTS

Abstract

Accurate cell-type annotation is fundamental to scRNA-seq analysis, yet rare and fine-grained cell types remain difficult to distinguish. In common workflows, experts iteratively determine clustering granularity, interpret marker evidence, and refine cell identities from broad lineages to fine-grained subtypes. LLM agents offer a new opportunity to incorporate biological knowledge into iterative annotation. However, current LLM-based annotators follow fixed prompting workflows: their annotation actions are rarely adapted to observed outcomes, and effective strategies are not explicitly summarized and reused across datasets. We present HiCellS, which casts hierarchical annotation as a learnable agent task under the guidance of a self-evolving skill bank. At each step, the agent observes the current annotation state (clustering outputs, marker evidence, biological knowledge) and decides the next actions to adaptively refine cell identities. Supervised fine-tuning provides the model with initial annotation capability, while reinforcement learning optimizes the model's annotation actions using final accuracy as feedback. Meanwhile, rollout trajectories are distilled into new skills that better guide future annotations, allowing the model and skill bank to co-evolve toward more accurate annotation. We demonstrate that HiCellS achieves strong performance across diverse scRNA-seq annotation benchmarks. Specifically, powered by a 14B-parameter backbone, HiCellS achieves an average Cell Ontology (CL)-informed matching score of 0.75, outperforming the strongest baseline CellTypist by up to 0.21. Beyond annotation accuracy, HiCellS demonstrates robust performance under varying levels of marker information.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Single-cell RNA sequencing, Cell Type Annotation, LLM-based Agents, Hierarchical Reasoning, Reinforcement, Self-evolving Skills

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DNpaOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
3(+1)
Presentation
3
Soundness
3(+1)
Read review
Paper summary

the paper presents an agentic cell type annotation method. it trains the llm with sft to intimate expert behavior, use grpo to improve its annotation capability, and employ a teach llm to distill skills for future use. experimental results, including ablation studies and case studies, highlights the advantage of the proposed method.

Critical strengths

S1: this paper proposes an innovative approach which combines both GRPO training and skill evolution, offering a promising solution with highly effective and the paradigm can also be applied in broader domains.

S2: the analysis on how evolving skills improve and degrade the annotation performance provides important information. the positive cases help readers understand how the skills help the agent deal with hard cases and the honest bad cases points out potential upgrades.

Critical weaknesses

W1: the term "(CL)-informed matching" is not expained. as an important concept, its full terminology should be presented early in the paper. also, as the main evaluation metric, its computations should be given in the main body.

W2: the design of skill evolution allows add, merge, and drop. however, it does not provide stats of the frequency of these operations. this could be a pie chart that show the distribution, and a line chart that shows the accumulation. also, it is unknown whether the number of skills grows infinitely as the agent passes more training samples.

Questions for authors

Q1: line 285 "HiCellS is trained on 113 separate samples", by 113 samples, do you mean 113 datasets (such as Tabula Sapiens), instead of 113 cells? this should be made clear.

Reviewer_wz1WOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

HiCellS formulates hierarchical cell-type annotation in single-cell RNA sequencing as a sequential decision-making problem controlled by an LLM agent. The framework combines supervised fine-tuning (SFT) on teacher-generated trajectories, GRPO-based reinforcement learning using hierarchical annotation accuracy as the terminal reward, and a skill-evolution mechanism that distills reusable rules from successful and unsuccessful trajectories. The agent performs quality control, clustering and resolution selection, and hierarchical cell-type annotation based on intermediate observations.

The authors evaluate the framework on 28 samples spanning eight dataset categories and compare it with several existing annotation methods. Ablation studies examine the contributions of RL and skill evolution. The results suggest improvements over some baselines, although the magnitude and consistency of the gains, the role of skill evolution, and the fairness of comparisons require further examination.

Critical strengths
  1. Relevant problem and integrated framework. Hierarchical cell-type annotation is a meaningful problem in single-cell analysis. HiCellS combines sequential tool use, intermediate observations, and hierarchical annotation in a unified agent framework, providing a coherent approach to coordinating multiple analytical stages.

  2. Component-level evaluation and analysis of failure cases. The paper includes ablations of RL and skill evolution and discusses cases in which skill evolution reduces performance. This provides useful initial evidence about the behavior and limitations of the proposed framework, rather than relying exclusively on aggregate improvements.

Critical weaknesses
  1. Evaluation fairness and potential QC-induced confounding. The paper acknowledges that QC filtering is applied only to HiCellS, leading to different evaluated cell populations. This makes it difficult to attribute the reported performance differences solely to annotation quality. Because the terminal reward is calculated over retained cells, the authors should also establish that the objective does not inadvertently reward excluding difficult-to-annotate cells. A common-cell-set evaluation and end-to-end analysis accounting for cell retention are needed to substantiate the principal performance claims.

  2. Insufficient evidence for RL generalization and the necessity of the action schema. The policy is trained on a limited number of biological samples, while the paper provides insufficient evidence of convergence and generalization across unseen samples. Moreover, the predefined action space may restrict the agent to a relatively coarse set of analytical decisions. The paper does not establish whether this RL-based framework offers measurable advantages over an LLM coding agent that can directly generate, execute, and revise a Scanpy pipeline. The independent contribution of skill evolution is also uncertain given its modest average gain and inconsistent effects across samples.

Questions for authors

HiCellS addresses an important problem in single-cell transcriptomics: hierarchical cell-type annotation requires iterative refinement of clustering and biological interpretation, and errors at coarse levels can propagate to finer-grained annotations. The paper presents an integrated framework combining supervised fine-tuning (SFT), GRPO-based reinforcement learning, and skill evolution. Its evaluation covers multiple datasets and annotation methods, and the ablation studies provide an initial investigation of the contributions of individual components. The analysis of cases where skill evolution degrades performance is also informative.

Despite these strengths, I have several concerns regarding the robustness of the learned policy, the granularity of the action space, and the validity of the empirical comparisons.

1. Is the RL training data sufficient to support robust policy learning?

The policy is trained on 113 biological samples for 18 RL epochs, using GRPO with a group size of four and terminal hierarchical annotation accuracy as the reward. Although multiple rollouts provide additional training signals, they do not necessarily capture the biological diversity of independent samples. The paper does not provide sufficient training dynamics to establish reliable convergence or generalization to unseen biological samples.

I encourage the authors to report training and held-out reward curves, per-level performance throughout training, rollout reward variability, and results across multiple random seeds. An analysis of performance as a function of the number of RL training samples would help clarify whether the observed gains reflect robust policy learning or adaptation to a limited training distribution.

2. Is the action space too coarse to justify the proposed RL framework?

HiCellS organizes the workflow around a predefined sequence of operations, including QC threshold selection, clustering and resolution selection at three hierarchical levels, and cell-type annotation. Although the policy can select parameters and react to intermediate observations, the available operations remain relatively constrained compared with the range of analytical decisions involved in scRNA-seq analysis.

For example, a code-execution agent using Scanpy could select highly variable genes with sc.pp.highly_variable_genes, normalize expression with sc.pp.normalize_total and sc.pp.log1p, perform dimensionality reduction with sc.tl.pca, construct neighborhood graphs with sc.pp.neighbors, perform Leiden clustering with sc.tl.leiden, and identify marker genes using sc.tl.rank_genes_groups with alternative differential-expression methods. Such an agent could inspect intermediate results, modify parameters or analytical methods, and iteratively revise the pipeline.

This alternative is not necessarily more accurate, but it raises an important question: what is gained by learning a policy over the predefined action schema rather than directly generating and executing analysis code? The current experiments do not establish whether the proposed RL and skill-evolution mechanisms offer advantages over such a flexible code-execution agent (e.g., Codex, Claude Code).

I encourage the authors to include a controlled comparison with a code-execution agent under matched data, computational budgets, and evaluation protocols. This would help establish the necessity and practical value of the proposed action schema. Given the limited number of training samples, the authors should also clarify whether the learned policy generalizes across tissues, datasets, and analytical settings, rather than relying on a small set of recurring parameter choices.

3. Are the baseline comparisons sufficiently fair and controlled?

The paper compares HiCellS with established reference-based, LLM-based, and agent-based methods using their respective workflows. However, the input representations and analytical procedures differ. For example, CellTypist receives normalized expression matrices, whereas several LLM-based baselines receive the top 10 differentially expressed genes per cluster. HiCellS additionally performs QC, adjusts clustering resolution, and makes sequential decisions based on intermediate observations. Comparisons using native workflows are useful, but they do not isolate the contribution of the proposed learning framework.

More importantly, the paper explicitly acknowledges that QC filtering is applied only to HiCellS. In the Tabula Sapiens example, HiCellS retains 118,067 cells, compared with 130,952 cells evaluated by the baselines. Consequently, the methods are not evaluated on identical cell populations. This confounds annotation quality with cell retention and task difficulty. Since the RL reward is computed over retained cells, the authors should also establish that the objective does not inadvertently favor removing difficult-to-annotate cells.

I encourage the authors to evaluate all methods on a common cell set where feasible, report cell-retention rates and losses across cell populations, and provide end-to-end results that account for excluded cells. The available tools, input information, interaction budgets, and access to skills should also be clarified for the API-based agent baselines.

4. Is the contribution of skill evolution sufficiently established?

The reported CL-informed score improves from 0.7335 without skill evolution to 0.7461 with the full method. However, the average gain is modest, and the paper reports substantial regressions on some samples. Because the distillation teacher can access ground-truth statistics unavailable to the policy at inference time, additional experiments are needed to distinguish the contributions of RL, oracle-guided supervision, and skill-library updates.

Multi-seed results, uncertainty estimates, and targeted ablations would help establish whether skill evolution provides a stable and independently meaningful benefit.

Finally, the asymmetric Cell Ontology matching metric may affect the interpretation of performance differences when reference labels vary in granularity. Strict-label metrics, per-level results, and sensitivity analyses under alternative matching rules would strengthen the conclusions. More complete training logs, reward specifications, and cell-level evaluation results would also improve reproducibility.

Overall, HiCellS presents a coherent and potentially useful framework. However, the current evidence does not sufficiently establish the robustness of RL training, the necessity of the constrained action space, or the fairness of the principal performance comparisons. I therefore lean toward Weak Reject, primarily because the central empirical claims require stronger controlled validation.

1 comment

Thank you for the detailed responses and additional experiments, and for pointing out the inconsistency between my written assessment and the recorded rating. My original written recommendation was Weak Reject, which was not accurately reflected in the recorded score.

The revised evaluation on a common post-QC cell set, the additional level-wise metrics, and the training analyses alleviate some of my concerns. I will therefore update my rating to 2 (Weak Reject). At the same time, I would appreciate further clarification on the following points.

  1. Codex baseline configuration and interpretation

I independently conducted a supplementary Codex experiment on PBMC68K. On 68,548 barcode-aligned cells, after fixed label mapping and hierarchical expansion, the cell-level accuracies were 89.88% at L1, 51.16% at L2, and 46.49% at L3, without ground-truth-optimized global relabeling. The corresponding HiCellS results in Table C.1 are 90%, 45%, and 32%.

The evaluated cells, label mapping, and hierarchy definitions have not yet been harmonized, so these results do not constitute a controlled direct comparison. They also cannot be directly compared with the Codex accuracy aggregated across eight datasets in Table 1. Nevertheless, this supplementary experiment suggests that Codex can achieve competitive level-wise performance on this dataset. To better understand the reported gap, please provide:

  • Codex L1/L2/L3 accuracies on PBMC68K and each evaluation dataset;
  • The underlying model version, reasoning settings, runtime and interaction budgets, and the tissue context, candidate labels, marker dictionaries, and reference resources actually provided;
  • Instantiated prompts, representative execution traces, and cell-level predictions. Appendix B.3 states that all methods use the same frozen label-mapping rules. Providing the mapping tables and evaluation code would help determine whether the Codex errors primarily arise from biological annotation, differences in label naming or granularity, or workflow execution.

If feasible, comparing the specialized harness and a free-form code-execution workflow using the same underlying model, input knowledge, evaluated cells, and explicit budgets would further clarify the contribution of the harness design itself.

  1. Training configuration and stability analysis

Appendix A.2 and Figure C.9 specify four rollouts per GRPO group, whereas the distillation prompt on page 15 describes selecting the best and worst trajectories from eight rollouts of the same task. These may correspond to different sampling or aggregation stages; please clarify their relationship and the configuration actually used.

Figure C.11 also reports workflow results across four random seeds. Please clarify whether these experiments involved independent SFT/GRPO training runs or used the same trained checkpoint with different inference or preprocessing seeds. This distinction would establish whether the results support training stability or workflow execution stability. An anonymous code repository, complete configurations, and raw training logs would also facilitate reproduction of the additional analyses.

  1. Scope of the skill-evolution and generalization conclusions

The multi-tissue skill-expansion experiment is informative for understanding factors that affect skill transfer. However, the additional tissues were selected based on performance declines already observed on the evaluation samples. I therefore view this as an exploratory analysis of negative transfer. Please clarify this experimental scope and distinguish it from generalization assessment on an independent test set.

Given that original skill evolution increases the overall CL score from 0.734 to 0.746, paired differences and uncertainty estimates on identical cell sets would help assess the stability of this gain. In addition, because the policy and skill bank change simultaneously during training, a fixed-skill-bank training control would help distinguish the contributions of policy optimization and skill updates.

The revision addresses some of my concerns, but these remaining questions still limit my confidence in the central conclusions. While recognizing the additional experiments and analyses, my current rating is 2 (Weak Reject).

Reviewer_gys7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

HiCellS trains an LLM agent to perform hierarchical cell-type annotation through quality control, clustering, and labeling. It combines supervised fine-tuning, reinforcement learning, and a skill bank updated from training trajectories. The task is relevant, but the evaluation leaves concerns about fairness, fine-grained accuracy, and the reliability of skill evolution.

Critical strengths
  1. The workflow reflects practical annotation procedures by jointly handling clustering and hierarchical labeling.

  2. A single trained model is evaluated across multiple tissues without dataset-specific fine-tuning. The paper also includes component ablations and informative failure cases.

Critical weaknesses
  1. The practical advantage is unclear. Training involves a 14B model, reinforcement learning, and repeated teacher calls, yet performance does not consistently exceed the strong API baseline. The approach requires fine-tuning a 14B model, reinforcement learning, and repeated teacher calls, but the paper does not provide enough cost measurements to establish when these investments are worthwhile.

  2. The main comparisons use different sets of cells. Section 4.1 explicitly states that HiCellS is evaluated on cells retained after its own quality filtering, whereas the baselines are evaluated without this additional filtering. As a result, the reported differences may reflect both annotation quality and changes in the difficulty of the evaluated population. A comparison on identical cells under a common QC procedure is needed, alongside retention rates and the cell-type composition of the excluded cells.

  3. The primary metric does not adequately establish fine-grained annotation accuracy. Appendix B.3 gives full credit whenever the predicted label is a descendant of the reference label. However, a coarse reference annotation cannot establish whether a particular predicted subtype is correct. This metric captures compatibility with the ontology, but it provides limited evidence for accurate subtype identification.

  4. Skill evolution yields small and unstable gains. It improves CL score by 0.0126 and macro-F1 by 0.0064, while accuracy on the Tabula Sapiens heart sample drops from 0.6917 to 0.2476. These regressions weaken the evidence for reliable transfer of learned skills.

Questions for authors

See Weaknesses for details.

1 comment

Thank you for the detailed responses. Some of my concerns have been addressed. However, two concerns remain.

First, the results in Response 1 do not establish a compelling practical advantage of the trained 14B model over inexpensive APIs such as DeepSeek. Given the substantial training cost and non-negligible deployment costs of a 14B model, I find the practical value of this contribution limited.

Second, the results in Response 4 suggest that the benefits of the skills are highly sensitive to the datasets used to construct the skill library. Changing these sources leads to inconsistent effects across evaluation samples. The evidence therefore remains insufficient to show that the proposed skill extraction and evolution method delivers reliable, transferable gains.

Overall, I acknowledge the effectiveness of the HiCellS harness. However, I remain unconvinced by the practical value of training the 14B model and the robustness of the skill evolution mechanism. I therefore maintain my original rating.

Comments on the paper

2 comments

thank you for your response.

In light of the clarification and skill evolution stats, i have raised the Informativeness and Soundness scores.

i kept my positive overall rating and increased my confidence.

Hi reviewers,

Thank you for your reviewing service. The authors have provided fairly extensive rebuttals. Please confirm that you have read them, state which of your concerns are or are not addressed, and whether that changes your recommendations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.