HiCellS addresses an important problem in single-cell transcriptomics: hierarchical cell-type annotation requires iterative refinement of clustering and biological interpretation, and errors at coarse levels can propagate to finer-grained annotations. The paper presents an integrated framework combining supervised fine-tuning (SFT), GRPO-based reinforcement learning, and skill evolution. Its evaluation covers multiple datasets and annotation methods, and the ablation studies provide an initial investigation of the contributions of individual components. The analysis of cases where skill evolution degrades performance is also informative.
Despite these strengths, I have several concerns regarding the robustness of the learned policy, the granularity of the action space, and the validity of the empirical comparisons.
1. Is the RL training data sufficient to support robust policy learning?
The policy is trained on 113 biological samples for 18 RL epochs, using GRPO with a group size of four and terminal hierarchical annotation accuracy as the reward. Although multiple rollouts provide additional training signals, they do not necessarily capture the biological diversity of independent samples. The paper does not provide sufficient training dynamics to establish reliable convergence or generalization to unseen biological samples.
I encourage the authors to report training and held-out reward curves, per-level performance throughout training, rollout reward variability, and results across multiple random seeds. An analysis of performance as a function of the number of RL training samples would help clarify whether the observed gains reflect robust policy learning or adaptation to a limited training distribution.
2. Is the action space too coarse to justify the proposed RL framework?
HiCellS organizes the workflow around a predefined sequence of operations, including QC threshold selection, clustering and resolution selection at three hierarchical levels, and cell-type annotation. Although the policy can select parameters and react to intermediate observations, the available operations remain relatively constrained compared with the range of analytical decisions involved in scRNA-seq analysis.
For example, a code-execution agent using Scanpy could select highly variable genes with sc.pp.highly_variable_genes, normalize expression with sc.pp.normalize_total and sc.pp.log1p, perform dimensionality reduction with sc.tl.pca, construct neighborhood graphs with sc.pp.neighbors, perform Leiden clustering with sc.tl.leiden, and identify marker genes using sc.tl.rank_genes_groups with alternative differential-expression methods. Such an agent could inspect intermediate results, modify parameters or analytical methods, and iteratively revise the pipeline.
This alternative is not necessarily more accurate, but it raises an important question: what is gained by learning a policy over the predefined action schema rather than directly generating and executing analysis code? The current experiments do not establish whether the proposed RL and skill-evolution mechanisms offer advantages over such a flexible code-execution agent (e.g., Codex, Claude Code).
I encourage the authors to include a controlled comparison with a code-execution agent under matched data, computational budgets, and evaluation protocols. This would help establish the necessity and practical value of the proposed action schema. Given the limited number of training samples, the authors should also clarify whether the learned policy generalizes across tissues, datasets, and analytical settings, rather than relying on a small set of recurring parameter choices.
3. Are the baseline comparisons sufficiently fair and controlled?
The paper compares HiCellS with established reference-based, LLM-based, and agent-based methods using their respective workflows. However, the input representations and analytical procedures differ. For example, CellTypist receives normalized expression matrices, whereas several LLM-based baselines receive the top 10 differentially expressed genes per cluster. HiCellS additionally performs QC, adjusts clustering resolution, and makes sequential decisions based on intermediate observations. Comparisons using native workflows are useful, but they do not isolate the contribution of the proposed learning framework.
More importantly, the paper explicitly acknowledges that QC filtering is applied only to HiCellS. In the Tabula Sapiens example, HiCellS retains 118,067 cells, compared with 130,952 cells evaluated by the baselines. Consequently, the methods are not evaluated on identical cell populations. This confounds annotation quality with cell retention and task difficulty. Since the RL reward is computed over retained cells, the authors should also establish that the objective does not inadvertently favor removing difficult-to-annotate cells.
I encourage the authors to evaluate all methods on a common cell set where feasible, report cell-retention rates and losses across cell populations, and provide end-to-end results that account for excluded cells. The available tools, input information, interaction budgets, and access to skills should also be clarified for the API-based agent baselines.
4. Is the contribution of skill evolution sufficiently established?
The reported CL-informed score improves from 0.7335 without skill evolution to 0.7461 with the full method. However, the average gain is modest, and the paper reports substantial regressions on some samples. Because the distillation teacher can access ground-truth statistics unavailable to the policy at inference time, additional experiments are needed to distinguish the contributions of RL, oracle-guided supervision, and skill-library updates.
Multi-seed results, uncertainty estimates, and targeted ablations would help establish whether skill evolution provides a stable and independently meaningful benefit.
Finally, the asymmetric Cell Ontology matching metric may affect the interpretation of performance differences when reference labels vary in granularity. Strict-label metrics, per-level results, and sensitivity analyses under alternative matching rules would strengthen the conclusions. More complete training logs, reward specifications, and cell-level evaluation results would also improve reproducibility.
Overall, HiCellS presents a coherent and potentially useful framework. However, the current evidence does not sufficiently establish the robustness of RL training, the necessity of the constrained action space, or the fairness of the principal performance comparisons. I therefore lean toward Weak Reject, primarily because the central empirical claims require stronger controlled validation.
Thank you for the detailed responses and additional experiments, and for pointing out the inconsistency between my written assessment and the recorded rating. My original written recommendation was Weak Reject, which was not accurately reflected in the recorded score.
The revised evaluation on a common post-QC cell set, the additional level-wise metrics, and the training analyses alleviate some of my concerns. I will therefore update my rating to 2 (Weak Reject). At the same time, I would appreciate further clarification on the following points.
I independently conducted a supplementary Codex experiment on PBMC68K. On 68,548 barcode-aligned cells, after fixed label mapping and hierarchical expansion, the cell-level accuracies were 89.88% at L1, 51.16% at L2, and 46.49% at L3, without ground-truth-optimized global relabeling. The corresponding HiCellS results in Table C.1 are 90%, 45%, and 32%.
The evaluated cells, label mapping, and hierarchy definitions have not yet been harmonized, so these results do not constitute a controlled direct comparison. They also cannot be directly compared with the Codex accuracy aggregated across eight datasets in Table 1. Nevertheless, this supplementary experiment suggests that Codex can achieve competitive level-wise performance on this dataset. To better understand the reported gap, please provide:
If feasible, comparing the specialized harness and a free-form code-execution workflow using the same underlying model, input knowledge, evaluated cells, and explicit budgets would further clarify the contribution of the harness design itself.
Appendix A.2 and Figure C.9 specify four rollouts per GRPO group, whereas the distillation prompt on page 15 describes selecting the best and worst trajectories from eight rollouts of the same task. These may correspond to different sampling or aggregation stages; please clarify their relationship and the configuration actually used.
Figure C.11 also reports workflow results across four random seeds. Please clarify whether these experiments involved independent SFT/GRPO training runs or used the same trained checkpoint with different inference or preprocessing seeds. This distinction would establish whether the results support training stability or workflow execution stability. An anonymous code repository, complete configurations, and raw training logs would also facilitate reproduction of the additional analyses.
The multi-tissue skill-expansion experiment is informative for understanding factors that affect skill transfer. However, the additional tissues were selected based on performance declines already observed on the evaluation samples. I therefore view this as an exploratory analysis of negative transfer. Please clarify this experimental scope and distinguish it from generalization assessment on an independent test set.
Given that original skill evolution increases the overall CL score from 0.734 to 0.746, paired differences and uncertainty estimates on identical cell sets would help assess the stability of this gain. In addition, because the policy and skill bank change simultaneously during training, a fixed-skill-bank training control would help distinguish the contributions of policy optimization and skill updates.
The revision addresses some of my concerns, but these remaining questions still limit my confidence in the central conclusions. While recognizing the additional experiments and analyses, my current rating is 2 (Weak Reject).