- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents CellCanon, a framework combining cellular perturbation prediction, source-grounded explanation, and observation-driven updating. Its predictive component combines condition-level biological knowledge with retrieval from a training-response atlas, followed by continuous-response and separate DE/direction readouts. These outputs guide single-cell expression generation. An explanation component checks biological claims against source evidence, while a fixed numerical updater incorporates newly observed responses to improve predictions for the same drug in other cell contexts. Evaluation covers held-out SciPlex3 drug–cell conditions, source-grounded explanation tasks, and retrospective observation-feedback experiments.
Critical strengths
-
CellCanon reports substantial improvements in differential-response prediction across a broad set of comparators. On the main evaluation, DE AUPRC improves from the strongest baseline’s 0.4938 to 0.8067, and joint unchanged/up/down Macro-F1 improves from 0.5400 to 0.7420. The component studies also provide useful evidence for biological annotations and separately supervised response and classification readouts. These results make the predictive approach worth considering beyond its broader agent framing.
-
Sharing condition-level evidence across genes avoids repeated language-model analysis, while the numerical updater provides a concrete way to use newly measured source-cell responses without test-time parameter updates. The explicit distinction between source evidence, predictions, and measurements is also useful for scientific applications. Although the individual techniques are largely established, their integration addresses a meaningful practical problem.
Critical weaknesses
-
The contribution of the emphasized LLM and agent components is not sufficiently isolated. The ablations establish the value of biological information and independent classification heads, but provide weaker evidence for LLM-based evidence compilation itself. In Table 15, raw-source encoding slightly outperforms LLM-compiled knowledge on DE AP and F1, while compilation improves direction metrics. Furthermore, comparisons against expression predictors do not fully isolate the effects of additional classification supervision and decoding. Matched controls using the same information, supervision, and readout would better establish which proposed mechanisms drive the gains.
-
The evidence for a tightly integrated “agentic cellular world model” is limited. The shared state tracks predictions, evidence, claims, and observations, but the demonstrated numerical dependencies mainly involve source measurements updating target predictions. The paper does not establish that explanation revision or claim-dependency management improves prediction or acquisition decisions; observation schedules are fixed, and explanation and numerical transfer are evaluated separately. The explanation benchmark primarily tests source-supported judgments rather than the faithfulness of explanations to the numerical predictor. Appendix F.1 also states that evaluation-panel error analysis informed refinement of the correction procedure, making an independent evaluation of the final pipeline important.
Questions for authors
-
Which outputs of Explain or the claim-dependency state are consumed by Predict or the numerical Update algorithm? With identical compiled knowledge, atlas, weights, and observation schedules, would removing Explain and claim management change any reported numerical result?
-
How do the predictive gains compare with a simpler knowledge-conditioned retrieval model equipped with the same DE/direction supervision and count decoder? Are any existing comparisons sufficient to separate these effects from the proposed LLM compilation and integration?
-
Which parts of the correction procedure were modified following evaluation-panel error analysis? Is there an untouched evaluation set, or another assessment performed after the final procedure was frozen, that confirms the reported explanation performance?
7 comments
Response to Reviewer 69D3 (Part 2/6):
2. How do the predictive gains compare with a simpler knowledge-conditioned retrieval model equipped with the same DE/direction supervision and count decoder?
CellCanon improved on the matched kNN model by 4.93 percentage points in sampled DE AP and 2.02 points in sampled Joint F1 after single-cell generation. We compared cosine-weighted kNN over the 20 nearest support examples, ridge regression with per-gene L2-regularized logistic heads, and a drug-agnostic prior using cell-line mean response and gene-wise prevalence. They share the same 96-dimensional biological and chemical features, 50-candidate validation budget, DE and direction supervision, count decoding, and frozen generator. We generated 341,202 cells per model and applied the same Wilcoxon and Benjamini–Hochberg tests.
Across 112 test drugs, the 95% interval for the sampled DE AP gain over kNN was 2.92 to 7.31 points, with p = 0.0001. The Joint F1 comparison had p = 0.0247. The dense gains were 6.12 points in DE AP and 5.13 points in Joint F1. The Systema correlation after removing systematic cell-line shifts was +0.0526 versus +0.0186, p = 0.0046. A single-seed CellCanon reached 0.8102 DE AP and 0.7582 Joint F1, exceeding deterministic kNN by 5.28 and 3.64 points, with p = 0.0001. CellCanon led matched Linear by 7.60 points in DE AP and 6.71 in Joint F1, with p = 0.0001 and p = 0.0004.
When we retrained on the SciPlex3 split with zero shared Bemis–Murcko scaffolds, cross-split Tanimoto similarity below 0.30, and 339 conditions, CellCanon reached 0.7414 sampled Joint F1 before generation. The gains were 13.42 points over Linear at 0.6072, with Holm-adjusted p = 0.0038, and 18.03 points over kNN at 0.5611.
SciPlex3 results after single-cell generation
| Method | Sampled DE AP | Sampled DE F1 | Sampled direction F1 | Sampled Joint F1 | Top 50 R² | Δ-Pearson | Systema correlation |
|---|---|---|---|---|---|---|---|
| CellCanon | 0.8067 | 0.8164 | 0.8472 | 0.7420 | 0.9145 | 0.3254 | +0.0526 |
| kNN | 0.7574 | 0.8040 | 0.8219 | 0.7218 | 0.8887 | 0.2910 | +0.0186 |
| Linear | 0.7307 | 0.7745 | 0.7781 | 0.6749 | 0.8953 | 0.2945 | +0.0056 |
| Drug-agnostic prior | 0.6677 | 0.7052 | 0.7460 | 0.5940 | 0.8969 | 0.2768 | +0.0012 |
| PRnet | 0.4057 | 0.6961 | 0.7006 | 0.5400 | 0.9039 | 0.1528 | +0.0001 |
| BioLORD | 0.2695 | 0.6184 | 0.7903 | 0.4900 | 0.9197 | 0.2538 | −0.0004 |
See the revised manuscript, Tables 1–2, Appendix C, and Appendix L, Table 36.
Response to Reviewer 69D3 (Part 3/6):
3. Are any existing comparisons sufficient to separate these effects from the proposed LLM compilation and integration?
Yes. The matched comparisons separate the prediction model from the shared compiled inputs. The additional tests then show what changes when we remove the classification heads, change the count decoder, or turn off probability guidance during generation.
- Prediction before generation. With matched features, supervision, and count decoding, CellCanon gained 3.84 percentage points in sampled DE AP over kNN, 0.8588 versus 0.8204, with raw p = 0.0058 and Holm-adjusted p = 0.0232. The Joint F1 gain was 3.90 points, 0.7242 versus 0.6852, with raw p = 0.0180 and adjusted p = 0.0350. Against Linear, the gains were 5.19 and 6.86 points, with adjusted p = 0.0350 and p = 0.0270.
- Classification supervision. Removing the heads from CellCanon reduced sampled DE AP to 0.1464, with Holm-adjusted p = 0.0036. Adding the same heads and count decoder to external models raised chemCPA from 0.1337 to 0.7675 DE AP. With those components matched, CellCanon reached 0.7242 Joint F1 versus 0.6359 for BioLORD, the strongest adapted deep model, a gain of 8.83 points. Adapted CMonge, PRnet, and chemCPA reached 0.5420 to 0.6326 Joint F1.
- Count decoding. On the dense panel of 678,000 queries with 1.28% true DE, count decoding called 15,480 genes as DE across queries, versus 53,022 to 82,707 calls using sampled probability thresholds. Count MAE improved from 132.8 to 33.5, and dense Joint F1 from 0.453 to 0.557. With per-model dense thresholds selected on validation data, CellCanon led Linear by 5.94 and kNN by 4.94 points, with 95% intervals above zero.
- Probability-guided generation. Holding the condition head fixed, guidance raised DE Macro-F1 from 0.6873 to 0.8315, a gain of 14.42 points with Holm-adjusted p = 0.0005. Continuous RMSE was 4.9090 without guidance and 4.9078 with guidance. The corresponding comparison for the submitted model showed a gain of 16.58 points.
SciPlex3 predictions before single-cell generation
| Model | Sampled DE AP | Sampled DE F1 | Sampled direction F1 | Sampled Joint F1 | Dense DE AP | Dense Joint F1 | Systema correlation |
|---|---|---|---|---|---|---|---|
| CellCanon | 0.8588 | 0.8077 | 0.8393 | 0.7242 | 0.3371 | 0.5472 | +0.0545 |
| kNN | 0.8204 | 0.7794 | 0.8227 | 0.6852 | 0.2983 | 0.5276 | +0.0194 |
| Linear | 0.8069 | 0.7670 | 0.7928 | 0.6556 | 0.2226 | 0.5007 | +0.0184 |
| Drug-agnostic prior | 0.7685 | 0.7099 | 0.7920 | 0.5699 | 0.1762 | 0.4576 | 0.0000 |
| BioLORD with matched heads and decoder | 0.8395 | 0.7316 | 0.8121 | 0.6359 | 0.2650 | 0.5368 | −0.0004 |
| CellCanon without classification heads | 0.1464 | 0.4540 | 0.8239 | 0.3040 | 0.0121 | 0.3298 | +0.0336 |
See the revised manuscript, Table 6 and Appendices D–E, including Table 17.
Response to Reviewer 69D3 (Part 1/6):
Thank you for highlighting the gains in differential-response prediction, the efficiency of sharing evidence across genes, and the practical value of keeping sources, predictions, and measurements distinct while updating with fixed model weights.
To address your concerns, we added the following comparisons and experiments.
On pre-cleaned sources, rules and LLM compilation gave similar discrete accuracy with the same structured representation. The LLM improved continuous prediction over raw text and improved one post-generation ranking metric over the rules. To separate the effects of compilation and source volume, all variants used the same upstream source records across 558 conditions. We compared raw text, shuffled judgments, a deterministic parser that fills the same evidence fields, and an open-weight end-to-end replication.
The deterministic parser and LLM compilation reached 0.7575 and 0.7532 sampled Joint F1. Across 20 discrete comparisons, all Holm-adjusted p-values were at least 0.4215. Both exceeded matched kNN at 0.7218 and Linear at 0.6749. The structured evidence, supervision, and readout are shared by both variants. The open-weight Qwen3.5-35B-A3B replication was within 0.0065 on the main prediction endpoints, including 0.7384 versus 0.7420 Joint F1. Separately, it reached 95.90% answerable Explain accuracy versus 100% for the submitted pipeline, and Update RMSE was 4.4475 versus 4.4969.
With the source records and text budgets matched, LLM judgments improved pre-generation Pearson correlation from 0.6240 for raw text to 0.6340 and lowered RMSE from 5.0056 to 4.9278, with Holm-adjusted p = 0.0014. Shuffling judgments within splits or across drugs gave Pearson correlations of 0.6252 and 0.6257. Relative to these shuffles, LLM judgments reduced RMSE by 0.0729 and 0.0714, with Holm-adjusted p = 0.0008. With the same balanced validation criterion, post-generation Top 100 Δ-Spearman was 0.3753 for the LLM and 0.3597 for the rules. The gain was 0.0156, with a 95% interval of 0.0036 to 0.0275 and unadjusted two-sided p = 0.0129. Compiled evidence links each claim to its sources. Reusing condition-level evidence across genes cut prediction token use by 99.77% relative to VCWorld.
Compiler comparisons on 339 SciPlex3 test conditions
For the DE AP comparison quoted in the review, 0.4938 is VCWorld's two-point trapezoidal PR area. Its step-wise AP is 0.2196. Under matched inputs and generation, the strongest baseline is kNN at 0.7574, and CellCanon reaches 0.8067, a gain of 4.93 percentage points.
See the revised manuscript, Tables 1–2 and 4, Appendix E, Tables 15–16, and Appendix I, Table 27.