acceptodds
Under review as a conference paper at ICLR 2027

Redundant Regulatory Programs Can Check Cell Groups They Locate Poorly

Abstract

Curated regulatory programs, such as pathway gene sets and regulons, are built into single-cell models, yet what they add beyond an expression encoder is unclear. We find that, used as linear features next to an encoder, programs estimate the encoder-defined state poorly but can test whether a group of cells fits the relations among programs seen in known cells. A geometric theorem explains why: the set of consistent states is wide along directions that program directions combine into only at high cost, whereas the threshold for detecting a shift of one program grows only with the cost of combining the others into it, which redundancy keeps small. Our conformally calibrated, state-invariant statistic, HellyCell, flags a group when no state fits all programs and returns a witness of at most programs that no state satisfies jointly (: state dimension); if data are exchangeable and few programs shift, a flag whose witness misses every shifted program has probability at most the level . Confidence sets from programs alone are 3.7 to 8.1 times wider than the encoder's with least squares, and 19.6 to 24.7 times as the intersections of intervals that the check uses. Estimated on pilot cells, the part of a mean shift that no change of state explains predicts which of 154 real perturbations are detected in held-out groups of ten cells (rank correlation 0.89; 0.91 in the largest screen, against 0.81 for the raw shift): interferon responses are detected, most single-gene CRISPR perturbations are not. Baarda's -test has more power on sparse or strong shifts, so HellyCell trades power for witnesses; invariance to the state fails under new sequencing protocols.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
single-cell genomics, regulatory programs, prior knowledge, conformal inference, consistency testing, novelty detection, Helly's theorem, linear programming, redundancy, perturbation screens

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_1jrVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper argues that predefined gene sets, although poor standalone encoders of cell state in the evaluated settings, can help detect discrepancies between new data and a reference dataset. The method learns relationships between predefined program readings and a low-dimensional cell state using reference data, then applies split conformal calibration to establish a consistency threshold. On new cells or cell groups, it tests whether any state can jointly explain the observed program readings. When no state satisfies the calibrated constraints, it flags the unit and returns a small set of programs certifying the incompatibility.

Critical strengths

The paper presents an interesting and novel application of gene sets as consistency checks. The geometric analysis provides a compelling explanation for how redundant programs can support discrepancy detection despite offering limited precision for state estimation. Returning a small, verifiable witness also makes the detected inconsistency easier to inspect.

Critical weaknesses

The practical utility for practitioners remains insufficiently clear. In particular, it is unclear what actionable biological conclusions a witness supports beyond identifying an unusual combination of program readings. A concrete example showing how a practitioner would interpret a flag, investigate its witness, and gain insight beyond standard differential expression or pathway analysis would strengthen the paper.

Questions for authors
  1. How does the method scale computationally with the number of reference cells, programs, and state dimensions? What would be the runtime and memory requirements for reference atlases containing millions of cells?
  2. How should practitioners use a massive, heterogeneous atlas as a reference for a smaller newly collected dataset? In particular, how much matched calibration data is required, and how should differences in sequencing protocol, donor composition, and cell-type representation be handled to preserve meaningful error control?
1 comment
AuthorsOpenReview ↗

How a practitioner reads a flag and its witness, cost on a million-cell reference, and how much matched calibration data is needed

Thank you for the constructive review. We added the requested practical evidence; changes are in blue in the revised PDF.

How a practitioner uses a flag and its witness (Section 7, Table 2).

  1. Fit the encoder and the view model on a reference atlas, and calibrate the tolerance q^\hat q on matched control groups of the new study (see below).
  2. Score each new group: one linear program of d+1d+1 variables and 2K2K constraints, 12 to 59 ms on one CPU thread for K=50K=50 to 800 programs at d=8d=8 (19 to 130 ms at d=32d=32), plus 99 to 153 ms for the witness of a flagged group at d=8d=8 (Table 12).
  3. A flag means that no cell state explains the group's program readings. It is rarely triggered by a different mix of cell types: control groups contaminated with 10% to 50% cells of another known type are flagged in 3% to 5% of cases, whereas comparing pathway scores with reference cells of the annotated type flags 38% at 30% contamination and 72% at 50% (Table 34). A new protocol does trigger flags (17% to 98% of groups), so matched technical controls come first.
  4. Read the witness. In a flagged group of IFN-β\beta-stimulated CD4 T cells chosen by a fixed rule (Table 2), the witness has nine regulons; IRF9 and STAT2, of the interferon-activated ISGF3 complex, lie 9.8 and 8.3 noise units above their fit at the group's encoder state, and the weighted contrast with the other seven regulons, which does not depend on the state, is 4.06 against a tolerance of 3.61. The conclusion is that no cell state satisfies these nine regulons together, so the group is not merely in a different state; the witness does not say which member changed (Section 7), but its two large interferon residuals direct the follow-up.

What this adds beyond differential expression or pathway analysis. Differential expression reports which genes differ from a comparison group, not whether the difference is a change of state or of regulation. A ranked list of deviating pathways cannot tell either: on 941 flagged IFN-β\beta groups, the nine most deviating programs were jointly satisfiable by some state in all but at most 3.2% of groups, but by a reference group's state in at most 1.9%, so the list cannot separate an unseen state from a regulatory change; the witness does. It is also reusable as a compact check: over 400 flagged groups, a witness's nine programs alone flag on average 68% of the other stimulated groups of its cell type (48% for CD4 T cells) and no control group (Table 33).

Scaling (Tables 12 and 36). Scoring cost does not depend on the reference size; it grows with the number of programs and the state dimension as in step 2. Fitting needs only sums over reference cells, so a reference can be streamed in chunks. We timed a streamed fit on references of 10410^4, 10510^5 and 10610^6 cells resampled from Kang control cells: the pass over 1,000,000 cells took 2.0 minutes (about 8,400 cells per second) after a one-minute setup on a 50,000-cell subsample, on a laptop CPU. Peak memory was 7.5 GB at every size, set by the setup, not by the reference size. Calibration costs one linear program per calibration group.

Using a large heterogeneous atlas (Section 7, Tables 14, 21, 35). The atlas can supply the encoder and the view model, but calibration must match the new data.

  • How much calibration data: at least 19 groups at α=0.05\alpha=0.05. With 20 groups, 10% of calibration draws give more than 10% false alarms; with 100 groups the false-alarm rate of a draw lies between 2.7% and 8.4% (5% and 95% quantiles); with 200, between 4.8% and 6.9%.
  • Protocol: calibrate, and preferably refit, on the new protocol; otherwise new protocols are flagged in 17% to 98% of groups.
  • Donor composition: calibrate on donors comparable to the test donors; calibrating on one donor instead of two moves the false-alarm rate to between 0.9% and 9.3%, and over 16 donor partitions it reaches 15.4%.
  • Cell-type representation: a quantile per cell type caps every type's false alarms (worst type 7% instead of 25%) at a cost in power for rare types (mean detection 56% instead of 98%); with few calibration groups per type, pool and report per-type rates.
Reviewer_2jKWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper considers the use of so-called curated regulatory programs for localization compared to an expression encoder. They show some experiments to say that used as linear readings programs do not add much to locating a cell's state. On the other hand, the authors claim that such programs work as checks, of whether a group of cells fits the relations in known cells. They attempt to explain the reason for this behavior under some theoretical assumptions in terms of the width of the set of states and the geometry. Further they propose a statistic, that is termed Helly's witness, which adds a short certificate to check consistency (joint infeasibility). Experiments are done to justify the above claims and theorems but some seem limited to synthetic setups.

Critical strengths

On the positive side, the paper seems to be relevant to a biological problem. It makes links between computational biology and statistics. For an expert reader it may offer some nuanced understanding of the usefulness and significance of regualatory programs. It also formalizes a new statistic that could provide witnesses of consistency of programs.

Critical weaknesses

On negative side, the paper is extremely unreachable for the non-expert. Very few terms are properly defined or motivated. My main concern is that it seems that even the problem definition is not formalized properly. What is the localization problem? Why is it interesting/important? What is the calibrated consistency test? Where can it be applied? Some details are under Implementation details in appendix, but even with that it was difficult to understand even the statements of the theorem's let alone the proofs. What is the precise problem being tackled in the paper and how does the algorithm solve it? Also, the algorithm is not explained clearly and many terms in are unclear. For instance, what is an optional width? The question of what is being ``flagged'' is defined in a very terse manner and does not give any intuition as to what it is supposed to do, even though this is mentioned vaguely in the abstract/introduction. Unfortunately, the paper has no preliminaries section and even if technical definitions are present, they are terse and written for someone who knows the area well.

I have to also note that many parts of the introduction are not even full sentences. For instance, in the Contributions, the second bullet point is just a set of phrases. I was neither able to parse this nor able to understand the terms involved.

Overall, I think the paper is probably written for an expert audience working in the area. But as a general AI researcher with some familiarity with biopathway and gene regulatory systems, I found it very difficult to judge both novelty and soundness of the paper. Also, as acknowledged by the authors, the papers seems to make heavy use of LLM tools, not just for the writing but also for the theory and experimental design. This perhaps adds to making it inaccessible to the non-expert.

Questions for authors

Already mentioned in the critical weakness section. Please state the technical problem statement in terms that a non-expert can understand. Can you have a clear preliminaries section with all terms defined and carefully motivated, perhaps with example when needed? This would be beyond the notations table in the appendix.

2 comments
AuthorsOpenReview ↗

A formal problem statement in plain terms, a new preliminaries section with a worked example, and a rewritten algorithm description

Thank you for the specific and actionable suggestions. We implemented each of them; the new and rewritten text is in blue in the revised PDF.

Problem statement (new Section 2, "Problem setup"). We have reference cells whose behaviour we take as known, and KK curated programs (gene sets). For a unit yy, a single cell or a group of nn cells of one type and donor, we observe its state z∈Rdz\in\mathbb{R}^d (a low-dimensional summary from an encoder, here d=8d=8 principal components) and its readings rj(y)r_j(y), the average standardized expression of program jj's genes. On reference cells each reading is close to a linear function of the state, rj≈aj⊤z+bjr_j\approx a_j^\top z+b_j, with noise scale σj\sigma_j. So program jj accepts a state zz at tolerance qq when ∣rj−aj⊤z−bj∣≤qσj|r_j-a_j^\top z-b_j|\le q\sigma_j, a band in state space; the feasible set Fq(y)F_q(y) is the set of states accepted by all programs.

  • Localization asks how small Fq(y)F_q(y) is, that is how precisely programs pin down the state. It matters because programs are built into single-cell models as features; if they localized well, they could replace or sharpen an encoder. We find they do not (Table 1).
  • Consistency checking asks only whether Fq(y)F_q(y) is empty. If it is, no state, seen or unseen, explains the readings, so the relations among programs have changed, as in a response to a stimulus or a technical shift. It applies whenever a new group of cells must be compared with a reference, for example in perturbation screens or when mapping new samples onto an atlas.
  • The calibrated test computes T(y)=min⁡zmax⁡j∣rj−aj⊤z−bj∣/σjT(y)=\min_z\max_j|r_j-a_j^\top z-b_j|/\sigma_j, one linear program, and flags yy when T(y)>q^T(y)>\hat q, that is when Fq^(y)F_{\hat q}(y) is empty. The tolerance q^\hat q is a quantile of TT on held-out reference groups, so a reference group is flagged with probability at most α\alpha (Theorem 1).
  • The witness is a set of at most d+1d+1 programs whose bands alone have no common state, found from the linear program's dual. Its weights combine the programs into a contrast in which the state cancels, and the contrast exceeds q^\hat q, so anyone can verify the conclusion.

Worked toy example (Section 2). With d=2d=2, three programs require z1∈[0,1]z_1\in[0,1], z2∈[0,1]z_2\in[0,1] and z1+z2∈[3,4]z_1+z_2\in[3,4]. Any two can be met, but not all three, because the first two give z1+z2≤2z_1+z_2\le2. Subtracting the first two constraints from the third leaves 0≥10\ge1, a certificate in which the state cancels. A real-data example of the same kind is in Table 2.

What is new. First, the two uses of programs depend on different geometric quantities that can be arbitrarily far apart (Theorem 2, followed by a plain sentence): the feasible set is wide along directions that are expensive to write from the programs' directions, whereas detecting a change in one program depends only on how cheaply the other programs reproduce it. This explains why programs that add little to an encoder (Table 1) still detect changed regulation. Second, a calibrated test whose every flag comes with at most d+1d+1 programs that provably no state satisfies, with a certificate in which the state cancels (Theorems 1 and 3); ranked lists of deviating programs almost never give one (Table 32).

"Optional width" and "flagged". The line in Algorithm 1 that returned widths was confusing: widths and thresholds are diagnostics used to test the theory and are not needed to run the test. Algorithm 1 now returns only "not flagged" (some state fits all programs) or "flagged" with the witness and its weights, and Section 3 explains the statistic, the calibration and the witness.

Readability. The abstract and introduction are rewritten in plain language and the contributions are full sentences. Related work moved after the experiments, so that a reader reaches the problem and the method first.

Synthetic setups. All main results use six real single-cell datasets. Synthetic data appear only in one controlled test of the theorem (Appendix C) and in planted shifts, where known changes are added to real cells in order to check guarantees whose ground truth is otherwise unknown.

AI tools. We disclosed our use of an LLM assistant in the AI use statement and the submission form. All AI-assisted work was reviewed by us: the code was tested end to end, every number is generated by script from the logged results, and each reference was checked; we take full responsibility for the theory, code and text. Accessibility is our responsibility, and the revision addresses it directly.

We would be grateful if you could look at the new Sections 2 and 3.

Reviewer_2jKWOpenReview ↗

Thanks for the revised version; but there are still too many issues

Thank you for the detailed response and the revision. The new Section 2 (Problem Setup) is a clear improvement: it defines the units, readings and reference model, motivates the problems, and its toy example is useful.

However, Section 3 and 4 still remain very hard to follow for a non-expert reader. Section 3 never writes out the Chebyshev LP or its dual (they appear only in the proof of Theorem 3), although the witness, its weights etc are all read off that dual. Section 4 opens with about ten definitions in one paragraph. Moreover the theorems are essentially unchanged, and no intuition is given, for example for why the width along u scales with κ(u). Terms like pivotality, infeasible subsystem, and the operating envelope are not sufficiently explained, at least in the main text.

I also re-read the introduction. I felt that the contributions in the introduction are still unclear: Bullet 1 uses κ(u), κ₋ⱼ and "program directions" before Section 4 defines them, and "these bounds" is unclear. Is "Arbitrarily far apart" an existence result or a computable result? Bullet 2 uses "state-invariant" (never defined) and "sparse shifts" (defined only in Assumption 2) and the soundness part is also unclear. Bullet 3 also seems to have two unrelated findings. It claims prediction "within every pilot/held-out split", but given the correction for Reviewer 6oJw, doesnt this only hold within the largest screen?

Given all this, I retain my score as of now.

Reviewer_QvweOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The major idea of this paper is that single cell transcriptomes can be represented by a state (e.g. a low dimensional representation) but also by a priori known gene sets. The paper examines the relationship between the former and the latter and the extent to which the cell states are covered by the projections from known gene sets.

Critical strengths

This is the first paper I have seen formally address the connection between the set of states a cell can exist in and a priori known gene sets/programs.

Critical weaknesses

As someone who works quite closely in the area of ML for single-cell, I made it as far as the second paragraph before becoming deeply confused by this paper. It is one of the most jargon-dense papers I have ever read and clearly mostly written by AI, which makes it both really difficult to read and subsequently evaluate. Even the AI use statement is clearly AI generated ("the synthetic world of Appendix C"!). So I'm left wondering what the point is, as I think it's unlikely someone will turn up to ICLR and leave with a new insight on the problem. Maybe there is some breakthrough in this paper that another reviewer will pick up on, but if the point in a research paper is (still?) to communicate some new concept or idea then I struggle to see how this succeeds.

Questions for authors

In section 5 the authors re-analyze several single-cell datasets. Is there a scenario where this work significantly alters the resulting conclusions from such a data analysis relative to the existing workflows?

1 comment

Thank you for the frank assessment, and for noting that this is the first formal treatment you have seen of how the possible states of a cell relate to known gene programs. You are right that the submission was hard to read. We rewrote the abstract and introduction in plain language and added a problem-setup section (Section 2) that defines the main terms in plain language, with a small worked example. We take full responsibility for the text, the code and the claims; every number in the paper is generated by script from the logged results in the supplement.

The point, in plain terms. Gene programs, such as pathway gene sets and transcription-factor targets, are often added to single-cell models to describe where a group of cells sits. In reference cells, each program's average expression is close to a linear function of the cell state, so each program confines the state to a band. We find that programs are poor at saying where the state is (their confidence sets are 3.7 to 8.1 times wider than an expression encoder's), yet good at a different question: whether any state at all fits all the bands. If none does, the relations among the programs have changed, which is what happens in a response such as interferon stimulation (97% of held-out groups of ten cells flagged at a 5% false-alarm rate). A geometric theorem explains why the two tasks differ, and the test returns a small set of programs (at most nine here) that provably no state can satisfy together.

Your question: does this change the conclusion of a standard analysis? Yes, in two situations we can quantify on the paper's data.

  1. Contamination or a new mix of cell types versus a regulatory change. A common analysis compares a group's pathway scores with reference cells of its annotated cell type. When control groups of one type contain 30% or 50% cells of another known type from the same donor, that comparison flags 38% and 72% of them, so contamination would be read as a pathway change. Our test flags only 3% of them, because contamination only moves the state (the least-squares tests, which also ignore changes of state, behave alike), while still flagging 99% of interferon-stimulated groups (Table 34). A flag therefore means that no state explains the readings, not that the cell mix differs.

  2. Ranked lists of deviating pathways versus a certificate. The usual explanation of a deviating group is a list of its most deviating programs. On 941 interferon-stimulated groups flagged by our test, the nine most deviating programs could be explained jointly by some cell state in all but at most 3.2% of groups, but by the state of a reference group in at most 1.9%, so the list cannot tell "this group is a cell state we have not seen" from "this group's regulation changed". Our witness can. In one group chosen by a fixed rule (Table 2), the targets of IRF9 and STAT2, two interferon-activated transcription factors, are 9.8 and 8.3 noise units above the value predicted from the group's state, and no state reconciles them with seven other regulons; the nine most deviating programs, by contrast, are satisfied together by some state.

A third example: in a CRISPR activation screen, a standard distribution test finds an effect for 102 of 105 activations, but for only 10 is the part of the effect that no change of cell state can explain above our threshold (Section 5). For most activations, the effect is compatible with a change of cell state within our tolerance; only a few show a detectable change of program relations.

We hope that the rewritten Sections 1 and 2 make the contribution accessible, and we would be grateful if you could take another look.

Reviewer_CYLqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether curated regulatory programs can provide useful consistency checks even when they poorly estimate an encoder-defined cellular state. It models program readings as noisy linear measurements of a low-dimensional state and introduces HellyCell, a split-conformally calibrated Chebyshev fitting statistic. A group is flagged when no state satisfies all calibrated program constraints, and linear-programming duality yields an infeasibility witness containing at most d+1 programs.

Critical strengths
  1. A coherent theoretical explanation connecting localization, detection, and certificates. Theorem 2 makes the distinction between poorly measured state directions and redundant individual program directions precise. Combined with the constructive witness result, this gives a useful account of why weak localization need not imply weak consistency checking. This conceptual contribution is the strongest reason to support acceptance; it does not require HellyCell to dominate competing detectors.

  2. Extensive, candid experiments that test both the proposed explanation and its boundaries. The evaluation includes matched calibration of classical baselines, alternative encoders, program randomization, planted and real perturbations, and distribution-shift stress tests. The pilot/held-out analysis provides evidence that the state-unexplained shift is empirically relevant, while the negative results prevent an overly broad biological interpretation. This makes the paper informative beyond the proposed method itself.

Critical weaknesses
  1. The practical value of short witnesses remains insufficiently demonstrated relative to their detection-power cost. Witnesses are the main distinguishing output because least-squares tests generally match or exceed HellyCell’s power, and HellyCell uniquely flags only a small fraction of perturbed groups. However, the sparse-shift interpretation is validated only for planted reading-level changes. Real expression changes affect many overlapping programs, making “the witness intersects the shifted set” potentially weak information. Table 2 further shows that witnesses contain relatively few predefined response programs compared with residual rankings. This is not a contradiction, the certificates are not intended as culprit rankings, but it leaves an unanswered practical question: what scientific decision becomes possible because a short infeasible subsystem is available? A concrete biological example demonstrating that benefit would materially strengthen the contribution.

  2. The empirical operating envelope and calibration results support a narrower claim than the strongest headline interpretations. The held-out correlation is substantial, but it predicts the detector’s response using pilot data from the same perturbation condition and a closely related residual statistic. Table 27 shows comparable or stronger prediction from least-squares residuals and direct pilot detection rates. The envelope is also acknowledged to be informative mainly at its extremes. Meanwhile, cell-type-specific false-alarm rates reach 25% under marginal calibration, and per-type calibration reduces mean detection at group size ten from 98% to 56%. These findings make the operational trade-off consequential. More explicit treatment of hierarchical sampling and pilot-estimation uncertainty is needed before interpreting the empirical envelope as a reliable prospective guarantee.

Questions for authors
  1. Clarification—sampling and calibration: What precise sampling model justifies the independence assertion in Appendix A? Are guarantees intended conditional on the observed donors/batches, or for new biological replicates? How do shared reference controls and reused donor rotations affect uncertainty estimates in Tables 26–27?

  2. Clarification—operating envelope: How sensitive is the reported envelope agreement to uncertainty in the pilot mean shift and to changes in residual covariance under perturbation? Does the agreement remain informative when using dataset-specific envelopes with uncertainty bands, particularly in the intermediate-signal regime?

  3. Assessment-changing evidence—witness utility: Can the authors provide a concrete real-data example where the certified joint inconsistency supports an interpretation or decision that a matched-size residual list does not? This would directly address whether the certificate’s practical value compensates for reduced detection power.

  4. Clarification—estimation-error diagnostic: Does Table 29 incorporate donor-level dependence or heteroscedasticity? If not, would the conclusion that estimation error is negligible remain supported by a donor-aware resampling analysis?

2 comments
AuthorsOpenReview ↗

What a witness adds on real data, a narrower reading of the envelope, the sampling model, and donor-aware estimation error

Thank you for the precise questions; we start with witness utility. Changes are in blue.

What decision a witness enables. We agree that for broad real responses Theorem 3(b) says little, since almost any witness meets the shifted set; Section 4 now says that what a witness offers there is the certificate of Theorem 3(a), and Section 7 states that witness soundness is verified only with planted shifts. Witnesses name fewer reference programs than lists (precision 0.16 against 0.82, Table 32; your Table 2 is now Table 31) because they include the programs that pin down the state. What a list cannot do is exclude an unseen state: on 941 flagged groups of ten IFN-β\beta cells, lists of the nine largest anchored residuals, ww-test residuals or raw program scores are jointly infeasible in at most 3.2% of groups, and a reference group's state satisfies them in at most 1.9%. So a list shows "far from known states", while a witness certifies "changed program relations" on at most nine named programs in every flag. Telling a new state (such as a new mix of cell types) from changed relations is a property of state invariance, which TT shares with the least-squares tests: control groups with 10% to 50% cells of another known type are flagged by TT, BB and GG in 2% to 5% of cases, against 38% at 30% and 72% at 50% contamination for a comparison with reference groups of the annotated type (Table 34). The witness adds the certificate, which can be rechecked on new samples from nine readings: over 400 flagged groups, a witness's nine programs alone flag on average 68% of the other stimulated groups of its cell type (48% for CD4 T cells) and no control group (Table 33). We claim the certificate, not power: TT flags a group that both least-squares tests miss in 1.1% of a perturbation's groups on average.

A concrete example (new Table 2, chosen by a rule fixed in advance: the flagged CD4 T-cell group with the median TT). Its witness has nine regulons, among them IRF9 and STAT2 of the interferon-activated ISGF3 complex, which lie 9.8 and 8.3 noise units above their fit at the group's encoder state. Their weighted contrast with the other seven regulons is the same for every state and equals 4.06>q^=3.614.06>\hat q=3.61, so no state reconciles them. The nine largest anchored residuals of the same group, seven of them interferon programs, are satisfiable together (their own statistic TL=0.51T_L=0.51).

A narrower reading of the envelope. We agree and narrowed the claims. Within each of six pilot/held-out splits (Table 27), ρ^\hat\rho ranks the 105 Norman activations at 0.80 to 0.88, above the raw shift in every split (0.71 to 0.77); the least-squares residual does as well or better (0.86 to 0.92), and the pilot flag rate, a direct measurement on other cells, is now the benchmark rather than a competitor. Pooled over screens, the raw shift predicts as well as ρ^\hat\rho; we dropped that claim. It predicts from pilot cells of the same perturbation, not for unseen ones (Section 7). With the uncertainty of ρ^\hat\rho and of the calibration distribution, the envelope constrains only 18 of 154 contexts, at the ends (Table 28). The per-type trade-off (worst type 25% false alarms, 7% with a quantile per type, at a mean detection of 56% instead of 98%) was in Section 5 and is now also in the practical-use paragraph of Section 7.

Sampling model (Appendix A). Exchangeability comes from randomly assigning the cells of the same donors and batches to calibration and test groups. The guarantees are over that split and conditional on those donors and batches, not for new biological replicates; new donors are a stress test (false alarms 1.8% to 7.3% over rotations, up to 15.4% over 16 donor partitions). For Tables 26 to 28, a new bootstrap resamples calibration groups, shared control cells and pilot and held-out groups jointly, with donors drawn as clusters first; it remains conditional on the fitted models. Example: the Norman correlation of ρ^\hat\rho with detection is 0.80 [0.75, 0.85], its gain over the raw shift 0.08 [0.01, 0.15].

Envelope sensitivity. The standard error of ρ^\hat\rho is a median 7% to 12% of ρ^\hat\rho (40% for Datlinger). The spread of perturbed groups around their own mean, relative to calibration groups, has median 0.96 to 1.10 per screen and exceeds 1.2 for 5 of 140 contexts, all Norman; this checks the scale of residual variability, not its full covariance, which Proposition 1 assumes unchanged. In the intermediate regime the bands are not informative, which we now state.

Table 29 (now Table 30). It used the OLS formula. We added the HC3 sandwich (largest term 0.04 slab half-widths) and a donor jackknife that refits the view model without each training donor (largest 0.18, medians at most 0.04). The term stays well below one half-width; Section 5 now reports the OLS and jackknife values (0.03 and 0.18).

Thanks for the detailed response and the revisions. I appreciate the clearer explanation of the witness’s role, the narrower interpretation of the envelope, and the clarification of the sampling assumptions and limitations. These changes improve the clarity of the paper. After considering the response, my overall recommendation and rating remain unchanged.

Reviewer_6oJwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether curated regulatory programs can provide useful consistency checks even when they estimate an expression-defined cell state poorly. HellyCell fits linear relations between program readings and a low-dimensional state, then uses a conformally calibrated Chebyshev residual to flag cell groups for which no state satisfies all program constraints. Each flag is accompanied by an infeasible subset of at most d + 1 programs. The theory relates localization widths and single-program detection thresholds through synthesis costs and gives a marginal witness-soundness guarantee under specified shift assumptions.

Critical strengths
  1. The central distinction is useful and mathematically well organized. Separating accurate state estimation from consistency testing is a worthwhile perspective for single-cell representations. Theorem 2 connects both tasks through an explicit geometry, and Theorem 3 makes the short certificate computationally accessible.
  2. The experimental coverage is substantial and includes strong comparisons. Negative results and limitations are reported clearly.
Critical weaknesses
  1. Preserve pilot and evaluation separation in the final prospective statistic. Within each directed split, the pilot and held-out cells are disjoint. However, the reported correlation is computed after averaging pilot estimates over both swapped halves and three splits and pooling detection over those same swapped halves (lines 411-419, 1603-1619; Fig. 3c; Tables 26-27). The implementation confirms this aggregation in code/experiments/e24_prospective.py, lines 105-129. Consequently, every eligible cell can contribute to both the aggregate predictor and aggregate outcome. The direct pilot flag-rate comparator has the same issue: averaging both directions makes it close to another grouping of the same full cell collection (e24b_predictors.py, lines 54-78). This does not invalidate the within-fold split or imply that the fitted model used test labels, but the final correlation does not establish prediction on a permanently untouched sample. Please compute correlations across perturbations separately for each directed pilot/test split and then summarize their variation, or retain a prespecified pilot/test partition throughout. Repeat the predictor comparisons under that evaluation. The direction and magnitude of any change in the headline result remain unknown until this is done.
  2. Distinguish the estimated empirical envelope from the guarantee in Proposition 1. The proposition assumes a fixed additive shift and the appropriate unchanged noise law (lines 239-249), whereas the real-data analysis substitutes an estimated mean shift into bounds computed from an empirical calibration distribution. Real perturbations may also change residual variability or produce heterogeneous responses. The paper acknowledges pilot-estimation inflation and that the envelope constrains detection mainly at its ends (lines 416-417, 1617-1619). Therefore, observing no points outside the estimated envelope is limited evidence for its practical predictive accuracy. Please report uncertainty in the pilot ratio and envelope, the widths of the resulting intervals, and a simple check of residual variability in perturbed versus reference groups. Apply this diagnostic to the separated evaluation requested in comment 1. A clear statement that the plotted envelope is an estimated diagnostic, rather than a finite-sample confidence guarantee for these biological perturbations, would align the empirical interpretation with the theory.
  3. Clarify the sampling unit behind the prospective confidence intervals. Resampling perturbation rows is preferable to treating every cell group as independent, and the manuscript explicitly describes this choice (lines 1612-1613). Nevertheless, the 154 rows include 28 cell-type-by-rotation evaluations of the same IFN intervention, while rows within screens share control samples and fitted calibration models (lines 995-997; Table 26). The row bootstrap does not propagate uncertainty from those shared sources. Please state whether the intervals are conditional on the particular reference cells and fitted models, and add a sensitivity analysis that preserves the available donor, replicate, and shared-control structure. Report within-screen results prominently and describe the pooled sample as perturbation-context evaluations.
Questions for authors
  1. Do the prospective correlations, especially the Norman improvement over raw shift size, persist when each correlation is calculated within one directed pilot/test split before summarizing across splits?
  2. What does the explanation comparison show when all methods are evaluated on the same T-flagged groups? Could one worked biological example demonstrate what the joint infeasibility certificate adds to the ranked residual lists on those groups?
2 comments
AuthorsOpenReview ↗

Within-split prospective test, dependence-aware intervals, the envelope as an estimated diagnostic, and explanations compared on the same flagged groups

Thank you for auditing the prospective test down to the code. All three points were right; the revision (changes in blue) implements each request.

1. Pilot and evaluation separation. Averaging over both swapped halves let a cell enter both aggregates. We now compute every correlation within one directed split (same seeds, halves and groups; averaging the per-split ρ^\hat\rho and pooling the per-split detections reproduces the submitted values exactly), and we fixed the first split as the prespecified partition before computing any per-split result. Rank correlation with held-out detection, range over the six splits (Table 27):

ρ^\hat\rhoraw shiftLS residual GGpilot flag rate
all 154 contexts0.79 to 0.850.80 to 0.820.79 to 0.840.83 to 0.91
Norman (105)0.80 to 0.880.71 to 0.770.86 to 0.920.82 to 0.88
Kang (28)0.32 to 0.510.23 to 0.410.32 to 0.500.37 to 0.72

Your question: yes, the Norman advantage over the raw shift persists in all six splits; in the prespecified split ρ^\hat\rho gives 0.80 [0.75, 0.85] against 0.72 [0.66, 0.78], paired difference 0.08 [0.01, 0.15]. The headline does change: pooled, the correlation is 0.79 to 0.85 rather than the submitted aggregate 0.89, and the raw shift does as well, because the screens differ more than the perturbations within them, so we removed the pooled claim; the abstract and Section 5 now report within-split, within-screen results. The least-squares residual does as well or better, so what matters is removing the state, not the ℓ∞\ell_\infty norm. Datlinger and Dixit show no ranking.

2. Envelope versus guarantee. We agree; Sections 4 and 5 now call the real-data envelope an estimated diagnostic, not a finite-sample guarantee. In the prespecified split (Table 28), the bootstrap standard error of ρ^\hat\rho is a median 7% of ρ^\hat\rho for Kang, 11% Norman, 12% Dixit and 40% Datlinger. Propagating it, together with the 95% DKW band of the calibration distribution (half-width 0.09 to 0.17 by dataset), the band is narrower than 0.9 (excludes more than a tenth of [0,1][0,1]) for only 18 of 154 contexts, all at the ends of the range of ρ^\hat\rho. Held-out detection lies inside every band, which is therefore weak evidence, as you anticipated. For residual variability, the spread of held-out perturbed groups around their own mean, measured by the same seminorm NN, divided by that of calibration groups has median 0.96 (Kang), 1.02 (Norman), 0.99 (Dixit) and 1.10 (Datlinger); it exceeds 1.2 for 5 of 140 contexts, all Norman activations.

3. Sampling unit. The submitted intervals resampled perturbation rows and were conditional on the reference cells, calibration units and fitted models; we now say so. The new dependence-aware bootstrap resamples jointly, for all contexts of a dataset and rotation, the calibration groups (hence q^\hat q), the test cells (shared control cells get one weight in all contexts) and the pilot and held-out groups, with donors (Kang) or replicates (Datlinger) drawn first as clusters. The intervals remain conditional on the fitted encoder and view model, which we state. The 154 rows are now called perturbation-context evaluations, and the 28 Kang contexts are described as one intervention.

Your second question: explanations on the same TT-flagged groups. On the same 941 flagged groups of ten IFN-β\beta cells (Table 32), the share naming a reference program and the precision are 96% and 0.16 for witnesses, 100% and 0.82 for the nine largest anchored residuals, 100% and 0.54 for the nine largest ww-test residuals, and 0.93 precision for the single largest ww-test residual. The certificate is what lists lack: the nine largest anchored residuals are jointly infeasible at q^\hat q in 0.1% of groups and the nine largest ww-test residuals in 3.2%, while a reference group's state satisfies them in 0.0% and 0.1%; an unseen state could explain a list, whereas a witness rules out every state. Worked example (Table 2; a rule fixed in advance chose the flagged group with the median TT among CD4 T cells): the regulons of IRF9 and STAT2, two members of the interferon-activated ISGF3 complex, lie 9.8 and 8.3 noise units above their fit at the group's encoder state, and no state reconciles them with seven other regulons. The certificate is the contrast ψ=∑jλjsjc~j=4.06>q^=3.61\psi=\sum_j\lambda_js_j\tilde c_j=4.06>\hat q=3.61, which is the same for every state because ∑jλjsja~j=0\sum_j\lambda_js_j\tilde a_j=0. The nine largest anchored residuals of that group, seven of them interferon programs, are satisfied together by some state (their own consistency statistic is TL=0.51T_L=0.51). Over 400 flagged groups (Table 33), a witness's nine programs, reused alone, flag on average 68% of the other stimulated groups of its cell type (48% for CD4 T cells) and no control group; the matched lists flag almost none (0.0%).

I appreciate the detailed response and the revised manuscript. The authors have addressed the methodological concerns I raised. After considering the paper's significance and the conceptual advance, I will keep my overall rating and recommendation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.