acceptodds
Under review as a conference paper at ICLR 2027

Which Neural Responses Can Intervention Data Validate?

Abstract

Which variation should a neural response model explain to make a finite intervention experiment more informative? We study state-conditioned mean stimulation effects under randomized assignment and bounded responses, with finite-sample error control during sequential monitoring. A variance decomposition separates predictable common activity from effect heterogeneity and residual response noise. Matching null laws establish an inverse-margin validation lower bound and an exact information frontier with full, partial, and zero-adjustment regimes. In an eight-state model with asymmetric noise and heterogeneous effects, learned adjustment with accurate history at margin 0.01 reduces restricted correct-decision cost from 7,262 trials for a no-history oracle to 1,818, a 75.0% saving. All 256 learned runs answer correctly by 16,384 trials, compared with 244 for the oracle. With 70% history replacement, the advantage reverses: learned cost is 9,306 trials versus 7,778 without history. Across 256 population-history questions, improving observation increases optimal information by a median factor of 1.53; common-mean adjustment with the finer observation exceeds the coarse observation's information optimum in 253 questions. In a matched beta–binomial condition, a parametric likelihood-ratio test costs 60 trials, compared with 124 for the bounded-response oracle. These results identify when auxiliary prediction changes supported response conclusions and distinguish gains from fitting, observing state, and restricting the response class.

Primary area:
applications to neuroscience & cognitive science
Keywords:
causal inference, regression adjustment, state-dependent neural responses

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rC67OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I found the paper hard to understand, so I'm not sure what the main contributions are. It begins with some timely and interesting questions in neuroscience modeling, sequential experimental design, and optical perturbations, but then after a couple pages switches modes into a description of some mathematical results that were not sufficiently well explained for me to follow.

Critical strengths

See above; I found this one hard to evaluate / figure out.

Critical weaknesses

As noted above, this paper was not possible for me to follow. It would need significant rewriting for clarity before I can review it positively.

Questions for authors

Unfortunately the paper was not sufficiently clear for me to formulate useful constructive questions here.

2 comments

Thank you for the clear diagnosis: the motivation ended on page 2 and the mathematics began without a bridge. The revision builds that bridge and states the contributions explicitly; the results are unchanged.

The contributions in brief (now listed at the end of the introduction).

  1. No predictor of pre-stimulus activity can reduce the trials needed to validate a stimulation effect below a floor of order one over the effect margin, because rare opposing responses cannot be ruled out early (Proposition 1).
  2. In a model with partly predictable background, subtracting the predicted background is the best use of a predictor only when unpredictable noise dominates the margin; otherwise part of the background should be kept, or none removed, and we solve this in closed form (Proposition 2).
  3. For any finite history, the fastest attainable rate is the optimum of a concave program, which puts better observation and better fitting on one scale; better observation wins in 253 of 256 questions built on recorded histories (Proposition 3, Section 4.2).
  4. On public mouse photostimulation recordings, two common questions fall into opposite regimes (Section 4.4), and Section 5 turns the results into a planning rule.

How the revision connects the topics you named to the mathematics.

  • Optical perturbations. The paper now starts from a photostimulation experiment. New Figure 1 shows one recorded trial, what a prediction removes, how evidence accumulates until the test stops, and the three regimes. Section 4.4 answers the opening questions on recordings, not only in simulation.
  • Sequential experimental design. New Section 2 explains in words what a trial is, what counts as a response, how evidence and stopping work and what validity assumes, with a notation table, before any result. Each result in Section 3 is then stated in one sentence before its formula; the closed-form solution comes before the general program, and proofs are in Appendix A.
  • Neuroscience modeling. Section 5 draws the consequence for evaluating neural response models: report the decision cost of prespecified questions, not only prediction accuracy. The recordings show why observation matters: with 128 rather than 16 neurons, the history explains a median 60% rather than 13% of the background.
  • From theory to data. Section 4 answers four questions in turn: the regimes in simulation, observing versus fitting, when the optimal adjustment pays and what standard tests cost, and where recorded experiments sit.

We would appreciate another look at Sections 1–2 and Figure 1.

If the revision now makes all things clear and accessible, we would sincerely appreciate it if you would reconsider your evaluation and score. We are happy to clarify any remaining step during the discussion.

Thank you again for your review. We have revised our paper and organized it into a clear format. If the paper is now accessible to you, we would sincerely appreciate it if you would reconsider your evaluation. If the sections are now clear, we would sincerely appreciate it if you would reconsider your evaluation; if some points are still unclear, we would be glad to explain it before the discussion closes.

Reviewer_msKLOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Apologies, but as a human reader, the writing of this paper is completely impenetrable to me. As such, I really have no idea what the paper is trying to say or do. I figure that this is still a more valuable summary than if I were to just copy a ChatGPT summary into this box.

Critical strengths
  • I really do not understand this paper at all
Critical weaknesses

The writing in this paper is jam packed with hyper dense jargon that lacks any contextualization whatsoever. Consider for example the first two lines of the abstract:

'Which variation should a neural response model explain to make a finite intervention experiment more informative? We study state-conditioned mean stimulation effects under randomized assignment and bounded responses, with finite sample error control during sequential monitoring. '

What is meant by a neural response model in this case? Which kinds of neurons? What is a finite intervention experiment? What does it mean for one to be informative? What variation is being referred to? What is a 'state conditioned mean stimulation effect'? What is being randomly assigned? What is a bounded response? What is finite sample error control? What is sequential monitoring?

It is possible that I am just not the right reviewer for this work. But more likely, I believe, is that the current presentation is simply not fit for human consumption.

Questions for authors

N/A

5 comments

Thank you for the candid review. We agree that the paper must be readable by a careful scientist outside the subfield, and we used your suggestions to make a checklist when rewriting the abstract, the introduction and the setup. The mathematical results are unchanged; what changed is how the paper introduces them.

What the paper is about. In optogenetic experiments, researchers stimulate a few neurons with light and record whether other neurons become more active. Each trial costs time with the animal, so they want to stop as soon as the answer is reliable. Neural activity fluctuates from trial to trial, and part of this fluctuation can be predicted from activity recorded just before the stimulus. We ask how many trials such a prediction can save and how it should be used. We prove that no prediction can reduce the number of trials below a minimum, show that subtracting the prediction is not always its best use, and test both results on recordings from mouse cortex.

Your ten questions. Each term is now defined before it is used (Section 2, Table 1; glossary in Appendix A.1):

TermMeaning in the revisionWhere
neural response modela model that predicts population activity from activity recorded just before the stimulusSec. 1, Fig. 1B
which neuronscortical neurons imaged with two-photon microscopy in mice; we analyze motor-cortex recordingsSec. 1, 4.4
finite intervention experimenta limited series of trials, each a light stimulation or a blankSec. 1–2, Fig. 1A
informativereaching a reliable conclusion with fewer trialsSec. 1
variationtrial-to-trial changes in the response: predictable background, differences in the effect, and unpredictable noiseSec. 3, Fig. 1B
state-conditioned mean stimulation effectthe average difference between stimulated and blank trials among trials in a given brain stateSec. 2
randomly assigneda fair coin decides whether each trial is a stimulation or a blankSec. 2, Fig. 1A
bounded responsea trial outcome between 0 and 1, such as the fraction of recorded neurons that become activeSec. 2
finite-sample error controlthe chance of a wrong conclusion stays below a set level at every sample size, not only asymptoticallySec. 2
sequential monitoringlooking at the data after every trial and stopping when the evidence suffices, without raising the error rateSec. 2, Fig. 1C

The two sentences you quoted have been replaced. The abstract now begins: "Optogenetic experiments ask whether stimulating some neurons changes the activity of others, and every trial costs time with the animal. A model that predicts population activity from its recent history could shorten such experiments by explaining away background fluctuations."

Other changes. New Figure 1 explains the experiment and the three results visually; the introduction ends with an explicit list of contributions; each result opens with a one-sentence statement; the experiments are organized around the questions they answer; and a five-step planning box in Section 5 states the practical recommendation.

We would be grateful if you could read the new abstract, Section 1, Figure 1 and Section 2 (about ten minutes).

If you found the revised paper is now accessible to you, we would sincerely appreciate it if you would reconsider your evaluation and scores in light of the revision. We are glad to clarify anything that remains unclear.

We hope these additions address the reviewer's concerns, and we would be grateful if the reviewer would reconsider the assessment in light of the revision. We would be glad to clarify anything further.

Thank you again for your candid review. We have revised our paper and organized it into a clear format. If the paper is now accessible to you, we would sincerely appreciate it if you would reconsider your evaluation. If parts are still unclear, please tell us which and we would be glad to clarify them before the discussion closes.

Reviewer_3Yh7OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks when a model that predicts neural responses can help an experiment reach a reliable conclusion with fewer trials. Information collected before an intervention is executed can explain some variation in neural activity, which makes the treatment effects easier to estimate.

Its main finding is that removing predicted variation as much as possible is not always optimal. Sometimes leave some variations helps to accumulate evidence faster. The authors derive when different types of adjustment is the best. It establishes a lower bound on the required observations and propose a method to learn the adjustment as the experiment proceeds. The simulations show substantial trial savings when the recorded history is informative.

Critical strengths

I find it an interesting and useful contribution that the predictor minimizing response-score variation need not minimize the time needed to reach a statistically valid conclusion. Instead, if one leaves some predictable variation unremoved, this can permit larger valid evidence increments. It derives different types of adjustment regimes and construct matching null distribution to establish optimality within its stated model.

Critical weaknesses

I am uncertain about the paper's practical value. Although leveraging past data is a big win by cutting the required trial by 75%, the new algorithm is not doing the heavy lifting. The paper's new method took 1818 trials while a simple baseline took 1817. They are essentially tied. Given this, I'm not sure what is the practical takeaway. It seems that simpler existing method like sequential inference with control variables is already sufficient.

Moreover, I think calling this a "neuroscience" application is a stretch. It reads like a simulation wrapped in brain science labels. The simulations are done in a controlled environments, and can be only proved working in these conditions. It does not tell us whether real brain data behaves similarly. On top of that, real-world neural data is messy, all of which violates the simplified assumptions, such as independent trials.

Questions for authors
  • What is the practical justification for using the more complex new objective over simple common-mean method?
  • Can you come up with realistic experiments or conditions where your objective provides improvements over the simple baseline?
  • Would it be possible to test the method on real-world neural recording dataset?
4 comments

We thank the reviewer for your careful summary and for identifying the question we had left implicit: what an experimenter should do differently because of these results. The revision answers it with four new experiments and a planning rule.

Q1 / W1. Practical justification and takeaway. The 1818 vs 1817 tie is what the theory predicts: in that condition, optimizing the adjustment raises the evidence rate by only 0.8% (Figure 3A), and the 75% saving comes from observing the history. The analysis tells an experimenter, before collecting data:

  • how many trials a predictor can save, and whether a session can resolve the question at all;
  • whether to subtract the prediction. Proposition 2 proves that common-mean adjustment is exactly optimal whenever unpredictable noise dominates the margin, which certifies the simple method there. At large margins it can be slower than ignoring the history, and the optimal adjustment keeps part of the background or none of it;
  • when better observation is worth more than better fitting.

Section 5 states this as a five-step planning rule.

Q2. Realistic conditions where the objective helps. New Section 4.3 (Figure 4A, Appendix B.4) tests the conditions the theory points to: margins 0.08–0.24, accurate or partly replaced histories, 1 or 32 questions monitored on the same trials, and coefficients learned from scratch or fitted on an earlier session.

  • Oracle savings match the predicted savings to within 2 percentage points.
  • With coefficients from an earlier session, the optimal objective saves 1.4–14.0% of trials relative to common-mean adjustment, more at larger margins; with a mismatched earlier session it still saves 4.4–6.5%.
  • From a cold start the saving is at most 0.7%, because start-up costs dominate short experiments.
  • At margin 0.24 with 32 questions, common-mean adjustment is slower than ignoring the history (52.1 vs 46.5 oracle trials), while the warm optimal learner needs only 46.3.

W1. Are existing sequential methods with control variates sufficient? For running a valid test, they are a strong baseline, and we now include them (Figure 4B, Appendix B.4). A betting confidence sequence with a control variate needs 1440±19 trials, fewer than our learners (1818±14), and is faster in five of six conditions: without coefficients from earlier sessions, how the stake is learned matters more than the adjustment objective (Section 4.3). We therefore do not claim a better learner, and the planning rule in Section 5 now recommends this procedure when no earlier session is available. With coefficients from an earlier session, both warm learners need about half its trials (Table 5). What these methods do not provide is the design analysis above: how many trials a predictor can save, whether to subtract the prediction at all, and when to improve the observation instead. Validity also matters: a fixed-sample z test planned by power analysis uses 155 trials but errs in 20.9% of experiments when rare opposing responses occur (4.5% without them), because any test valid for all bounded responses needs at least 249 trials at that power (Proposition 1).

Q3 / W2. Real recordings and the neuroscience framing. Currently new Section 4.4 (Figure 5, Appendix D) analyzes the public all-optical recordings of Daie et al. (2021): 16 sessions from 8 mice and 64 questions about targeted and distant neurons. Histories are recorded, each question's response distribution is calibrated to its recorded trials, and pre-stimulus activity is balanced between stimulated and blank trials in every session (permutation p ≥ 0.10). The recordings illustrate each point above:

  • Distant neurons: pre-stimulus activity explains a median 60% of held-out response variation and cuts the trials needed for a 2-point change by a median 31%, while optimizing adds 0.8%, as predicted for the noise-dominated regime. Yet all 32 distant questions need more trials than their session collected.
  • Targeted neurons: subtracting the prediction slows the test in 31 of 32 questions, as predicted for large margins.
  • Observing 16 rather than 128 neurons lowers the explained variation from 60% to 13%.

We also narrowed the framing to an experimental-design analysis for perturbation experiments; the recordings show where real questions fall on its map.

W2. Messy data and independent trials. Validity needs only that each assignment is a fair coin given the past and that responses are bounded; no distributional form is assumed, so outliers in the raw signal cannot break it. Under any null in which each trial's effect given the past is at most the threshold, the evidence is a nonnegative supermartingale (Section 2, Appendix B.5); independence is used only to compute rates and prove optimality. Under state persistence, carryover and slow drift, valid procedures err in at most 2.3% of null experiments and need up to 17% more trials.

We hope these additions address the reviewer's concerns, and we would be glad to clarify anything further.

Thank you again for your review. As the discussion period progresses, we wanted to check whether our response and the revised version address your main concerns. Briefly:

  1. Practical value (Q1, Q2). With coefficients from an earlier session, the optimal objective saves 1.4–14.0% of trials over common-mean adjustment in the conditions where the theory predicts gains (Section 4.3, Figure 4A); the original 1818 vs 1817 tie occurs where the theory predicts only 0.8%.

  2. Existing methods (W1). We added a betting confidence sequence with a control variate, AsympCS and a fixed-sample z test (Figure 4B); the planning rule in Section 5 now recommends the betting confidence sequence when no earlier session is available.

  3. Real recordings (Q3, W2). Section 4.4 analyzes 64 questions from public all-optical recordings of mouse motor cortex: distant and targeted neurons fall in opposite regimes, as the theory predicts. Appendix B.5 shows error control under dependent trials.

If any concern remains, we would be glad to address it before the discussion closes.

Thank you for responding to my comments. I would like to maintain my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.