acceptodds
Under review as a conference paper at ICLR 2027

PROBE HORIZON IS NOT PREDICTABLE MEMORY: MEASURING THE HISTORY A SURROGATE NEEDS ON A COMPLETE CONNECTOME MODEL

Abstract

A surrogate model must be told how much of the past to see, and practice takes that number from a probe of the simulator, typically an impulse horizon, an autocorrelation time or a settling time read at a threshold such as . Such a probe reports how long one chosen excitation leaves a visible trace in one observed statistic. What a surrogate needs is different: the horizon over which past input still changes the noise-averaged response it is trained to predict. On a leaky integrate-and-fire model of the first complete connectome of an adult male *Drosophila* central nervous system, the three probes return 30, under 10 and 150 ms, while a windowed MLP student keeps gaining fidelity out to 960 ms of history and the memoryless MLP trails a GRU by 0.15 in fidelity. We quantify what the student needs with a shared-suffix measurement whose estimator subtracts the teacher's own stochastic floor in squared distance. On this connectome the measurement returns a tail that is still above a tenth of its initial amplitude at 1 s, the last lag we measure, matching the range of history the student keeps using and well beyond every probe. On a second complete brain connectome under identical neuron dynamics the same measurement returns 83 ms, and its students need 80 ms. On a synthetic teacher with a planted 500 ms state it returns a fitted slow timescale of 521 ms [429, 693]. The student's required history survives a sixteenfold larger training bank, a change of input noise and a 9.6-fold change in capacity, and wherever the input contains transitions it moves with the predictable tail across teachers and interventions. A gain-matched intervention makes the separation causal. Read after subtracting its stochastic floor, the teacher's step response lengthens from 220 ms to past the end of the step-response window, while the conventional reading shortens. The history a surrogate needs is a measurement of predictable memory, and predictable memory is not the horizon a probe reports.

Primary area:
applications to neuroscience & cognitive science
Keywords:
surrogate modeling, teacher-student distillation, memory timescales, stochastic simulators, connectome, Drosophila, spiking neural networks, system identification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QWDXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether conventional probe timescales indicate how much input history a surrogate model needs. It proposes a shared-suffix measurement that compares responses to different past inputs while holding recent inputs fixed, and subtracts an estimated noise floor. Experiments on connectome models and synthetic recurrent networks relate this measurement to surrogate performance.

Critical strengths

The paired synthetic teachers, which differ only in whether a planted slow state contributes to the output, provide a clean test of the proposed measurement against a known mechanism.

The study controls for training-bank size, input noise, and student capacity. The constant-amplitude family also shows why intervention-based dependence is not necessarily history that is useful for prediction.

Critical weaknesses

The released code and the paper disagree on how the main metrics were computed. The code computes fidelity in channel-standardised log-rate space rather than on the untransformed rates specified in Appendix C, which changes the prediction target and channel weighting. It also includes the identity endpoint in the crossing and timescale fits although Section 4 excludes it, and implements a different floor-corrected step-response calculation from the squared-distance 10% criterion described in Appendix E. These discrepancies should be reconciled to establish which implementation generated the reported results and whether the estimates are affected.

The statement that a student scored against the D_TT/2 floor “carries every predictable component of the response” is too strong as written. Merely evaluating a student against this floor does not establish that it has recovered every predictable component; that interpretation requires attaining the population noise floor. The discussion should distinguish the scoring objective from what the trained students actually achieve.

Questions for authors

Please clarify whether the manuscript descriptions or the released implementation require correction, and whether any reported values change.

Please state the stimulus and training conditions under which the shared-suffix tail is expected to predict c*, and whether the connectome experiments satisfy them.

Reviewer_nScROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper explores how much past input is required for building surrogate models. The paper argues that the probes commonly used for establishing this window are not measuring the memory needed and propose an alternative method (shared-suffix measurement) for establishing the appropriate memory window. They validate their method on an artificial dataset with fixed memory and evaluate it using two connectome-constraints models with different memory requirements. The question is interesting and the method appears sound, but the paper is very hard to read, and I have some concerns about how strongly the main result supports the claims.

Critical strengths

S.i) Interesting question & clear solution: The topic of the paper is very specific and addresses a real issue that surrogate models have and the paper makes a clear conceptual distinction between what a probe reports and what a surrogate needs. The shared-suffix measurement estimates the latter directly, and computing the noise correction in squared distance is simple and well motivated.

S.ii) Ground-truth validation. The planted-timescale teacher is a very good control. The method recovers the planted state while the impulse probe cannot distinguish the network from one without it. This makes the authors’ point convincingly.

Critical weaknesses

W.i) Clarity. The paper is extremely dense. Many quantities are defined late, in scattered places, or only in the appendices. Some examples:

Fig. 1 uses c* before it is defined in §2.

The abstract uses “fidelity,” “stochastic floor” and “gain-matched” without explanation.

§3 reports a 150 ms settling time and Table 3 reports 140 ms; the reason appears only in Appendix F.

The lag grid of the shared-suffix measurement is specified only in Appendix H.

The motivation for several analysis choices, such as the 1/e and 10% thresholds and the choice of τ90 as the summary statistic, is not given where they are introduced.

The question and method are focused enough to lend themselves to a concise, readable structure, and I think a substantial rewrite is needed.

W.ii) Possible episode-onset confound. Episodes are 1.5 s long, the student window reaches 960 ms, and only the first 200 ms are excluded from scoring. Part of what both the student and the shared-suffix measurement capture may be slow relaxation from the start of the episode rather than memory. A stationary-state control (long burn-in before measurement) would address it.

W.iii) Strength of the baselines. The standard probes as implemented are fairly weak: a short pulse at 24 seeds, and autocorrelation on noise-dominated bins. Moreover, a floor-corrected reading of the existing settling probe already orders the four teachers almost correctly. This raises the question of whether a new measurement is needed, or whether floor-correcting existing probes would suffice.

W.iv) Validation. Only one planted timescale and one variance share are tested. A sweep over the planted timescale and the slow component’s share of variance would better demonstrate the methods capabilities.

Questions for authors

Q.i) MaleCNS and FlyWire are connectomes of the same species, yet one shows a strong slow component and the other none. They differ in sex, and MaleCNS includes the ventral nerve cord. The paper’s controls partly address this: the VNC lesion shortens the tail, a random lesion of equal size removes it even more, and the gain-matched control rules out firing rate alone. However, none of these localises the slow component. Is it a property of the fly’s wiring, or of this LIF model in this regime? Would the degree-preserving rewired MaleCNS from Appendix B also show long predictable memory? At least some discussion of possible causes would strengthen the paper.

Q.ii) Can you show the MaleCNS results with a longer window grid and lag grid, so that neither quantity is censored?

Q.iii) The shared-suffix measurement costs several thousand simulated episodes, while a student fit takes about a minute on a CPU. In what setting would one run the measurement rather than sweeping across the student window?

Reviewer_qsJLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper explores whether time scales based on traditional probes can reliably determine the historical window required for alternative models. The authors distinguish between probe time scales and predictable memory and propose a shared suffix measurement method: by comparing the outputs generated from input sequences with the same recent suffixes and performing noise-floor correction in the squared-distance space, they estimate historical input dependencies. This method was evaluated on synthetic recurrent systems and two connectome-based simulations of the fruit fly brain, incorporating student models trained using different historical windows.

Critical strengths
  1. The paper identifies a meaningful problem in surrogate modeling.
  2. This paper makes some useful attempts to examine robustness and causal evidence.
Critical weaknesses
  1. Methodological innovation is limited. The proposed shared suffix method primarily involves scanning suffix lengths, measuring output deviations, and correcting for the noise floor. The paper does not demonstrate that this approach offers advantages over simpler methods that directly measure historical dependence under controlled conditions. A comparison with traditional probes alone is insufficient.
  2. Historical dependence does not necessarily imply predictive value. The detectable influence of past inputs may not translate into useful predictive information under the target input distribution. In the constant-amplitude experiment, the predictable tail extended beyond 1,000 milliseconds, yet students using only a 10-millisecond window achieved a fidelity of 0.980, which is nearly on par with the best result of 0.981. If students are not trained on multiple window lengths, how can the proposed metric distinguish between persistent historical dependence and the historical information actually required for prediction?
  3. The main results exhibit right-tail truncation. In the MaleCNS dataset, at least 960 milliseconds of historical data are required, yet the predictable tail remains above the threshold even at the 1,000-millisecond limit. The fitted 824-millisecond timescale does not resolve this truncation issue and may depend on the fitting process. The authors should expand the measurement range and student window range, and test the sensitivity to the fitting interval and function form.
  4. Evidence regarding generalizability and predictive value is limited. The experiments encompassed two connectome-based simulations, synthetic systems, and selected input conditions. The paper demonstrates a qualitative consistency between the measured memory and students’ historical requirements but does not establish a reliable quantitative mapping between the two. Can this metric predict an appropriate historical window applicable to unseen systems or input distributions?
  5. The contributions of the individual components remain unclear. The effects of shared suffix construction, repeated simulations, noise floor correction, and decay curve fitting have not yet been fully elucidated. In addition to the existing comparison of noise-corrected estimators, the authors should test other historical input control conditions and stimulus generation procedures to determine which components drive the results and to what extent the conclusions are robust.
Questions for authors

Stated in Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.