- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper asks whether conventional probe timescales indicate how much input history a surrogate model needs. It proposes a shared-suffix measurement that compares responses to different past inputs while holding recent inputs fixed, and subtracts an estimated noise floor. Experiments on connectome models and synthetic recurrent networks relate this measurement to surrogate performance.
Critical strengths
The paired synthetic teachers, which differ only in whether a planted slow state contributes to the output, provide a clean test of the proposed measurement against a known mechanism.
The study controls for training-bank size, input noise, and student capacity. The constant-amplitude family also shows why intervention-based dependence is not necessarily history that is useful for prediction.
Critical weaknesses
The released code and the paper disagree on how the main metrics were computed. The code computes fidelity in channel-standardised log-rate space rather than on the untransformed rates specified in Appendix C, which changes the prediction target and channel weighting. It also includes the identity endpoint in the crossing and timescale fits although Section 4 excludes it, and implements a different floor-corrected step-response calculation from the squared-distance 10% criterion described in Appendix E. These discrepancies should be reconciled to establish which implementation generated the reported results and whether the estimates are affected.
The statement that a student scored against the D_TT/2 floor “carries every predictable component of the response” is too strong as written. Merely evaluating a student against this floor does not establish that it has recovered every predictable component; that interpretation requires attaining the population noise floor. The discussion should distinguish the scoring objective from what the trained students actually achieve.
Questions for authors
Please clarify whether the manuscript descriptions or the released implementation require correction, and whether any reported values change.
Please state the stimulus and training conditions under which the shared-suffix tail is expected to predict c*, and whether the connectome experiments satisfy them.