- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this work the author analyse an in-silico digital twin of mouse visual cortex (developed by MICrONS Consortium; Wang et al. 2025) by investigating the relationship between neural selectivity (i.e. how big is the pool of stimuli for which a given unit shows substantial responses) and tolerance (i.e. how stable is the unit's response as a given transformation is applied to the provided visual stimulus). The authors leveraged a diffusion-based model to general most exciting dynamic input (MEDI) by optimising the latent codes of an auto-encoder-based generator via gradient descent (the entire setup is fully differentiable) and showed how the resulting optimisation successfully elicited stronger responses then standard alternatives (dynamic Gabors, natural movies, MESI). The generated pool of MEDI stimuli was then use to characterise both sensitivity and tolerance of individual model units: MEDIs were most effective within their target area, selectivity showed the ranking V1 > LM/AL > RL, while tolerance showed various trend depending on the chosen transformation. Finally, selectivity and tolerance were generally anti-correlated and selectivity was stronger in layer L4/L5 neurons than L2/3 neurons.
Critical strengths
The paper presents a comprehensive evaluation of the considered in-silico model, and while the MEDI methodology is not new, its application to the study of the interplay of sensitivity and video-transformations tolerance is interesting. The authors provide good evidence for the success of the optimisation problem and in general offer well-controlled statistical evidence for their results. The paper is in general well written, with clean Figures and a detailed Appendix, furthermore the reproducibility efforts and the nice visualisation website are much appreciated.
Critical weaknesses
While the authors well characterised the model response to MEDI stimuli, it remains unclear how much the derived sensitivity and tolerance trends depend on the chosen methodology. The author compared the methodology to other baselines only to assess MEDI optimisation success, however elevated model responses are by themselves somewhat expected. What would have been perhaps more interesting to assess was how much the derived sensitivity and tolerance ranking depended on the derived stimulus pool. In general, while many results are presented, the author offer little in terms of interpretations, especially for the reported results inconsistent with the known literature, i.e. negative association between selectivity and tolerance. For the latter, the authors point precisely to a difference in the stimulus set, but never test it.
Finally, the proposed technique leveraged a fully differentiable setup to synthesise the MEDI stimulus set. It remains unclear how applicable this technique is to an experimental settings with limits on neuron sampling, optimisation time/number of iterations, noise etc... especially considering the requirement of full-video optimisation. Recent works relied on fast overnight tuning of digital twin (Ding et al. 2026) or turned to gradient-free approaches (Ponce et al. 2023; Tausani et al. ICLR 2026). A comparison of the optimisation convergence time between MEDI and MESI for example would strengthen the claims of applicability to lab experiments.
Questions for authors
The following are questions related to the identified weakness which, if addressed, would further strengthen the paper:
- Are the core sensitivity and tolerance results dependent on the derived stimulus set?
- Related to (1), In Fig. 9e we see that natural responses, while still less eliciting than MEDI, are the most competing alternatives and arguably escape the stated limitation of optimisation exploiting model-specific errors. How do the results (e.g. tolerance to speed-up, slow-down etc...) look like for naturalistic stimuli?
- How do the full-video MEDI optimisations relate to single-frame MESI optimisation (or dynamic Gabors) in terms of number of iterations and computational complexity?
- Do the authors estimate the proposed pipeline to be successfully deployed for in-vivo experiments? Would a fine-tuning of the foundational baseline be required?
- Some of the explored tolerance transformations (contrast, rotation, translation) are inherently static and can thus be studied in MESI or other single-frame baselines. Does the time-dimension show interesting effects of is the same tolerance profile retrieved in the other conditions?
7 comments
Response to Reviewer e5wa #2
Question 1-2 — Stimulus-set dependence and natural-reference transformations
The core sensitivity and tolerance results are partly dependent on the derived stimulus set. See Critical Weakness 1 and Appendix F.4, Figure 25: under natural-reference stimulus set, several profiles persist, but LM preference and some temporal associations change.
Question 3 — Iterations and computation
MEDI generation uses 20 updates per overlapping window; MESI and dynamic-Gabor generation use 10 and 30 iterations, respectively. Their parameters and the work per update differ, so measured wall time and peak memory provide the practical comparison.
Appendix F.8 reports time and memory at these settings. Pilot experiments would establish an appropriate iteration schedule for a new animal.
Question 4 — In-vivo deployment
Deployment requires fine-tuning the animal-specific components of the pretrained model on recorded responses. The pretrained core supplies visual features while the trained components adapt predictions to the recorded animal. The cited overnight digital-twin work provides a precedent for model adaptation, while gradient-free methods offer a complementary approach to stimulus search.
At the measured generation rate, 500-1,000 MEDIs would require approximately 8-16 GPU-hours. With ideal parallel scaling on four comparable GPUs, synthesis alone would take about 2-4 hours; model training, data transfer, and recording add to that time.
Our planned experiment would record natural-movie responses, train the animal-specific model and generate stimuli overnight, then test optimized and conventional stimuli on subsequent days.
Pilot recordings will establish prediction quality, optimization settings, repetition counts, and total time consumption. The Discussion describes this planned validation, and Appendix F.8 provides the measured synthesis costs.
Question 5 — Static spatial tolerance
Yes. We tested paired MESIs and MEDIs in the random 500-per-area cohort. MESIs were repeated images; each pair shared a rotation center. Retention used each family's own untransformed response as denominator.
MESIs also showed the lowest translation retention in V1 and the highest rotation retention in AL. At 48 pixels, medians were 0.039, 0.091, 0.131, and 0.159 in V1, LM, RL, and AL; at , they were 0.097, 0.148, 0.149, and 0.189 (Appendix F.7, Figure 28).
The close spatial similarity between MESIs and MEDI mean frames offers a plausible explanation for this agreement. The additional value of the video is its temporal sequence: in the separate 100-readout ablation, repeating the strongest individual frame retained a median 35.6% of the MEDI response and shuffling retained 49.6%. MEDIs therefore share spatial structure with MESIs while adding response-driving temporal structure and enabling reversal and playback-speed interventions (Appendix F.6, Figure 27). We now make this division of roles explicit in the Discussion.
I thank the authors for the rebuttal and the new analyses responding to my questions, in particular the natural-movie reference analysis (now Appendix F.4), the comparison with MESIs on tolerance, and the report of the computational costs relevant for in-vivo deployment. I believe these additions have improved the paper; however, as I understand them, they have also reinforced my original concerns. In particular:
- The new experiments show that the selectivity–tolerance association depends strongly on the chosen stimulus set: with preferred natural movies, the within-area correlations roughly halve for contrast and translation and largely vanish for reversal and speed-up. Moreover, retention above one under speed-up suggests that retention partly reflects how far the reference stimulus is from the readout’s optimum. Since MEDIs are by construction local optima of the model response, a readout with a narrowly peaked response would appear both selective and intolerant. This makes it hard to tell whether the negative association reflects real properties of the model’s visual areas (and might generalise to visual-cortex recordings), or is simply a consequence of measuring tolerance around a response maximum. A dose-response check (e.g., using partially optimised MEDIs) could help separate the two.
- The MEDI–MESI comparison shows that MESIs reproduce the spatial tolerance profiles (in Fig. 28 the dashed and solid curves closely track each other). For the tolerance analysis, the distinctive contribution of MEDIs therefore lies in the temporal manipulations. While the area-level temporal profiles are robust, the temporal selectivity–tolerance associations (reversal, speed-up) are precisely the ones that weaken across response models or depend on the stimulus set.
For these reasons I keep my original score: the new analyses improved the work, but they leave these original concerns open.
Response to Reviewer e5wa #1
Thank you for asking how the findings depend on the stimulus set and how the method could be used experimentally. We added natural-movie and static-reference comparisons, expanded the interpretation, and measured generation time and memory.
Critical Weakness 1 — Stimulus-set dependence
The relationship partly depends on the reference stimuli. We repeated the measurements with preferred natural movies and tested response offsets separately, directly addressing both dependencies you identified.
For a random 2,000-readout cohort (500 per area), we used each target's 60-frame preferred natural video as described in the paper. We evaluated all 2,000 references against all targets. A reference's source area is the area of the readout that selected it. We used the same definitions of selectivity and tolerance and the same transformation settings as for MEDIs.
An interesting finding is that, speed-up increased the median response in every area, giving ratios of 1.049-1.147. The originals were selected among untransformed natural clips, so a sped-up version could elicit a stronger response. V1 had the smallest increase.
Different areas may select similar or identical natural clips, making the source groups less distinct and offering one possible explanation for weaker area specificity. Several temporal associations also changed when natural references replaced MEDIs (Appendix F.4, Figure 25).
Therefore, these differences show that within-area preference and the selectivity-tolerance relationship depend on the stimulus set and reference. The MEDI framework complements predefined stimulus families by identifying strongly driving dynamic references for individual readouts, and applying the same selectivity and transformation protocol across areas.
Critical Weakness 2 — Generation cost and deployment
MEDI generation took about one minute per target on the measured GPU. We generated all four stimulus types for ten readouts on an NVIDIA RTX 4090 at the settings in Appendices B.1-B.2. Entries are medians (minimum-maximum):
Timing excludes loading and saving; peak tensor allocation includes resident models and the decoder when used (Appendix F.8).
We optimize an autoencoder's latent representation by gradient descent. MEDI generation uses eight overlapping 15-frame windows, with 8-frame overlap and 20 updates per window, before stitching and trimming to 60 frames. MESIs use 10 iterations; both Gabors use 30. The table reports the costs of these procedures at their stated settings.