- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Image2MUA models the conditional distribution of simultaneously recorded macaque V1/V4/IT multi-unit activity from visual stimuli. It uses rectified flow matching, population self-attention, and four frozen visual feature sources combined by a channel-specific router with a shared image-dependent adjustment. Training uses one response per image; repeated responses are reserved for channel-reliability screening and evaluation. On a class-disjoint split of TVSD from two macaques, the model improves average-response correlation over the evaluated deterministic baselines and captures marginal trial variability and pooled residual correlations. The paper also compares a matched-architecture diffusion formulation and tests whether area-associated routing structure persists when explicit area information is removed.
Critical strengths
- The distributional evaluation separates important aspects of success. Table 2 reports both CRPS and pooled noise-correlation similarity over five seeds. Collapsing samples to their own conditional mean worsens CRPS from 0.434 to 0.605 for Monkey F and from 0.400 to 0.553 for Monkey N. Channel-wise trial shuffling preserves the marginals while eliminating correlation similarity. These controls show that the model contributes more than a good mean or independently variable channels. The class-disjoint validation/test construction and matched-architecture DDPM comparison further strengthen the evidence. Appendix B also correctly treats the recorded-trial leave-one-out reference as neither an image-only baseline nor a formal bound.
- A useful extension to joint multi-area neural encoding, with strong mean predictions. One model generates a simultaneous response vector across V1, V4, and IT while retaining per-channel visual feature mixtures. On the class-disjoint split, overall mean-response correlations are approximately 0.87 and 0.89, compared with 0.84 and 0.75 for the evaluated TBE baseline (Table 1). The region-blind control removes both area inputs and area-balanced loss before recovering area-associated preferences, making the observed organization more informative than an analysis of a model directly supplied with area labels alone.
Critical weaknesses
- The incremental value over established or simpler probabilistic population models is insufficiently characterized. The principal generative comparator is a DDPM using the same architecture. Mean collapse and channel shuffling are useful diagnostic controls, but neither is a competitive alternative distribution model. The paper itself cites the flow/factor-analysis model of Bashiri et al. (2021), which already learns image-conditioned population distributions without repeated training presentations, but does not compare against that family. A strong shared-feature mean predictor with a regularized correlated-residual model would also be informative: pooled residual-correlation similarity can be high even when covariance does not depend on the image. Thus, the current results establish useful marginal distributions and pooled dependence, but leave uncertainty about how much benefit requires the proposed conditional generator. This limits the significance of the methodological advance, rather than invalidating the reported scores.
- The router is interpretable as a model mechanism, but its predictive necessity and stable area organization are only partially demonstrated. In Table 6, Global Static is better than the full router on Monkey F in both MCP (0.8746 versus 0.8708) and CRPS (0.4311 versus 0.4341); the full router improves both on Monkey N. Image-term interventions have very small effects, and channel-routing interventions barely change noise-correlation similarity (Table 3). Furthermore, the fully region-blind analysis uses one checkpoint per animal, whereas the five-seed no-area-embedding control retains the area-balanced loss and is a different experiment. Main/region-blind CLR correlation also retains global expert preferences, as the authors acknowledge. These results support an interesting association, but not yet a robust claim that the factorized router is essential to either distributional performance or reproducible cortical organization.
Questions for authors
- Assessment-changing comparison: How does Image2MUA compare with a correlated probabilistic readout using the same visual sources, or an adapted flow/factor-analysis population model, under the same single-presentation split? A comparison of MCP, CRPS, and pooled correlation error would clarify the generator's incremental value. Please also distinguish image-dependent covariance from pooled covariance if drawing conclusions about conditional population dependence.
- Assessment-changing control: Do the source-centered area preferences and area-associated variance remain consistent across independently trained fully region-blind models? Can a channel-only static router recover essentially the same results as the full channel-plus-image router? These controls would clarify which part of the routing design is consequential and which interpretation is reproducible.
- Clarification / evaluation uncertainty: Since the repeated test responses also determine the retained-channel set, how sensitive are the conclusions to the reliability threshold or to estimating reliability from a separate subset of repetitions? Please distinguish seed variability from uncertainty across held-out images. This would help quantify how strongly the reported gains depend on the evaluated sample rather than on optimization alone.
3 comments
(Continued)
-
Region-blind reproducibility. We expanded to three independent seeds per monkey (42–44), excluding area labels from inputs and loss. Figure 5 (p. 9) reports means and SD; Appendix Figure 7 (p. 25) shows every seed. It shows a stable preferences as below:
Across both monkeys and all three seeds, V1 favors E0, V4 favors E1/E2, and IT strongly downweights E0. Source-centered profiles correlate across seeds at r = .879–.898 (F) and .802–.871 (N). Area-associated routing variance spans 15.38%–24.92% in F and 16.50%–29.52% in N. These are fully region-blind models, distinct from the no-area-embedding control.
-
Image-sampling uncertainty and channel selection. We evaluated image-sampling variability separately from training-seed SD. For each monkey and main-model seed 42–45, we drew 50 image IDs with replacement from the 100 test images, in five independent batches of 1,000 resamples. Metrics were recomputed on matched draws and averaged over the four fixed seeds.
| Metric | F: all 100 | F: resampled 95% interval | N: all 100 | N: resampled 95% interval |
|---|---|---|---|---|
| MCP | .8720 | [.8568, .8838] | .8915 | [.8719, .9049] |
| CRPS | .4323 | [.4269, .4375] | .3998 | [.3934, .4062] |
| NC | .8111 | [.6471, .7171] | .7719 | [.6217, .6975] |
| CorrMAE | .0515 | [.0553, .0643] | .0815 | [.0754, .0983] |
| AverageCov MSE | .6127 | [.4835, .8328] | .9638 | [.6425, 1.4564] |
| ImageCov MSE | 1.1251 | [.8638, 1.4309] | .6386 | [.3250, 1.0296] |
Implanted channels may not all carry task-relevant visual responses. Higher thresholds favor reproducibility but may exclude useful, unstable signals; lower thresholds may admit task-irrelevant channels. Scores can vary with channel selection, but prediction accuracy alone cannot establish biological relevance. We therefore retained TVSD's mean pairwise reliability threshold >0.3, rather than exploring cutoffs based on model performance. Reliability averages over repetition pairs across 100 images with 30 presentations each. Repeats serve screening and evaluation; fitting and checkpoint selection use single presentations (Section 3.1; Appendix B, p. 14). The image-resampling analysis fixes this channel set; screening on separate repetitions was not tested.
Thank you again for your valuable feedback. We welcome further discussion.
Thank you for the response. I'll keep my score.
Thank you for your thoughtful review, which strengthened our comparisons and reproducibility analyses. References are to the revised manuscript.
We agree that Gaussian factor is a competitive generative framework, performing close to flow matching with strengths on different metrics. This does not undermine our central contribution: the routed cross-attention expert fusion design provides strong neural encoding and interpretable cortical associations; the competitive Gaussian factor baseline itself uses this same backbone with Gaussian output heads and shared latent loadings, demonstrating its encoding capability beyond flow matching.
Average and stimulus dependent covariance. Following your suggestion, we added AverageCov MSE for average covariance error and ImageCov MSE for error in each image's deviation from that average (Section 4, p. 6). Both use 20 fixed 15/15 trial splits and cross-products of covariance errors to account for sampling noise.
Retrained channel-only router. Channel (C) was retrained without image-routing logits (Table 2B, p. 8), achieving MCP/CRPS of .875/.429 in F and .889/.401 in N. C+I has lower mean ImageCov MSE in both animals: 1.1229 versus 1.1418 in F, and .6334 versus .6386 in N. Image-logit shuffling also produces small changes (Table 3, p. 10). These controls identify stable channel-specific feature selection as the more consequential routing component, with image reweighting contributing to stimulus specific covariance. All settings retain image features as expert inputs.