- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes a spatial-temporal transformer for video snapshot compressive imaging (SCI), built on a slimmed STFormer-style backbone with parallel windowed spatial and temporal attention, physics-guided initialisation, and a final data-consistency step. The main contribution is a Mask-Conditioned Ising Refinement Block, integrated into each attention branch. It generates token biases and signed pairwise couplings from pre-softmax logits and conditioning signals, then uses mean-field iterations to gate a residual MLP correction to the attention output. Experiments show modest PSNR gains over baselines across three benchmarks, with approximately 20% fewer parameters and FLOPs, substantially lower latency and peak memory, and qualitative validation on real data.
Critical strengths
- The model achieves competitive reconstruction quality across three benchmarks while substantially reducing latency and peak memory, supporting practical large-scale reconstruction.
- The evaluation is rigorous and includes well-designed and controlled ablations, and the main claims are generally supported by empirical evidence.
- The proposed conditioning mechanism appears to promote generalization, with comparable or better performance on an unseen mask than on the training mask.
Critical weaknesses
- With regards to novelty, MC-IRB is closely related to established techniques, namely sigmoid gating of attention outputs, signed attention, Hopfield-style inference, and unrolled mean-field methods. The Ising formulation does not deliver additional insight. In particular, the paper provides no explicit energy function, no analysis of the three-step mean-field iteration, and no interpretation of the learned couplings.
- Moreover, the reconstruction quality gains are modest, and it seems that the efficiency improvements stem largely from the slimmed backbone rather than from MC-IRB (Table 1). Without a fully trained backbone-only baseline, the incremental contribution of MC-IRB cannot be assessed.
- Regarding prior work, the paper omits relevant ML prior art, as well as major SCI approaches (e.g., deep unfolding, mask-conditioned reconstruction, and efficient video SCI methods).
- Lastly, the exposition is heavy in physics terminology (spins, couplings, magnetisations) that is neither introduced nor properly motivated for the SCI readership. A self-contained introduction would clarify the method.
Questions for authors
- Could the authors please report the fully trained backbone without MC-IRB in Tables 4 - 6, under the same training schedule and evaluation protocol?
- Do the ablation gains persist at full training, given that they decrease between 40 and 80 epochs?
- What does the mean-field iteration provide beyond a sigmoid gate on the attention output (e.g., Gated Attention, NeurIPS 2025), or beyond signed attention (e.g., Differential Transformer, ICLR 2025)?
- Is there an underlying energy function derivation? Do the iterations contract to a meaningful fix point?
- How does SpinSCI perform across mask distributions, compression ratios, and calibration errors?
3 comments
I thank the authors for their detailed response and for clarifying all the issues that I raised in my review. I am therefore raising my recommendation to weak accept.
I would recommend revising the abstract and introduction to state explicitly that the efficiency gains come predominantly from the backbone, and the accuracy gains from MC-IRB.
Thank you for the updated recommendation
Thank you for raising your recommendation. We have addressed your suggestion in our revised manuscript as follows:
- Abstract. We kept the abstract unchanged, since inserting the breakdown there would break the flow between its sentences.
- Introduction. We appended the following sentence to the second contribution: "This efficiency comes mainly from the STSFormer backbone design, while the quality gains come from MC-IRB."
Thank you again, and please let us know if anything else should be added.
full-training backbone, energy and fixed point, and prior work
W1. Novelty and the Ising formulation. Appendix E compares MC-IRB with gating, signed attention, Hopfield-style and mean-field methods. None of them ties the gates of different tokens through signed couplings built from attention logits and the sensing mask, which is what MC-IRB adds. The paper provides all three items listed:
W2 / Q1 / Q2. Gains and the fully trained backbone. Read together with cost, the gains are not modest: SpinSCI-B moves beyond the previous accuracy–cost frontier on all three benchmarks (see the general response).
W3. Prior work. New Appendix E covers unrolled mean-field inference (CRF-as-RNN, DPN), energy-based attention (modern Hopfield, Energy Transformer), signed and gated attention, deep equilibrium models, deep unfolding for SCI, and recurrent, reversible and graph-based video SCI networks. It also defines mask-conditioned reconstruction and adds MST, DAUHST and Yin et al. (2025), which modulate individual features or stages. To our knowledge, SpinSCI is the first video SCI method in which the masks condition the pairwise interactions of a refinement module; removing the mask channels costs 0.30–0.38 dB on an unseen mask (Table 3). We already compare the efficient EfficientSCI-B and MambaSCI-B and will add any method the reviewer names.
W4. Physics terminology. Section 3.2 now defines each term: a spin is a latent binary state of a token, a coupling favors agreement or disagreement between two tokens, a field is a token's own evidence, and the magnetization is the mean spin that sets the token's gate.
Q3. Beyond a sigmoid gate and signed attention. MC-IRB couples the gates of different tokens; neither alternative does.
Q4. Energy function and fixed point. Yes to both. Eq. 2 is the energy; Appendix A.5 derives the update as the mean-field stationarity condition (Proposition A.7). Theorem A.9 shows that if β∥J∥2<1, the iteration has a unique fixed point and converges geometrically; on the released checkpoints it holds for 9 of 12 coupled temporal blocks in every refinement group, and the other three are locally contractive in over 99.999% of groups. The reached state is meaningful: it nearly minimizes the pairwise energy and separates measured from unmeasured tokens (Appendix B.5).
Q5. Mask distributions, compression ratios and calibration.