- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes S2E, a dense feed-forward network (FFN) architecture that aims to achieve functional differentiation through prototype-guided subspace modulation. The method partitions hidden representations into multiple subspaces and dynamically modulates their features using similarity-based weighting. Experiments on ImageNet classification, distribution-shift benchmarks, and COCO detection and segmentation report improvements in predictive accuracy and computational efficiency.
Critical strengths
- Promising accuracy–efficiency trade-off. The paper reports improved classification accuracy across multiple vision backbones while reducing FLOPs and, in several configurations, parameter counts.
- Broad experimental coverage. The evaluation covers multiple architectures, distribution-shift benchmarks, object detection, and instance segmentation. The paper also includes ablation studies and subspace intervention analyses.
Critical weaknesses
- Unverifiable references and concerns about experimental reliability. The arXiv identifier (2206.02351) provided for G-FFN: Grouped Feed-Forward Network for Efficient Vision Transformers actually corresponds to The Synergy of Electromagnetic Effects and Thermophysical Properties of Metals in the Formation of Laser Induced Periodic Surface Structures, a paper in optics. Similarly, the identifier (2303.07975) provided for Dynamic Gated Feed-Forward Networks for Vision Transformers corresponds to Software-based Security Approach for Networked Embedded Devices. Neither cited work could be verified using the provided titles and author information. Notably, G-FFN is included as an important baseline in Table 2, with specific experimental results. These issues raise serious concerns about the reliability and reproducibility of the corresponding baseline results. The authors should provide verifiable references, baseline implementations, and original experimental records.
- Substantial inconsistencies between the supplementary code, the described method, and the reported results. The paper claims to modify only the Transformer FFN, whereas the supplementary Swin-T/S configurations set the attention window size to 56, compared with the standard Swin window size of 7. The resulting attention computation is also inconsistent with the reported 3.9G FLOPs for the entire Swin-T+S2E model. Furthermore, the implementation introduces spatial convolution and normalization components that are not adequately described in the main method, and contains missing imports, configuration entries, and training configuration files. Consequently, the provided supplementary materials do not allow reliable reproduction of the reported results. The authors should clarify these discrepancies and provide executable code and original experimental records that correspond to the reported experiments
- Insufficient evidence for the claimed novelty and functional differentiation mechanism. S2E essentially combines feature grouping, learnable prototypes, and input-dependent dynamic feature modulation. Although the representation-diversity, category-selectivity, and subspace-ablation analyses demonstrate differentiated responses, these observations do not establish a fundamentally new functional differentiation mechanism distinct from conventional FFNs or dynamic gating architectures. The paper also lacks sufficient comparisons with simple dynamic-gated FFN baselines under matched parameter and computational budgets, and does not establish a clear causal relationship between the claimed functional differentiation and the observed performance improvements.
Questions for authors
- Could the authors provide verifiable references for G-FFN and Dynamic Gated FFN and explain why the cited arXiv identifiers correspond to unrelated papers? Could the authors also provide the G-FFN implementation and original experimental records to substantiate the results reported in Table 2?
- Could the authors explain the discrepancies between the attention window configurations, additional architectural components in the supplementary code, and the method described in the paper? Could they provide executable code and evaluation records consistent with the reported FLOPs, accuracy, and latency?
- Compared with conventional dynamic-gated FFNs under matched parameter and computational budgets, does S2E demonstrate a distinct functional differentiation mechanism? What evidence establishes that the claimed differentiation is responsible for the observed performance improvements?