- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents a systematic framework that targets real problems in the deployment of vision transformers under resource constraints, where it compresses the model at the macro level through progressive block dropping with entropy initialized gates, and at the micro level by dynamically slimming the FFN width. It then combines these approaches with established early exit techniques for additional system level benefits. Table 1 reports large accuracy gains for the macro level block dropping method against its peers, but unclear comparison budgets and text–table inconsistencies make these gains difficult to assess. Section 3.2 provides a more nuanced picture of the interaction between the two pruning methods that is worth further investigation. Finally, the reported results suggest that combining these methods with existing early exit techniques could yield system level benefits at approximately the same level of predictive performance, but there is ambiguity about how certain claimed benefits are measured, with limited supporting data or elaboration.
Critical strengths
- The research question is real and well motivated, and this paper provides a useful exploration of combinations of different methods that could each bring practical benefits.
- The method is tested on several different datasets and compared against competing methods, providing evaluation beyond a single benchmark.
Critical weaknesses
- Insufficient details are provided about the experimental results. Important information regarding comparison budgets, speedup definitions and timing measurements, memory reduction, training and recovery schedules, the detailed configurations of competing methods, and hardware is missing or unclear. This severely limits the verifiability and reproducibility of the claims.
- Insufficient ablation studies are provided to isolate the contributions of progressive block dropping and two-width training. While the exploratory investigation in Section 3.2 is useful, it doesn’t establish which parts of the training recipe account for the observed gains, and it limits the confidence in selecting reliable deployment configurations without further empirical validation.
- Several accuracy claims in the text are inconsistent with the paper’s own tables. Section 3.1 reports 91.37% on Food-101, while table 1 has 90.78%. The claimed improvements of 4.05% and 2% on Stanford Cars and Flowers-102 also don’t match the corresponding entries of the same table. Section 3.4’s claim of consistently outperforming EnViT in accuracy is again not supported by the Food-101 result in Table 3 (89.03% vs 89.15%). These inconsistencies on top of missing experimental details unfortunately raise concerns about the reliability of the reported results.
Questions for authors
- Could you provide more details on how these metrics are measured, what budgets and baseline settings are used, and how much the results vary across runs? This would help establish whether the claimed advantages are reliable.
- Beyond the exploratory analysis in Section 3.2, could you provide focused ablations of progressive block dropping and two-width training under comparable training and resource budgets? Further investigation of how they interact and when they may fail would help identify reliable configurations for deployment.
- Could you check the discrepancies between the text and Tables 1 and 3 against the experiment records? Please clarify the correct values and settings, and whether the corrections affect the conclusions.