acceptodds
Under review as a conference paper at ICLR 2027

Risk-Controlled Post-Compression Verification for Visual Token Reduction with Exact Dense Fallback

Abstract

When a system trusts a compressed visual prediction on a particular input, how often does that choice alter the prediction of a frozen dense teacher? Aggregate accuracy and analytical compute do not answer this deployment-contract question. We study post-compression verification: a frozen vision transformer runs a token-merged continuation, accepts its output only when a calibrated confidence rule passes, and otherwise restores an exact dense continuation from an uncompressed activation saved before the first merge. Six merge ratios, one hundred thresholds, and four targets are calibrated jointly as one finite family of 2,400 candidates with per-candidate error probability 2.08 × 10^-5. Across four dataset–backbone combinations and three seeds, the 1% target accepts 79.5%–95.4% of examples; the largest per-seed calibration upper bound is 0.991%, while observed conditional dense-teacher disagreement is 0.42%–0.55%. A separately implemented program that does not import the production selector recomputes every reported policy and result. The selected policies use 0.74–0.82 of the analytical transformer compute. On the measured hardware platform, wall-clock gains appear only at batch one and reverse at larger batches, underscoring that analytical compute reduction is not a hardware-speedup guarantee. The study therefore reports four non-interchangeable layers of evidence: certified conditional disagreement, analytical transformer cost, compute saved by exact prefix reuse, and measured end-to-end latency.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
visual token reduction, risk control, selective prediction, vision transformers, adaptive inference, dense fallback

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_gmpgOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a risk-controlled inference framework for Vision Transformer (ViT) visual token compression, aiming to reduce computational overhead while limiting the rate of prediction inconsistency between the compressed model and the original dense model.

Critical strengths
  • A practical problem to improve efficency.
Critical weaknesses
  • This is an AI-generated paper. The bibliography contains several instances of author names that are incorrect when checked against the original publication records. For instance, the citation for SpecVLM (EMNLP 2025) and Fast yet Safe (NeurIPS 2024).

  • The paper primarily combines existing techniques. Its technical contribution remains modest when compared to existing methods such as Risk-Controlled Early-Exit or Selective Inference.

  • Although the paper reports a theoretical reduction in Transformer computation to approximately 74%–82% of the original model, empirical hardware evaluations show a speedup of only about 1.093× when the batch size is 1. At batch sizes of 16 and 64, the performance actually drops to 0.738× and 0.762× of the original dense model, respectively.

  • Internal inconsistencies in the experimental reporting raise questions regarding the validity of the "independent audit."The authors should clearly provide materials sufficient to verify the key experimental results.

Questions for authors

Can the authors explain the errors in author names for several cited papers and the inconsistencies in statistical results for the same experiments between Table 2 and Table 7?

Reviewer_ddFROpenReview ↗
Rating
1(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets token merging compression for Vision Transformers and proposes a post-compression verification mechanism. After inference on the compressed branch, the model decides whether to accept the compressed prediction based on confidence. If the risk exceeds the threshold, it falls back losslessly to dense-model inference using pre-saved shared prefix activations, ensuring outputs are numerically identical to those from a full dense forward pass. Combining a calibration set with the union-bound statistical framework, the paper provides a provable upper bound on conditional risk. It also highlights that lower theoretical FLOPs do not guarantee hardware speedup, distinguishes four mutually independent evaluation metrics, and adopts independent auditing and blind-test protocols to ensure experimental credibility. Its limitations include experiments restricted to small-scale image classification tasks, a simple scoring function, and statistical guarantees built on the exchangeability assumption of samples.

Critical strengths
  1. Instead of reducing the compression objective simply to average accuracy or FLOPs, this work explicitly controls the conditional risk that accepted compressed outputs disagree with the dense teacher. It stores uncompressed attention outputs prior to the first merge, so rejected samples can resume dense inference from this shared prefix without recomputing the entire network. The paper also verifies that fallback logits are equivalent to those from an independent dense forward pass.
  2. Evaluations are conducted across four combinations of CIFAR-100 / Tiny-ImageNet and DeiT-Tiny / DeiT-Small with three random seeds. The authors separate calibration and blind test sets, and recompute reported results via an independent pipeline that does not rely on the original selector. Under the 1% risk target, the acceptance coverage of compressed outputs ranges from 79.5% to 95.4%. The calibrated upper bound for every setting stays below 0.991%, the observed conditional disagreement rate is 0.42%–0.55%, and the theoretical Transformer compute is 0.74–0.82 times that of the dense baseline. The paper clearly differentiates theoretical compute from measured latency, reporting speed gains under batch size 1 and slower runtime than the dense model at batch 16 / 64. It avoids marketing reduced FLOPs as universal hardware acceleration.
Critical weaknesses
  1. Experiments are limited to CIFAR-100, Tiny-ImageNet and two small DeiT backbones. The results cannot directly support generalization claims on ImageNet, large-scale Vision Transformers or real-production visual models.
  2. The authors adopt the existing ToMe algorithm; their main contributions lie in risk calibration, verification and the fallback protocol. Accordingly, the innovation of this paper leans toward deployment methodology and evaluation framework rather than a novel token merging algorithm. All selected strategies hit the predefined upper merge ratio of 0.30, indicating that “dynamic compression budget selection” within the candidate family is effectively not activated. The search space of 2400 candidates also introduces an extra multiplicity cost. Although the authors report this observation, it weakens the contribution regarding adaptive budget selection.
Questions for authors
  1. Validate the risk guarantees and coverage on larger-scale datasets.
  2. Re-run calibration with several pre-fixed merge ratios, or add experiments that genuinely select compression budgets across different inputs, to disentangle the contributions of threshold selection and budget selection.
  3. Wrap fixed-compression methods such as DynamicViT and EViT with the same teacher-verification/fallback pipeline, then compare coverage, conditional disagreement rate and expected compute.
  4. Report the tightness of guarantees and policy stability under varying calibration sample sizes, target risk levels, and candidate family sizes.
Reviewer_htWpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

After compressing away a portion of the tokens, the overall test accuracy may remain virtually unchanged, yet the predicted classes for certain individual images can still shift. This paper frames the problem as follows: when the system chooses to rely on the compressed output, can the probability of it yielding a different prediction than the frozen, full-scale model be effectively controlled? To address the limitations of the compressed branch, the authors propose a strategy that first executes the compressed branch, then decides whether or not to proceed with the dense branch.

Critical strengths

None

Critical weaknesses

This article was most likely generated by AI; the number of decimal places used in the table is inconsistent, and there are contradictions in the data. It is highly problematic and requires a thorough manual revision.

Questions for authors

It requires a thorough manual revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.