- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Test-time Token Deletion (TTD), a novel "first-generate-then-delete" framework for test-time reasoning intervention in Multimodal Large Language Models (MLLMs). The authors observe that not all generated intermediate thinking tokens (Chain-of-Thought traces) are equally beneficial when subsequently used as a conditioning context for final-answer decoding. To address this, TTD first allows the model to produce a complete reasoning trace, then selectively prunes a subset of thinking tokens, and finally re-decodes the final answer based on the remaining trace. To implement token pruning without modifying the base model parameters, the authors introduce two instantiations:
- Training-free Entropy-Guided Deletion: Uses truncated top- token entropy during generation as an uncertainty signal to filter out high-entropy thinking tokens (often function words or discourse markers that act as distractors during final-answer decoding).
- Test-time Mask Learning: Optimizes an instance-adaptive mask via forward-only Simultaneous Perturbation Stochastic Approximation (SPSA), guided by a joint objective combining output entropy minimization and a local trace-perturbation robustness term.
Extensive experiments across eight multimodal mathematical and general reasoning benchmarks demonstrate that TTD consistently improves reasoning accuracy (by up to 4.7%) while simultaneously reducing response lengths by roughly 38%. Furthermore, the method exhibits strong transferability across model scales, pure-text reasoning tasks, and tool-augmented multimodal workflows.
Critical strengths
-
A Counter-Intuitive and Insightful Perspective: Traditional test-time scaling methods focus on generating more tokens (e.g., via longer CoTs, tree search, or self-consistency). This paper challenges the dogma by demonstrating that deleting redundant or distracting thinking tokens from a completed trace can improve accuracy while shortening response lengths.
-
Effective Training-Free and Forward-Only Design: By keeping the base MLLM frozen and relying either on a lightweight entropy threshold or a forward-only SPSA mask-learning strategy, TTD avoids the heavy computational or memory burdens typically associated with gradient-based model fine-tuning or full parameter adaptation at test time.
-
Thorough and Broad Empirical Validation: The evaluation is exceptionally comprehensive, spanning multiple model families and parameter sizes (Qwen2.5-VL, ThinkLite, MM-Eureka, LLaMA3.2-Vision), diverse benchmark categories (math, general reasoning, science), pure-text tasks, and tool-augmented workflows.
Critical weaknesses
-
Inference Latency Overhead of Mask Learning: While the training-free entropy-guided variant is efficient (adding only ~0.125s per example), the test-time mask-learning variant requires SPSA optimization iterations—taking around 20 seconds per example. This high latency overhead diminishes its practical appeal for real-time interactive applications despite reducing token length.
-
Sensitivity to Hyperparameters and Thresholds: The performance of the training-free variant relies heavily on the deletion ratio/threshold , while the mask-learning variant involves several sensitive hyperparameters (such as perturbation scale , step size , and robustness weight ). The paper lacks a deeper theoretical analysis or automatic tuning mechanism to ensure stability across out-of-domain inputs.
Questions for authors
-
Error Propagation in Mask Learning: Since the test-time mask learning relies on forward-only SPSA optimization guided by entropy and local perturbation robustness, how does the framework prevent cases where the model is confident in a wrong pseudo-answer, thereby locking in an incorrect deletion mask?
-
Granularity of Deletion: TTD evaluates deletion primarily at the individual token level. Have the authors explored chunk-level or sentence-level token pruning (e.g., removing entire reasoning sub-steps or redundant clauses)? Would coarser granularity better preserve logical continuity while reducing optimization search space?
4 comments
Response to Reviewer 1SL9 (2/3)
W2. Clarification of hyperparameter sensitivity and automatic selection.
- We appreciate the reviewer’s valuable feedback.
- We would first like to humbly highlight that our evaluation covers 16 benchmarks with 23463 evaluation examples, spanning multimodal mathematical reasoning, general multimodal reasoning, tool-augmented multimodal reasoning, and text-only mathematical and scientific reasoning. We also evaluate TTD across eight base models. The experiments provide empirical support for the applicability of TTD across the evaluated tasks, model families, and scales.
- We would also like to humbly highlight that we have examined hyperparameter sensitivity in Figure 5 and Table 5. The results show TTD's relatively stable performance over moderate deletion ratios and within a neighborhood of the selected optimization settings. For baselines, we use the same threshold sweep protocol, as described in Appendix C.5. We also compare deletion-based methods under matched deletion budgets to control for the number of thinking tokens removed.
- Following the reviewer’s suggestion and previous work [1], we conducted additional experiments using 30 randomly sampled, labeled validation examples from each benchmark to select hyperparameters based on validation accuracy. For entropy-guided TTD, we select the deletion ratio from {0.1,0.2,0.3,0.4}. For mask-learning TTD, the candidate values are {0.1,0.3,0.5,0.7} for the perturbation scale, {0.1,0.2,0.3,0.4} for the step size, and {1,2} for the robustness weight. The selected hyperparameters are then fixed for all remaining evaluation examples.
- The results, shown in the table below, indicate that the automatically selected configurations achieve comparable or even better accuracy than our defaults.
Method MathVista MMMU Entropy-Guided TTD 69.50 52.78 Entropy-Guided TTD + automatic selection 69.80 52.55 Mask Learning TTD 71.53 53.41 Mask Learning TTD + automatic selection 71.90 53.67 [1] Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference. In CVPR, 2022
Response to Reviewer 1SL9 (3/3)
Q1. Error propagation in mask learning.
- We appreciate the reviewer’s valuable feedback. Entropy minimization alone can be unreliable at test time, motivating us to introduce the additional robustness term. This term penalizes changes in predictive confidence under perturbations of the retained reasoning trace, discouraging solutions whose confidence depends strongly on a particular arrangement of the generated context. Predictions are also recomputed for each candidate mask, so the optimization does not enforce agreement with a fixed initial pseudo-answer. As shown in Table 1, adding this robustness constraint improves accuracy over entropy-only mask learning and narrows the performance gap to the oracle learner.
Q2. Further analysis of deletion granularity
We appreciate the reviewer’s insightful suggestion and have conducted additional experiments to examine the effect of deletion granularity.
- Specifically, we compare the original token-level mask learning with two sentence-based variants: (1) sentence mask learning, which optimize one mask variable shared by all tokens within each sentence; and (2) token mask learning + sentence voting, which first learn individual token masks and then apply majority voting within each sentence to retain or remove it as a whole. For the latter, a sentence is retained if at least half of its tokens are selected by the learned token masks. We regenerate the final answer after applying the resulting masks to the same reasoning context.
- The results, shown in the below table, indicate that token-level masking achieves the highest accuracy on all four benchmarks, outperforming direct sentence masking and post-hoc sentence voting. Sentence voting retains improvements over the base model, but reduces accuracy relative to the original token masks on every benchmark. The result suggests that useful and redundant or distracting thinking tokens can coexist within the same sentence, and thus assigning a single decision to the entire sentence may retain unnecessary tokens or discard useful conditioning context.
- We agree that direct sentence masking reduces the number of learnable mask variables, from one per token to one per sentence. Post-hoc sentence voting, by comparison, changes only the final deletion granularity and leaves the token-level optimization unchanged. The results suggest a trade-off between reduced mask dimensionality and finer control over context deletion, supporting our choice of token-level deletion.
- We have incorporated these comparisons and discussion into the revised manuscript.
Method MathVision MathVerse MathVista WeMath Average Base Model 25.06 43.64 66.83 63.54 49.77 Mask Learning TTD (token mask) 27.16 46.30 71.53 67.64 53.16 Mask Learning TTD (sentence mask) 25.36 42.72 69.00 64.77 50.46 Mask Learning TTD (token mask + sentence voting) 25.82 45.23 70.90 66.15 52.03
Dear Reviewer 1SL9,
Thank you again for your time and constructive feedback. We would greatly appreciate your feedback, and we would be happy to provide additional clarification.
Thank you again for your time and consideration.
Best regards, The authors
Response to Reviewer 1SL9 (1/3)
We appreciate very much your constructive comments on our paper. Please kindly find our response to your comments below, and all revisions made to the paper are highlighted in blue for your ease of reference. We hope that our response satisfactorily addresses the issues you raised. Please feel free to let us know if you have any additional concerns or questions.
W1. Further Discussion on the Complementary Roles and Practical Applicability of TTD.