- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TTE-Flash, which replaces the explicit CoT in Think-Then-Embed with a fixed number of latent think tokens. Within one MLLM backbone the method produces two kinds of continuous tokens, think and embed; it compares loop-based and register-based extraction and adopts per-layer registers. For training, the think hidden vectors serve as an information bottleneck from which a frozen LLM reconstructs the explicit CoT, embed tokens are supervised with a contrastive loss, and the two use decoupled positional parameters. With Qwen3-VL-2B, MMEB-V2 overall is 64.1, above the explicit-CoT TTEs-2B (63.1), with about 70× the throughput. The paper also analyzes think-token scaling, interventions, CoT form, visualization, and adaptive budgets.
Critical strengths
A clear and practical efficiency–accuracy analysis: Table 1 compares loops and registers within one framework (8 registers: 71 ms, 14.1 sample/s; 8 loops: 179 ms, 5.6 sample/s; PLUME: 298 ms, 3.3 sample/s; TTEs: about 5k ms, 0.2 sample/s), and offers a low-cost per-layer-register improvement, which is directly valuable for retrieval systems with strict latency constraints.
A simple training objective with fairly thorough behavioral analysis: Think tokens act as a bottleneck for CoT generation, supervised by a frozen LLM decoder; masking/shuffling/noising the think states clearly degrades performance, and increasing the number of think tokens helps more steadily on reasoning-heavy zero-shot video tasks (ActivityNet-QA, NExT-QA, VideoMME), suggesting the think states are genuinely used by the embedding.
Critical weaknesses
"Outperforming explicit CoT" comes mainly from the video sub-score; image and document sub-scores still lag, and Table 2 contains a numerical inconsistency. In Table 2, TTE-Flash's Image overall of 68.3 is below TTEs' 70.1 (QA 61.7 vs. 66.6, GD 81.3 vs. 84.1), and VisDoc 68.1 is below 68.8; the overall advantage comes from video. However, TTEs' video All column is 32.1, below all four of its video sub-columns (47.3, 49.1, 34.4, 33.2), which is arithmetically impossible for a weighted average; weighting by the number of MMEB-V2 datasets (36/18/24, 78 in total) reproduces TTE-Flash's 64.1 and UME-R1's 60.1, whereas plugging in the table's 32.1 with the same weights gives about 60.9 for TTEs, and obtaining 63.1 would require a video average of about 41.5. This cell may therefore be wrong, and the size of the video advantage needs to be verified. At the same scale, appendix Table 7 lists Embed-RL-2B at 66.8 and Qwen3-VL-Embedding-2B at 73.4, while TTE-Flash is compared only against "open-data" models, with differences in training data and backbone left unquantified.
Key design conclusions lack numeric and statistical support, and the "bottleneck" interpretation is loose. Loop > register (about 3% relative gap), per-layer registers narrowing the gap, pairwise > sum-of-max ("comparable performance"), and decoupling > sharing are mainly shown as trends in figures (averaged over 20 checkpoints), while the main table and other analyses are single-run results; none has confidence intervals. The effects of the number of think tokens and embed tokens are confounded (with think=8, adding embed tokens saturates). The "information bottleneck" is only an analogy, with no capacity/mutual-information measurement; caption supervision is almost as good as full CoT (70.4 vs. 70.8), suggesting that think tokens encode descriptive semantics rather than reasoning.
Questions for authors
Is the comparison with TTEs/UME-R1/PLUME done under the same training data, backbone and number of epochs? Can you provide an explicit-CoT baseline on the same backbone?
Please check the TTEs video All (32.1) and overall 63.1 in Table 2, and explain where the video improvement comes from (video training data, number of think tokens).
Please turn the key results of Figs. 4–9 into tables with mean ± standard deviation, and provide multi-seed variance for the main table.
For the 70× comparison, does TTEs' 5k ms use inference acceleration (e.g., vLLM)? What is TTE-Flash's latency with long inputs (video, visual documents) and larger batches?
What happens in a control with the generation loss removed and only registers kept (same parameter count), to demonstrate the value of generation supervision?