- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes STROP, a discrete visual tokenizer that learns image-dependent token sequence lengths. It combines a Transformer-based program generator, vector quantization, a feature interpreter, and a lightweight length-prediction head. The core idea is to train the length head using local rate–distortion signals measured against frozen DINOv3 features, enabling the model to select an appropriate token prefix in a single forward pass without inference-time search or a predefined set of compression rates. STROP presents a technically plausible and empirically promising approach to learning image-dependent discrete visual representations. The rate-matched experiments provide convincing evidence that adaptive token allocation can improve downstream segmentation performance over fixed-length representations under the evaluated setup.
Critical strengths
-
Relevant problem and clear motivation. Allocating representation capacity according to image complexity is a meaningful direction for visual tokenization. The paper appropriately focuses on preserving task-relevant features rather than optimizing pixel reconstruction alone.
-
Controlled evaluation against fixed-length tokenizers. The rate-matched experiments compare STROP with separately trained fixed-length models under a shared teacher, architecture, and training setup. STROP improves segmentation mIoU by approximately 1.6–3.1 points at a similar nominal bit budget across four benchmarks. The use of multiple tokenizer and probe seeds strengthens this evidence.
-
Broad empirical and analytical evaluation. The paper includes component ablations, multiple downstream tasks, token-erasure analyses, and experiments investigating the relationship between program length and scene complexity. The authors also discuss several limitations and provide implementation code.
Critical weaknesses
1. Limited methodological originality. Flexible-length visual tokenization has been explored by FlexTok, One-D-Piece, ALIT, and other adaptive methods, while KARL also predicts image-dependent token lengths in a single forward pass. STROP’s main distinction is its local rate–distortion supervision in frozen teacher-feature space. Still, the lack of a direct comparison with KARL or other controlled alternatives makes the advantage of this design insufficiently clear.
2. Limited comparability of adaptive-tokenizer baselines. Comparisons with FlexTok, One-D-Piece, and ALIT use a shared readout but involve different training data, teachers, and objectives, making it difficult to attribute performance gains specifically to adaptive length selection. Controlled comparisons under matched training and evaluation settings would better isolate the contribution of STROP’s proposed mechanism.
3. Insufficient evidence of computational efficiency. The paper primarily evaluates efficiency through nominal bit rate and token count, which do not necessarily translate into lower end-to-end computation. Since STROP generates a fixed maximum-length candidate sequence before truncation, reporting inference latency or FLOPs would clarify whether it reduces actual computational cost.
4. Limited evidence for semantic interpretability. Token-erasure analyses and correlations between program length and scene complexity suggest that STROP captures meaningful visual structure, but they do not establish that individual codes correspond to interpretable concepts or form compositional visual programs. The reported difficulty of reliably decoding object attributes further limits the claim that these representations provide a symbolic interface for downstream reasoning.
Questions for authors
-
It is necessary to provide a controlled comparison with KARL or otherwise isolate the benefits of local rate–distortion supervision over existing adaptive-length prediction mechanisms.
-
How much of STROP's improvement over public adaptive tokenizers comes from adaptive length selection, as opposed to the DINOv3 feature-distillation objective? Can this be tested under a shared teacher, training dataset, and readout protocol?
-
Does STROP reduce the actual computation required to generate visual tokens, or primarily reduce the number of tokens passed to the interpreter and downstream models? What are the corresponding latency or FLOPs measurements?
-
What evidence supports interpreting the learned codes as compositional visual programs, rather than compressed latent representations with localized effects? How reliably can individual tokens or token combinations be associated with semantic objects and attributes?
3 comments
Thank you for the detailed response. My major concerns have been addressed, and I will maintain my original positive decision.
Thank you for your time and constructive feedback throughout the review and discussion period. We are glad our response addressed your main concerns. The revision will include what we committed to in our reply:
• the distinction from KARL in how the length head is supervised and in needing no quality target at inference;
• the confounders of the comparison with public tokenizers in Table 2;
• the compute and latency measurements of prefix generation;
• a definition of "program", with the limits of the semantic evidence stated plainly.
Since our previous reply, we have also measured how much the length head loses by allocating from per-crop reconstruction error rather than with an oracle at the same token budget; the results are in our latest reply to DXCD and will be added to the paper.
Thank you for the detailed review. We do not claim single-pass length prediction to be new; we differ from KARL in how the length head is supervised and in needing no quality target at inference. On compute, STROP mainly shortens the representation it passes on, and generating the shorter program saves only a modest amount; we also narrow "program" to what the evidence shows. More details follow.
Limited methodological originality
KARL predicts halting in a single pass that is conditioned on a reconstruction-quality target supplied at inference. STROP's length head is trained inside a prefix curriculum from teacher-feature reconstruction and needs no target at inference.
We have not run a KARL control aligned with our configurations. A fair one would involve training KARL's halting on DINOv3 features and calibrating its quality target to our rate, which would change its published recipe. However, we did run the closest controlled alternative, i.e., supervision from reconstruction error alone (see the reply to DXCD, section "Length supervision and the training budget"): the probe factor sets the operating point, and a fixed scale at the converged value is a valid, simpler variant.
Limited comparability of adaptive-tokenizer baselines
We expect most of the gap w.r.t. the public tokenizers originates in the feature-reconstruction objective and the teacher, and our discussion of Table 2 does not attribute it to adaptive length. Our reply to ijYv ("Comparability of public tokenizers") discusses the confounders and evidence. Adaptive length is tested inside STROP under a shared teacher, data and readout in two ways: against separately trained fixed-length models (Table 1), and by randomly reassigning predicted lengths across images while keeping the representation and the exact token total fixed (reply to DXCD, "Image-specific allocation").
Insufficient evidence of computational efficiency
STROP mainly reduces the number of tokens it passes on; the computation needed to generate them falls only modestly. The length head reads pooled encoder features, so the length is known before the program is generated. Because the generator is causal, generating only the first L tokens gives the same tokens as generating all 64 and truncating. The released implementation generates 64 tokens and then truncates; generating only the prefix needs no retraining, only a different inference loop. Measured on an H100 (DINO-S, 256-pixel crops, from GPU-resident input to interpreted features), prefix generation lowers the expected compute from 22.4 to 21.0 GFLOP per crop, or from 9.9 to 8.5 GFLOP (14%) for the STROP part after DINO. Expected latency drops by only 0.3–1.0% (FP32 and BF16, batch sizes 1 and 16, with fixed-length timings weighted by the observed length distribution).
The saving is small for two reasons: DINO's 12.5 GFLOP do not depend on length, and the generator processes the 256 patch tokens together with the program positions, so a 24-token program shortens its input sequence by only about 12%. The larger change is in the output. At about 250 bits per crop, STROP passes on 24 codes on average instead of 64 (640 bits), 2.6 times fewer. This reduces storage and the input length of any downstream model whose cost grows with the number of tokens it reads. A dense readout such as our segmentation probe gains little time, because its 256 output positions dominate its cost. We will add these measurements to the final version.
Limited evidence for semantic interpretability
By "program" we mean a discrete, ordered description executed by a learned interpreter. We will state this definition in Section 1 and avoid wording that suggests symbolic composition we have not shown. What the evidence does show:
At the current stage of STROP advancement, individual codes cannot yet be reliably related to visual semantics. Color and material are hard to read on CLEVR (Table 20), and frozen LLMs did not beat a dataset prior on attributes. The representation is spatially organized and ordered, with parts that can be located and manipulated. Forming codes with interpretable semantic meanings reused across scenes remains a further goal, and we will state this limit more plainly in the revised paper.