acceptodds
Under review as a conference paper at ICLR 2027

D-Tok: Single-Pass Dynamic-Length 1D Video Diffusion Tokenizer

Abstract

Fixed-length visual tokenization with 2D/3D inductive bias is suboptimal in learning rich semantics and facilitating visual autoregressive modeling. 1D visual tokenizers have mitigated these issues; however, test-time search for optimal token length locks its dynamics and efficiency. We propose D-Tok, a single-pass ynamic-length 1 video iffusion tokenizer. The core idea is to map visual content to a set of latent tokens and then discard redundant tokens using a single-pass end-of-sequence detector. The remaining tokens are decoded via the diffusion decoder. Experimental results show that D-Tok surpasses all baselines, setting a new state-of-the-art rFVD/gFVD among video tokenizers with the highest token compression rate. It also achieves comparable performance to its fixed-length counterparts while enabling semantic representations. Downstream tasks on video generation demonstrate that our method achieves better results on autoregressive modeling compared to 2D or 3D methods. Code will be released.

Primary area:
generative models
Keywords:
Diffusion, 1D video tokenizer, Dynamic-length tokenizer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SF1TOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

D³-Tok predicts a content-dependent prefix length for video tokenization using complexity-based supervision and reconstruction feedback, followed by diffusion decoding.

Critical strengths
  1. Addresses a practical problem: adaptive token allocation without reconstruction-based search.

  2. Reports promising reconstruction and generation quality with approximately 31% fewer tokens than its fixed-length counterpart.

  3. Provides component ablations and additional reconstruction metrics.

Critical weaknesses
  1. Insufficient positioning against prior work. Relevant missing references include CAT[1], One-D-Piece[2], VaporTok[3], and One-DVA[4]. The contribution beyond prior complexity-aware allocation and diffusion decoding needs clarification. The characterization of KARL as requiring inference-time search is inaccurate, and comparison with EVATok is missing.

  2. The benefit of content-adaptive allocation is not isolated. Comparing 1024 fixed tokens with 706 adaptive tokens cannot distinguish adaptive allocation from sequence shortening. Matched-budget fixed-length and input-independent variable-length baselines are needed.

  3. The efficiency comparison is unclear. ElasticTok uses a custom sequential search with an rFVD-based stopping criterion. Since rFVD is a distribution-level metric, its application to individual videos requires explanation. Comparisons should match reconstruction quality.

  4. The frame-prediction protocol is underspecified. Given the bidirectional video encoder, the construction of conditioning tokens using only observed frames needs clarification. Essential AR training and FVD evaluation settings are also missing.

[1] CAT: Content-Adaptive Image Tokenization

[2] One-D-Piece: Image Tokenizer Meets Quality-Controllable Compression

[3] VaporTok: RL-Driven Adaptive Video Tokenizer with Prior & Task Awareness

[4] Adaptive 1D Video Diffusion Autoencoder

Questions for authors
  1. What specifically distinguishes this method from existing adaptive tokenizers, and which differences account for its improvements?

  2. Do the gains persist against fixed-length and randomly allocated baselines at the same average token budget and training budget?

  3. How is the rFVD stopping criterion implemented, and does the speedup persist against alternative search strategies at comparable quality?

  4. How are K600 conditioning tokens constructed without access to future frames, and what are the evaluation settings?

I would consider raising my score if the authors adequately address the concerns above.

Reviewer_t8bQOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces D$^3$-Tok, a dynamic-length 1D video diffusion tokenizer that determines the number of tokens needed for each video in a single pass. Unlike previous approaches that require test-time search, D$^3$-Tok uses an EOS detector to predict where to truncate the token sequence based on video complexity. The detector is trained using H.264-based complexity estimates and reconstruction feedback, while a diffusion decoder reconstructs the video from the retained tokens. Experiments on UCF-101 and Kinetics-600 demonstrate improvements in reconstruction and autoregressive generation quality with fewer tokens, alongside faster inference than search-based methods.

Critical strengths
  • Simple and effective formulation of dynamic-length video tokenization. Predicting a global EOS position is an intuitive way to allocate tokens based on video complexity without test-time search. The method is well motivated and fits naturally into 1D autoregressive modeling.
  • Promising experimental results with clear presentation. D$^3$-Tok achieves strong reconstruction and generation results while using fewer tokens than competing methods. The qualitative examples show better preservation of local details, and the ablations support the benefit of reconstruction-aware EOS training. The paper is also well written and easy to follow.
Critical weaknesses
  • Insufficiently controlled comparisons for autoregressive (AR) generation. Although the paper reports strong gFVD results, it is unclear whether the baselines use the same AR architecture, training budget, and generation settings. This makes it difficult to attribute the improvements specifically to dynamic-length tokenization. A controlled comparison between fixed-length, random-drop, pseudo-label-only, and full D$^3$-Tok under the same AR setup would better support the claims.
  • Incomplete discussion of related work. The claim of being the first single-pass dynamic-length 1D video diffusion tokenizer would benefit from clearer comparisons with recent work [1-4]. These comparisons are important for establishing the novelty of the proposed method.

[1] Teng, Yao, Minxuan Lin, Xian Liu, Shuai Wang, Xiao Yang, and Xihui Liu. "Adaptive 1D Video Diffusion Autoencoder." arXiv preprint arXiv:2602.04220 (2026).

[2] Ge, Yuying, Yizhuo Li, Yixiao Ge, and Ying Shan. "Divot: Diffusion Powers Video Tokenizer for Comprehension and Generation." In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13606-13617. IEEE, 2025.

[3] Jang, Huiwon, Sihyun Yu, Jinwoo Shin, Pieter Abbeel, and Younggyo Seo. "Efficient Long Video Tokenization via Coordinate-based Patch Reconstruction." In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22853-22863. IEEE, 2025.

[4] Jia, Mingkai, Mingxiao Li, Zhijian Shu, Anlin Zheng, Liaoyuan Fan, Jiaxin Guo, Tianxing Shi et al. "DINO-Tok: Adapting DINO for Visual Tokenizers." arXiv preprint arXiv:2511.20565 (2025).

Questions for authors
  • Were the baseline gFVD results obtained using the same AR architecture, training budget, and generation settings? Could the authors provide a controlled comparison between fixed-length, random-drop, pseudo-label-only, and full D$^3$-Tok?
  • Could the authors clarify how D$^3$-Tok differs from the closely related methods listed above, particularly in terms of novelty and dynamic token allocation?
Reviewer_sENEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes D3-Tok, a discrete 1D video tokenizer that keeps a content-dependent number of tokens per video in a single forward pass, avoiding the test-time length search of ElasticTok and AdapTok. A fixed-length 1D tokenizer (ViT encoder, VQ, flow-matching decoder) is trained first. A lightweight " detector" is then trained jointly to predict where to truncate the token sequence. Since no length labels exist, the detector uses a noisy pseudo-label from the H.264 compression ratio (smoothed into a soft distribution, with cross-entropy plus reverse cross-entropy) together with the reconstruction loss, propagated through the hard truncation by a straight-through estimator. On UCF-101 and Kinetics-600, the paper reports rFVD/gFVD competitive with or better than 1D and 3D tokenizers at roughly 700-930 average tokens, about 4x lower latency than ElasticTok, and better linear-probing accuracy than fixed-length and random-drop variants.

Critical strengths

A simple, practical single-pass mechanism. Replacing search with a 33M-parameter detector trained from a cheap pseudo-label and reconstruction feedback is lightweight and fits a standard AR pipeline through a natural token. The prior is only a starting point, which is a sensible design.

Strong reported results with an informative detector ablation. At 128x128, D3-Tok-L reaches rFVD 18 / gFVD 52 on UCF-101 with 706 average tokens, versus LARP-L-Long at 20 / 102 with 1024 tokens, and Tab. 3 shows a clear progression from hard labels to the full objective (gFVD 105 to 52).

Critical weaknesses

The benefit of adaptive length is not isolated. In Tab. 3, a fixed-length D3-Tok with 1024 tokens already reaches rFVD 17 (vs 18 for the dynamic model), so the rFVD advantage over LARP likely comes largely from the diffusion decoder. The large gFVD gain (132 to 52) compares a 1024-token model with a ~706-token model that also has an AR-side and masked padding, so it may come from shorter sequences rather than content-adaptive allocation. A matched-budget fixed-length baseline and a "prior-only" baseline (length set directly from the H.264 ratio, also single-pass) are missing, and the learned lengths are never analyzed. At 256x256 the average is 933 of 1024 tokens, only about a 9% saving.

The comparison with other variable-length tokenizers is incomplete and not like-for-like. Only ElasticTok and AdapTok are compared. KARL, ALIT, FlexTok and VideoFlexTok are described but absent from the results, and the concurrent single-pass EVATok is dismissed with an unsupported claim of "semantic fragmentation". KARL is the closest single-pass competitor. Most baseline numbers appear to come from the original papers, with different data, model sizes and codebooks. The 4x latency claim covers only ElasticTok, under a stopping rule chosen by the authors, and AdapTok is not timed. No seeds or confidence intervals are reported, although the differences with the closest 1D baselines are small.

Questions for authors

The related work says KARL predicts the token count in a single forward pass, but then says all dynamic-length methods require search. Which statement do you intend for KARL?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.