- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Gated Token Recurrence (GTR), a softmax-free recurrent vision backbone for efficient dense prediction. GTR combines gated linear attention, alternating spatial scan directions, and spatially enhanced SwiGLU blocks, and is distilled from a detection-specialized DINOv3 teacher using final-layer patch-token alignment. The authors also develop a specialized chunkwise CUDA operator and integrate GTR into TensorRT for edge deployment. Experiments across six dense prediction tasks and four model scales demonstrate competitive accuracy and practical inference efficiency, including favorable scaling at higher input resolutions.
Critical strengths
- The paper is clearly written, with detailed and easy-to-follow descriptions of the method and experiments.
- The work addresses both accuracy and practical efficiency, with a custom CUDA implementation and edge deployment demonstrating latency improvements.
- The experiments cover six dense prediction tasks and four model scales, including an analysis of performance at higher resolutions.
- The appendix provides detailed training, deployment, and timing protocols, along with ablations of the main design choices.
Critical weaknesses
- Limited architectural novelty. GTR largely combines established components, including GLA, directional scanning, local convolutions, and feature distillation. The contribution is primarily engineering-oriented, and controlled comparisons with ViG and EdgeCrafter would help establish the value of the proposed design choices.
- The baseline comparisons could be more complete. The Objects365-pretrained ECDet models were released one month before the submission deadline. Including these results would help clarify the accuracy–latency trade-off under more comparable pretraining settings.
- Accuracy and latency are measured under different inference settings for some tasks. Segmentation and depth evaluation use sliding-window or multi-view inference, whereas latency measures only a single forward pass. Reporting the runtime of the full evaluation pipeline would better reflect the practical accuracy–efficiency trade-off.
Questions for authors
- Could you provide accuracy and latency comparisons between GTR and RF-DETR at the same input resolution? This would help distinguish the effects of architectural choices from those of input resolution.
- Could you report the training time or GPU-hours for each training stage, and compare the training throughput with a comparable softmax-attention baseline such as EdgeCrafter under matched settings? Since the custom CUDA kernel is used only for inference, it remains unclear how GLA affects training efficiency.
6 comments
Response to Reviewer pg2P (2)
3. Additional controlled backbone comparisons directly demonstrate the architectural advantage of GTR.
To further isolate the contribution of the GTR architecture, we conducted additional experiments by replacing only the GTR backbone with ViG and EdgeCrafter, while keeping the training configurations and Python environment identical. The results are as follows:
| Model | Median Latency (ms) | Params (M) | FLOPs (G) | AP | AP$_{50}$ | AP$_{75}$ | AP$_S$ | AP$_M$ | AP$_L$ |
|---|---|---|---|---|---|---|---|---|---|
| GTR | 1.215 | 12.07 | 33.87 | 44.9 | 60.6 | 48.2 | 27.6 | 48.3 | 60.7 |
| ViG | 1.395 | 12.32 | 32.92 | 42.2 | 57.9 | 45.7 | 24.3 | 45.3 | 57.7 |
| EdgeCrafter | 1.404 | 11.97 | 57.15 | 43.4 | 59.1 | 47.1 | 26.1 | 46.2 | 59.7 |
Under the same training configuration, GTR achieves +2.7 AP over ViG and +1.5 AP over EdgeCrafter, while simultaneously reducing latency by 12.9% and 13.5%, respectively. Notably, GTR also achieves lower latency with fewer FLOPs than EdgeCrafter. These results provide direct evidence that the performance is not merely inherited from the training recipe or distillation, but also benefits from the GTR architectural design.
4. The single-scan design is principle-driven rather than search-driven.
Unlike architecture search approaches that explore multiple configurations to identify favorable architectures, GTR's single-scan design follows directly from our architectural principle of decoupling local spatial interaction from long-range recurrent modeling. It is therefore not an isolated hyperparameter choice, but a consequence of the proposed computational decomposition.
[1] Guoyizhe Wei and Rama Chellappa. ViT-Linearizer: Distilling quadratic knowledge into linear-time vision models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 20737–20747, 2025.
[2] Yifan Li, Seunghyun Yoon, Viet Dac Lai, Franck Dernoncourt, Jason Kuen, Yu Kong, and Trung Bui. ViT-AdaLA: Adapting vision transformers with linear attention. To appear in Advances in Neural Information Processing Systems (NeurIPS), 2026.
[3] Huaiyuan Qin, Muli Yang, Gabriel James Goenawan, Shiqi Huang, Min Kass Chong, Wahyu Wiratama, Peng Hu, Chen Gong, Wu Liu, Xi Peng, Chun Jian Ho, and Hongyuan Zhu. Copy the same, distill the difference: Initializing linear vision transformers. arXiv preprint arXiv:2609.35745, 2026.
[4] Dongchen Han et al. FLatten Transformer: Vision Transformer using Focused Linear Attention. ICCV, 2023.
[5] Dongchen Han et al. Agent Attention: On the Integration of Softmax and Linear Attention. ECCV, 2024.
[6] Dongchen Han et al. Demystify Mamba in Vision: A Linear Attention Perspective. NeurIPS, 2024.
[7] Bencheng Liao et al. ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attention. AAAI, 2025.
[8] Weikang Meng et al. PolaFormer: Polarity-aware Linear Attention for Vision Transformers. ICLR, 2025.
[9] Qwen Team. Qwen3-Next, 2025.
[10] Kimi Team. Kimi Linear: An Expressive, Efficient Attention Architecture, 2025.
Response to Reviewer pg2P (3)
Weakness 2: The baseline comparisons could be more complete. The Objects365-pretrained ECDet models were released one month before the submission deadline. Including these results would help clarify the accuracy–latency trade-off under more comparable pretraining settings.
Thank you for your advice. To better address this concern, we provide additional comparisons with Objects365-pretrained ECDet models across different model scales.
| Model | Params (M) | GFLOPs | Latency (ms) ↓ | AP ↑ | AP$_{50}$ ↑ | AP$_{75}$ ↑ | AP$_S$ ↑ | AP$_M$ ↑ | AP$_L$ ↑ |
|---|---|---|---|---|---|---|---|---|---|
| ECDet-S | 10.0 | 50.0 | 1.652 | 53.6 | 71.0 | 58.0 | 35.1 | 58.3 | 71.8 |
| GTR-S | 12.1 | 33.8 | 1.225 | 53.6 | 71.1 | 58.3 | 36.4 | 58.3 | 70.2 |
| ECDet-M | 19.0 | 85.0 | 2.095 | 56.8 | 74.3 | 61.8 | 38.8 | 61.6 | 74.0 |
| GTR-M | 22.7 | 62.6 | 1.462 | 57.3 | 75.0 | 62.4 | 41.1 | 62.0 | 74.1 |
| ECDet-L | 33.0 | 147.0 | 2.730 | 59.0 | 77.0 | 64.4 | 42.1 | 64.2 | 76.0 |
| GTR-L | 37.2 | 106.0 | 1.908 | 58.9 | 76.8 | 64.3 | 42.4 | 64.0 | 76.0 |
| ECDet-X | 49.0 | 204.0 | 3.021 | 59.9 | 77.5 | 65.2 | 42.4 | 64.8 | 77.0 |
| GTR-X | 46.5 | 130.2 | 2.115 | 59.4 | 77.3 | 64.7 | 42.1 | 64.6 | 76.4 |
We have updated Table 1 on page 6 to include the Objects365-pretrained ECDet results.
Weakness 3: Accuracy and latency are measured under different inference settings for some tasks. Segmentation and depth evaluation use sliding-window or multi-view inference, whereas latency measures only a single forward pass. Reporting the runtime of the full evaluation pipeline would better reflect the practical accuracy–efficiency trade-off.
Thank you for your careful reading! We clarify that our reported latency measures a single forward pass, following the same evaluation convention as YOLO26, ensuring a consistent comparison across methods. Nevertheless, to further address the concern, we additionally evaluate the full inference runtime (). The measurements use the same hardware, software environment, and timing protocol as in the original paper.
| Dataset | Model | Single-forward (ms) | (ms) | FPS |
|---|---|---|---|---|
| Cityscapes | GTR-S | 1.495 | 13.08 | 76.4 |
| Cityscapes | GTR-M | 1.970 | 15.35 | 65.1 |
| Cityscapes | GTR-L | 2.958 | 19.34 | 51.7 |
| Cityscapes | GTR-X | 3.482 | 21.47 | 46.6 |
| NYU | GTR-S | 1.296 | 15.78 | 63.4 |
| NYU | GTR-M | 1.508 | 19.20 | 52.1 |
| NYU | GTR-L | 1.948 | 25.61 | 39.1 |
| NYU | GTR-X | 2.158 | 27.79 | 36.0 |
Response to Reviewer pg2P (1)
Thank you for raising this important concern! We respectfully clarify that GTR is not an incremental combination of existing components, nor is its novelty merely a different scan configuration obtained through hyperparameter tuning. Its design is motivated by a fundamentally different computational paradigm and three key insights.
1. GTR adopts a different computational paradigm from ViG and other existing architectures.
ViG integrates 2D depthwise convolution into the input of bidirectional GLA, allowing local features to participate directly in bidirectional global context modeling:
O_{\mathrm{local}} &= \mathrm{DWConv}(X), \\\\ O_{\mathrm{global}} &= \mathrm{BiGLA}(O_{\mathrm{local}}), \\\\ O &= G_{2D} \odot O_{\mathrm{local}} + (1 - G_{2D}) \odot O_{\mathrm{global}}. \end{aligned}$$ In contrast, GTR places DWConv in the value branch of an independent Spatial SwiGLU block: $$\begin{aligned} {[G, V]} &= \mathrm{Linear}_{g,v}(X), \\\\ V_{\mathrm{local}} &= \mathrm{DWConv}_{2D}(V), \\\\ Y_{\mathrm{spatial}} &= \mathrm{Linear}\big(\mathrm{SiLU}(G) \odot V_{\mathrm{local}}\big). \end{aligned}$$ By separating local spatial interaction from recurrent token mixing, GTR eliminates the need for bidirectional recurrence within each block, while retaining a fixed-size recurrent state for efficient inference. **2. GTR is motivated by three complementary insights.** **```Insight 1: Long-range dependency modeling and local spatial modeling should be decoupled.```** We argue that long-range propagation and local spatial interaction are distinct problems best addressed by complementary operators: recurrence captures long-range dependencies, while DWConv models local interactions. By explicitly handling locality, GTR eliminates the need for additional scan directions, enabling a single causal recurrence per block without sacrificing performance [Table 4]. Thus, our contribution goes beyond simply adding DWConv; we demonstrate that decoupling locality from recurrence enables an effective purely causal architecture. **```Insight 2: GTR demonstrates an effective knowledge transfer strategy across different computational paradigms.```** Transferring representations from non-causal Transformers to causal recurrent models is challenging due to their different computational structures. Existing approaches often require multi-stage alignment or multiple distillation objectives [1–3]. In contrast, we demonstrate that a single L2 loss is sufficient to transfer knowledge [Table 3]. The contribution is not a new loss function, but the finding that such a simple feature-level supervision strategy effectively bridges this architectural gap. **```Insight 3: GTR validates the effectiveness of purely linear-attention modeling for modern real-time detection.```** GTR is a purely linear-attention architecture. Most linear-attention backbones [4–8] are validated only on legacy frameworks (e.g., Mask R-CNN, Faster R-CNN), which cannot reveal their upper bound, and recent LLMs often adopt 3:1 linear–softmax hybrids [9,10]. In contrast, GTR achieves strong performance in a modern real-time detection setting without any softmax attention. As shown in [Table 5], the 3:1 hybrid changes AP by only ±0.1 while increasing latency by ~10%.