acceptodds
Withdrawn from ICLR 2027

Where to Interact and When to Skip: Efficient Transformer for Underwater Visual Tracking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Underwater visual tracking is challenged by degraded imaging conditions such as light attenuation and scattering, as well as by limited computational resources on underwater platforms that restrict real-time deployment. Existing Transformer trackers typically perform dense template–search interactions and execute all Transformer layers for every frame, resulting in considerable redundant computation. To address these issues, we propose WTrack, an efficient Transformer-based underwater tracker that allocates computation adaptively in both token interaction and network depth, determining where token interactions should occur and when redundant Transformer blocks can be skipped. Specifically, WTrack contains two key components: (i) Stage-aware Attention Interaction (SAI), which reorganizes template-search interactions across Transformer stages. In shallow stages, SAI removes search-to-template interactions to prevent ambiguous information propagation caused by dense and visually similar underwater targets. In deeper stages, SAI eliminates redundant intra-frame interactions and focuses on bidirectional template-search correspondence to improve target matching and localization. (ii) Adaptive Layer Execution (ALE), which dynamically adjusts Transformer depth according to the tracking state. Since underwater sequences exhibit varying target visibility, motion patterns, and image quality, ALE estimates scene complexity and skips redundant deep-layer computation in easy cases where additional layers provide limited benefits. Experiments on UTB180, UOT100, and VMAT demonstrate that WTrack achieves competitive tracking accuracy and real-time efficiency, reaching 210.52 FPS on GPU while outperforming existing real-time trackers.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Underwater visual tracking, Vision Transformers, Lightweight tracking

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_17AVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article proposes a lightweight Transformer method W2Track for underwater single target tracking, which adjusts calculation allocation through two modules. Stage aware Attention Interaction uses asymmetric templates for search interaction in the shallow layer, while retaining only bidirectional cross frame attention in the deep layer. Adaptive Layer Execution uses intermediate features to predict layer execution probability and adjusts the deep layer execution threshold based on cross frame attention entropy. Accuracy, speed, and computational complexity were reported on UTB180, UOT100, and VMAT.

Critical strengths
  1. The experimental results have advantages, with W2Track reaching 61.35 AUC on UTB180, reporting GPU speed of 210.52 FPS, CPU speed of 27.09 FPS, and model parameters of approximately 4.87M. In the lightweight model group in Table 1, the accuracy on all three datasets is superior.
  2. The ablation coverage is relatively comprehensive. The author compared the independent combinations, different attention modes, skip layer positions, and threshold strategies of SAI and ALE, and provided density and difficulty analysis. These experiments provide a foundation for evaluating various designs, rather than just comparing the final results with external methods.
Critical weaknesses
  1. The explanation of core attention design contradicts the actual information flow, and the definition of ALE execution is incomplete, resulting in insufficient evidence of the mechanism and innovation. The author claims that deleting $Q_st ^ \ top $can prevent information pollution templates in the search area. But according to standard attention calculation, the template output is weighted by $Q_tK_s ^ \ top $on the search value $V_s $, and the search output is weighted by $Q_SK_t ^ \ top $on the template value $V_t $. Therefore, equation (3) actually preserves the path for search information to enter the template, while deleting the path for template information to enter the search feature. This contradiction directly relates to the core motivation of SAI and cannot be resolved solely by renaming the interaction direction; We need to explain why the reserved path can suppress the interference mentioned. ALE performs skip layers on consecutive candidate layers, but equation (13) relies on the previous layer's attention $A_ {i-1} $. How to obtain attention when the current layer is skipped? The weighted output description of equation (12) itself cannot prove that block calculations are truly omitted during inference, and it is necessary to clarify the actual condition branches and their timing ranges. In addition, the regularization term $L_ {\ mathrm {reg}} $only references AVTrack and does not provide the actual formula and hyperparameters used. In addition, this article has cited works such as FERMT (ECCV 2024) and AVTrack (ICML 2024), but lacks direct comparisons and comparisons between existing execution strategies and the proposed entropy threshold on the same backbone. The current results are not sufficient to clearly define the contribution of the new design.

  2. Some experiments cannot correspond to the overall results, weakening the main performance conclusions. Table 3 reports SAI gains of 2.354 and 1.195 AUC points for dense and sparse subsets, respectively. If both constitute a complete partition of UTB180 and are weighted in a manner consistent with the overall result, the total gain should be between these two numbers. Is it a specially selected subset? Therefore, this analysis currently cannot reliably support the conclusion that 'SAI is more effective for dense targets'. Table 4 also lacks difficulty grouping criteria and sample size. On page 6, the method is classified as non real time according to "over 20 GMACs", but in Table 1, multiple methods in this group still significantly exceed 30 FPS on the same GPU. For example, PUTrack is 60.15 FPS and UTB180 AUC is higher than W2Track; AQATrack reaches 133.59 FPS. Therefore, if GPU 30 FPS is used as a criterion, the statement 'better than existing real-time trackers' is not valid. For dynamic models, it is also necessary to explain how individual GMACs values are calculated, and whether the speed includes attention entropy, routing judgment, pruning, and post-processing is also not specified. The average speed on RTX 5090 can support desktop GPU efficiency, but it is not sufficient to directly demonstrate the deployment performance of resource constrained underwater platforms.

Questions for authors
  1. Please specify which branch to update for each cross frame path in equation (3). Since \ (Q_tK_s ^ \ top V_s ) is still writing search information into the template, how can deleting \ (Q_stK_t ^ \ top ) achieve the claimed template interference suppression? Is this a textual error or is the formula/implementation different from the description?

  2. How to obtain the attention entropy of equation (13) when continuously jumping layers? Please provide the pseudocode for inference, the masking method for prohibited attention blocks, and the actual use of \ (L_ {\ mathrm {reg}} ) and hyperparameters. Does the attention module truly omit matrix calculations instead of calculating the complete matrix and then masking it?

  3. Which model configuration in Table 2 does Table 3 correspond to? Does Dense/Sparse cover all UTB180 sequences? Please provide the number of sequences, grouping criteria, and overall indicator aggregation method, and explain the difference between subset gain and overall gain. How to determine the difficulty of Table 4?

  4. Why wasn't FERMT compared? What is the independent benefit of the proposed entropy threshold relative to the AVTrack fixed threshold under the same backbone, training data, and execution budget? Are the layer positions and thresholds selected through independent validation sets?

  5. Which dataset and execution rate does 1.553 GMACs correspond to? What batch size, numerical accuracy, synchronization and preheating settings are used for FPS testing, and what steps are included in timing? Please specify 20 GMACs as the basis for real-time grouping boundaries.

Reviewer_fECdOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes W2Track, a lightweight one-stream Transformer tracker tailored for underwater visual tracking. It introduces two main components to reduce computational overhead: (1) Stage-aware Attention Interaction (SAI) and (2) Adaptive Layer Execution (ALE). The tracker is evaluated on three small-scale underwater tracking datasets (UTB180, UOT100, and VMAT).

Critical strengths

The paper provides an engineering integration of block-wise attention pruning and dynamic layer skipping on a ViT-Tiny backbone, demonstrating empirical accuracy-speed trade-offs across three small-scale underwater tracking benchmarks (UTB180, UOT100, and VMAT).

Critical weaknesses

1.The proposed method largely combines and fine-tunes attention masking and dynamic layer skipping from existing general-purpose and UAV lightweight trackers (e.g., AVTrack, SGLATrack, and FERMT), without novel theoretical insights into underwater physical characteristics.

2.Although optical degradation (light attenuation and scattering) and limited edge-device computing resources are emphasized, the architecture lacks specific designs addressing underwater imaging physics or wavelength-dependent attenuation.

3.In Lines 280–283, the authors argue that high entropy indicates uncertain matching and requires deeper refinement, while low entropy indicates clear matching and permits layer skipping. However, since Δi≥0\Delta_i \geq 0, the adaptive threshold can only increase, encouraging layer skipping. It cannot fall below the fixed baseline τf\tau_f to enforce deeper refinement in severely degraded underwater frames. Moreover, high-entropy attention from background template tokens may significantly distort the global average entropy.

4.The manuscript has notable writing and presentation issues. For example, the core architecture diagram (Figure 2) contains spelling errors, such as “Satge 2.”

Questions for authors

1.Aside from the static binary attention masks (SAI) and the entropy term Δi\Delta_i in Eq. (13), how does the ALE routing mechanism and training objective fundamentally differ from AVTrack (Li et al., 2024a)?

2.In Eqs. (3) and (4), how are the inactive blocks implemented in practice? Are they masked with −∞-\infty inside a full (mt+ms)×(mt+ms)(m_t+m_s)\times(m_t+m_s) attention matrix (which would save zero FLOPs/GMACs), or decomposed into separate sub-matrix multiplications and concatenations?

3.The proposed method is evaluated only on underwater datasets. How does it perform on general-purpose terrestrial and UAV tracking benchmarks, such as LaSOT, GOT-10k, and UAV123?

Reviewer_D2mzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes W²Track, an efficient Transformer-based tracker for underwater visual tracking. The core contribution is Stage-aware Attention Interaction (SAI): the search-to-template interaction is removed in shallow layers to avoid ambiguous matching caused by dense, visually similar targets, while deep layers retain only bidirectional template–search correspondence. In addition, Adaptive Layer Execution (ALE) decides whether subsequent Transformer blocks are executed based on intermediate feature states, using a fixed threshold in shallow layers and an entropy-driven adaptive threshold in deep layers. The overall architecture is one-stream with a ViT-Tiny backbone, evaluated on three underwater tracking benchmarks.

Critical strengths
  1. The stage-wise allocation of attention is a reasonable design. Shallow layers preserve intra-frame modeling while deep layers focus on cross-frame matching, which answers the where-to-interact question of the title; together with ALE, the two components address the two redundancies identified in the motivation.
  2. Component validation goes beyond overall gains. SAI is examined on target-density subsets and ALE on sequences of varying difficulty, and the observed trends are consistent with their respective underwater-specific motivations.
  3. The ablation coverage is broad. Component contributions, attention combinations, gate placements, and threshold configurations all have dedicated comparisons; the competing trackers are retrained under the same protocol, and the claimed margins are consistent with the tables.
  4. The method achieves a favorable accuracy–efficiency trade-off within its compute class.
Critical weaknesses
  1. The grouping underlying the headline claim is defined by computation rather than measured speed. Baselines exceeding 20 GMACs are categorized as non-real-time, yet on the same GPU all of them run above conventional real-time frame rates, and OSTrack and PUTrack surpass the proposed method in accuracy on all three benchmarks. The claim of outperforming existing real-time trackers holds only under this self-defined grouping.
  2. Several counter-intuitive ablation results are left unexplained. Enabling ALE alone noticeably improves accuracy although layer skipping forgoes computation by design; a lower execution threshold lowers accuracy despite consuming more computation, single-stage gate placements are all worse than no gating at all, and dual adaptive thresholds underperform the deep-stage-only variant. No seed variance is reported, and the scaling factor α and the sharpness γ are fixed without sensitivity analysis, so these results cannot be attributed to the design. The claims that each component independently improves accuracy and that ALE saves computation without sacrificing accuracy therefore become doubtful.
  3. One explanation of the shallow-stage design is inconsistent with the information flow of standard attention. The removed search-to-template term updates the search tokens, whereas the retained template-to-search term writes information from the search region into the template representations, which runs counter to the stated motivation. The configuration itself may still be effective, but the authors need to clarify why it works.
  4. The paper does not clarify how its adaptive-depth mechanism differs from prior adaptive-layer-selection trackers, leaving its independent contribution unclear. Adaptive layer selection has already proven effective for tracking, the routing regularizer is directly adopted from prior work, and the remainder is an incremental refinement without a controlled comparison against that prior routing design.
  5. The complexity summary in the paragraph following Eq. (6) misstates the saved cost as the resulting complexity. The presented expression is the difference between the two complexity bounds, yet it is introduced as the complexity that remains.
Questions for authors

see above weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.