acceptodds
Withdrawn from ICLR 2027

Keyword Enhanced Generative Priors for Vision Language Tracking in Underwater Conditions

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision Language Tracking (VLT) localizes target objects in videos by jointly exploiting visual and textual cues, ensuring stronger reliability than traditional tracking methods. However, existing VLT methods still suffer from severe performance degradation in underwater scenarios because the strong noise and color distortion resemble the target and its surroundings closely. Additionally, the lack of corresponding training data prevents this limitation from being addressed through a pure data-driven approach. In this paper, we address this limitation through a more efficient method, transferring the strong prior of generative models learned from the pretraining on large-scale natural images to underwater scenarios. Specifically, we incorporate a keyword enhancement strategy. It extracts intermediate multi-modal features from the generation process to assist tracking. During this, it extracted the keywords related to the target and modifies cross-attention maps by increasing the corresponding attention weights to steer the features to place greater emphasis on the tracking target. With this in mind, we propose KeyTrack, a generative-model-based VLT framework, which uses strong priors from generative models and enables target-focused representation learning while suppressing irrelevant environmental noise, thereby achieving more accurate tracking. Extensive experiments demonstrate that KeyTrack achieves state-of-the-art performance on underwater tracking benchmarks and maintains competitive results on general datasets, validating its robustness and generalization capability.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision-language tracking, underwater tracking, diffusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HGAkOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes KeyTrack, a generative tracking framework for underwater vision-language tracking. The method leverages the visual priors of a pre-trained diffusion model, extracts target-related keywords from textual descriptions via a Key Extraction Module, and amplifies the cross-attention weights of keyword tokens during the diffusion process to suppress background interference and generate target-aware multimodal features. A Multi-Modal Decoder then fuses the target-aware multimodal features with visual features for tracking. Experiments cover underwater benchmarks such as WebUOT-1M, UW-COT220, UTB180, and UVOT400, as well as conventional benchmarks including LaSOT, LaSOText, TNL2K, and MGIT, with ablations on the keyword scaling factor and attention visualizations.

Critical strengths
  1. Reasonable motivation and clear writing. Underwater scenes suffer from low visibility, color attenuation, background interference, and image degradation, and exploiting generative model priors to enhance target-related features is an appealing direction. The paper is well structured, and the method pipeline and experimental setup are clearly presented.

  2. Broad experimental coverage. The paper evaluates on both underwater and conventional VLT benchmarks and achieves competitive results on multiple underwater datasets; in Table 1, metrics on WebUOT-1M and UW-COT220 outperform several existing trackers.

  3. Relatively complete framework design. From keyword extraction and attention enhancement to multimodal decoder fusion, the modules are logically connected; the keyword enhancement strategy is intuitive and easy to implement, and can provide a reference for future generative VLT research.

Critical weaknesses
  1. The necessity of the keyword extraction module is insufficiently validated. The authors train an MLP for keyword identification using target-word annotations, but lack a systematic comparison with simpler, training-free baselines. For example, would using an off-the-shelf POS tagger to extract all nouns and amplify their attention achieve comparable results? If simply extracting nouns is sufficient, the design value and annotation cost of the MLP need further justification.

  2. The baseline comparison is incomplete, especially the lack of results from the latest general SOTA trackers on underwater data. Table 1 contains few 2025 comparison methods and lacks evaluations of the latest general SOT/VLT trackers such as MCITrack on underwater benchmarks. As a method tailored for underwater environments, KeyTrack should demonstrate its advantages over strong general trackers, not only improvements over 2025 methods.

  3. The justification for underwater specificity is insufficient. The core of the method is textual keyword attention enhancement and generative priors, neither of which is an underwater-specific design. The paper does not design dedicated mechanisms for underwater physical degradations (e.g., color attenuation, scattering, low contrast), nor does it sufficiently analyze how these degradations are mitigated by the method. Therefore, there is a gap between the motivation of "designed for underwater environments" and the method.

Questions for authors
  1. Compared with training-free baselines such as a POS tagger or simply "extracting all nouns and amplifying their attention," what are the specific advantages of the MLP keyword extractor in terms of tracking accuracy, robustness, and efficiency? Can the authors provide additional ablations?

  2. Can the authors add results of the latest general trackers such as MCITrack on underwater datasets such as WebUOT-1M and UW-COT220?

  3. Which designs in the paper specifically address underwater image degradations? Can the authors use visualizations or ablations to show how keyword attention enhancement and generative priors mitigate these degradations?

Reviewer_3fdFOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

KeyTrack addresses underwater vision-language tracking by using generative visual priors to cope with degraded appearance and background interference. Its main contribution is a keyword-guided attention mechanism that emphasizes target-related information during diffusion-based feature extraction. These representations are combined with visual tracking features to localize the target. Experiments on underwater and conventional tracking benchmarks, together with scaling ablations and qualitative analyses, report improvements in tracking accuracy and examine the method's sensitivity to textual guidance.

Critical strengths
  • The underwater application is relevant, with evaluation on eight benchmarks.
  • Table 3 includes a no-enhancement control: keyword scaling improves WebUOT-1M and TNL2K AUC by 0.7 and 1.0 points, respectively.
Critical weaknesses
  • The evidence does not establish a sufficiently distinct and consequential contribution. The manuscript credits prior work for diffusion-based template representations (Appendix A) and supervised keyword extraction (Section 3.1); the central addition is multiplying selected attention columns by 1.1 (Eq. 5). Table 3 only varies this multiplier, without distinguishing keyword-specific benefit from generic attention/feature amplification. Moreover, KeyTrack receives additional WebUOT-1M training (Section 4.1), while matched baseline training and language access are unspecified. GLAD, the cited generative tracker, is absent from the underwater comparison. These results do not establish a substantial advance over existing generative tracking.
  • The proposed mechanism and executable method are not adequately connected. Eq. (5) leaves non-keyword attention entries unchanged and specifies no renormalization, so the claimed background suppression is not directly established by the operation; qualitative maps in Figure 6 do not isolate its cause. More fundamentally, Algorithm 1 starts from Gaussian noise without a template input, whereas Figure 4 and Appendix A describe an encoded, noised template. This changes what visual information conditions the generative features. The denoising schedule, extracted layers, and caching policy are also unspecified, leaving both the actual method and the reported 65.2 FPS difficult to assess.
Questions for authors
  1. Under identical underwater training and language inputs, how does KeyTrack compare with GLAD and with uniform attention scaling, to isolate the contribution of keyword selection?
  2. Clarification: What diffusion initialization and attention normalization are actually implemented? Please specify the steps, extracted layers, caching, and whether FPS includes diffusion feature extraction.
Reviewer_sANdOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a vision-language tracking framework called KeyTrack, designed to address challenges in underwater scenarios—such as the lack of color contrast between targets and the background, image blur, poor lighting, and a scarcity of training data. By integrating visual priors from a pre-trained diffusion model with keyword-driven attention modulation, the authors aim to improve target localization accuracy without relying on large volumes of annotated underwater data. Extensive experiments demonstrate that Key Track achieves state-of-the-art performance on unde rwate rtracking benchmarks and maintains competitive results on general datasets.

Critical strengths
  1. The method is designed with the specific characteristics of underwater tracking in mind, and it is clearly problem-oriented.
  2. Leverage keywords to strengthen textual guidance and enhance the precision of target positioning.
  3. Good experimental results were achieved on several underwater tracking benchmarks.
Critical weaknesses
  1. In the paper, KeyTrack is compared with methods such as OKTrack, ODTrack, and GRM on underwater tracking datasets; however, the current experimental description fails to adequately specify the sources of the results for each baseline, nor does it clarify whether a consistent training and testing protocol was applied across all methods.
  2. A core design element of KeyTrack is the integration of Stable Diffusion v1.5, which employs a diffusion process to extract multimodal features and adjusts the attention weights associated with specific keywords within the cross-attention layers. While this design may enhance representational capacity, it also introduces additional computational overhead. Consequently, the paper should not rely solely on tracking accuracy to demonstrate the method's utility; it must also quantify the computational cost incurred to achieve that level of accuracy.
  3. The paper reports performance improvements across several underwater benchmarks. However, the stability of these gains has not been fully evaluated. For smaller performance improvements, conducting multiple repeated runs, estimating variance, or calculating confidence intervals would be highly informative.
  4. This framework incorporates generative priors, keyword extraction, attention enhancement, and multimodal feature fusion. Current ablation analyses have yet to fully clarify the individual contributions of each component. Further controlled experiments are required to disentangle the gains derived from diffusion features versus those resulting from "keyword-guided attention reweighting."
  5. Although the paper reports a figure of 65.2 FPS and specifies the GPU environment used, it remains unclear whether the different methods follow consistent timing protocols. The authors should explicitly state the GPU configuration, input resolution, warm-up procedure, and the number of frames used for evaluation for each speed measurement. Furthermore, they should clarify whether the timing encompasses the entire pipeline—including preprocessing, text encoding, diffusion inference, and post-processing.
Questions for authors
  1. Were the baseline results taken from original papers or reproduced by the authors? Were consistent training data and evaluation protocols used across methods?
  2. Could the authors report KeyTrack's parameter count, FLOPs, and inference latency to quantify the overhead introduced by Stable Diffusion?
  3. Were experiments repeated with different random seeds? Could the authors report variance or confidence intervals to support the reported improvements?
  4. Could the authors isolate the contributions of diffusion features and keyword-guided attention reweighting through controlled ablation experiments?
  5. Were all methods benchmarked under the same hardware and timing protocols? Does the reported FPS include the entire pipeline, including diffusion inference and preprocessing?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.