- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes S²TTA, a test-time adaptation method for multimodal object tracking. The method introduces modality-specific prompts into pretrained trackers. It uses the annotation of the first frame to optimize the prompts and construct a multi-level feature statistics prior. During subsequent tracking, the discrepancy between the current frame and the template prior is used to select which prompts to update. Meanwhile, prediction uncertainty is used to determine whether the template prior should be updated.
Critical strengths
- This paper uses the first-frame annotation available in object tracking to construct a target-specific template prior and uses it as the optimization objective for test-time adaptation. Compared with methods that directly use prediction confidence or source-domain statistics as adaptation objectives, this idea offers a different approach.
- The experimental evaluation is relatively comprehensive. The authors evaluate the method on tracking tasks with different modalities and use two trackers with different architectures as baselines. The paper also includes comparisons with other TTA methods and ablation studies of different components.
Critical weaknesses
- The overall network architecture and some important implementation details are not clearly explained. For example, the relationship between the template and search regions in Figure 2, as well as the prompt optimization process in the first frame, lacks sufficient explanation. This makes the proposed method difficult to understand.
- Although the paper includes several comparative experiments and ablation studies, it lacks qualitative visualizations of the tracking results. For example, there are no visual comparisons of tracking results before and after applying S²TTA under challenging conditions such as illumination changes, occlusion, and target appearance variations.
Questions for authors
- In Figure 2, some modules and labels overlap, such as the Encoder and Layer labels. I suggest improving the layout of the framework diagram.
- In the left part of Figure 2, only the template images are provided as input, but the prediction head is still used to predict bounding boxes and compute the tracking loss. How are the target bounding boxes predicted without a search region? In addition, the right part of Figure 2 only shows the search regions, without the original template features. Are the original templates still used for target matching during subsequent tracking? Is this due to a simplification of the framework diagram? I suggest clarifying the complete tracking process.
- Appendix A.3 mentions that geometric transformations and color perturbations are used to construct the augmented template pool, but the specific augmentation operations and parameter settings are not provided. I suggest adding these details.
- The proposed method performs test-time adaptation by aligning the feature statistics of the current frame with the evolving template prior. However, feature discrepancies may arise not only from environmental changes, such as illumination and weather variations, but also from target deformation or viewpoint changes. In the latter case, why would reducing the feature distribution discrepancy between the current frame and the template prior improve tracking performance? I suggest providing further explanation or experimental analysis.
- The paper lacks some intuitive visualization results. I suggest adding visual comparisons of tracking results before and after applying S²TTA under different scenarios. In addition, could the authors provide visualizations of feature responses or feature distributions before and after prompt updates?