acceptodds
Withdrawn from ICLR 2027

A Simplified DINOv3-Driven Discriminative Framework with Siamese Template Fusion for Visual Tracking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual object tracking requires representations that separate target from background with high precision while remaining cheap enough for real-time deployment. On UAV platforms, model size and latency constraints make this trade-off especially acute. Lightweight vision transformers achieve the required speed but their limited capacity weakens discriminative power, whereas self-supervised architectures such as DINOv3 provide rich semantic hierarchies that remain underexplored in tracking and, lacking an official tiny variant, cannot be deployed directly on constrained devices. We present DinoSDTrack, which reconciles these two regimes. First, we introduce a simplified, unified Siamese-discriminative framework in which the Siamese template prior is fused into a discriminative tracking head within a one-stream architecture, enabling target-background separation to be learned jointly rather than assembled from independent modules. Second, we show that DINOv3-Base fine-tuned on tracking objectives supplies task-adapted representations, and we formulate a task-adaptive distillation scheme that transfers these representations into a custom DINOv3-Tiny, preserving discriminative accuracy under strict real-time budgets. Experiments on six general tracking benchmarks and six UAV benchmarks show that DinoSDTrack attains state-of-the-art accuracy while maintaining real-time inference speed, indicating that task-adapted self-supervised hierarchies can be effectively compressed for discriminative tracking. Code will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Object Tracking, Discriminative Framework, Siamese Template, DINOv3

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HqCnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DinoSDTrack, a one-stream visual tracking framework combining Siamese template information with discriminative learning. DisEmbed encodes Gaussian foreground labels and LTRB spatial information into training tokens, while GSWTF compresses template features through spatial weighting and gated fusion with the classification token. The prediction head combines target-specific filtering with center-based regression. A DINOv3-Base model is fine-tuned for tracking and subsequently used to distill feature, score, and bounding-box knowledge into a custom Tiny student. The approach is evaluated on six general tracking benchmarks and six UAV benchmarks, with component ablations and model complexity measurements.

Critical strengths
  1. A coherent and practically motivated tracking system. Integrating target-label prompts, compact template fusion, and a lightweight prediction head is a reasonable engineering approach to balancing discrimination and efficiency. The component ablations provide evidence that these design choices improve performance within the proposed system.

  2. Broad empirical evaluation and useful student-model improvements. Evaluation across general and UAV tracking benchmarks provides substantial evidence of practical competitiveness. In Table 8, distillation improves the Tiny model from 66.5 to 69.0 AUC on UAV123 and from 66.8 to 69.8 on UAVTrack112. These gains support the practical value of teacher supervision for compact trackers.

Critical weaknesses
  1. The technical novelty of the proposed “unification” appears limited. DisEmbed implements label and spatial prompt encoding; GSWTF combines spatial attention pooling with gated token fusion; the prediction head explicitly combines ToMP-style target-specific filtering with OSTrack-style center regression; and the distillation objective combines feature, score, and box supervision. These are largely established building blocks. Their integration is sensible, but the paper does not clearly establish what constitutes a substantial methodological advance beyond existing one-stream discriminative tracking approaches. The current contribution appears stronger as a carefully engineered tracking system than as a new tracking formulation or representation-learning method. This concern is consequential because the unified paradigm is presented as a central conceptual contribution.

  2. The ablations do not sufficiently isolate the two central conceptual claims. First, Table 8 compares training without a teacher, distillation from task-adapted Base/Large teachers, and direct use of the Base model. It establishes that teacher supervision helps, but does not isolate the benefit of tracking-specific teacher adaptation over a generic pretrained teacher under a comparable distillation setup. Consequently, the claim that task adaptation avoids deep-layer capacity or semantic mismatch is not directly substantiated. Table 7 also keeps feature distillation enabled throughout, leaving its independent contribution unclear. Second, Table 3 incrementally adds DisEmbed and GSWTF to a Siamese baseline, but does not provide a clearly defined, controlled comparison among Siamese-only, discriminative-only, and combined configurations. The DisEmbed-equipped baseline in Table 5 might partially address this issue, but its retained template pathways are insufficiently specified. The reported gains therefore support the usefulness of the added modules, while providing weaker evidence for the claimed complementarity of the two tracking paradigms. These gaps affect the interpretation of the paper’s main contributions, rather than merely the completeness of its experimental coverage.

Questions for authors
  1. Clarification of novelty: What is the precise methodological distinction between the proposed unified framework and the closest existing one-stream discriminative trackers, particularly JDTrack? Which contribution extends beyond changing the backbone, template fusion, and prediction head?

  2. Evidence for task adaptation: Can you provide a controlled comparison that isolates teacher task adaptation from the general benefit of distillation? Since a generic pretrained DINOv3 model lacks tracking score and box outputs, please explain how the comparison would keep feature supervision and output-level supervision comparable. What evidence specifically supports the claim of avoiding deep-layer capacity mismatch?

  3. Clarification of the complementarity ablation: Does the DisEmbed-equipped baseline in Table 5 remove all Siamese template information? Please specify its inputs and retained pathways. If it does not, can you provide a comparison of Siamese-only, discriminative-only, and combined configurations using the same backbone, prediction head, and training budget, with input and computation differences reported?

Reviewer_j6seOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces DinoSDTrack, a DINOv3‑based one‑stream visual tracker for real‑time UAV tracking, which unifies Siamese template matching and discriminative learning via DisEmbed discriminative prompt encoding and the GSWTF template‑compression module, and leverages a multi‑objective task‑adaptive distillation pipeline to transfer representations from fine‑tuned DINOv3‑Base into a custom lightweight DINOv3‑Tiny student; validated on six general and six UAV tracking benchmarks with comprehensive ablation studies, complexity measurements and qualitative visualizations, DinoSDTrack‑Base achieves state‑of‑the‑art tracking accuracy while DinoSDTrack‑Tiny obtains 254.7 FPS for edge‑friendly deployment, though it still suffers from limitations including degraded performance on extreme tiny targets, empirically tuned distillation hyper‑parameters and exclusive RGB‑input support.

Critical strengths
  1. The contributions are well motivated. This paper leverages the well known DINOv3 vision foundation model to construct a sota one-stream tracker and distill a tiny model for fast uav tracking.
  2. Both the DINOV3 and DONOV3-tiny model based trackers acheive sota tracking performance on both general and uav tracking benchmarks.
Critical weaknesses

1.There is no controlled ablation using a standard ViT backbone of matching parameter size to verify whether DINOV3 is better than a standard ViT backbone, which is the base motivation behind this paper. 2. The main contributions, e.g. DisEmbed and GSWTF, are not dedicatedly designed for the DINOV3 foundation model. Without feature statistics or visual evidence to interpret knowledge transfer, the contributions of DisEmbed, GSWTF and distillation read as high‑quality engineering increments rather than deep machine‑learning insights expected at ICLR. 3. This paper didn't report FPS of the tiny tracking model on edge device, which is very import for high-speed tracking for UAVs. 4. The text formats are not consistent in all the tables.

Questions for authors

Is the DINOV3 backbone better than the standard ViT of matching parameter size for the tracking task?

Reviewer_rX1MOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents a Siamese tracker that operates in the DINOv3 space. The design of the tracker is driven by efficiency concerns. Hundreds of papers with similar architecture have been published, and the novelty is incremental.

Critical strengths

The performance of the tracker is solid on most of the reported benchmarks.

Critical weaknesses

The tracker has not been evaluated on any of VOTS benchmarks, which are the golden standard in the field. The conclusion from the recent VOTS reports is that SAM2 and SAM3-based tracker are the state of the art, yet they are not mentioned in the paper. The omission of the VOTS evaluation is a rejection-implying weakness.

Questions for authors
  1. Run the VOTS challenge and report results. This is critical. Note that VOTS has a subchallenge that looks at efficiency (real-perfr ormance), which is important in UAV use case.
  2. For the benchmarks that are reported in the paper, report the state-of-the-art trackers. Table 2. is not complete in this sense.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.