- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes UniMergeTrack, a model-merging framework for unified RGB-Thermal, RGB-Depth, and RGB-Event visual object tracking. Starting from independently trained task-specific trackers, it merges their backbones using high-pass-filtered task vectors while retaining sparse modality-specific experts. Within each task, it combines tracking heads from different training checkpoints through optimal-transport-based channel alignment and parameter averaging. A final fine-tuning stage adapts the heads to the merged backbone while keeping the backbone frozen.
The main contribution is this hierarchical strategy of cross-task backbone merging and intra-task head merging. The paper evaluates the framework on multiple RGB-X tracking benchmarks, comparing it with task-specific trackers, unified trackers, and existing model-merging methods. It reports improved tracking performance and reduced storage relative to maintaining separate models, supported by component ablations and comparisons of merging and channel-alignment strategies.
Critical strengths
-
Practical consolidation of modality-specific trackers. The framework combines RGB-Thermal, RGB-Depth, and RGB-Event trackers into a shared backbone with lightweight task-specific components. The reported storage reduction—from three 340 MB models to approximately 379 MB—while improving over independently trained trackers demonstrates a useful accuracy–storage trade-off. This gives the approach practical value beyond improvements in tracking scores alone.
-
Broad empirical evaluation with targeted ablations. The paper evaluates multiple RGB-X tracking benchmarks and compares against both tracking and model-merging baselines. It also examines alternative backbone/head merging strategies and channel-alignment methods, rather than only removing individual modules. The reported improvements over FREE-Merging, together with these controlled comparisons, provide empirical support for the proposed tracking-specific integration, even though its individual techniques largely build on prior work.
Critical weaknesses
-
Limited methodological novelty beyond existing model-merging techniques. The backbone-merging pipeline closely follows FREE-Merging, including frequency filtering, sparse task-specific experts, and rescaling. Likewise, aligning neurons with optimal transport before averaging parameters was established by Model Fusion via Optimal Transport (NeurIPS 2020). The main remaining contribution is therefore the particular combination of cross-task backbone merging and intra-task head merging for RGB-X tracking. Although this combination produces improvements, the paper does not sufficiently establish a new methodological insight beyond adapting and integrating existing techniques. This limits the significance of its claimed contribution.
-
Insufficient evidence that the gains require the proposed pipeline under matched training budgets. Performance depends strongly on post-merging fine-tuning: without it, AUC falls from 61.8 to 57.7 on LasHeR and from 64.1 to 60.9 on VisEvent, below the independent-training baselines of 58.8 and 62.2. This does not invalidate merging, but makes budget-matched adaptation controls particularly important. The paper does not clearly establish whether the independent/joint-training baselines receive comparable additional training or whether all merging baselines use identical post-merging adaptation. Despite the component ablations, it remains unclear whether the hierarchical pipeline offers a meaningful advantage over simpler merging or checkpoint-averaging approaches with equally tuned heads. This weakens confidence in attributing the reported gains to the proposed design.
Questions for authors
- The main merging techniques in Stage 2—including frequency filtering, sparse expert extraction, and OT-based channel alignment—have precedents in existing work. Across the three stages, what do the authors consider the single most important technical innovation? Authors should clarify how it differs from prior methods and whether the contribution lies primarily in a new algorithm, the specific combination of existing techniques, or their adaptation to RGB-X tracking.
- The pipeline appears to introduce several corrective steps: filtering addresses interference from cross-task merging, sparse experts preserve task-specific information, and subsequent head adaptation addresses incompatibility with the merged backbone. This sequence is plausible, but what evidence shows that the full combination is necessary, rather than a simpler design avoiding some of these intermediate problems? The component ablations and alignment comparisons demonstrate improvements within the proposed configuration; could the authors explain which results establish the advantage of the overall design over simpler alternatives, and which specific failure modes make its additional steps necessary?
- Table 3 shows that, without post-merging fine-tuning, the merged model performs worse than the independent-training baseline on both LasHeR and VisEvent. Since the final stage uses the original tracking objective to adapt the heads, what evidence distinguishes the benefit of hierarchical merging from that of subsequent adaptation? In particular, do the merging baselines in Table 5 receive the same post-merging fine-tuning, and do the independent- and joint-training baselines receive comparable training budgets? Clarifying these conditions would help determine whether the reported gains specifically support the proposed merging strategy.