- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces CD-RMOT-Bench, a benchmark for cross-domain referring multi-object tracking. Its main dataset contribution is Refer-vKITTI, constructed by transferring language–trajectory annotations from five real-world scenes to their synthetic counterparts across ten weather, lighting, and viewpoint conditions. It contains 50 videos, 20,560 frames, and 27,730 expression instances, with 272,043 expression–track pairs retained after filtering. These counts include variants of the same scenes and expressions across domains. Together with the existing Refer-KITTI-V2 and Refer-BDD datasets, the benchmark comprises 121 videos, 38,573 frames, and 42,118 expression instances, supporting controlled domain-shift and synthetic–real transfer evaluation. The paper also proposes QCA, which combines teacher–student consistency, adversarial domain alignment, and language-guided query calibration, and evaluates it across six transfer settings with ablations and multi-seed experiments on selected settings.
Critical strengths
Useful controlled evaluation. Synthetic variants of the same scenes reduce confounding differences in scene content, allowing weather and viewpoint shifts to be studied more directly.Clearly described construction and evaluation protocols. The paper explains identity mapping, visibility filtering, and human auditing, and uses separate scenes for adaptation and evaluation.
Supporting experimental evidence. Experiments cover six transfer settings and include component ablations and three-seed evaluations on two target domains, supporting QCA’s improvements over the source-only baseline.
Critical weaknesses
Insufficient differentiation from recent benchmarks. Table 1 mainly describes the benchmark’s composition rather than establishing its advantages over existing datasets. CRTrack and AerialMind already include cross-domain evaluation, so the additional value of the controlled paired data and unsupervised adaptation protocol should be clarified.Limited independent scene diversity. Refer-vKITTI contains only five underlying scenes, two of which are used for evaluation. Its 50 videos and 27,730 expression instances include repeated variants across domains and therefore do not directly reflect independent visual or linguistic diversity.
Insufficient validation of transferred expressions. Correct identity correspondence does not guarantee that the original descriptions remain valid after viewpoint changes. The audits report annotator agreement but do not quantify semantic correctness across domains and expression types.Confounding factors in comparisons and ablations. The main baselines use source data only, whereas QCA accesses unlabeled target data; the generic adaptation controls also use different training budgets. Additionally, removing CDT-MT stops gradient propagation through ACDA’s target-domain features, making it difficult to isolate individual component contributions.
Questions for authors
- Benchmark novelty: How does CD-RMOT-Bench differ from the existing cross-domain evaluation protocols in CRTrack and AerialMind? Could you clarify which additional capabilities arise from the paired scene variants and the target-domain adaptation protocol?
- Dataset diversity: How many unique expressions remain after removing repetitions across domain variants? Given that the controlled benchmark uses only five underlying scenes, could you provide per-scene results or an alternative scene-disjoint split to assess sensitivity to the selected training and test scenes?
- Annotation validity: How do you verify that transferred expressions, particularly spatial descriptions such as “vehicles on the left,” remain valid after viewpoint changes? Could you report the semantic error rate observed in the 300 audited expression–track instances, together with the correction procedure, rather than annotator agreement alone?
- Experimental fairness and component contributions: Could you compare QCA with generic adaptation baselines under the same target-data access and comparable training budgets? Additionally, could you repeat the CDT-MT ablation while preserving ACDA’s target-domain gradient flow, so that the effect of removing CDT-MT can be isolated?