- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
QTrack generates target trajectories from a short clip, a text query, and reference-frame boxes. It operates offline, with the boxes identifying the requested targets. The model is initialized from VisionReasoner-3B and trained with GRPO using format, localization, and motion rewards. TAPO contrasts the likelihood under the observed clip with that under a clip repeating the reference frame. Evaluation uses RMOT26, a diagnostic dataset assembled from existing tracking datasets. The paper reports 0.30 MCP, 0.75 MOTP, 44.61-pixel CLE, and 0.39 NDE. It also includes component ablations, a reasoning-output ablation, and an exploratory four-clip evaluation at 50 frames.
Critical strengths
- The task definition is explicit about what the model is given: initial boxes specify target membership, and all clip frames are available offline. This distinguishes the task from language-only target discovery and online tracking. A controlled benchmark for propagating these explicitly grounded targets could help assess VLMs that generate coordinates.
- The ablations show effects that an overall baseline comparison would miss. In Figure 4, GRPO raises MOTP from 0.44 to 0.65. Adding the MCP reward raises MCP from 0.22 to 0.25 but reduces MOTP to 0.61, and the full combination reaches 0.30 MCP and 0.75 MOTP. Appendix Table 9 removes structured reasoning, directly testing its contribution beyond a correlation between rationale quality and accuracy. These controls are useful, subject to the evaluation concerns below.
Critical weaknesses
- The RMOT26 description does not yet support an independent, reproducible test set. Appendix B describes a sequence-level split, but Table 3 lists hundreds of “sequences” from MOT16 and MOT17. These appear to be sampled clips rather than original source videos. Section 6 leaves source-video-disjoint splits to future work. Disjoint clips can still share video content and identities, and MOT17 extends the MOT16 videos. Checking overlap separately by dataset name would therefore be insufficient. This does not establish leakage, but the description does not exclude it. Clip lengths are also inconsistent: Table 4 lists 1,000 test sequences and 4,008 frames, giving 4.008 frames per sequence. Appendix B.1 instead describes a reference frame followed by 5–6 future frames, while Appendix C describes 5–6-frame evaluation. The source-video mapping, frame ranges, and counting convention are needed to interpret the benchmark.
- The evaluation MCP definition needs clarification, and the evidence for query and temporal use is incomplete. Main-text Eq. (8) defines the training reward as max(0, A)SP, while Appendix Eqs. (13)–(14) define evaluation MCP as AS. These can serve different purposes. The concern is the [0,1] range stated in Appendix C.1: A is a cosine similarity, so AS can be negative. Under Eq. (7), zero ground-truth displacement also gives A = 0, including when P = 1. Please clarify the evaluator’s handling of negative alignment and stationary targets. The introduction also announces a fixed-box language intervention, but I could not locate its quantitative results in the main paper or appendix. Figure 10 removes the initial box, and Table 9 removes structured reasoning. Neither changes query content while keeping boxes and video fixed. The TAPO training ablation is useful, but sensitivity to the input alone provides limited evidence that motion is used correctly. The current results therefore leave the query-use and temporal-use claims only partly supported.
Questions for authors
- Please provide a train/test manifest mapping each RMOT26 clip to its original video and frame range. Were original videos and identities kept disjoint, including duplicates shared by MOT16 and MOT17? How does the stated 5–6-frame protocol reconcile with 4,008 frames across 1,000 test sequences?
- Appendix C.1 defines evaluation MCP as AS with a [0,1] range. Is A clamped or transformed in the evaluator, and how are stationary ground-truth intervals scored? Please identify the implementation used for Tables 1–2 and Figure 4. Do the reported evaluation metrics use the Hungarian matching, normalization, and invalid-output penalties described for the rewards in Sections 3.3 and 5? How is the supplied reference frame treated?
- Where are the fixed-box language-intervention results announced in the introduction? Is there an evaluation comparing original clips with reference-frame repetition while holding the initial boxes and output protocol fixed? Please identify which experiments have been completed and which are proposed.
- Algorithm 1 and Eq. (11) specify a sampled log-ratio estimate of the KL value. Could you provide the corresponding loss or gradient pseudocode? Since the sampling policy depends on the parameters, differentiating the expected KL includes a sampling-distribution term. Please clarify how the implementation handles this dependence, including any score-function, importance-weight, or stop-gradient operations.