acceptodds
Under review as a conference paper at ICLR 2027

TrackEVT: Dual-State Target Maintenance for Embodied Visual Tracking

Abstract

Embodied visual tracking (EVT) requires an agent to continuously follow a designated moving target from egocentric visual observations. However, high-confidence localization alone cannot confirm that the tracker still follows the designated target. An identity error can steer the agent away and be reinforced by later observations. We refer to this closed-loop failure as target-state takeover. To address it, we propose TrackEVT, which maintains two complementary states. A dense pixel state provides target geometry for visual servoing. In parallel, persistent target and distractor trajectories check whether the propagated pixels still belong to the selected target. When the states disagree, identity trajectories cannot overwrite the control state directly. Instead, identity-constrained rebinding verifies a current-frame target mask before reinitializing pixel propagation. TrackEVT combines pretrained visual components with explicit control and requires no task-specific training. Experiments cover several simulated scenes, target categories, and a real-world deployment. On established benchmarks, TrackEVT achieves the highest mean scores among the compared methods: a mean success rate of 0.97 on Gym-UnrealCV and a mean tracking success rate of 0.98 for aerial vehicle tracking on DAT. A targeted identity stress test further shows that persistent trajectories and verified rebinding improve identity preservation and recovery.

Primary area:
applications to robotics, autonomy, planning
Keywords:
embodied visual tracking, identity preservation, dual-state target maintenance, visual servoing, training-free tracking

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_6g81OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TrackEVT addresses cases where a robot tracks a region confidently but follows the wrong instance. A propagated pixel mask drives control, while stored target and distractor trajectories check identity. Before correcting the tracker, the system verifies a target mask in the current frame and then reinitializes propagation. The evaluation covers three simulation benchmarks, a same-clothing distractor test and a real robot, without task-specific training.

Critical strengths
  1. Checking target identity while the tracker remains confident is a useful idea. Separating control from identity memory gives the system a way to correct persistent tracking of the wrong person.
  2. The experiments cover people, vehicles, simulation and a real robot. The Cutie-safe comparison and fixed-step rerun are useful controls and make the reported improvements more convincing (Tables 5 and 9).
Critical weaknesses
  1. Table 5 provides useful evidence for rebinding. I would still like to know how reliably the audits detect identity errors while pixel confidence remains high. The paper does not report how many such errors are caught, or how often an accepted correction selects the wrong instance. These measurements would directly support the claimed benefit of identity auditing and verified rebinding.
  2. The smaller ablation gains need uncertainty estimates. On Cross-ID, the full method succeeds in seven more episodes than Cutie-safe: 66/90 versus 59/90. Paired results on the same seeds would help judge how consistent this gain is. The calculation of the confidence intervals in Table 7 is also unspecified.
Questions for authors
  1. How many identity errors do the audits detect while pixel confidence remains high? Among accepted corrections, how many recover the original target and how many select a distractor?
  2. Could you provide paired per-seed results for Tables 5 and 9 and explain how the confidence intervals in Table 7 were calculated? For Crossing IDSW and PN-Rec@30, could you provide uncertainty estimates that account for repeated crossings within the same episode? If the additional results show that the audits make reliable corrections and the gains hold in the paired analysis, I would raise my rating from 2 to 3.
Reviewer_dWFWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TrackEVT, a novel dual-state target-maintenance framework for embodied visual tracking. By decoupling the executable pixel state from the persistent identity trajectory state (used for background auditing and identity verification), the method effectively addresses the closed-loop failure in EVT. Extensive experiments across Gym-UnrealCV, DAT, EVT-Bench, and real-world Unitree Go2 Pro robot deployments demonstrate good performance and robustness without task-specific training.

Critical strengths
  1. Solid Evaluation. The paper conducts extensive experiments across diverse simulation benchmarks (Gym-UnrealCV, DAT, EVT-Bench) and achieves state-of-the-art performance on Gym-UnrealCV and DAT. Furthermore, real-world deployment on a Unitree Go2 Pro robot strongly validates the practical efficacy and robustness of the proposed framework.
  2. Clear Motivation and Well-Designed Architecture. The core concept, decoupling control from identity verification via an asynchronous auditing mechanism and an identity-constrained rebinding strategy, is extremely well-motivated. It effectively tackles a critical bottleneck of applying pre-trained perception models in closed-loop embodied tasks while maintaining high real-time efficiency.
Critical weaknesses
  1. Limitation on Control and Planning. The framework relies on a basic PID-based visual servoing controller, which performs well on benchmarks with minimal planning demands but limits its performance ceiling in complex scenarios requiring long-range navigation (such as EVT-Bench). We suggest the authors explicitly discuss this control limitation in the manuscript and evaluate the potential upper bound of performance by integrating advanced local planning modules.
  2. More Evaluation on Diverse Target Categories. The current experiments primarily focus on human tracking and specific aerial/ground vehicles, leaving the generalization capability across broader, unseen object categories or extreme lighting and weather conditions unexplored. We suggest the authors discuss or provide more results regarding category generalization and environmental robustness.
Questions for authors
  1. Limitation on Control and Planning. The framework relies on a basic PID-based visual servoing controller, which performs well on benchmarks with minimal planning demands but limits its performance ceiling in complex scenarios requiring long-range navigation (such as EVT-Bench). We suggest the authors explicitly discuss this control limitation in the manuscript and evaluate the potential upper bound of performance by integrating advanced local planning modules.
  2. More Evaluation on Diverse Target Categories. The current experiments primarily focus on human tracking and specific aerial/ground vehicles, leaving the generalization capability across broader, unseen object categories or extreme lighting and weather conditions unexplored. We suggest the authors discuss or provide more results regarding category generalization and environmental robustness.
Reviewer_xMz4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TrackEVT addresses identity drift in embodied tracking by separating the target estimate used for immediate control from a longer-term record of target and distractor trajectories. This record helps detect identity conflicts, while verified observations allow the system to correct and restart target propagation. The approach combines pretrained components without task-specific training. Simulation benchmarks, targeted identity-conflict tests, and robot trials report improved target maintenance, with matched ablations examining the contributions of persistent trajectories and verified recovery.

Critical strengths
  • Table 5 isolates persistent trajectories against Cutie-safe, which already retains anchor verification and current-frame recovery: Cross-ID completion increases from 0.656 to 0.733 and identity switches decrease from 0.429 to 0.307. Appendix E additionally preserves the ablation ordering under fixed simulator timing (Table 9), strengthening the causal evidence.
  • The approach transfers across people, vehicles, and a physical robot. Tables 1–2 report mean Gym-UnrealCV success of 0.97 and strong DAT target retention, while Table 7 directly connects the Cross-ID improvement to identity recovery rather than completion alone.
Critical weaknesses
  • The central verification mechanism remains partly operationally unspecified in the manuscript: association weights/gates, sufficient-history criteria, anchor-reservation thresholds, and reference acceptance rules are not numerically defined in Sections 3.3–3.4 or Appendices A–B. Table 6 supplies several implementation settings but not these decisive rules, limiting independent assessment of what makes rebinding trustworthy.
  • The real-world success rates lack explicit trial duration and success/failure criteria (Section 4.3; Appendix C.1). This limits interpretation of the 85%/100% results, particularly because EVT-Bench distracted tracking still has 28.9% success and 14.5% collisions (Table 3), a limitation the authors acknowledge.
Questions for authors
  1. Clarification: Where are the exact association and rebinding acceptance rules specified, including thresholds and positive-reference updates, and how were these selected before evaluation?
  2. Clarification: What defines robot-trial success, how long are trials, and were target trajectories and control interfaces matched across methods?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.