Hold the Evidence, Then Look Back: Resolving Identity Conflicts in Memory-Based Visual Object Tracking

Memory-based visual object trackers built on promptable segmentation models still confuse identities when similar objects overlap, occlude one another, or return after long absences. These trackers typically decide such conflicts in the frame where they appear, although the overlap that reveals a conflict usually follows the identity error and the evidence that resolves it arrives during and after the conflict. We present PCIA-Track, a training-free, plug-in framework built on one principle: keep the evidence during a conflict and decide the identity after it. Specifically, Hold-and-Report (HaR) holds the memory of a conflicting track but keeps reporting its prediction, and relocates the target among class-agnostic proposals when that prediction duplicates another object. Post-conflict identity arbitration (PCIA) waits until the tracks separate, replays the buffered video backward from two fresh, memory-free probes, and hands the identity over only when the candidate, and not the incumbent, reconnects to the incumbent's last stable window. A small-target confidence floor (SCF) removes spurious outputs of vanished small targets. PCIA-Track improves on SAM 3 on LVOSv2, LVOSv1, VOST, DiDi and LaSOT_ext, where it is the best of the compared trackers, and reaches a quality score of Q = 0.735 on VOTS2026, above the best entry of the challenge. It keeps the model weights and the architecture unchanged.