Reliable Identity Memory for RGB-T Multi-Object Tracking

RGB-thermal multi-object tracking aims to localize multiple targets and preserve their identities across paired visible and thermal videos. Despite effectively fusing multimodal information, existing methods struggle to maintain consistent identity associations over time. This challenge is particularly acute for tiny objects, whose weak spatial evidence often leads to intermittent detections, disrupting temporal continuity and preventing reliable identity association. To overcome this challenge, we introduce Reliable Identity Memory Tracking (RIMTracking), a joint detection-and-tracking framework that preserves identity consistency by updating trajectory memory only with reliability-screened observations. Specifically, we employ an Observation-Filtered Identity Memory (OFIM) module to recursively consolidate cross-frame observations into a temporally evolving trajectory representation for stable identity association under changing modality reliability. We further develop Scale-Adaptive Cross-Modal Detail Refinement (SCDR), which sparsely samples fine-grained visible and thermal features and injects the aggregated evidence into object queries according to target scale, enhancing small-target localization. Extensive experiments and comparative analyses on VT-Tiny-MOT and VT-MOT demonstrate that our method achieves state-of-the-art performance while consistently improving identity preservation and tiny-object tracking.