Track2Recon: Persistent 3D Object Detection and Tracking for Dynamic Scene Reconstruction

Instance-level dynamic 3D Gaussian Splatting (3DGS) for autonomous-driving scene reconstruction typically relies on per-frame 3D bounding boxes and persistent track IDs, whereas unlabeled driving logs contain only multi-camera images, LiDAR point clouds, calibration, and ego poses. Recovering persistent 3D objects from such data poses two main challenges: sparse and incomplete LiDAR observations make reliable association between 2D instances and 3D components difficult, while limited viewpoints and occlusion lead to underestimated object sizes and shifted centers. To address these issues, we present Track2Recon. Its hull-supported 2D–3D object association measures object-level spatial consistency using the projected envelope of an entire 3D component and combines it with support from actual LiDAR points to determine object assignment. Its complete-geometry recovery for partial observations estimates the full object extent from complementary observations across frames and uses visible-surface anchoring to complete missing geometry primarily toward unobserved regions, thereby jointly recovering object size and center. Track2Recon requires neither manual 3D annotations in the target domain nor additional supervised training. Experiments on PandaSet and Waymo demonstrate strong performance in 3D detection and persistent tracking, and the recovered 3D bounding boxes and track IDs can be directly used for instance-level dynamic scene reconstruction and editing.