Dense-Temporal Surgical Video Object Segmentation via Agentic Trajectory Recovery

High-frame-rate surgical video object segmentation (VOS) exposes finer-grained intraoperative dynamics over densely sampled sequences. Existing VOS methods often fail to detect unreliable propagation and recover once tracking drift contaminates memory. We propose SurgATR, an Agentic Trajectory Recovery framework for dense-temporal surgical VOS. SurgATR first performs trajectory reflection to identify propagation failures from recent target evolution. Once triggered, an agent directly intervenes in both spatial and temporal tracker states: spatially, it re-anchors the target with corrective prompts; temporally, it rolls back and reconstructs tracker memory from a reliable state to prevent error accumulation. To avoid incorrect recovery toward visually similar distractors, cycle validation further filters identity-inconsistent recoveries, while global Viterbi selection determines a globally coherent trajectory. To support systematic study of dense-temporal surgical VOS, we introduce SurgDVOS, the largest high-frame-rate surgical VOS benchmark, with 330,246 annotated frames and 869,922 pixel-level masks, supporting both mask-guided and referring VOS. Experiments on SurgDVOS and three public natural-video benchmarks show consistent improvements, with SurgATR achieving state-of-the-art performance in mask-guided and referring VOS while demonstrating strong cross-domain generalization.