CARE3R: Contact-Aware Feed-Forward 3D Reconstruction for Surgical Endoscopy

Three-dimensional reconstruction from endoscopic video supports surgical planning, intraoperative navigation, augmented visualization, and robotic assistance. Yet a reconstruction can be accurate pixel by pixel and still place the instrument tip at the wrong distance from the tissue. We show that this instrument–tissue clearance is dissociated from standard geometry metrics: three variants of our model trained with different instrument supervision have near-identical depth, pose, and reconstruction metrics, including the same tissue depth error on our stereo validation clips, yet their contact AUCs range from 0.74 to 0.93; and fine-tuning the prediction heads of Depth Anything 3 on our training data reduces its C3VD depth error sevenfold without improving contact discrimination. To make clearance measurable and learnable, we introduce ESD-Contact, a multimodal ex-vivo endoscopic submucosal dissection (ESD) dataset with synchronized stereo and side views, verified contact/hover labels, and resistance recordings for a subset of clips, together with a contact benchmark that scores contact/hover discrimination from primary-view RGB. We further propose Contact-Aware Supervision (CAS), which supervises the instrument with targets taken from independent measurements rather than from the model's own prediction: a stereo-referenced depth profile along the instrument, and perpendicular tip-to-tissue targets that a triangle construction derives from the side view. CARE3R, trained with CAS jointly across public and in-house datasets, predicts metric-scale depth, pointmaps, camera poses, and instrument segmentation from RGB frames in one feed-forward pass. On the validation clips of ESD-Contact, it reaches a contact AUC of 0.933, compared with at most 0.806 for seven general and surgical 3D baselines, while remaining competitive on conventional depth, pose, reconstruction, and segmentation benchmarks. Code and model weights will be released upon acceptance.