TomoDETR: Point-Set Prediction with Cross-Window Refinement for Cryo-ET Particle Localization

Particle localization is a critical step in cryo-electron tomography (cryo-ET), providing the coordinates needed for downstream structural analysis. State-of-the-art supervised methods have achieved strong localization performance by first predicting dense particle masks or response maps and then extracting center coordinates. Yet the target itself is a labeled set of particle centers. We therefore develop TomoDETR, a DETR-based framework that formulates particle localization as point-set prediction from point annotations. Large tomograms require overlapping-window inference, causing the same particle to be predicted repeatedly from different window contexts. We find that these repeated observations differ in localization quality, while classification confidence does not reliably identify the most accurate one. Motivated by this finding, TomoDETR incorporates a Cross-Window Query Refiner (CWQR), which learns interactions among nearby candidates from different windows to jointly refine candidate centers and class scores before final selection. On the CZII particle localization benchmark, TomoDETR achieves the highest weighted F4 among the evaluated methods. Under a shared backbone and matched training setup, direct point-set prediction improves weighted F4 by 2.30 points over dense prediction with center extraction, while CWQR provides a further 2.85-point improvement with only 2.9% measured inference-time overhead. Additional evaluations on cellular and simulated tomograms further support the effectiveness of cross-window refinement. Our code will be released publicly upon acceptance.