acceptodds
Under review as a conference paper at ICLR 2027

SurgURDF-Calib: Learning Robust Geometric Optimization for Surgical Robot Camera Calibration

Abstract

Camera-robot extrinsic calibration is essential for aligning surgical robot kinematics with endoscopic observations, yet remains challenging under specular appearance, articulated motion, occlusion, and partial visibility. We introduce SurgURDF-Calib, a geometry-structured learned optimization framework that preserves an analytical SE(3) solver rather than directly regressing camera extrinsics. Given a surgical video, robot joint states, URDF geometry, and an initial extrinsic estimate, we first construct a compact optimization candidate pool using motion diversity and instrument visibility. A learned coarse calibration module then predicts a sequence-level correction to move the estimate into a favorable convergence basin. Starting from this initialization, the second stage performs geometry-guided refinement through observability-aware D-optimal frame scheduling and a learned robust sparse Levenberg-Marquardt (LM) solver, which adaptively estimates geometric factor weights and anisotropic damping while retaining analytical pose updates. We further introduce SurgURDF-100K, a geometry-consistent benchmark containing 1,000 surgical sequences and 100K annotated observations. Experiments on synthetic and real surgical videos demonstrate improved calibration accuracy and robustness under challenging initialization and unreliable observations, together with generalization to unseen instrument geometries.

Primary area:
optimization
Keywords:
Learning-to-Optimize, Geometric Optimization, Robot–Camera Calibration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ofwuOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SurgURDF-Calib, a method for estimating a sequence-level rigid correction to an initial camera-to-robot extrinsic state. The methods works by selecting a 64-frame candidate pool and trains a cores initializer followed by learned robust LM stage. The LM stage augments the classical analytical pose update with a learned per factor reliability weighting, 6D damping vectors, where frames are chosen by a greedy D-optimal schedule. The authors also introduce SurgURDF-100k, a synthetic benchmark with URDF, joint-state and extrinsic annotations. Extensive ablations confirm the design choices and the resulting model performs well on the holdout test set.

Critical strengths
  • Geometry preserving Pipeline: The introduced pipeline combines classical LM update rules with learning based parts to initialize, weight and damp the optimization, making it accurate, robust and interpretable. The pipeline is well motivated and design choices are clearly ablated. The model shows strong performance on synthetic data and generalizes to held-out-instruments.

  • Synthetic benchmark with full calibration annotations: A surgical dataset shipping with URDF, joint states, and camera extrinsics. If this dataset gets released it would enable controlled evaluations of calibration methods and help train future models.

Critical weaknesses
  • Concerns about baseline comparisons: The proposed method takes gt instrument masks as inputs while some baseline consume RGB. It is unclear to me if or how baselines were retrained on SurgURDF-100k. If not retrained, the synthetic benchmark is in domain for the proposed method and out of domain for every baseline and the results could reflect domain gap rather than method performance.

  • Evaluation currently does not quantify robustness vs accuracy: All results are means only. In order to get a real understanding of the convergence of the method, the authors should report success rates or full error distributions.

Questions for authors
  • It is unclear to me whether the baselines are retrained on the SurgURDF-100k dataset. Could you clarify if the baselines are retrained on the dataset and if so how? If not, the synthetic benchmark measures domain gap rather than calibration quality. In that case, please add an evaluation on a public benchmark if possible for a fair comparison to the baselines.

  • In order to get a deeper understanding of robustness and accuracy, it would be helpfull to report sequence distributions: median errors, histograms and success rates at fixed thresholds (e.g. 1°/2 mm, 2°/5 mm) for Table 2 and fig 6.

  • LM100: is the network applied outside its training schedule since its trained up to 64 frames only or is the model retrained for 100 frames? Would this be a more plausible explanation for the degradation? Does classical LM also degrade from 64 to 100 frames?

5 comments
AuthorsOpenReview ↗

Response to Reviewer

Response to Reviewer

We sincerely thank the reviewer for the thoughtful and constructive feedback, particularly for highlighting important concerns regarding baseline fairness and evaluation robustness. We have conducted additional experiments and expanded our statistical analyses to address these concerns. We hope our responses will clarify the evaluation protocol and strengthen your confidence in our work.

W1/Q1. Baseline Retraining and Fair Comparison

We clarify that the task-specific modules of four baselines were retrained on the SurgURDF-100K training split, rather than directly using out-of-domain pretrained models. Specifically:

  • DREAM: The dual-arm keypoint network was trained for 10 epochs.
  • RoboPose and RobotArena∞: The pose-refinement network and instrument renderer were each trained for 10 epochs, respectively.
  • SurgCalib: The SFC and MFC frontends were trained separately for each instrument, for 10 epochs each.
  • EasyHeC: Directly optimizes extrinsics from masks without additional training.
  • CalibAll: Uses frozen pretrained CoTracker3 trajectories with PnP, without additional training.

All methods follow the same training/test split, initialization conditions, camera intrinsics, and evaluation metrics. We acknowledge that different methods retain their respective RGB or mask inputs, so these comparisons evaluate complete calibration systems rather than identical input modalities. The controlled mask-based comparison in our response to R1-W2/Q2 further evaluates the LM refinement stage under identical observation conditions, isolating the contribution of the learned solver.

Furthermore, Table 2 includes evaluations on three public real-world datasets (SurgSync, SutureBot, and ImitateCholec), with all learned weights kept fixed and no additional fine-tuning. We have also expanded the appendix with detailed baseline training and implementation protocols.

W2/Q2. Error Distributions and Calibration Success Rates

We conducted additional statistical analyses to quantify calibration accuracy and robustness, including mean ± standard deviation, median errors, error histograms, and success rates under fixed accuracy thresholds.

1. Error statistics and distributions.

StatisticE_R (°) ↓E_T (mm) ↓E_2D (px) ↓E_3D (mm) ↓E_tip (mm) ↓IoU ↑
Mean ± Std4.844 ± 4.17913.110 ± 11.68510.142 ± 41.7667.304 ± 8.1704.310 ± 7.2960.836 ± 0.177
Median3.5819.6452.8954.7192.1530.894

The median errors are lower than the means, indicating that a subset of challenging cases produces relatively large errors. We have additionally included histograms of all six metrics in the appendix to illustrate their distributions.

2. Calibration success rates.

A calibration is considered successful only when both rotation and translation errors satisfy the specified thresholds.

Method1° / 2 mm ↑2° / 5 mm ↑3° / 10 mm ↑
DREAM6.56%10.28%20.06%
RoboPose3.84%7.16%13.81%
EasyHeC6.31%18.09%28.75%
CalibAll3.88%5.88%9.94%
RobotArena∞2.16%9.38%18.19%
SurgCalib6.06%16.88%25.75%
SurgURDF-Calib (Ours)11.69%23.19%40.97%

Our method achieves the highest success rates across all three thresholds. At 3° / 10 mm, it achieves 40.97%, outperforming the strongest baseline, EasyHeC (28.75%), by 12.22 percentage points.

We have also updated Figure 6 with standard-deviation error bars across initialization perturbation levels. Together, these additions provide a more comprehensive assessment of calibration accuracy, variability, and robustness beyond average performance.

AuthorsOpenReview ↗

Response to Reviewer

Q3. LM-100 Evaluation and Training-Schedule Generalization

We clarify that LM-100 uses the original trained checkpoint without retraining. After the 64-frame stage, it performs one additional refinement update using all 100 frames. The shared factor encoder and masked mean pooling support variable-length observations, while the analytical normal equations remain 6×6.

However, LM-100 extends beyond the four-stage refinement schedule used during training, which may contribute to the observed degradation. We additionally evaluated classical Robust LM under the same frame expansion.

Classical Robust LMLM-64LM-100
Rotation Error (°) ↓7.9598.179
Translation Error (mm) ↓17.42617.539

Classical Robust LM also exhibits a slight performance decrease when extending from 64 to 100 frames. This suggests that additional weakly informative or occluded observations may adversely affect optimization. Nevertheless, since LM-100 changes both the observation set and update count while extending beyond the training schedule, we cannot attribute the degradation to a single factor. We will clarify this limitation in the revised discussion of Figure 5.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer ofwu,

Thank you for your time and constructive feedback on our paper. As the author–reviewer discussion period will end in about one day, we would greatly appreciate it if you could review our rebuttal and consider whether your concerns have been addressed. Thank you again for your time and consideration.

Regards from the authors.

Reviewer_uPY9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduce SurgURDF-Calib, a framework for refining surgical robot camera extrinsics using instrument masks, robot joint states, URDF geometry, camera intrinsics, and an initial calibration. The framework combines motion–visibility-based candidate selection, a learned coarse initializer, fixed D-optimal frame scheduling, and a sparse geometric solver withlearned factor weights and anisotropic damping. Pose increments are obtained through an analytical six-dimensional linear solve. The authors also introduce SurgURDF-100K, a synthetic benchmark containing 1,000 sequences and 100K observations.

Critical strengths
  1. The learned solver addresses unreliable contour constraints while preserving a explicit geometric update. Each factor’s reliability is predicted from its residual, local geometry, Jacobian, and kinematic context, allowing the solver to distinguish constraints that a residual-only robust kernel would treat similarly.

  2. The frame scheduler explicitly considers complementary constraints on extrinsic parameters. Using projection Jacobians derived from URDF geometry, the scheduler prioritizes combinations of frames that constrain different pose directions. This addresses a concrete limitation of visibility-based selection: clearly visible instruments can still provide repetitive or weak geometric constraints.

Critical weaknesses
  1. Real-world calibration accuracy is not independently established. Appendix A.2 explicitly states that the same masks are used for calibration and scoring, and ground truth extrinsics are unavailable. Consequently, real-world IoU measures agreement with the fitting observations; improved silhouette alignment does not necessarily establish accurate depth or all six extrinsic parameters. Temporal motion consistency provides complementary evidence, but remains an indirect metric. This limits confidence that the substantial synthetic accuracy gains translate into comparable real-world calibration gains.

  2. The evidence for robustness to unreliable observations is incomplete. Real-world masks are produced using SAM3 with manual correction, whereas the proposed optimizer relies directly on mask boundaries and mask-derived visibility. Since learned factor reliability is a central contribution, its ability to handle imperfect observations is consequential to assessing the method’s practical significance.

Questions for authors
  1. How are the initial extrinsics and nominal camera-from-root transformations obtained for the real datasets? Is there any independent calibration reference, or evaluation on frames excluded from fitting, that supports accuracy beyond agreement with the calibration masks?

  2. Are results available using automatically generated masks without manual correction, or under controlled boundary errors, missing regions, and occlusion? How does the advantage of learned weighting over classical robust LM change under these conditions?

  3. Since the final stage uses all 64 candidate frames, do the scheduling variants in Table 4 share the same candidate pool, update count, and final observation set? Can the authors clarify whether the gain primarily reflects informative early subsets and optimization order, rather than a different final set of observations?

5 comments
AuthorsOpenReview ↗

Response to Reviewer

Response to Reviewer

We sincerely thank the reviewer for the careful and constructive feedback, particularly for identifying important aspects that were insufficiently clarified in our original submission. We have conducted additional experiments and hope our responses will address your concerns and strengthen your confidence in our work.

W1/Q1. Real-World Initialization and Independent Calibration Evaluation

Initial extrinsics and nominal transforms. For SurgSync, we initialize each arm's camera-from-root transform (B_{0,a}) using the per-case OpenCV hand–eye registration provided by the dataset. For SutureBot and ImitateCholec, which do not provide hand–eye calibration for the evaluated sequences, we use fixed, dataset-specific initial transforms obtained by fitting projected URDF silhouettes to instrument masks in pilot sequences. For these two datasets, we use the SurgSync left-camera intrinsics rescaled to the target resolution. For all three datasets, recorded endoscope poses propagate the initial transform as Bt,a=Et−1E0B0,a,B_{t,a}=E_t^{-1}E_0B_{0,a}, where (E_t) maps the endoscope frame at time (t) into a fixed reference frame. The resulting (B_{t,a}) serves as the nominal camera-from-root transform for sequence-level extrinsic refinement.Disjoint-frame evaluation.** Since independent ground-truth extrinsics are unavailable for these real datasets, we conducted additional evaluations using non-overlapping calibration and evaluation frames.

First, we constructed a long-video test set containing 100 sequences of 400 frames each, calibrating only on frames 1–100 and evaluating independently on frames 101–400.

Evaluation FramesInitial IoUCalibrated IoUIoU Gain
1–100 (Calibration)0.18640.8020+0.6156
101–400 (Held-out)0.18640.7993+0.6129

The held-out IoU decreases by only 0.0027, indicating consistent geometric alignment on observations excluded from optimization.

We further evaluated this protocol on three real-world datasets, using only the first 100 frames for calibration and evaluating separately on subsequent frames.

DatasetFull-video Calibration (All Frames)First-100 Calibration (Held-out Frames)First-100 Calibration (All Frames)
SurgSync0.7960.7720.755
SutureBot0.7430.7370.717
ImitateCholec0.7080.6980.662
Overall0.7240.7110.707

Our method achieves 0.711 IoU on held-out real-world frames, compared with 0.724 under full-video calibration. These results provide evidence of geometric consistency beyond the fitted masks. We acknowledge that absolute six-degree-of-freedom calibration accuracy cannot be directly established without ground-truth extrinsics, and will clarify this distinction in the revised manuscript.

W2/Q2. Robustness to Automatically Generated Masks

We clarify that manual correction of SAM3-generated masks was limited to filling small internal holes, without refining mask boundaries. To assess robustness without manual intervention, we additionally evaluated calibration using raw automatically generated masks.

Calibration MasksTuned Classical Robust LM ↑Learned Weighting ↑Full Learned LM (Ours) ↑
With hole filling0.3860.6380.724
Raw automatic masks0.3770.6280.719

All values report IoU (↑).

Both calibration conditions are evaluated using the same reference masks and identical evaluation frames. When calibrated using raw automatic masks, our full method achieves 0.719 IoU, only 0.005 lower than with hole filling. Moreover, learned weighting substantially outperforms tuned classical Robust LM (0.628 vs. 0.377), maintaining nearly the same advantage as with hole filling (0.251 vs. 0.252 IoU).

These results demonstrate that learned factor reliability retains its advantage under the segmentation imperfections present in raw automatic masks, without relying on manual boundary refinement. We will clarify that this evaluation does not separately isolate controlled boundary perturbations or occlusion severity.

AuthorsOpenReview ↗

Response to Reviewer

Q3. Fairness and Effect of D-optimal Frame Scheduling

We clarify that Candidate-uniform, Single-frame Information, Visibility-only, and D-optimal share the same 64-frame candidate pool, initialization, learned LM solver, and four refinement updates with progressively increasing frame budgets (8, 16, 32, 64). All four strategies use the identical final 64-frame observation set. Random-uniform instead samples from the original 100 frames.

To examine the effect of early frame selection, we report the calibration performance after each refinement stage.

Selection StrategyLM-8LM-16LM-32LM-64
Random-uniform21.094 / 0.55918.989 / 0.59017.399 / 0.61916.312 / 0.632
Candidate-uniform20.608 / 0.56118.227 / 0.59916.539 / 0.62315.450 / 0.636
Single-frame Information21.335 / 0.58819.553 / 0.63517.964 / 0.66516.349 / 0.693
Visibility-only19.811 / 0.60417.142 / 0.66214.844 / 0.70312.197 / 0.740
D-optimal (Ours)12.046 / 0.6509.865 / 0.7458.299 / 0.8027.304 / 0.836

Each cell reports E_3D (mm, ↓) / IoU (↑).

D-optimal achieves better alignment from the earliest stage, reaching 0.651 IoU with only 8 frames. Even with the identical final 64-frame observation set, it achieves 0.836 IoU and 7.304 mm E₃D, compared with 0.740 and 12.197 mm for Visibility-only.

Since geometric factors are re-linearized at each refinement stage, informative early subsets influence subsequent optimization updates. These results support that the improvement arises from early frame selection and its effect on the optimization trajectory, rather than differences in the final observation set, under the fixed four-update budget.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer uPY9,

Thank you for your time and constructive feedback on our paper. As the author–reviewer discussion period will end in about one day, we would greatly appreciate it if you could review our rebuttal and consider whether your concerns have been addressed. Thank you again for your time and consideration.

Regards from the authors.

Reviewer_7682OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The work is about calibrating the camera to a surgical robot when an initial extrinsic, a URDF model, joint states, and instrument masks are available. The authors do not regress the extrinsic directly. They keep a damped geometric solver and learn the parts around it: which frames to keep, a coarse pose correction, which frames to schedule for refinement, and how much to trust each geometric factor and each pose dimension. The pose update itself is still solved analytically. This work also release a synthetic benchmark with URDF, joint states, instrument poses, and camera annotations. Experiments compare with several calibration baselines on synthetic sequences and three real surgical video sources.

Critical strengths
  1. Learning only the optimizer controls rather than the pose is sensible: the analytical update stays, and the network only handles frame selection, factor reliability, and damping.

  2. SurgURDF-100K fills a real gap. Existing datasets rarely combine URDF, joint states, and camera extrinsics needed for geometry-guided calibration. Synthetic data cannot replace real annotations, but it gives the community a reproducible starting point.

Critical weaknesses

1.The core ingredients are not very new. D-optimal frame selection is already standard, and learning factor weights or damping around an analytical Gauss-Newton or LM solver has been explored in the learned optimizer and differentiable solver literature. The paper does not really discuss that line of work, so the contribution reads more like a careful engineering combination than a methodological advance.

2.The need for learning is not tested against a strong classical baseline. There is no hand-tuned non-learned baseline with factor-level robust weights and anisotropic damping scaled from the Hessian diagonal. Since the learned scalar damping variant is basically tied with classical robust LM in IoU, I am not convinced the gain comes from learning rather than from the structural prior of factor-level weighting plus anisotropic damping.

3.Related work is incomplete. SurgCalib and the main calibration baselines are covered, but the broader learned optimization, differentiable solver, and classical robust estimation literature is largely missing. The motivation is reasonable, but the paper does not fully defend why learning is necessary over a better robust kernel and a better damping heuristic.

Questions for authors
  1. Could you separate the frames used for calibration from those used for scoring, or at least report results when the masks used for calibration and evaluation are disjoint? This would clarify how much of the real-data IoU reflects genuine calibration quality rather than within-video consistency.

  2. Could you add a hand-tuned baseline with factor-level robust weights and anisotropic damping scaled from the Hessian diagonal, without any learned network? If this baseline comes close to the full learned LM, the argument for learning would need to be reframed around engineering convenience rather than necessity.

  3. The D-optimal schedule uses link-origin Jacobians, while the optimization uses signed-distance residuals. Near singular configurations or heavy occlusion, could the selected high-information frames actually be poor for the final residual? Any statistic linking scheduled frames to residual reduction would help.

  4. Table 5 only gives averages across held-out instruments. Could you report per-instrument variance or the worst held-out instrument? If one instrument is clearly worse, the zero-shot claim should be softened accordingly.

3 comments
AuthorsOpenReview ↗

Response to Reviewer

Response to Reviewer

We sincerely thank the reviewer for the insightful and constructive feedback, which reflects a strong understanding of geometric calibration and learned optimization. We have conducted additional experiments to directly address your concerns and hope that our responses will resolve your doubts and strengthen your confidence in our work.

W1/W3. Methodological Novelty and Related Work

We agree that D-optimal selection and learned optimization are established techniques. Our contribution lies in geometry-conditioned refinement for surgical robot calibration, rather than introducing these components individually. Specifically, our method incorporates SDF residuals, Jacobians, kinematic features, inter-arm depth information, and aggregated context to learn factor-specific reliability beyond conventional robust kernels. It also predicts six anisotropic damping coefficients while retaining analytical LM pose updates.

Unlike generic learned optimization, our formulation explicitly exploits articulated URDF geometry and observation-dependent factor reliability. We will expand the related work to include learned nonlinear optimization (e.g., LS-Net), differentiable solvers (e.g., BA-Net), and classical robust estimation, while clarifying that D-optimal selection is a fixed scheduling component. The necessity of learning is further examined in W2/Q2.

Q1. Disjoint Calibration and Evaluation Frames

We conducted additional evaluations using non-overlapping calibration and evaluation frames. Since SurgURDF-100K contains only 100 frames per sequence, we constructed an additional test set with 100 long sequences of 400 frames each. Only frames 1–100 are used for calibration, while frames 101–400 are reserved for evaluation.

Evaluation FramesInitial IoUCalibrated IoUIoU Gain
1–100 (Calibration)0.18640.8020+0.6156
101–400 (Held-out)0.18640.7993+0.6129

The held-out IoU decreases by only 0.0027, indicating consistent geometric alignment on observations excluded from calibration.

We further evaluated this protocol on three real-world datasets, calibrating with the first 100 frames and evaluating independently on subsequent frames.

DatasetFull-video Calibration (All Frames)First-100 Calibration (Held-out Frames)First-100 Calibration (All Frames)
SurgSync0.7960.7720.755
SutureBot0.7430.7370.717
ImitateCholec0.7080.6980.662
Overall0.7240.7110.707

Our method achieves 0.711 IoU on held-out real-world frames, compared with 0.724 under full-video calibration. These results support generalization beyond the fitted masks, although absolute real-world extrinsic errors remain unavailable without ground-truth calibration.

W2/Q2. Necessity of Learning vs. Classical Robust LM

We implemented the requested hand-tuned classical LM baselines with factor-level Cauchy/Huber weighting and Hessian-diagonal-scaled anisotropic damping, without learned components in the LM refinement. All variants share the same frozen coarse initialization, D-optimal frame schedule, and four LM updates for a controlled solver-level comparison.

MethodE_R (°) ↓E_T (mm) ↓E_3D (mm) ↓IoU ↑
Original Classical Cauchy LM7.95917.42621.8880.467
Tuned Cauchy LM7.43516.92114.4430.521
Tuned Huber LM7.39516.88514.3720.525
Full Learned LM (Ours)4.84413.1107.3040.836

The strongest tuned classical baseline improves IoU from 0.467 to 0.525, confirming the benefit of improved robust weighting and anisotropic damping. Nevertheless, our learned solver achieves 0.836 IoU (+0.311) under the same refinement budget. Furthermore, Table 4 shows that learned scalar damping alone achieves 0.467 IoU, whereas learned factor weighting reaches 0.803. These results demonstrate that context-dependent learned factor reliability contributes beyond the tested classical weighting and damping strategies.

AuthorsOpenReview ↗

Response to Reviewer

Q3. Relationship Between D-optimal Scheduling and SDF Residual Reduction

We acknowledge that link-origin Jacobians approximate geometric observability rather than directly measuring SDF residual sensitivity. Under severe occlusion or near-singular configurations, high-information frames may therefore provide weak SDF constraints.

To examine this relationship, we additionally measure SDF residual reduction at refinement stage (k) as

ΔRk=RF(T^k−1)−RF(T^k),\Delta R_k=R_{\mathcal F}(\hat T_{k-1})-R_{\mathcal F}(\hat T_k),

where (R_{\mathcal F}) denotes the SDF residual evaluated on the same fixed observation set (\mathcal F) across scheduling strategies, and (\hat T_{k-1}) and (\hat T_k) are the estimates before and after the stage. We report (\Delta R_k) for the 8-, 16-, 32-, and 64-frame refinement stages, with total reduction (\sum_k\Delta R_k).

Selection StrategyΔR(8) ↑ΔR(16) ↑ΔR(32) ↑ΔR(64) ↑Total ΔR ↑
Random-uniform1.9390.4990.2220.1142.774
Candidate-uniform1.9450.5460.2220.1112.824
Single-frame Information2.0270.5210.2320.1162.896
Visibility-only2.0940.5390.2390.1232.995
D-optimal (Ours)2.4330.6030.2550.1283.419

D-optimal achieves the largest SDF residual reduction at every stage, with a cumulative reduction of 3.419, exceeding Visibility-only (2.995) by 14.2% and Random-uniform (2.774) by 23.3%. Moreover, D-optimal and Visibility-only share the same final 64-frame observation set, yet achieve final IoUs of 0.836 and 0.740, respectively. These results support the effectiveness of information-guided early frame selection for SDF-based optimization, while not guaranteeing optimal selection under every degenerate configuration.

Q4. Per-Instrument Variance and Worst-Case Generalization

We additionally report mean ± standard deviation across evaluation sequences for all five held-out instruments under the leave-one-instrument-out protocol, together with aggregate statistics across all held-out instruments.

Held-out InstrumentE_R (°) ↓E_T (mm) ↓E_2D (px) ↓E_3D (mm) ↓E_tip (mm) ↓IoU ↑
Large Needle Driver6.96 ± 5.3717.96 ± 13.7315.62 ± 45.6810.79 ± 10.136.63 ± 9.160.771 ± 0.232
Maryland Bipolar Forceps5.62 ± 4.4816.09 ± 14.1114.66 ± 22.698.90 ± 8.525.40 ± 7.040.775 ± 0.197
ProGrasp Forceps6.22 ± 5.0416.65 ± 13.4611.50 ± 30.879.48 ± 9.515.92 ± 8.470.781 ± 0.181
Cadiere Forceps6.94 ± 4.9818.05 ± 13.6621.73 ± 27.7911.34 ± 10.487.60 ± 9.890.752 ± 0.210
Medium-Large Clip Applier5.12 ± 4.6014.94 ± 13.4010.64 ± 35.748.53 ± 9.005.27 ± 7.710.828 ± 0.171
All Held-out Instruments6.17 ± 4.9616.74 ± 13.7214.83 ± 37.339.81 ± 9.626.16 ± 8.560.782 ± 0.201

Cadiere Forceps is the most challenging held-out instrument, exhibiting the lowest mean IoU (0.752) and highest mean 2D alignment error (21.73 px), 3D alignment error (11.34 mm), and tip error (7.60 mm). Nevertheless, all five unseen instruments achieve mean IoUs between 0.752 and 0.828, with an aggregate IoU of 0.782 ± 0.201, without target-instrument training. These results demonstrate cross-instrument transferability while revealing instrument-dependent variability and sequence-level performance fluctuations. We will clarify these limitations rather than claiming uniformly reliable performance across all unseen geometries.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer 7682,

Thank you for your time and constructive feedback on our paper. As the author–reviewer discussion period will end in about 2 hours, we would greatly appreciate it if you could review our rebuttal and consider whether your concerns have been addressed. Thank you again for your time and consideration.

Regards from the authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.