- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
InstantRefine is a one-reference 6D pose refinement pipeline: given a query RGB-D image, a reference RGB-D image, masks, and an initial coarse pose, it back-projects and samples both views, transforms the query points into a Normalized Anchor Frame (NAF), tokenizes each view by concatenating some off-the-shelf features, and feeds the resulting point tokens into a Transformer. Two linear heads regress the delta rotation and delta translation, and the pose is updated iteratively. The core contribution is a Transformer-based delta-pose estimation network operating on NAF-anchored tokens, trained on synthetic data and evaluated on LINEMOD, YCB-V, and TUD-L against ICP, UNOPose, COG, and FoundationPose.
Overall, the pipeline is coherent and the results are promising, but the paper still contains some unclear claims and analysis. The multi-reference contribution is not discussed in detail. The two core components, NAF and tokenization, are defined rather than analyzed, with no ablation over normalization schemes, token counts, or individual feature sources. The training procedure is underspecified—the loss, rotation parameterization, and data scale are missing. Overall, several issues are fixable with additional experiments, but the multi-reference claim and the ablation inconsistency affect core credibility.
Critical strengths
1.A complete, modular one-reference refinement pipeline that removes reconstruction, explicit matching, and geometric solvers from the loop. 2. An end-to-end Transformer delta-pose network with decoupled [R]/[T] tokens and 4D positional encoding, naturally extensible to extra reference views. 3. Credible single-reference gains over correspondence- and reconstruction-based refiners, with reconstruction overhead eliminated, across three datasets and three upstream coarse-pose sources.
Critical weaknesses
-
Pipeline, Network, and Training Are Only Partially Presented The training objective is not mathematically specified: the rotation parameterization (6D/quaternion/axis-angle), whether the rotation loss is geodesic, and what λmax = 10 actually weights are all missing. The target delta pose is expressed in NAF (Eq. 4, Fig. A7), but the network's output parameterization within NAF—specifically how the 6-vector output maps to the NAF translation component—is not specified. The scale of the training data is never quantified: there are no object/scene/view-pair counts, no split, and no deduplication, so the 400K iterations cannot be judged. Moreover, the training data are synthetic, and there is no discussion of whether the synthetic-to-real gap influences performance; no mixed or adapted training is reported. The single-view vs. multi-view recipe is unclear: the single-reference setting is "trained with V = 2," whereas the multi-view setting uses V = 2–9; yet testing uses 16 references with no explanation.
-
NAF and Tokenization Are Defined, Not Analyzed Sensitivity is reported only at the end-to-end level (Table A6 for SAM masks, Fig. A9 for depth noise), not at the level of NAF's center and diameter estimates. How d and the frame origin respond to mask misses, depth bias, and missing regions is not analyzed; the robustness boundary of the frame itself is never discussed. NAF–initial-pose coupling is quantified only through binned analysis over COG's natural error distribution (Fig. A10); there is no controlled sweep over initial rotation/translation error, so the rotational error threshold beyond which refinement degrades is unknown. There is no ablation over alternative normalization schemes (unit sphere, PCA, learned). The per-view token count (128) has no ablation or justification, and the per-view budget is never varied. Multi-view token scaling is not discussed; 16 references would yield ~2210 tokens with quadratic attention cost, yet Table 2 reports FPS and memory without decomposing the token-count contribution to accuracy or cost. Tokens are the core of the pipeline; Table 3 provides only partial evidence—B1 removes PointNet geometry while DINOv2/SD-VAE remain, and A1 removes SD-VAE—but there is no independent removal of DINOv2 and no controlled comparison of feature-source contributions under a fixed token budget. Which of PointNet, DINOv2, or SD-VAE contributes most is therefore not answered and should be discussed.
Questions for authors
-
Are the 16-reference results (Tables 1–2) produced by the same model trained with V ≤ 8, or by a separately trained multi-view model? If the former, what allows the model to handle 16 references unseen during training (e.g., permutation-invariant attention over views)? If the latter, please give its training configuration. This directly determines whether the multi-reference contribution stands.
-
In Table 3, A1 (no SD-VAE) leaves YCB-V ADD(-S) unchanged and drops AR by only 0.3, and D1 (entangled translation) differs from Full by ≤1.5 AR, yet the text claims "larger degradation" and a "consistent performance drop." Is this a table error, a metric error, or overstatement? If the gaps are within noise, please report multi-seed variance.
-
Please state the training data scale and model capacity of the UNOPose, COG, and FoundationPose pretrained models, and whether they are comparable to InstantRefine. Can a same-class direct-regression refiner (DeepIM-style or an adaptation with the same interface) be added?
-
Please explain the basis for 128 per-view tokens, and whether token count versus accuracy and attention cost was analyzed at ~2210 total tokens; whether PointNet, DINOv2, and SD-VAE contributions can be isolated under a fixed token budget, since Table 3's B1/A1 only partially separate them.
-
Please state whether any analysis shows the synthetic-to-real gap is not a bottleneck (e.g., synthetic-only vs. mixed training). If no such data exist, this is future work and will not count against the current assessment.