- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper presents a 6D pose tracking framework named EgoTrack in extreme visual degradation scenarios. This framework retains multiple pose hypotheses and passes them to the diffusion model for time trajectory reconstruction, thereby addressing the issue of unreliable single-frame estimation in extreme visual degradation scenarios. In the EgoXtreme benchmark test, this method outperforms existing methods significantly.
Critical strengths
-
The motivation of this paper is very clear, and it addresses the issue that the performance of traditional 6D object pose tracking significantly deteriorates under extreme degraded conditions.
-
The quantitative results of the paper have been significantly improved, and the components have been thoroughly validated through ablation studies.
Critical weaknesses
-
The 6D object pose estimation has high requirements for real-time performance in practical applications, but this paper does not evaluate the computational efficiency of the framework at all. It is suggested to conduct experiments on the inference time, memory usage, and the trade-off between accuracy and efficiency.
-
The paper claims that the NULL marker enables the model to learn to reject unreliable candidates, but this mechanism lacks direct verification. The ablation results in Table 4 also show that the AUC improves after adding NULL, but 5 out of 6 RPE metrics decrease, and its actual help for temporal consistency remains unclear.
-
When EgoTrack is compared with the baseline, both the candidate generation method and the model architecture change simultaneously. It is impossible to determine whether the advantage shown in Table 1 is due to the proposed reasoning method. It is suggested to conduct additional control variable experiments, comparing different models under the condition of fixed candidate inputs.
-
This paper only used the EgoXtreme dataset in the quantitative experiments and did not verify its generalization ability on different datasets; moreover, all the experiments relied on real object masks, but in the case of severe visual degradation, the segmentation itself might fail. It is suggested to supplement the experiments by introducing predicted masks or adding segmentation noise, and to conduct validations on more datasets.
Questions for authors
Please see the weaknesses.
8 comments
The paper claims that the NULL marker enables the model to learn to reject unreliable candidates, but this mechanism lacks direct verification. The ablation results in Table 4 also show that the AUC improves after adding NULL, but 5 out of 6 RPE metrics decrease, and its actual help for temporal consistency remains unclear.
Response:
To examine NULL's role more directly, we analyzed its attention weights and its effects on pose accuracy and temporal consistency under different candidate-quality conditions.
NULL behavior under candidate uncertainty. We divide frames into two groups based on the best Top-K candidate's ADD(-S) error: reliable (≤0.3d) and unreliable (>0.3d). NULL attention is higher in the unreliable group across all three EgoXtreme scenarios. For example, its average weight increases from 0.0815 to 0.1375 on Sports and from 0.1592 to 0.2511 on Maintenance.
Contribution to pose accuracy. Adding NULL improves ADD(-S) AUC in the unreliable group by 2.20, 5.88, and 1.48 percentage points on Sports, Maintenance, and Emergency, respectively. The improvement is larger than in the reliable group for Sports and Maintenance, suggesting that NULL helps reduce reliance on inaccurate candidate evidence.
Temporal consistency. NULL participates in state-conditioned, per-frame candidate aggregation at each denoising step, before temporal modeling. It provides a learned alternative to unreliable candidate evidence but does not explicitly regularize inter-frame motion. While NULL improves AUC, RPE-T and RPE-R do not consistently improve. We therefore clarify that NULL primarily supports pose reconstruction, while temporal dependencies are modeled by the trajectory denoiser, without claiming that NULL directly improves temporal consistency.
NULL analysis on EgoXtreme-Sports.
| Top-K quality | NULL weight | AUC w/o ↑ | AUC w/ ↑ | RPE-T w/o ↓ | RPE-T w/ ↓ | RPE-R w/o ↓ | RPE-R w/ ↓ |
|---|---|---|---|---|---|---|---|
| Oracle ≤0.3d | 0.0815 | 49.48 | 50.75 (+1.27) | 0.0728 | 0.0774 | 5.24 | 5.29 |
| Oracle >0.3d | 0.1375 | 41.58 | 43.78 (+2.20) | 0.0526 | 0.0542 | 5.53 | 5.98 |
NULL analysis on EgoXtreme-Maintenance.
| Top-K quality | NULL weight | AUC w/o ↑ | AUC w/ ↑ | RPE-T w/o ↓ | RPE-T w/ ↓ | RPE-R w/o ↓ | RPE-R w/ ↓ |
|---|---|---|---|---|---|---|---|
| Oracle ≤0.3d | 0.1592 | 28.56 | 30.70 (+2.14) | 0.0077 | 0.0081 | 4.27 | 4.44 |
| Oracle >0.3d | 0.2511 | 16.70 | 22.58 (+5.88) | 0.0090 | 0.0093 | 5.09 | 5.70 |
NULL analysis on EgoXtreme-Emergency.
| Top-K quality | NULL weight | AUC w/o ↑ | AUC w/ ↑ | RPE-T w/o ↓ | RPE-T w/ ↓ | RPE-R w/o ↓ | RPE-R w/ ↓ |
|---|---|---|---|---|---|---|---|
| Oracle ≤0.3d | 0.1954 | 37.70 | 39.54 (+1.84) | 0.0154 | 0.0154 | 2.85 | 3.00 |
| Oracle >0.3d | 0.2152 | 14.33 | 15.81 (+1.48) | 0.0231 | 0.0226 | 4.65 | 4.31 |
Candidate quality is determined using GT poses for post-hoc analysis only. GT information is not used to compute NULL attention during inference. "w/o" and "w/" denote configurations without and with NULL, respectively. Values in parentheses indicate AUC improvements in percentage points.
We will include this analysis in the appendix and clarify NULL's role in the main paper.
When EgoTrack is compared with the baseline, both the candidate generation method and the model architecture change simultaneously. It is impossible to determine whether the advantage shown in Table 1 is due to the proposed reasoning method. It is suggested to conduct additional control variable experiments, comparing different models under the condition of fixed candidate inputs.
Response:
Isolating Stage 2. We evaluate EgoTrack using Clean Aug-1 Rank-5, which disables learned augmentation selection and uses GigaPose's Top-5 hypotheses from the unaugmented template bank. This configuration achieves ADD(-S) AUCs of 43.87, 27.07, and 20.70 on Sports, Maintenance, and Emergency, respectively. Adding learned augmentation selection improves these results to 47.07, 27.24, and 23.29, demonstrating that substantial gains persist without the proposed selection policy.
Comparison under a fixed candidate pool. We additionally compare against FreePose (Top-5 Oracle) using the same Selected Top-5 candidate pool. FreePose selects the GT-best hypothesis for initialization, providing a privileged-initialization reference. Despite this advantage, FreePose achieves AUCs of 12.51, 17.38, and 14.08, compared with 47.07, 27.24, and 23.29 for EgoTrack.
Controlled comparisons on EgoXtreme.
| Scenario | Method | AUC ↑ | RPE-T ↓ | RPE-R ↓ |
|---|---|---|---|---|
| Sports | GigaPose Top-1 | 8.33 | 3.6018 | 114.28 |
| Sports | FreePose (Top-5 Oracle) | 12.51 | 1.9399 | 80.30 |
| Sports | EgoTrack (Clean Aug-1 Rank-5) | 43.87 | 0.2729 | 29.49 |
| Sports | EgoTrack (Selected Aug-1 Rank-5) | 47.07 | 0.2721 | 26.45 |
| Maintenance | GigaPose Top-1 | 5.52 | 0.9705 | 91.47 |
| Maintenance | FreePose (Top-5 Oracle) | 17.38 | 0.0958 | 38.55 |
| Maintenance | EgoTrack (Clean Aug-1 Rank-5) | 27.07 | 0.0311 | 19.97 |
| Maintenance | EgoTrack (Selected Aug-1 Rank-5) | 27.24 | 0.0328 | 20.75 |
| Emergency | GigaPose Top-1 | 4.02 | 3.0906 | 92.28 |
| Emergency | FreePose (Top-5 Oracle) | 14.08 | 0.5531 | 34.00 |
| Emergency | EgoTrack (Clean Aug-1 Rank-5) | 20.70 | 0.0675 | 12.90 |
| Emergency | EgoTrack (Selected Aug-1 Rank-5) | 23.29 | 0.0668 | 14.42 |
Clean Aug-1 Rank-5 disables learned augmentation selection. FreePose (Top-5 Oracle) and full EgoTrack share the same Selected Top-5 candidate pool, but FreePose uses a single GT-selected initialization, whereas EgoTrack jointly processes the full candidate set.
Together, these results show that the improvements cannot be attributed solely to candidate generation. Although the FreePose comparison does not provide identical model inputs or training supervision, it controls the underlying candidate pool while preserving each method's refinement architecture. We will clarify these comparisons in the revised manuscript.
Weaknesses
Response:
We thank the reviewer for highlighting the importance of computational efficiency. We conducted additional inference-time and GPU-memory profiling of EgoTrack and the compared baselines.
Inference time and memory. GigaPose requires 63.37 ms/frame (5.777 GiB). Our policy runs before GigaPose and requires 10.20 ms/frame (0.235 GiB), while DDIM-10 diffusion takes 10.08 ms/window (0.056 GiB). Conservatively counting one complete diffusion-window evaluation per frame without amortization, the estimated EgoTrack pipeline cost is 83.65 ms/frame for inference computation, or 143.27 ms/frame including component-level online overhead. These are component-based estimates, not directly measured end-to-end latencies.
For comparison, FreePose refinement requires 532.83 ms/unique object-frame (4.746 GiB), and GoTrack tracking requires 3149.19 ms/frame (1.513 GiB), excluding GigaPose initialization. The profiling scopes differ across methods.
Runtime and memory comparison.
Pipeline times are estimated by summing separately profiled components, with different timing protocols across methods. Pipeline memory values are sums of component peak GPU allocations, not directly measured end-to-end peaks.
These results demonstrate that EgoTrack introduces relatively low computational and memory overhead beyond its GigaPose backbone, although we do not claim real-time operation. We will include the runtime and memory analysis in the revised manuscript.