acceptodds
Withdrawn from ICLR 2027

ASTRA: A Large-Scale Multi-Satellite Benchmark for Onboard 6-DoF Spacecraft Pose Estimation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurate six-degree-of-freedom (6-DoF) pose estimation of non-cooperative satellites is critical for on-orbit servicing, assembly, and manufacturing missions, yet progress is bottlenecked by a severe scarcity of labeled in-orbit imagery and by the narrow target coverage of existing public datasets, most of which focus on only one or two spacecraft. We introduce ASTRA, a large-scale multi-satellite benchmark for onboard 6-DoF pose estimation that spans twenty structurally diverse targets drawn from NASA reference CAD models. ASTRA pairs every target with two complementary data regimes under a common annotation format, comprising an independent-frame training subset rendered under broad parameter sweeps for diverse pose coverage and a sequential test subset of rendezvous trajectories. To close the sim-to-real gap without relying on physical hardware-in-the-loop (HIL) facilities, ASTRA is released with two orthogonal real-style generation pipelines. FRESCO is a neural operator that performs band-selective Fourier amplitude mapping while preserving pose-critical geometry, and a world-model pipeline based on NVIDIA COSMOS turns sequential edge maps from synthetic renderings into photorealistic frames with diverse space environments.

Primary area:
datasets and benchmarks
Keywords:
sim2real, spacecraft pose estimation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LQEzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces a multi-satellite benchmark for 6-DoF spacecraft pose estimation, covering 20 spacecraft with synthetic training images and sequential test trajectories. It incorporates two real-style generation pipelines, FRESCO and COSMOS, and evaluates several pose estimators across different visual domains and embedded platforms.

Critical strengths
  • The paper addresses the limited target diversity and realistic evaluation of existing spacecraft pose estimation benchmarks.
  • The paper provides a unified dataset covering multiple spacecraft, camera configurations, and sequential trajectories, offering useful resources for evaluating spacecraft pose estimation methods.
  • The experiment is relatively comprehensive, covering multiple pose estimation architectures, synthetic and generated visual domains, and onboard-oriented inference efficiency.
Critical weaknesses
  • Some results in Table 3 appear inconsistent with Eq. (3), the reported unified pose errors are smaller than the corresponding rotation errors converted to radians, even though the unified metric includes an additional non-negative translation term.
  • Preserving Fourier phase does not necessarily guarantee unchanged image geometry, and edge-conditioned generation may introduce structural deviations.
  • The evidence for real-world generalization remains limited, since most evaluations are conducted on synthetic or generated images, while the real-image comparison includes only 37 Hubble photographs without pose accuracy evaluation.
  • The architecture description of SPNv3 differs between the main text and Appendix F. The FRESCO training procedure and spectral filter constraints should also be specified more clearly to support reproducibility.
Questions for authors

Please refer to the Weaknesses.

Reviewer_7LPJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ASTRA, a UE5 benchmark for monocular 6-DoF spacecraft pose estimation built from 20 NASA 3D Resources models. It has a 1M-frame independent training split and a sequential test split (per target 2 ROE scenarios x 10 trajectories x 240 frames x 2 FOVs), with pose and up to 39 keypoints. Instead of HIL data the authors produce two "real-style" test sets: FRESCO, a learned band-limited Fourier amplitude filter trained on SHIRT synthetic/TRON pairs, and COSMOS-Transfer2.5 conditioned on edge maps and text prompts. Five baselines trained on synthetic data degrade strongly on both sets. There is also a domain interpolation test, a comparison to 37 real Hubble photos and Jetson timings. The claim is that ASTRA is the most target-diverse benchmark and that the two pipelines can replace HIL facilities.

Critical strengths
  1. Going beyond Tango-only benchmarks is well motivated, 20 very different targets is a real asset (Sec 3.1, Tab 6).
  2. Unified schema, depth-buffer visibility, two FOVs and sequential trajectories (App C) make it usable for tracking too (AUKF example, App G).
  3. Two very different generators is a sensible idea, and Tab 4 honestly reports FRESCO is worse than raw renders on FID/CMMD.
  4. Data, code and weights will be released.
Critical weaknesses
  1. HIL-free protocol is not validated (fundamental to the main claim). Nothing shows that errors or rankings on FRESCO/COSMOS predict real performance. FRESCO learns TRON lab style of Tango, not in-orbit style. Applying both pipelines to SPEED+ synthetic and comparing rankings with lightbox/sunlamp would adress this. Fig 7 also doesnt really match: TRON is flat until α≈0.8, FRESCO rises from ~0.3, and picking the 100 hardest frames at α=1 biases the curves.
  2. Cross-target generalization motivates the paper (Sec 1) but is never tested, train and test share all 20 targets. A held-out-target split is needed.
  3. FRESCO is underspecified. As written, Eq 1 is one multiplicative filter, i.e. a global linear convolution, "neural operator" oversells it. Loss, ρ, number of pairs and an FDA comparison are missing. Freq-L1 is close to what FRESCO optimizes and FID on 37 images is unreliable.
  4. COSMOS pose preservation is only shown qualitatively (Fig 6). Edge chamfer or mask IoU against the conditioning would help. ~20° rotation error with small std for all models at 45° may indicate a systematic misalignment.
  5. Table 3 has errors. Epose >= ER[rad] must hold, but YOLO-26n FRESCO 45° (27.06° = 0.47 rad, Epose 0.36), YOLO-26n FRESCO 90° and SPNv3 COSMOS 90° violate it. Synthetic Et is "a few centimeters" in text but KRN is 0.27/0.67 m. SPNv3 is ViT in Sec 5.1 but EfficientDet-D3 in App F.
  6. Onboard results use FP32 PyTorch on random input, Orin latency is ~400 ms for both 5M and 13M param models so overhead probably dominates. 2.50 vs 2.48 FPS does not support "only YOLO-26n approaches closed-loop".
  7. Related work: SPE3R (Park & D'Amico, 2024) already has 64 satelite models from NASA 3D Resources and ESA, so the "most target-diverse" claim does not hold. URSO (Proença & Gao, ICRA 2020) and SwissCube (Hu et al., CVPR 2021) are also missing. SPARK 2024 stream 2 has real lab images (https://huggingface.co/datasets/CVI2-UniLU/SPARK-2024).
Questions for authors
  1. Any evidence, even small scale on SPEED+, that rankings on FRESCO/COSMOS correlate with HIL rankings?
  2. Please explain Epose < ER in Tab 3. Is ± over targets or frames?
  3. Is FRESCO a single global filter? Which loss, ρ and training pairs, is R real-valued?
  4. At what resolution was COSMOS run and how were outputs mapped back to 2048x2048 and original intrinsics?
  5. One model per target or a joint model? Is target identity given at test time?
  6. Which model is in Fig 7? The 100 hardest frames end at ~14° under FRESCO, similar or below the all-frame means in Tab 3.
  7. 240 frames at 24 fps is 10 s. How does this relate to real ROE time scales?
Reviewer_VvJWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces ASTRA, a benchmark for 6-DoF pose estimation of non-cooperative spacecraft. A unified UE5 pipeline renders twenty spacecraft, providing approximately one million independent training images and 576,000 sequential test images across the original synthetic domain and two generated appearance domains. FRESCO learns a Fourier-amplitude mapping from paired synthetic and real images, while COSMOS-Transfer2.5 generates videos conditioned on edge sequences. The evaluation includes five pose estimators, distributional comparisons with real Hubble photographs, a temporal-filtering example, and inference speed and energy measurements on Jetson platforms. The intended contribution is a unified resource combining multiple targets, sequential trajectories, and controlled appearance shifts.

Critical strengths

ASTRA provides a unified benchmark covering twenty spacecraft with diverse geometries and scales, including camera parameters, pose annotations, keypoints, and visibility information. Its frame-based training and sequential testing data facilitate cross-target evaluation and integration with navigation filters.

The paper evaluates original renderings, FRESCO outputs, and COSMOS outputs, alongside inference speed and energy on two Jetson platforms. These measurements offer a useful starting point for studying sim2real gap and computational tradeoffs.

Critical weaknesses

Internal inconsistencies in the reported metrics. Under Equation (3), mean pose error should be no smaller than mean rotation error in radians, assuming identical samples and aggregation weights. However, Table 3 reports 27.06° (0.472 rad) rotation error but only 0.36 pose error for YOLO-26n with FRESCO at 45°. This case is not single. This discrepancy exceeds rounding error and raises concerns about the reliability of the reported results.

Ambiguous baseline definitions and multi-target evaluation protocol. Section 5.1 describes SPNv3 as ViT-based, whereas Appendices F and I specify EfficientDet-D3. It is also unclear whether models are trained jointly across spacecraft or separately per target, and how target-specific keypoints and correspondences are handled. These ambiguities hinder reproducibility and interpretation of cross-target performance.

Insufficient validation of the generated benchmark. Edge conditioning and phase preservation alone do not guarantee pose-label consistency after image translation. Moreover, comparisons with only 37 real HST photographs rely on image-distribution metrics rather than task-level agreement with real-domain pose errors or model rankings. FRESCO improves frequency and edge distances but worsens FID and CMMD relative to raw UE5 renderings, leaving its practical relevance uncertain. Independent real-domain evaluation and a simple baseline such as FDA would help establish validity.

Questions for authors

Please explain the discrepancies with Equation (3) and provide corrected or disaggregated results.

Which SPNv3 architecture was evaluated? Are models trained jointly or separately across targets, and how are target identity and differing keypoint sets handled?

Was pose-label consistency independently verified after image translation? What are the keypoint displacement statistics and invalid-frame rates?

Which SHIRT pairs were used to train FRESCO, and were evaluation trajectories held out? Is there evidence that generated-domain pose errors or model rankings correlate with independent real HIL results, preferably compared against FDA?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.