acceptodds
Under review as a conference paper at ICLR 2027

Fast Active Reconstruction by Omnidirectional View

Abstract

In this work, we explore panoramic vision for fast active reconstruction, leveraging its omnidirectional observations to enable efficient UAV-based 3D scanning beyond existing perspective-based methods that require frequent camera adjustments due to their limited fields of view. Specifically, we propose FAR, a closed-loop perception–planning–capture framework with two key components. A Spherical Ray Adapter (SRA) incorporates synchronized camera poses into perspective-based VGGT-Ω, enabling streaming metric panoramic depth prediction with cross-window consistency. Adaptive Initialization automatically configures planning parameters from the scene scale for efficient exploration across diverse environments. To support systematic evaluation, we further develop a unified panoramic simulation platform with real-time point-cloud output and standardized planner interfaces, and construct a large-scale benchmark with panoramic observations, accurate poses, and ground-truth geometry. Extensive experiments show that FAR outperforms existing methods in both planning and reconstruction; with RGB panoramas only, it reaches 83.3% coverage with a 1017 m mean path, exceeding MAGICIAN with ground-truth depth (79.4%, 1400 m). The proposed simulator, benchmark, and models will be publicly released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Next-Best-View, Active Exploration

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_eJ88OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper presents FAR, a closed-loop framework for active UAV reconstruction using panoramic RGB images and known metric camera poses. A Spherical Ray Adapter conditions a pretrained geometry model on viewing rays and camera poses, while triangulation-based scale alignment and persistent voxel refinement produce consistent metric geometry across successive image windows. The planner uses this geometry to identify structure-anchored frontiers and select the next capture position, with voxel resolution and viewpoint-sampling parameters initialized from the scene scale. The paper also introduces a simulation platform and the FAR26 benchmark. Experiments on Macarons++ and FAR26 report improved depth and reconstruction accuracy, while planning experiments show higher coverage per unit flight distance. Component ablations and downstream Gaussian Splatting results provide additional evaluation of the reconstruction pipeline.

Critical strengths
  1. The integration addresses a useful problem. Panoramic observations simplify viewpoint selection, while metric alignment and persistent refinement provide a practical connection between feed-forward geometry prediction and active reconstruction.
  2. The reconstruction results are encouraging. Table 2 shows consistent improvements in depth and geometry on both datasets. Table 3 also supports the usefulness of the reconstruction components.
Critical weaknesses
  1. The contribution appears incremental. SRA closely follows UCPE, the voxel backend builds on AMB3R, and routing adapts FUEL. The integration is sensible, but the methodological advance beyond these components remains limited. The paper should clarify which adaptations are essential for panoramic active reconstruction and what additional benefit they provide.
  2. Hyperparameter sensitivity analysis. The scale coefficients in Eq. (5) and the stopping ratio in Eq. (8) affect map resolution, viewpoint selection, and the balance between coverage and flight distance. Using one setting across scenes is encouraging, but sensitivity study would help establish how robust the reported gains are.
  3. The effect of scale adaptation on fine details is unclear. Voxel size grows with the overall scene extent, whereas local structures do not necessarily become larger. Coarser aggregation may therefore weaken the representation of small structures and the frontier signals used to observe them. A comparison with a finer fixed resolution would help show whether the scaling rule preserves local detail while maintaining efficiency.
  4. The ablation does not sufficiently explain the coverage gains. Table 1 compares geometry sources, while Table 3 evaluates reconstruction components only through depth and geometry metrics. The contribution of Adaptive Initialization to coverage remains untested. Comparing fixed versus scale-adaptive parameters, and persistent versus per-window voxel refinement, using coverage–path curves would better explain the system’s improvements.
Questions for authors
  1. Parameter selection. What criterion was used to choose the coefficients in Eq. (5) and the stopping ratio in Eq. (8)? Are they tied to a desired spatial resolution or coverage target, or selected through empirical tuning?
  2. Scene scale. How is the task boundary B specified in practice: as a tight region around the target structure or a broader flight region? Since its volume determines voxel size, this distinction matters for understanding the scale-adaptation rule and its ability to retain local detail.
6 comments
AuthorsOpenReview ↗

Response to Reviewer eJ88, Q1

Reply to Q1:

Thank you for your careful assessment. Our adaptations address the unavailability of ground truth (GT) depth in practice and the planning requirements of omnidirectional observations across different scene scales.

Planning. We introduce two designs for this setting:

  • Adaptive Initialization (Eq. 5): Metric planning parameters scale with the characteristic scene length L through dimensionless coefficients, allowing one coefficient set across scenes. Replacing this with fixed parameters reduces coverage from 85.6% to 81.5% (Response 4).
  • Termination based on the peak ratio (Eq. 8): The active frontier count is compared with its historical peak, giving a criterion independent of scene scale. Removing it adds only 0.4 percentage points (pp) in coverage but increases path length by 21% (Response 4).

Responses 5, 2, and 3 respectively explain parameter selection, sensitivity, and the effect on local detail.

Reconstruction. Our adaptations are as follows:

  • Spherical Ray Adapter (SRA) and UCPE: SRA adopts UCPE's relative ray encoding: each token has a local frame whose z axis follows its viewing ray and whose x axis is constructed from the camera's downward axis. Attention uses relative transformations between these frames, as in GTA. UCPE uses this encoding to control cameras in video diffusion models, with pinhole, wide angle, and fisheye views. We adapt it to condition feedforward geometry prediction on known metric poses of equirectangular projection (ERP) panoramas covering the full sphere. Three changes are needed. First, the image plane component uses rotary positional encoding with different longitude offsets across attention heads to preserve adjacency across the ERP seam. Second, UCPE's frame becomes undefined when the viewing ray is parallel to the downward axis near the poles; we use the camera's horizontal axis to construct the frame. Third, camera centers are expressed as metric offsets from a window reference point to keep the numerical values moderate in large outdoor scenes. Removing SRA increases AbsRel from 0.1196 to 0.1914 (Table 3 in the main paper).
  • Metric alignment: Each window's raw depth can have a different scale, causing inconsistent points when fused into one map. We triangulate spherical correspondences using known camera baselines to anchor each window to metric scale, without learnable parameters. Removing alignment increases AbsRel from 0.1196 to 0.2073 and reduces the fused point cloud F@1m from 0.586 to 0.370 (Table 3 in the main paper).
  • Persistent refinement and AMB3R: AMB3R rebuilds its voxel map for each window. Our global map persists, retaining features from earlier windows observing the same structure and improving consistency across windows. Adding the voxel backend reduces AbsRel from 0.1337 to 0.1196 (Table 3 in the main paper).

These adaptations also benefit active reconstruction. In the closed loop, FAR with SRA and alignment achieves 83.3% coverage and 0.634 AUC, retaining 97% of the coverage and 96% of the AUC obtained with GT depth. With the same planner, PanoVGGT achieves 69.2% coverage and 0.421 AUC (Table 1 in the main paper).

AuthorsOpenReview ↗

Response to Reviewer eJ88, Q2, Q3 and Q4

Reply to Q2:

Thank you for this suggestion. We performed a 5 × 5 sweep of α (voxel size ρ = αL) and η with GT depth over 11 scenes, using multiple start positions per scene (Table R10 and Table R11). Each setting is shared across all scenes. The default is α = 0.0273 and η = 0.07, with 85.6% coverage in this sweep.

Changes in coverage at ε = 1.0 m relative to the default, in pp:

Table R10. Coverage changes relative to the default setting (pp).

α / default αη = 0.03η = 0.05η = 0.07η = 0.10η = 0.15
×0.5−1.0−1.2−1.4−1.5−1.9
×0.7+1.6+1.6+0.9+0.70.0
×1+0.5+0.30 (default)−0.3−1.5
×1.4−1.6−1.8−2.0−2.3−2.9
×2−3.6−3.6−3.7−3.8−4.4

Path length relative to the default:

Table R11. Path length relative to the default setting.

α / default αη = 0.03η = 0.05η = 0.07η = 0.10η = 0.15
×0.51.761.631.541.411.25
×0.71.511.441.331.251.14
×11.191.091.00 (default)0.940.87
×1.40.880.860.830.790.76
×20.670.670.660.650.63

α controls resolution and viewpoint selection. At η = 0.07, varying α over a fourfold range changes coverage by −3.7 to +0.9 pp. Coarse voxels can represent small structures with too few cells, exhausting frontiers prematurely. Finer voxels increase path length by 33–54% without a comparable coverage gain. The default balances coverage and path length.

η controls the stopping point. At the default α, increasing η from 0.03 to 0.15 shortens the path by approximately one quarter while reducing coverage by 2.0 pp, allowing adjustment to the flight budget.

Across all 25 settings, coverage changes range from −4.4 to +1.6 pp. The lowest coverage is 81.2%, still above MAGICIAN's 79.4% and EPIC's 65.6%. These results indicate that the performance advantage does not require precise parameter tuning.

Reply to Q3:

Thank you for this observation. We compared adaptive voxels with two finer fixed resolutions in large scenes, using the stricter ε = 0.5 m threshold to assess detail coverage (Table R12).

Table R12. Detail coverage and path length at different voxel resolutions.

SceneVoxel size (m)Coverage at ε = 0.5 mAUC at ε = 0.5 m under the same path budgetPath length (m)
Colosseum2.2564.6%0.5122477 (+53%)
Colosseum3.1563.3%0.5182103 (+30%)
Colosseum4.5 (adaptive)62.0%0.5331615
Dunnottar2.2298.8%0.7421883 (+34%)
Dunnottar3.1198.0%0.7831470 (+5%)
Dunnottar4.45 (adaptive)98.4%0.8131401

The finest voxels improve final detail coverage by at most 2.6 pp but require 34–53% longer paths. Adaptive voxels achieve the highest detail AUC in both scenes under the same path budget. Scale adaptation thus balances detail and efficiency: finer fixed voxels provide small coverage gains at the cost of longer paths.

Reply to Q4:

Adaptive Initialization contributes 4.1 pp in coverage. Following your suggestion, we compared adaptive parameters with a fixed set, taking the adaptive values for Colosseum and applying them to every scene (Table R13). The added results are averaged over multiple start positions per scene; reference results are taken from Table 1 in the main paper.

Table R13. Planning component ablations.

SettingCoverage at ε = 1.0 m ↑Path length (m) ↓
FAR85.6%1111
w/o Adaptive Initialization (one fixed parameter set for all scenes)81.5%888
w/o termination based on the peak ratio (η = 0)86.0%1345
MAGICIAN79.4%1400
EPIC65.6%1807

The shorter path with fixed parameters results from coarse settings exhausting frontiers and ending exploration early in small scenes. Redeemer's coverage drops from 93.2% to 72.8%, and Pantheon's decreases by approximately 12 pp. Removing termination based on the peak ratio gains only 0.4 pp in coverage while increasing path length by 21%.

We will include the coverage versus path length curves in the appendix. With fixed parameters, the curves stop earlier with final coverage 4.1 pp lower. Without termination based on the peak ratio, they extend after coverage nearly saturates, with little additional gain.

AuthorsOpenReview ↗

Response to Reviewer eJ88, Q5 and Q6

Reply to Q5:

α controls relative spatial resolution subject to planning efficiency, while η defines coverage saturation. Both objectives are independent of scene scale, and both parameters are shared across all 11 scenes without individual tuning.

α is chosen for mapping and planning efficiency. With ρ = αL and α = 0.0273, the characteristic scene length spans approximately 37 voxels. This count does not grow with scene scale, keeping mapping and planning costs bounded. The default is selected where coverage approaches saturation at an acceptable planning cost. Across the sweep, coverage gains are at most +1.6 pp; at η = 0.07, finer settings increase path length by 33–54% (Response 2).

η defines the saturation criterion. Acquisition stops when the active frontier count falls below 7% of its historical peak. This ratio is independent of metric scene scale.

Reply to Q6:

We use the official Macarons++ evaluation box as B, a region closely enclosing the target structure rather than a broader flight region. Consequently, ρ = αL scales with the target structure's extent.

Reply to the AI Disclosure Concern:

Thank you for raising this concern. The central "Omnidirectional View" panel in Figure 1 combines three elements: a 3D scene generated with OpenAI's ChatGPT Image based on one of our existing scenes, a photograph of our actual UAV, and a panoramic sphere in the upper left corner created from visual materials in PowerPoint. This composition illustrates omnidirectional acquisition, with the generated scene designed to enhance the sense of depth and convey this concept more clearly. All other result images in the paper were produced by our experiments, and we remain fully confident in the validity of our method and the accuracy of the reported results. We apologize for omitting this use of image generation from our AI Use Statement. We will revise this statement in the main manuscript to disclose the specific use of ChatGPT Image, and update the Figure 1 caption to identify the generated scene and explain the composition.

Reviewer_zcV1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FAR, a closed-loop system for fast UAV-based 3D scanning with a 360° panoramic camera. The main idea is that omnidirectional observations remove camera-heading search, so the planner selects where to capture next rather than jointly optimizing position and orientation. The system has two main components. First, a Spherical Ray Adapter (SRA) injects known camera poses and spherical-ray geometry into a pretrained perspective feed-forward model, followed by triangulation-based metric scale alignment and persistent voxel refinement. Second, Adaptive Initialization scales metric planning parameters with the cube root of the task-boundary volume, and a frontier-based planner selects position-only viewpoints from structure-anchored frontiers. The paper also introduces a panoramic simulator and a 20-scene FAR26 benchmark. Experiments on 11 Macarons++ scenes report better coverage efficiency than EPIC and MAGICIAN, stronger metric reconstruction than panoramic feed-forward baselines, improved downstream 3DGS rendering, and perception ablations supporting SRA, metric alignment, and persistent voxel refinement.

Critical strengths
  • Perception and planning exchange metric world points, which makes the decomposition sensible and potentially reusable.

  • SRA is pose-aware but invariant to the global world frame, zero initialization preserves pretrained behavior, and metric alignment adds no learned parameters. The large degradation without alignment convincingly shows that cross-window scale consistency is a central streaming problem.

  • The method performs best across the reported depth and point-cloud metrics on both evaluation sets, with visibly better boundaries in the qualitative results.

  • A panoramic simulator with online point-cloud output, a standardized planner interface, and released benchmark scenes would be valuable to the community.

  • The 3DGS experiment shows that the acquired observations improve rendering quality, not only the paper’s coverage metric.

Critical weaknesses
  • From Sec 4.1's setting, table 1 compares EPIC, MAGICIAN, and FAR with ground-truth depth but through their native sensing configurations: a 50 m LiDAR with a limited vertical band, a 60° perspective camera, and a full-sphere panorama with 130 m depth range. This is a reasonable native-system comparison, but it cannot determine how much of FAR’s advantage comes from the panoramic sensor versus Adaptive Initialization, the frontier definition, routing, or termination. A matched-sensor comparison, or at least ablations of FAR’s planning components, is needed before attributing the gain primarily to the planner.

  • The proposed model is trained on 16 FAR26 scenes together with Holo360D, while the main baselines are evaluated off the shelf. The w/o-SRA ablation is useful, but it does not establish how much of the gain over other panoramic models comes from the adapter versus task-specific training.

  • SRA’s main advantage is that it uses known poses inside the network, whereas the reported baselines use poses only for post-hoc metric alignment. Pow3R, OmniVGGT, and MapAnything are conceptually relevant because they accept camera or geometric priors. I recognize that ERP adaptation may be nontrivial, but the paper should compare against such models where feasible, or explain clearly why a direct comparison is not possible.

  • The paper describes real-time simulator interfaces, but does not report per-window perception latency, planning latency, GPU memory, or achieved closed-loop rate. These quantities matter for a UAV system.

Questions for authors

Please refer to my Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer zcV1, Q1 and Q2

Reply to Q1:

Thank you for this observation. We added controlled comparisons and planning ablations to examine the contributions of sensing and planning.

Same sensor. All three methods use an omnidirectional field of view, a 50 m range, and ground truth (GT) depth (Table R4). The added results are averaged over multiple start positions per scene.

Table R4. Comparison with the same sensor configuration.

Sensor configurationMethodCoverage at ε = 1.0 m ↑Path length (m) ↓AUC ↑
Native (Table 1 in the main paper)EPIC65.6%18070.377
Native (Table 1 in the main paper)MAGICIAN79.4%14000.500
Native (Table 1 in the main paper)FAR85.6%11110.663
Omnidirectional, 50 mEPIC67.4%17560.437
Omnidirectional, 50 mMAGICIAN80.5%17950.604
Omnidirectional, 50 mFAR85.2%15710.623

With the sensor fixed, FAR exceeds MAGICIAN and EPIC by 4.7 and 17.8 percentage points (pp) in coverage, with paths shorter by 12% and 11%, respectively. Changing each planner's sensor changes coverage by at most 1.8 pp, with the ranking unchanged. EPIC gains 1.8 pp with a slightly shorter path when its field of view becomes omnidirectional. FAR loses only 0.4 pp but flies 41% farther when its range decreases from 130 m to 50 m. MAGICIAN gains 1.1 pp and flies 28% farther when both its field of view and range change. The coverage advantage mainly comes from the planner, while sensing contributes to the shorter paths.

Planning ablations. We evaluated the two modified planning components with native sensing and GT depth (Table R5).

Table R5. Planning component ablations.

SettingCoverage at ε = 1.0 m ↑Path length (m) ↓
FAR85.6%1111
w/o Adaptive Initialization (one fixed parameter set for all scenes)81.5%888
w/o termination based on the peak ratio86.0%1345
MAGICIAN79.4%1400
EPIC65.6%1807

Adaptive Initialization contributes 4.1 pp in coverage. Fixed parameters are too coarse for small scenes, exhausting frontiers prematurely and ending exploration early; Redeemer's coverage drops from 93.2% to 72.8%. Removing termination based on the peak ratio gains only 0.4 pp but increases path length by 21%, as exploration continues after coverage nearly saturates.

The frontier formulation follows existing surface frontier methods, and routing follows FUEL (Section 3.3.2). The ablations therefore focus on our two modifications: Adaptive Initialization improves coverage, while termination based on the peak ratio reduces flight distance.

Reply to Q2:

We agree that comparisons with models used without additional training cannot separate the effects of training data and the Spherical Ray Adapter (SRA). The Macarons++ scenes reported in Table 2 in the main paper are absent from our training data (16 FAR26 scenes and Holo360D), so evaluation is on unseen scenes. We further trained PanoVGGT with the same data, number of steps, and data mixture as our model, and evaluated it using the protocol for Table 2 in the main paper. All rows in Table R6 use the same metric alignment.

Table R6. Training data and SRA comparisons.

ModelAdditional training dataAbsRel ↓RMSE (m) ↓δ₁.₂₅ ↑
PanoVGGTNone0.166727.180.7684
PanoVGGTOurs (FAR26 + Holo360D)0.127424.360.8473
VGGT-Ω w/o SRAOurs (FAR26 + Holo360D)0.191431.460.6821
VGGT-Ω + SRA (ours)Ours (FAR26 + Holo360D)0.119623.290.8647

Training on our data improves PanoVGGT's AbsRel from 0.1667 to 0.1274. Our model still achieves lower AbsRel (0.1196) and RMSE, and higher δ₁.₂₅. With SRA and the same additional training, VGGT-Ω, pretrained only on perspective images, outperforms PanoVGGT, which was designed for panoramas.

With the same backbone and data, updating only the depth head without SRA gives 0.1914 AbsRel; adding SRA reduces it to 0.1196, a reduction of approximately 37%. This supports SRA's contribution.

AuthorsOpenReview ↗

Response to Reviewer zcV1, Q3 and Q4

Reply to Q3:

We added a comparison with MapAnything (Table R7), which accepts known intrinsics and metric poses. Since it only supports pinhole cameras, each equirectangular projection (ERP) frame is resampled into six 90° cubemap faces with exact intrinsics and metric poses mapping camera coordinates to world coordinates. Its radial depth is then resampled back to the ERP grid.

Table R7. Comparison with MapAnything.

MethodUse of posesAbsRel ↓δ₁.₂₅ ↑RMSE (m) ↓
MapAnything (cubemap)Within the network0.35550.348155.8
Ours w/o alignmentWithin the network0.20730.587237.72
MapAnything (cubemap) + alignmentWithin the network and for alignment0.21890.638536.54
Ours (SRA + alignment)Within the network and for alignment0.11960.864723.29

Without alignment, both models use poses only within the network. Our model has lower AbsRel (0.2073 vs. 0.3555), performs better on all three metrics, and outperforms MapAnything on all test scenes. With the same alignment, MapAnything improves to 0.2189 AbsRel, still 83% higher than our 0.1196, with worse δ₁.₂₅ and RMSE.

Both methods receive the same metric poses; the difference is how they use them. MapAnything processes perspective crops with a pose shared within each image, while SRA encodes the metric camera position and viewing direction for each ray on the full panorama. We will add MapAnything to Table 2 in the main paper as a baseline accepting pose inputs.

Reply to Q4:

Thank you for highlighting these metrics. We measured all methods on the same machine: an AMD EPYC 9T24 with 32 vCPUs and an NVIDIA RTX 5880 Ada with 48 GB of GPU memory. Latencies are averages of the median step times for each scene; RAM and GPU memory are averages of the peak usage for each run (Table R8).

Table R8. Computational costs and memory usage.

MethodPerception (ms) ↓Mapping (ms) ↓Planning (ms) ↓RAM (GB) ↓GPU memory (GB) ↓
EPIC—55592.20
MAGICIAN128217538434.115.1
FAR912149442.58.7

FAR's complete loop, from RGB input through depth prediction and mapping to planning, runs on a single GPU. Median and P95 latencies are reported in Table R9:

Table R9. FAR latency and closed loop rate.

MetricMedianP95
Panorama rendering and readback (ms)4272
Perception per window (ms)9121002
Mapping (ms)149221
Planning per decision (ms)44133
Closed loop rate (Hz)0.78—

The 0.78 Hz rate measures viewpoint decisions, from capture to issuing the next viewpoint. The underlying controller tracks the trajectory between viewpoints. Across all 11 scenes, this rate ranges from 0.72 to 0.81 Hz.

Planning takes 44 ms, less than EPIC's 59 ms and nearly two orders of magnitude less than MAGICIAN's 3843 ms. Perception accounts for approximately 70% of the decision cycle. EPIC directly uses point clouds and needs no perception network. MAGICIAN still predicts occupancy despite using GT depth, taking approximately 5.3 s per decision (0.19 Hz); FAR's decision rate is approximately four times higher.

EPIC has the lowest memory requirements. FAR's RAM usage is close to EPIC's (2.5 GB vs. 2.2 GB), and every reported computational and memory cost is lower than MAGICIAN's. FAR's lower decision rate relative to EPIC primarily reflects the cost of predicting depth from RGB; planning itself is not the bottleneck.

AuthorsOpenReview ↗

Revised Paper Uploaded: Summary of Changes

Dear Reviewer,

Thank you for your constructive comments. We have added the following to the revised manuscript, with additions marked in blue:

  • Section 4.2, Table 2 (page 8): The same sensor comparison addressing Q1 (rebuttal Table R4).
  • Section 4.5, Figure 7 (page 10): Planning ablation curves supporting our response to Q1.
  • Appendix A.7, Table 7 (page 18): The MapAnything comparison addressing Q3 (rebuttal Table R7), referenced in Section 4.3.

We appreciate your time and helpful suggestions.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up from the Authors

Dear Reviewer,

Thank you again for your time and for the thoughtful comments on our submission. We have posted our responses above and would like to follow up briefly.

To make our method and the additional results easier to inspect, we have prepared an anonymous supplementary page: https://far-rebuttal-iclr27.github.io/far-rebuttal-anonymous/

It opens with video demonstrations and additional qualitative results, which we hope give a clearer picture of how FAR works. These are followed by our responses to the questions shared across reviewers and to each reviewer's specific questions.

The page is fully anonymous, uses only local assets, and contains no analytics or visitor tracking.

As the discussion period draws to a close, we would be grateful for the chance to address any remaining concerns, and we will do our best to clarify anything that is still unclear.

Best regards,

The Authors

Reviewer_qiHDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents FAR, a closed-loop framework for panoramic UAV-based active 3d reconstruction. The key idea is that panoramic observations eliminate orientation search, reducing trajectory planning time. FAR uses a spherical ray adapter that adapts a pretrained 3d geometry model to posed panoramas, metric scale alignment, and persistent voxel refinement for streaming 3D reconstruction. Experiments on Macarons++ show that FAR achieves higher coverage with shorter flight paths.

Critical strengths
  1. The framework takes advantage of panoramic images as input to simplify next-best-view planning.
  2. Well-motivated reconstruction design. The Spherical Ray Adapter (SRA) adapts VGGT-Ω to panoramic inputs, while metric alignment and persistent voxel refinement support streaming reconstruction. The historical information helps refine current depth predictions and improve cross-window consistency.
  3. The experimental results are promising, showing improved coverage per meter flown with the proposed framework, and also showing better depth prediction and reconstruction quality.
Critical weaknesses
  1. FAR combines panoramic sensing and streaming reconstruction, but both directions have been explored in prior work, and several components build on existing methods. The contribution appears to lie primarily in component combination. The novelty is limited.
  2. The experiments in Macarons++ compare panoramic, non-panoramic, and LiDAR sensors with different fields of view and sensing ranges, which directly affect performance. It is difficult to determine how much of the improvement comes from the planner rather than the panoramic setting.
  3. The evaluation evaluates flight distance without reporting planning/inference time or memory usage. Shorter paths do not necessarily imply faster acquisition given the overhead of window-based reconstruction and persistent voxel refinement.
  4. FAR and MAGICIAN are evaluated with different 3d gs splatting methods (SPaGS and FastGS), so rendering-quality differences cannot be attributed solely to the planning strategy. An evaluation using a common reconstruction pipeline would provide stronger evidence.
Questions for authors

See Weakness 2/3/4.

4 comments
AuthorsOpenReview ↗

Response to Reviewer qiHD, Q1 and Q2

Reply to Q1:

Thank you for your careful review. FAR addresses two issues in panoramic active reconstruction: obtaining depth in practice and adapting planning to omnidirectional observations.

The compared next best view systems directly use supplied depth: MAGICIAN uses simulator ground truth (GT) depth, while EPIC is designed to use LiDAR measurements. GT depth is unavailable in real flights, and LiDAR accuracy and range depend on the hardware, particularly outdoors at altitude. FAR predicts depth from RGB panoramas. Omnidirectional observations remove orientation search but expand the observed space beyond a limited field of view, requiring corresponding planning rules. Adaptive Initialization adjusts planning parameters to different scene scales.

The results show improvements in coverage, path length, and planning time:

  • Depth from RGB: FAR reaches 83.3% coverage with a 1017 m mean path, compared with 79.4% / 1400 m for MAGICIAN and 65.6% / 1807 m for EPIC with GT depth (Table 1 in the main paper).
  • Same sensor: With an omnidirectional 50 m sensor and GT depth, FAR achieves 85.2% / 1571 m, compared with 80.5% / 1795 m for MAGICIAN and 67.4% / 1756 m for EPIC (Table R1).
  • Planning efficiency: FAR takes 44 ms per planning step, compared with 59 ms for EPIC and 3843 ms for MAGICIAN (Table R2).

Reply to Q2:

We added a controlled comparison with an omnidirectional field of view, a 50 m range, and GT depth for all three planners (Table R1). The added results are averaged over multiple start positions per scene.

Table R1. Comparison with the same sensor configuration.

Sensor configurationMethodCoverage at ε = 1.0 m ↑Path length (m) ↓AUC ↑
Native (Table 1 in the main paper)EPIC65.6%18070.377
Native (Table 1 in the main paper)MAGICIAN79.4%14000.500
Native (Table 1 in the main paper)FAR85.6%11110.663
Omnidirectional, 50 mEPIC67.4%17560.437
Omnidirectional, 50 mMAGICIAN80.5%17950.604
Omnidirectional, 50 mFAR85.2%15710.623

With the sensor fixed, FAR retains higher coverage and shorter paths. Its coverage exceeds MAGICIAN and EPIC by 4.7 and 17.8 percentage points (pp), respectively, with paths shorter by 12% and 11%.

With each planner fixed, changing the sensor mainly affects path length. Coverage changes by at most 1.8 pp, with the ranking unchanged. Expanding EPIC's field of view at the same range increases coverage by 1.8 pp and slightly reduces its path. Reducing FAR's range from 130 m to 50 m lowers coverage by only 0.4 pp but increases its path by 41%, as each viewpoint observes a smaller region. Changing both MAGICIAN's field of view and range increases coverage by 1.1 pp and path length by 28%.

The coverage advantage primarily comes from the planner, while sensing contributes to the shorter paths. FAR retains its coverage advantage over MAGICIAN and the shortest path with the same sensor.

AuthorsOpenReview ↗

Response to Reviewer qiHD, Q3 and Q4

Reply to Q3:

Thank you for this observation. We measured all methods on the same machine: an AMD EPYC 9T24 with 32 vCPUs and an NVIDIA RTX 5880 Ada with 48 GB of GPU memory. Latencies are averages of the median step times for each scene; RAM and GPU memory are averages of the peak usage for each run (Table R2).

Table R2. Computational costs and memory usage.

MethodPerception (ms) ↓Mapping (ms) ↓Planning (ms) ↓RAM (GB) ↓GPU memory (GB) ↓
EPIC—55592.20
MAGICIAN128217538434.115.1
FAR912149442.58.7

Planning: FAR takes 44 ms per step, less than EPIC's 59 ms and nearly two orders of magnitude less than MAGICIAN's 3843 ms. EPIC evaluates eight yaw directions per candidate viewpoint. MAGICIAN uses a network for beam search over candidate views, accounting for approximately 3.6 s of its planning time. FAR only selects positions because each panorama covers all directions.

Perception and decision rate: FAR predicts depth from RGB in approximately 0.9 s per window. Although MAGICIAN uses GT depth, its network still predicts scene occupancy to evaluate candidate views, taking approximately 1.3 s. EPIC directly uses GT depth and requires no perception network. FAR's viewpoint decision rate is approximately 0.78 Hz, below EPIC's 5 Hz replanning rate but approximately four times MAGICIAN's 0.19 Hz.

Mapping and memory: EPIC's lightweight LiDAR pipeline has the lowest mapping and memory costs. FAR uses 2.5 GB of RAM, close to EPIC's 2.2 GB, and has lower mapping latency, RAM usage, and GPU memory usage than MAGICIAN.

FAR has lower costs than MAGICIAN in every reported component. Relative to EPIC, it adds the cost of predicting depth from RGB but avoids LiDAR and plans faster. Its 1017 m mean path is 44% shorter than EPIC's 1807 m and also shorter than MAGICIAN's 1400 m.

Reply to Q4:

We agree that using different 3DGS methods confounds the effects of data acquisition and reconstruction. FastGS only supports pinhole cameras, while FAR captures panoramas. We repeated the experiment with a common pipeline (Table R3).

Training: Each 1024 × 512 FAR panorama is resampled into six 90° cubemap faces of 256 × 256 pixels, matching its angular resolution. The faces share the panorama's camera center and use the corresponding known poses. We train FastGS with the same configuration as the publicly released MAGICIAN model.

Evaluation: Both models are rendered with the same equirectangular projection (ERP) rasterizer from SPaGS at the same ten test positions per scene, selected independently of either trajectory.

Table R3. 3DGS rendering quality comparison.

Method3DGS trainingInitial geometryPSNR ↑SSIM ↑LPIPS ↓
MAGICIANFastGSGT depth13.140.3970.447
FAR (original paper)SPaGSPredicted depth15.940.4520.429
FAR (FastGS)FastGSPredicted depth15.290.4340.443

With the same 3DGS method, FAR improves mean PSNR by 2.15 dB and SSIM by 0.037, and reduces LPIPS by 0.004, while flying 27% less (1017 m vs. 1400 m). This comparison is conservative for FAR: its initial point cloud comes from depth predicted from RGB, while MAGICIAN uses GT depth. The advantage with the 3DGS method fixed supports the contribution of our planning and panoramic reconstruction.

Thank you for your response. The rebuttal has addressed most of my concerns. I suggest including Tables R1 and R3 from the rebuttal in the main paper. I will raise my score.

AuthorsOpenReview ↗

Revised Paper Uploaded: Summary of Changes

Dear Reviewer,

Thank you for your constructive feedback and your positive assessment of our additional results. Following your suggestion, we have incorporated both tables into the main text of the revised manuscript. The additions are marked in blue:

  • Section 4.2, Table 2 (page 8): The comparison under the same sensor configuration, previously Table R1 in our rebuttal.
  • Section 4.4, Table 4 (page 9): The rendering quality comparison using a common FastGS pipeline, previously Table R3 in our rebuttal.

We appreciate your time and helpful suggestions.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.