- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces ActiveGeo, where an agent sequentially selects new viewpoints and determines when to stop. It proposes ActiveMoPT with three-stage training: Multi-View Prompt-Preserving Adaptation, Trajectory-Guided Policy Initialization, and Cost-Aware Policy Refinement.
Critical strengths
- ActiveGeo introduces a new task formulation that brings active perception to CVOGL through viewpoint selection and stopping.
- ActiveMoPT integrates a multi-view localizer and an action policy while reusing the initial prompt across subsequent unprompted UAV views.
- Experiments demonstrate a promising trade-off between localization accuracy and observation efficiency, including zero-shot evaluation on ActiveGeo-858.
Critical weaknesses
- The experiments rely on 54 predefined UAV viewpoints, with all corresponding images collected in advance. The agent selects among these viewpoints rather than actively navigating in an online environment. However, in practical UAV applications, the target location is initially unknown, raising the question of how these candidate viewpoints can be determined and accessed before the target has been localized.
- The action policy does not consider spatial dependencies between viewpoints. The agent can directly select any unobserved viewpoint without considering the physical movement required to reach it. In practice, moving to a selected viewpoint may involve passing through intermediate locations where useful observations could already be obtained. Moreover, the proposed reward assigns a fixed acquisition cost to each additional view, regardless of the travel distance or movement cost.
- The current experiments mainly compare ActiveMoPT with passive localization methods and random viewpoint acquisition. Comparisons with simpler selection strategies, such as fixed viewpoint priorities or heuristic-based selection, would help establish the advantages of the learned policy.
- A central contribution of ActiveMoPT is its ability to select informative viewpoints based on the current localization state. However, Table 8 reveals substantial preferences for specific viewpoint IDs, with View 54 accounting for 30.44% of second-view selections. Consequently, the evidence supporting the learned selection policy remains incomplete.
- Several implementation details make the reported improvements difficult to interpret precisely. First, Figure 2 describes the initial localizer as identical to MoPT, despite different single-view results in Table 2, leaving the extent of parameter adaptation unclear. Second, the Preserve-Improve loss is presented without weights in Eq. (6), whereas Appendix C.1 specifies nonuniform default weights. These ambiguities affect the reproducibility of the experiments and the interpretation of the proposed adaptation strategy.
Questions for authors
- In practical UAV applications where the target location is initially unknown, how can the 54 candidate viewpoints be determined and accessed?
- How would the spatial relationships between viewpoints, including intermediate observations during UAV movement and viewpoint-dependent travel costs, affect the learned action policy and its accuracy-efficiency trade-off?
- Could the authors compare ActiveMoPT against a fixed-priority viewpoint selection strategy (e.g., following the most frequently selected viewpoints in Table 8) under the same multi-view localizer and stopping mechanism?
8 comments
Response by authors (2)
Weakness 2 & Question 2 (Part 2): Viewpoint-Dependent Travel Costs and the Accuracy–Efficiency Trade-off.
Thanks for asking how viewpoint-dependent travel costs affect the learned policy. Without changing the architecture, training with a distance-based movement cost leads to more observations over a shorter estimated travel distance while maintaining localization accuracy. This demonstrates that the acquisition strategy adapts to the cost objective.
We keep the architecture, frozen localizer, and localization gain term unchanged, initialize the policy from SFT, and replace the fixed observation cost for CONTINUE with a movement cost. In this experiment, we compute movement cost using the normalized straight-line distance between consecutively selected viewpoints:
Here, and is the mean distance between viewpoints in the training scenes. We retrain only the policy with GRPO and compare on the same 100 MoP-UAV test scenes and 17,447 targets using P+B+L. For the view-count row, accuracy and Avg. Views are from the paper; distance is estimated separately. We estimate travel distance by summing straight-line distances between consecutively selected viewpoints and report the average over targets:
| Cost | mIoU ↑ | [email protected] ↑ | Avg. Views ↓ | Avg. Distance (m) ↓ |
|---|---|---|---|---|
| View-count cost | 0.414 | 0.478 | 1.450 | 75.761 |
| Movement cost (based on distance) | 0.416 | 0.480 | 1.822 | 48.077 |
The policy trained with movement cost has a shorter estimated travel distance, uses more observations, and maintains high localization accuracy. Reducing observation count and reducing movement distance are different objectives; the action policy changes with the cost definition. These results show that the same framework can train the action policy to account for movement costs while maintaining localization accuracy.
We have added the results and corresponding analysis to the main manuscript (Table 5, page 10).
Response by authors (3)
Weakness 3 & Question 3: Learned Viewpoint Selection Improves Localization over Simpler Strategies.
Thanks for suggesting comparisons with simpler selection strategies. With the same multi-view localizer and stopping mechanism, the learned policy achieves higher localization accuracy than frequency-weighted random selection and Fixed order at similar average budgets; these gains persist when observation counts are fixed. Below, we compare against two strategies favoring the frequently selected viewpoints in Table 12 (Table 8 in the original submission), then fix view counts to compare viewpoint selection strategies.
(1) With the same multi-view localizer and stopping mechanism, the learned policy achieves higher localization accuracy than both simpler strategies. To examine the reviewer's suggestion of favoring frequently selected viewpoints in Table 12, we use a frequency-weighted random baseline: sampling probabilities follow the table's selection frequencies, making frequently selected viewpoints more likely to be chosen, with the same fixed probabilities for all targets. We also include Fixed order, which follows a fixed priority ranking based on the learned policy's viewpoint-selection frequencies, selecting the highest-priority available, unobserved viewpoint at each step. All methods use the same initial view, multi-view localizer, candidate viewpoints, and GRPO stopping policy, changing only the viewpoint selection strategy. We use P+B+L on MoP-UAV and P+B on ActiveGeo-858.
| Dataset | Viewpoint selection strategy | mIoU ↑ | [email protected] ↑ | Avg. Views ↓ |
|---|---|---|---|---|
| MoP-UAV | ActiveMoPT (learned selection) | 0.4139 | 0.4778 | 1.4499 |
| MoP-UAV | Frequency-weighted random | 0.4136 | 0.4761 | 1.4478 |
| MoP-UAV | Fixed order | 0.4130 | 0.4765 | 1.4486 |
| ActiveGeo-858 | ActiveMoPT (learned selection) | 0.4222 | 0.4755 | 1.3706 |
| ActiveGeo-858 | Frequency-weighted random | 0.4211 | 0.4714 | 1.3648 |
| ActiveGeo-858 | Fixed order | 0.4214 | 0.4732 | 1.3648 |
Frequency-weighted random selection and Fixed order both inherit global viewpoint preferences from the learned policy's selection frequencies. Their competitive performance is consistent with these learned preferences being useful for localization. Both baselines apply the same global preferences across targets, whereas ActiveMoPT selects viewpoints using the current localization state and observation history. ActiveMoPT achieves additional improvements in mIoU and [email protected] on both datasets at similar average observation budgets, supporting the benefit of selecting viewpoints based on the current localization state.
We have added the results and corresponding analysis to the main manuscript (Table 4, page 9).
(2) Learned viewpoint selection improves localization at fixed view counts. Even with the same stopping policy and similar average budgets, different viewpoint choices can lead to different observation counts for individual targets. To isolate viewpoint selection more directly, we fix the observation count for each target. We group MoP-UAV targets by the number of views acquired before the policy stopped in the original paper evaluation. All viewpoint selection strategies are re-evaluated on the same targets, with identical initial observations and the same frozen localizer. For each subset, all acquire exactly K views, including the initial image, and then terminate. Only the viewpoint selection strategy differs.
| Number of views K | Number of targets | Learned selection mIoU ↑ | Frequency-weighted random mIoU ↑ | Fixed order mIoU ↑ |
|---|---|---|---|---|
| 2 | 7,721 | 0.4090 | 0.4083 | 0.4070 |
| 3 | 55 | 0.2479 | 0.2413 | 0.2404 |
| 4 | 4 | 0.3478 | 0.3443 | 0.3414 |
With the same targets, initial observations, frozen localizer, and exactly K views per target, ActiveMoPT achieves higher localization accuracy than frequency-weighted random selection and Fixed order at each evaluated K. These differences reflect the selected viewpoints rather than differences in stopping behavior. These results support the policy’s ability to select more informative views for these targets based on the current localization state and observation history, even when the alternatives already use its learned global preferences.
We have added the results and corresponding analysis to Appendix B.2 (Table 8, page 15).
Response by authors (1)
We thank the reviewer for recognizing the ActiveGeo formulation that brings active perception to CVOGL, the reuse of the initial target prompt across subsequent unprompted UAV views, and the promising accuracy–observation-efficiency trade-off, including zero-shot evaluation on ActiveGeo-858. We provide additional experiments and clarify practical deployment, simpler selection strategies, and implementation details below.
Thanks for raising this deployment question. The 54 UAV views are defined around the building at the center of the satellite image, which is not necessarily the target specified by the prompt. Defining these candidate viewpoints therefore does not require knowing the prompted target's location. The building at the center of the satellite image is used to define the candidate viewpoints, while the prompt specifies the target to be localized.
We have revised the viewpoint description in Appendix A (page 14).Our current policy follows a decision loop similar to graph-based VLN. In graph-based VLN, an agent selects the next navigation node using language instructions, current observations, and history, then continues deciding from observations obtained upon arrival (Bridging the Gap [1], CVPR 2022). Our policy similarly uses the target prompt, localization state, and observation history to select the next candidate viewpoint or stop. In the current evaluation, selecting a viewpoint reveals its pre-collected image. For UAV flight, this loop could be extended so that a VLA policy determines observation positions from online feedback and a flight controller executes the corresponding movements, as explained below.
Thanks for highlighting useful observations available at intermediate positions. A VLA-style extension could use these observations to update localization, adjust movement, or stop before reaching the intended viewpoint. Our framework provides a foundation for anticipating which observations are useful and deciding when to stop.
VLA models map visual observations and language instructions to robot actions (OpenVLA [2], CoRL 2024). For UAV deployment, a VLA policy could take the target's language description and initial visual prompt, current onboard image, satellite reference image, localization history, and UAV state as inputs, and output relative movement and orientation commands. A flight controller could execute short movements from these commands, allowing the policy to decide again from newly acquired images. Observation positions could thus be determined and reached incrementally through online feedback.
For example, if a building partly blocks the target, the policy could anticipate that moving slightly left would reveal more. Images acquired during movement could then be used to update localization and decide whether to continue. If an intermediate observation already provides sufficient localization evidence, the policy could stop before reaching the intended position; if ambiguity remains, it could adjust its direction to seek a more informative view. Intermediate observations could therefore change both where the UAV observes and when it stops during movement.
The key decision throughout this loop is whether another observation is likely to improve localization enough to justify its cost. Our framework learns to make this decision through multi-view localization updates and policy training with the gain-cost reward, connecting the current localization state to viewpoint selection and stopping. These capabilities provide a foundation for extending active localization to continuous UAV control, as discussed in Appendix C.5. We examine viewpoint-dependent movement costs in Part 2 below.
We have revised the discussion in Appendix C.5 (page 19).