- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PanoAgent, an active panoramic understanding framework that enables VLMs to select task-relevant observations through two tools: PlanarView for local perspective rendering and PanoRotate for reference-heading adjustment. The model is trained using trajectory-based supervised fine-tuning followed by GRPO with outcome and geometry-aware rewards. Experiments on multiple panoramic benchmarks demonstrate substantial improvements over base models and competitive panoramic adaptation methods.
Critical strengths
-
Strong empirical performance. PanoAgent achieves substantial improvements over its corresponding base models across three panoramic benchmarks at both 4B and 9B scales. The results demonstrate the practical effectiveness of the framework.
-
Effective training framework. The combination of trajectory supervision and geometry-aware reinforcement learning is well motivated. Ablation studies show consistent improvements from trajectory SFT and geometric rewards.
Critical weaknesses
-
Insufficient isolation of the active observation contribution. The comparison between VQA SFT and trajectory SFT simultaneously changes supervision richness and access to additional visual observations. It remains unclear how much improvement specifically comes from adaptive observation rather than additional visual evidence. A compute or observation-budget-matched fixed multi-view baseline would strengthen the contribution.
-
Limited evidence for efficiency claims. Training-data efficiency is measured primarily by the number of VQA examples without accounting for trajectory generation and teacher model costs. Furthermore, the PanoWorld inference time comparison uses a backbone proxy rather than the complete model, limiting the fairness of the efficiency analysis.
Questions for authors
-
How does PanoAgent compare against a fixed multi-view or non-adaptive observation baseline under matched visual-token and inference budgets?
-
In the Pixel versus Spherical interface comparison, could the approximate coordinate conversions introduce inconsistencies between training observations and actual inference-time views? How much do these inconsistencies affect performance?
-
Could the authors provide more comprehensive training and inference cost comparisons, including trajectory synthesis overhead and actual end-to-end latency?
2 comments
Thanks to the authors for their response. I will maintain my original positive score.
Response to Reviewer B8pD
Thank you for recognizing the empirical results and for identifying the need to better separate observation adaptivity from additional evidence and computational cost.
1. Fixed observations and inference budgets. We evaluated ERP and fixed multi-view VQA-SFT baselines with increased visual budgets. Both remain less accurate than PanoAgent on 360Bench, ODI-Bench, and PanoSpace-Bench; General Response G3 (Table 2) reports the settings and results.
2. Pixel training–execution consistency. Both interfaces use the same source demonstrations. For supervised fine-tuning with the Pixel interface, we convert spherical local-view actions into ERP bounding boxes but retain the original observation images. At inference, the boxes are decoded into viewing directions and FoVs for rendering. This approximate conversion may produce views that differ from the training observations.
We compare the spherical coverage of the original training views and the views produced by the converted Pixel actions. The mean IoU is 0.9605. Differences are concentrated in views covering the poles, where ERP distortion limits how accurately rectangular boxes represent spherical views; other views match within numerical precision. Spherical actions specify viewing directions and FoVs directly, avoiding this conversion. Our comparison therefore evaluates the practical suitability of the interfaces, including their geometric constraints.
3. Training and inference costs. We use Qwen3.6-27B to generate and validate trajectory reasoning, producing approximately 40,000 trajectories in about 11 hours on eight H20 GPUs. For comparison, an SFT run of the 4B model using these data takes approximately 10 hours on the same GPU configuration. Since synthesis takes about as long as one SFT run and the generated data can be reused, we consider the upfront cost manageable.
For inference, the full PanoWorld-9B model, including its spherical module, takes 1.94 s per case. The updated measurement and the comparison with PanoAgent are reported in General Response G1 and G2 (Table 1).