- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies view understanding and interactive camera planning in vision-language models. It introduces ViewSuite, a benchmark comprising Path-to-View, View-to-Path, and Interactive View Planning tasks constructed from shared initial–target view pairs. The evaluations reveal a substantial performance gap between interpreting supplied paths and planning camera movements through interaction. To improve learning from a low-success initial policy, the authors propose View Graph Distillation, which organizes self-exploration experience into a persistent graph and constructs supervision through goal relabeling and cross-trajectory path sampling. Alternating supervised fine-tuning with reinforcement learning improves Qwen2.5-VL-7B’s IVP success from 2.5% to 47.7% on held-out scenes.
Critical strengths
1.The benchmark and diagnostic analyses provide useful insights. Constructing three complementary tasks from shared view pairs provides a useful framework for studying supplied-path understanding and interactive camera control. The analyses of viewpoint distance, interaction budgets, reasoning settings, and failure types help characterize the specific difficulties current VLMs face.
2.The training framework effectively reuses exploration experience. Turning unsuccessful exploration into valid supervision substantially improves learning over direct reinforcement learning. Although this idea is related to hindsight relabeling, its application to the learning bottleneck identified in this paper is well motivated and effective. The inclusion of a trajectory-level relabeling baseline is particularly helpful for distinguishing the benefits of reusing failed trajectories from the additional contribution of the graph.
Critical weaknesses
1.Appendix A.3 shows that benchmark construction already produces executable action sequences between initial and target views. These could be used for SFT followed by RL, potentially addressing the low-success initialization problem without self-exploration. Without this baseline, the additional benefit of the proposed pipeline over simpler supervised training remains unclear.
2.Removing rotation snapping reduces the trained model's success rate from 47.8% to 19.6%. This suggests that the trained policy is more sensitive to changes in camera execution rules, which deserves further investigation.
3.The graph-based approach improves IVP success from 41.7% to 47.7% over trajectory-level relabeling. Since the comparison uses single training runs, repeated experiments would help establish the consistency of this improvement.
4.The final model shows lower P2V and V2P accuracy than the base model, despite the auxiliary view-understanding supervision. Further analysis of this performance drop would be helpful.
Questions for authors
1.Is the improvement over trajectory-level relabeling consistent across random seeds? Are any repeated-run results available to support this comparison?
2.Why does removing rotation snapping cause such a large performance drop? Could the authors provide a brief analysis to help explain it?
3.Appendix A.3 indicates that benchmark construction already produces executable action sequences connecting the initial and target views. Could the authors compare their method with a simple SFT-plus-RL baseline using these existing training-split paths as demonstrations? This would help establish the additional benefit of self-exploration and graph construction over readily available supervision.
4.Does the ScanNet camera environment enforce collision constraints with walls and objects, or can the camera move freely through scene geometry? Please clarify whether any special restrictions or handling are applied when a camera movement would result in a collision.
4 comments
Rebuttal-2
[W3, Q1] Consistency of Gains over Trajectory-Level Relabeling
The gain is consistent. We retrained both variants from scratch under an updated environment configuration (mesh rendering, ground-aligned camera actions, strict success thresholds) on the same 530 test pairs. Because the configuration changed, we compare the gap within each round.
| Split (n) | Run 1 | Run 2 | Pooled [95% CI] |
|---|---|---|---|
| Short (187) | +2.1 | -1.1 | +0.5 [-4.9, +5.9] |
| Long (343) | +8.1 | +7.3 | +7.7 [+3.8, +11.6] |
| All (530) | +6.0 | +4.3 | +5.2 [+2.0, +8.3] |
Δ = IVP success of view graph (ours) minus trajectory-level relabeling
Three observations show that this gap is not run-to-run noise:
- It replicates. Across rounds, the gap changes by only 1.7 points overall and 0.8 points on Long targets, less than the sampling error of a single comparison on this test set. On Long targets, the graph variant alone solves 59 test episodes and relabeling alone solves 34 (paired McNemar p = 0.012).
- It grows with target distance. The two variants are level on nearby targets, which a single trajectory already reaches, and the graph leads on distant ones:
| View distance d | < 2 | 2 to 3 | 3 to 4 | 4 to 5 | ≥ 5 |
|---|---|---|---|---|---|
| Δ (pts), re-run | 0.0 | -1.9 | +3.8 | +4.7 | +12.8 (p = 0.009) |
[W4] P2V and V2P Accuracy of the Final Checkpoint
The drop is answer-format collapse in a checkpoint trained without auxiliary supervision, not weaker view understanding. The reported 6.0 / 20.9 come from a checkpoint trained on IVP alone; the paper wrongly attributes them to the full model, and we will correct this. A P2V score far below the 25% chance level already signals a format failure.
New analysis: format audit of the test rollouts. In this IVP-only checkpoint, about four-fifths of P2V replies and over a third of V2P replies are camera-action strings that the answer(letter) parser rejects (almost none for the base model). On replies that do give an option, accuracy is comparable to the base model.
Auxiliary supervision prevents the drift. In the paper's ablation, distilling planning alone halves single-turn accuracy, while adding the auxiliary view tasks keeps it at the base level:
| Distilled supervision | P2V | V2P |
|---|---|---|
| Untrained base | 30.0 | 27.1 |
| Planning only | 14.0 | 11.1 |
| + Auxiliary view tasks (ours) | 28.0 | 29.0 |
The full model, trained with auxiliary supervision through final RL in our re-run (W3), scores 27.0 / 33.0 against 28.5 / 27.9 for the base model, with only 26 and 25 of 530 replies off-format.
[Q4] Collision Handling
The environment does not enforce collisions, by design, and re-scoring with a collision rule leaves our conclusions unchanged. ViewSuite studies viewpoint planning with a free virtual camera rather than embodied navigation. Poses therefore update deterministically, and no restriction is applied when a movement passes through scene geometry. When building training data, the view graph discards renders that are mostly empty (void fraction > 0.7), such as views from inside geometry.
New analysis: collision re-scoring. We replayed every IVP test episode, checked each translation step against the ScanNet scene mesh, and counted an episode as failed if the camera collided with the scene before submitting.
| IVP, All (%) | Default success | Failed if any collision before submitting | Episodes with at least one collision |
|---|---|---|---|
| Ours | 47.7 | 44.5 | 13.0 |
| Gemini 3.1 Pro | 21.3 | 17.0 | 40.8 |
| GPT-5.1 | 6.2 | 5.3 | 37.2 |
| Qwen2.5-VL-7B base | 2.5 | 2.3 | 37.9 |
Success drops by at most 4.3 points for every model and the ranking is unchanged. Our lead over Gemini 3.1 Pro even grows slightly, from 26.4 to 27.5 points. Our policy also collides far less often: in 13.0% of episodes, compared with about 40% for the other models, including its own base model. A 5 cm collision radius gives the same trend.
We thank Reviewer vCDu again and hope these clarifications address the concerns. We are happy to provide further analysis during the discussion period.
Dear Reviewer vCDu,
Thank you again for your thoughtful reviews. As the discussion period is ending soon, we wanted to kindly check whether our responses and the added results have addressed your concerns. If they have, we would be grateful if you could consider updating your assessment. We are also happy to clarify anything further.
Best, The Authors
Dear Reviewer vCDu,
Thank you again for your detailed review. Before the discussion closes, we would like to briefly summarize how we addressed your concerns. In short, our main conclusions hold across every suggested additional experiments, and the variation between runs is small. If these results address your concerns, we would be grateful if you would consider updating your score.
| Your concern | What we did |
|---|---|
| W1 / Q3: Comparison with SFT + RL on the benchmark-construction paths | We ran this baseline. Oracle SFT + RL reaches 30% success, while our method reaches 46% without using any expert paths. |
| W2 / Q2: Drop without rotation snapping | The drop reflects a train-test shift: the policy was trained only with snapped rotations and is specialized to these dynamics. Even under this shift, it remains the strongest evaluated model (19.6% vs. 15.7% for Gemini 3.1 Pro). The revised PDF states this in Section 5, Appendix B.4 and the limitations. |
| W3 / Q1: Consistency of the gain | The gain is robust. An independent training round reproduces it on Long targets (+8.1 and +7.3 points), and the 0.8-point difference between the two rounds is smaller than the sampling error of the test set. Pooled over both rounds, the gain is +7.7 points (95% CI +3.8 to +11.6) and is significant in a paired test (p = 0.012). |
| W4: Lower P2V / V2P after training | The drop comes from answer-format collapse, not from weaker view understanding. With auxiliary supervision, the final model stays at the base level (27.0 / 33.0 vs. 28.5 / 27.9). |
| Q4: Collision handling | When any collision counts as a failure, success changes only slightly and the ranking of models stays the same. Our policy also collides much less often, in 13.0% of episodes compared with about 40% for the other models. |
Please let us know if you have any further concern, we are happy to clarify anything further.
Best, The Authors
Rebuttal-1
We thank Reviewer vCDu for recognizing the insights from our benchmark and diagnostic analyses, as well as for noting that our training framework and trajectory-level relabeling baseline effectively isolate the benefits of reusing exploration experience. We address your specific questions and concerns below.
[W1, Q3] Training on Benchmark-Construction Solver Paths vs. Self-Exploration
We thank the reviewer for this suggestion and ran the requested baseline: SFT on the training-split solver paths from Appendix A.3, followed by the same RL stage as our method, on Qwen2.5-VL-7B.
As shown in the table below, while the Oracle SFT baseline's performance plateaus around 0.30, whereas our proposed method reaches 0.47.
[W2, Q2] Performance Drop without Rotation Snapping
The drop mainly reflects a train-test mismatch in execution rules, and our policy is still the strongest model under No-Snap. Our policy was trained only with the default interface, which snaps rotations to multiples of 30°. Removing snapping at test time changes how every rotation executes, so the policy meets camera poses it never saw during training. The zero-shot models were never adapted to either rule, which is why their drop is smaller.
Without training, Qwen2.5-VL-7B reaches 2.5%, far below our 19.6% without snapping.