acceptodds
Under review as a conference paper at ICLR 2027

Learning to Think and Explore in 3D World

Abstract

We study how vision-language models (VLMs) can learn to think and actively explore in 3D space. Instead of assuming informative input and providing multi-view images, videos, or complete 3D representations directly to the model, we preserve the native visual encoding of VLMs: a single image at a time, just as humans perceive the world. We treat the 3D representation only as an environment with which the model can interact through actions, such as moving left, and observations in the form of rendered images. Each rollout begins from a fixed initial first-person view together with a visual question. The model can then iteratively reason, navigate, and reorient its viewpoint, with each action inducing a new observation, before committing to a final answer. Following an initial stage of trajectory-supervised fine-tuning, we optimize the interactive policy with reinforcement learning (RL), using rewards driven primarily by final-answer correctness. Crucially, RL provides no direct supervision over which viewpoints the model should visit or which exploration actions it should take. Instead, the model must learn an effective exploration policy solely from the extent to which its interactions improve performance. Across six 3D benchmarks, our policy achieves the highest average score among active models at 54.1%, improving upon its base Qwen model by 22.9% and surpassing the proprietary Gemini 3.6 Flash model by 2.6% under the same interaction protocol. The evaluation includes four out-of-domain benchmarks, on which our policy achieves an average improvement of 7.2%. It also outperforms every evaluated open-source 3D model despite using 50–70% fewer input frames and lower-resolution renders. Further analysis shows that RL improves accuracy across all interaction levels by 3.2–6.2% and induces question-conditioned exploration: the policy searches for occluded targets, briefly confirms visible ones, and gathers multiple viewpoints for relational comparisons. Our RL design enables VLMs not only to reason over observed evidence, but also to decide when and how to acquire the evidence needed for generalizable 3D question answering.

Primary area:
reinforcement learning
Keywords:
embodied reasoning, vision-language models, reinforcement learning, GRPO, 3D question answering

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_iPE7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies 3D question answering through active viewpoint exploration. Starting from a single rendered image, a VLM moves a virtual camera to gather additional observations before answering. Training consists of supervised fine-tuning on teacher-generated trajectories followed by GRPO, with rewards for answer correctness, tool use, object visibility, and output format. Across six benchmarks, the final model improves average accuracy from 48.6% after SFT to 54.1% after RL. The paper also studies several design choices, including relative camera actions, rendered versus captured images, and supervised initialization.

Critical strengths

The paper addresses a meaningful problem: answering spatial questions often requires obtaining information that is missing from the initial view. The interaction setup makes it possible to study observation selection alongside question answering.

The RL model improves over its SFT initialization on all six benchmarks. The comparisons with general-purpose VLMs under the same interaction interface are also useful and show that the trained model performs competitively in this setting.

Some of the design studies are informative. In particular, the comparison between rendered views and nearby photographs shows the importance of returning observations that match the requested camera pose. The relative-versus-absolute action comparison provides another useful implementation finding.

Critical weaknesses

The novelty over existing active-view methods needs to be better established. CoV already alternates reasoning, camera actions, and rendered observations for 3D QA. Think3D-RL already uses GRPO with terminal answer and format rewards to train multi-step viewpoint selection. The single-view starting condition and access to an external scanned scene distinguish this paper, but I am not yet convinced that these differences establish a substantial methodological advance. Both works are discussed, yet neither is included in the experiments. Comparing against compatible versions of these methods under matched conditions would help establish what the proposed approach adds.

The claim about learning exploration from answer correctness is stronger than the evidence supports. The reward includes a bonus for each tool call in a successful episode and an object-visibility term based on ground-truth 3D annotations. Under the stated weights, twelve calls contribute 0.6, while the two answer-correctness terms contribute at most 0.5. Thus, the objective explicitly encourages longer successful trajectories, even if some calls are redundant. Table 2 examines the tool bonus, but the paper does not report an ablation of the grounding reward or remove both terms together. Learning without intermediate action labels is a reasonable claim; learning useful exploration solely from final-answer correctness has not been demonstrated.

It remains unclear how much of the gain comes from better exploration. RL changes both the observation policy and the model’s ability to answer from those observations. Figure 5 does not separate these effects: grouping questions by the RL policy’s call count does not ensure that SFT and RL receive the same amount of evidence. The qualitative examples are plausible, but do not show whether RL makes better viewpoint choices than SFT or a simple exploration policy. Evaluating SFT and RL answer models on the same collected views, and comparing different exploration policies with a fixed answer model and matched budgets, would make the main conclusion more convincing.

The OOD evaluation needs clarification. Figure 3 includes 7,534 ARKit questions in the RL data, and Section 4.1 confirms that ARKit is used for training. However, Table 2 labels ARKit as OOD, and its improvement contributes substantially to the reported four-benchmark OOD gain. Held-out scenes or question types could still provide a useful generalization test, but that is different from transfer to an unseen dataset domain. The paper should specify these distinctions and document scene overlap across all training and evaluation sources, particularly for the generated ScanNet and ARKit questions. As written, the cross-domain generalization claim is difficult to assess.

Questions for authors

What defines OOD in Tables 1–2? How does the ARKit evaluation set differ from the ARKit training data, and are scenes disjoint across all training and evaluation sources?

How is visibility determined for the grounding reward, particularly when the target is occluded? What proportion of training examples receives this reward, and are results available without both grounding supervision and the tool-call bonus?

In Figure 5, are SFT and RL evaluated on the same questions grouped by the RL policy’s call count? What are the SFT call counts within these groups?

Are all Table 1 outputs scored with the same evaluation prompt and judge settings? Since the same judge model is used during RL and evaluation, has the improvement been checked with an independent judge or a human-reviewed sample?

13 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer iPE7 1/5

We thank the reviewer for the in-depth review. We appreciate the recognition that the paper addresses a meaningful problem, that the RL policy improves over its SFT initialization on all six benchmarks and is competitive with general-purpose VLMs under the same interaction interface, and that the design studies on rendered versus captured views and relative versus absolute actions are informative. We address each point below.

>>> W1

Novelty over CoV (reasoning + camera actions + rendered observations) and Think3D-RL (GRPO with answer and format rewards for viewpoint selection). Neither is included in the experiments; compare them under matched conditions.

>>> A1 Thanks for raising this concern. Below, we compare Act3D with CoV and Think3D-RL in detail to better establish our contribution and novelty. The key distinction between these approaches and ours lies in their assumption that complete 3D information is provided as input to the model. Under this assumption, both Think3D-RL and CoV start from a set of input images. Specifically, Think3D-RL takes multiple images as input and uses Pi3 to reconstruct a 3D point cloud. Renderings of this point cloud can reveal details from new perspectives, but the available evidence is bounded by what Pi3 recovers from the input images. Think3D-RL selects from a finite set of overhead viewpoints at the corners of the scene (in 72.0% of the tasks we tested below, it chooses the same view: 0° azimuth, looking down at 60°), whereas Act3D supports a continuous range of camera poses (free exploration). CoV similarly has access to prerecorded video frames and even a bird’s-eye view of the scene before refining the camera position and orientation. These initial inputs expose 99.7% of the target objects in the ScanQA validation set. Furthermore, CoV is a training-free, prompt-based framework and therefore relies on the underlying model’s existing ability to explore and reason in 3D. As a result, CoV cannot equip the model with an intrinsic capability to explore and search in 3D space.

Act3D formulates 3D reasoning under partial observability and exposes the scene solely as an interactive environment. Each rollout begins with a single egocentric image. Receiving only the resulting first-person renderings, the model must iteratively navigate and reorient the camera to acquire new information, while reasoning about the spatial relations between observations to answer the question. Through interaction with this otherwise hidden 3D environment, Act3D learns when and how to acquire missing visual evidence.

To further address your concern, we compare against these methods under strictly matched conditions. Specifically, we evaluated Act3D, Think3D-RL, and CoV on the same six 3D QA benchmarks and measured target exposure on ScanQA to assess how often each method brought the target object into view. Act3D achieved the highest accuracy on all six benchmarks, averaging 54.1%, compared with 41.4% for Think3D-RL and 41.9% for CoV. On ScanQA, Act3D exposed 95.1% of the target objects, compared with 86.8% for Think3D-RL and 33.3% for CoV. The exposure results show that Act3D’s learned policy more frequently acquires task-relevant visual evidence.

Comparison with Think3D-RL and CoV under matched conditions.

Model (active)ScanQASQA3DSPB MVSPB SIMP3DARKitAvgScanQA Target Exposed (%)
Think3D-RL*31.443.336.733.048.455.841.486.8
CoV on Qwen3.5-9B**23.737.742.037.948.461.941.933.3
Act3D (mixed, ours)42.547.750.541.262.979.654.195.1

* Instead of running Think3D-RL's official pipeline, which starts from 3D reconstruction with Pi3, we skip reconstruction and give it the ground-truth 3D mesh, so that its results are comparable to ours in terms of exploration ability. Its ScanQA target exposure is high despite requesting only 0–2 viewpoints, because its selectable viewpoints include overhead views.

** CoV's official evaluation uses privileged starting input, including a bird's-eye view, up to 15 viewpoints of the scene, and a starting-view selector. For comparability, we reduced the starting view to a single viewpoint and removed the viewpoint selector.

AuthorsOpenReview ↗

Rebuttal for Reviewer iPE7 2/5

>>> W2 & Q2

The claim of learning exploration from answer correctness is too strong: the tool bonus (12 calls = 0.6 vs. at most 0.5 for correctness) encourages long trajectories, and the grounding term uses ground-truth 3D annotations. How is visibility determined when the target is occluded, what fraction of examples receive the grounding reward, and are there results without both the grounding term and the tool bonus?

>>> A2

In the current reward, the tool bonus is gated by judged answer correctness: Rjudge∈{0,1}R_{\mathrm{judge}}\in\lbrace 0,1\rbrace, the LLM judge’s verdict on whether the final answer matches the ground truth, acts as an on/off switch for the tool bonus. Answer correctness therefore governs the tool-bonus term as well, so its influence on the reward is not limited to its own, smaller weight.

The grounding reward applies only to questions whose answer object has an annotated 3D location. This covers about 75.6% of the RL corpus (38,981 of 51,543 questions): all ScanQA and curated ScanNet questions, 64.1% of curated ARKit questions, and no MP3D questions. For the remaining questions, the grounding term is zero for every rollout and cancels out of the GRPO advantage, so their updates are driven by the other reward terms, mainly answer correctness. Grounding is 1 if the target objevt falls inside the image boundaries of any view in full trajectrary and is unoccluded (casting a ray from the camera toward the object, if the first mesh hit is more than 0.30 m before the object, it is still considered occluded).

To further quantify the contribution of the grounding reward and the tool-call bonus in Eq. 7, we compare the full-reward policy with two ablations in the mixed-data setting. The first removes the grounding condition while retaining the interaction bonus. The second removes both. We report answer accuracy, the average number of views, and how often the policy brings the annotated target into view. We evaluate all policies on the full benchmark pools (11,522 questions) using the native harness, a budget of 12 interactions, and a sampling temperature of 0.7. As shown in Table R2, removing the grounding reward reduces averaged accuracy from 54.1% to 53.2% and ScanQA target exposure from 95.1% to 91.6%, with both policies averaging 11.0 views per question across the six benchmarks. With both rewards removed, RL still improves accuracy from 48.6% to 53.3% and target exposure from 78.7% to 84.7% over the SFT initialization, with similar average view counts (5.0 vs. 5.3).

Table R2. Grounding-reward ablation in the mixed-data setting. #Views includes the initial view and is averaged across the six benchmarks. ScanQA Target Exposed is the percentage of ScanQA questions for which at least 5% of the target object is visible and unoccluded in any observed view.

ScanQASQA3DSPB MVSPB SIMP3DARKitAvg#ViewsScanQA Target Exposed (%)
SFT init39.347.439.237.761.865.948.65.078.7
Act3D, full reward (ours)42.547.750.541.262.979.654.111.095.1
Act3D, no grounding reward41.847.845.440.364.779.453.211.091.6
Act3D, no grounding reward, no tool bonus38.947.248.139.164.781.653.35.384.7
Act3D, maximize tool bonus when grounded41.248.348.841.367.180.454.58.491.0

Tool bonus supports our goal of active exploration that yields higher capture of targeted object (+) more than pure accuracy gain. If we also want to further optimize efficiency, we experimented with one more reward interaction: tool bonus credited only with a correct answer, but verified grounding now acts as a shortcut to the full credit rather than as a separate bonus:

\begin{cases} N_{\max}, & R_{\mathrm{gnd}}=1, \cr \min\lbrace n_{\mathrm{int}},N_{\max}\rbrace , & \text{otherwise,} \end{cases}$$ where $n_{\mathrm{int}}$ is the number of interactions in $\tau$, and $R_{\mathrm{gnd}}\in\lbrace 0,1\rbrace$ uses the annotated 3D location of the object of interest, taking the value 1 when that object is visible in some rendered view of $\tau$. A rollout that is judged correct *and* has actually brought the target object into view therefore receives the full credit $N_{\max}$ at once, after as few as a single call. The result for "Act3D, maximize tool bonus when grounded" in Table R2 shows that gives matching or slightly better average accuracy over the current policy (**54.5% vs. 54.1%**), while using fewer views (**8.4 vs. 11.0**) at a lower exposure (91.0% vs. 95.1%). This suggests that tying the tool bonus to grounding encourages more efficient exploration. Yet, higher accuracy at lower target exposed rate may be attributed to better answering in low-exploring problem as it contains shorter interleaved chat history. We will include this result in the revised paper.
AuthorsOpenReview ↗

Rebuttal for Reviewer iPE7 3/5

>>> W3 & Q3

It's unclear how much of the gain comes from better exploration. Fig. 5 groups by the RL policy's call count, so SFT and RL don't receive the same evidence. Evaluate SFT and RL answerers on the same views, and compare exploration policies with a fixed answerer and matched budget. In Fig. 5, are SFT and RL evaluated on the same questions, and what are SFT's call counts within each group?

>>> A3

Yes. In Fig. 5, the base, SFT and RL models are evaluated on the same 4,649 ScanQA validation questions, grouped by the RL policy's number of moves. To avoid a grouping that depends on any particular policy's behaviour, we instead compute, from scene geometry alone, the number of interactions needed to bring the answer object into view from the initial pose: ⌈|bearing| / 30°⌉ default turns (at least 1), plus forward moves if the object is still more than 2.5 m away. Table R11 below re-plots Fig. 5 with this new grouping.

Under this scene-grounded grouping, SFT makes 2.0–3.5 calls in groups 2–6 and 5.4 in group 7+, rising with difficulty but well below the number of interactions needed in groups 3 and above. RL’s gain over SFT is therefore largest where SFT under-explores most: +4.4 to +5.4 points in groups 3, 4 and 6, and +10.9 in group 7+, where the object is farthest from the initial view.

Table R11. Change in accuracy relative to the base model on ScanQA validation, grouped by the number of interactions needed.

Interactions neededSFT − baseRL − baseRL − SFT
2-−1.1+0.8+1.9
3+4.0+8.4+4.4
4+5.6+11.0+5.4
5+5.5+8.0+2.5
6+6.5+11.3+4.8
7++5.4+16.3+10.9

To further isolate the effect of interaction in a controlled way, we conducted two analyses.

First, we measure whether the policy finds relevant objects using ScanQA target exposure, the percentage of questions for which at least 5% of the annotated target object is visible and unoccluded in any observed view during the model's inference rollout.

Table R3. Answer accuracy and target exposure on the full benchmark pools. #Views counts the initial view and all views returned by interactions, averaged across the six benchmarks. The initial view alone exposes the target in 50.6% of ScanQA questions.

Model (active)ScanQASQA3DSPB MVSPB SIMP3DARKitAvg#ViewsScanQA Target Exposed (%)
Qwen3.5-9B (base)36.733.521.324.730.940.131.25.462.9
Qwen3.5-27B27.540.233.231.433.833.833.37.076.6
InternVL3.5-38B30.941.543.241.650.155.843.92.155.1
GLM-4.6V30.642.143.639.540.445.840.35.367.5
Gemini 3.6 Flash39.153.845.544.453.473.051.57.789.5
Gemini 3.1 Pro40.752.750.343.964.472.254.08.589.7
SFT init (mixed)39.347.439.237.761.865.948.65.078.7
Act3D (mixed, ours)42.547.750.541.262.979.654.111.095.1

Second, we hold the answerer fixed and vary only the explorer that collects the views. On a 300-question-per-benchmark subset, we give Gemini 3.6 Flash the question and the view sequence collected by each explorer. Each explorer retains its original trajectory length. As shown in Table R6, the fixed answerer achieves its highest mean accuracy with Act3D’s views (52.2%), compared with 49.5% using views from the SFT initialization and 49.8% using views collected by Gemini 3.6 Flash itself. Act3D’s views improve accuracy over SFT views on all six benchmarks. With the answerer held fixed, these gains show that the observations collected by the RL policy provide more useful evidence for answering.

Table R6. QA accuracy (%) with Gemini 3.6 Flash as the fixed answerer on a 300-question-per-benchmark subset.

Explorer (views from)ScanQASQA3DSPB MVSPB SIMP3DARKitAvg
Act3D (RL)36.355.043.434.463.081.052.2
SFT init35.053.035.731.562.379.749.5
Base Qwen3.5-9B31.347.732.926.659.073.045.1
Gemini 3.6 Flash35.351.734.636.462.778.349.8
Random walk31.046.029.231.760.374.045.4
Fixed turn-in-place sweep35.051.732.933.960.379.348.9
Reviewer_ZraeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how vision-language models (VLMs) can learn to reason and actively explore in 3D environments. Starting from a single rendered view and a question, the model interacts with a pre-existing 3D scene through camera movements to gather evidence before producing a final answer. The authors formulate the task as a partially observable decision process and adopt a two-stage training approach: supervised fine-tuning on curated teacher-generated interaction trajectories, followed by reinforcement learning to optimize the interactive policy. Across six 3D benchmarks, the resulting policy achieves the highest average performance among the evaluated active models. The reported results also show accuracy improvements over the SFT initialization across different levels of tool use.

Critical strengths
  1. The paper studies how VLMs can select camera movements to collect missing evidence before answering questions. This setting addresses an important aspect of spatial reasoning with missing visual evidence by deciding what to observe, when to take the action and how to interpret available observations.
  2. The mixed-data RL policy improves average performance from 48.6% to 54.1% with gains on all 6 evaluated benchmarks. This provides concrete evidence that reinforcement learning adds value beyond the trajectory-supervised initialization and is the strongest support for the proposed training approach.
Critical weaknesses
  1. The experiments do not sufficiently isolate the contribution of the learned exploration strategy. The paper includes active and passive baselines and shows that RL improves answer accuracy. However, the mixed-data RL policy also takes more camera actions and receives more images, while the reward explicitly encourages tool calls on correct rollouts. The gains may therefore reflect additional observations or improved answer generation, as well as better exploration. The active/passive comparisons do not fully separate these effects because the answering models or observation budgets differ. A comparison using the same answering model checkpoint and observation budget would help distinguish better evidence acquisition from these other sources of improvement. This would strengthen the evidence for the paper’s claim that RL enables task-directed exploration.

  2. The link between the task definition and the implementation is unclear. Section 3.1 defines the decision process, action space, interaction history, and implicit belief. Sections 3.2 and 3.3 do not clearly explain how these concepts connect to SFT and RL. Although the paper states that the model uses the full interaction history, it does not sufficiently explain how this history is constructed and managed. The authors should also clarify how model outputs become executable camera actions and whether implicit belief has any operational role beyond describing the model’s internal representation. A step-by-step algorithm would help readers understand and reproduce the method.

Questions for authors
  1. Can you provide a comparison of learned exploration with predefined or random action policies, using the same answering model checkpoint and observation budget? This would help determine whether the gains come from better evidence acquisition, rather than additional images or improved answer generation.
  2. How is the interaction history constructed and updated during SFT, RL, and evaluation? Is implicit belief only a conceptual interpretation of VLM’s internal representation, or does it have an operational role in the method?
  3. Why is ARKit out-of-domain when generated ARKit questions are included in RL training? Please clarify what “out-of-domain” means here and how training and evaluation scenes and questions are separated.
8 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer Zrae 1/3

We thank the reviewer for the detailed review. We appreciate the recognition that learning to choose camera movements to collect missing evidence addresses an important aspect of spatial reasoning, and that the RL gains over the SFT initialization on all six benchmarks show reinforcement learning adds value beyond trajectory supervision. We address each point below.

>>> W1 & Q1 The contribution of learned exploration isn't isolated: RL also takes more actions and sees more images, and tool calls are rewarded. Compare against predefined or random policies with the same answering checkpoint and observation budget.

>>> A1 Thanks for the constructive question. To answer this question, we first measure whether the policy finds relevant objects using ScanQA target exposure, the percentage of questions for which at least 5% of the annotated target object is visible and unoccluded in any observed view during the model's inference rollout.

In the "6-view budget (mixed)" setup in Table R3, we train Act3D in the mixed-data setting with a six-view limit during RL, which forces the model to explore within only 6 interactions before answering the question. We apply the same limit during evaluation. The other policies use the standard 12-interaction protocol. In Table R3, the six-view Act3D policy achieves 52.6% averaged accuracy and 87.0% ScanQA target exposure using 4.73 views per question. This policy outperforms most of the compared methods and the SFT init on both metrics while using fewer views, and approaches the two Gemini baselines on both metrics with 39–44% fewer views on average. This illustrates that Act3D can learn to acquire useful and relevant observations both effectively and efficiently.

Table R3. Answer accuracy and target exposure on the full benchmark pools. #Views counts the initial view and all views returned by interactions, averaged across the six benchmarks. The initial view alone exposes the target in 50.6% of ScanQA questions.

Model (active)ScanQASQA3DSPB MVSPB SIMP3DARKitAvg#ViewsScanQA Target Exposed (%)
Qwen3.5-9B (base)36.733.521.324.730.940.131.25.462.9
Qwen3.5-27B27.540.233.231.433.833.833.37.076.6
InternVL3.5-38B30.941.543.241.650.155.843.92.155.1
GLM-4.6V30.642.143.639.540.445.840.35.367.5
Gemini 3.6 Flash39.153.845.544.453.473.051.57.789.5
Gemini 3.1 Pro40.752.750.343.964.472.254.08.589.7
SFT init (mixed)39.347.439.237.761.865.948.65.078.7
Act3D, 6-view budget (mixed)43.748.046.341.860.974.952.64.7387.0
Act3D (mixed, ours)42.547.750.541.262.979.654.111.095.1

Additionally, we hold the answerer fixed and vary only the explorer that collects the views. On a 300-question-per-benchmark subset, we give Gemini 3.6 Flash the question and the view sequence collected by each explorer. Each explorer retains its original trajectory length.

As shown in Table R6, the fixed answerer achieves its highest mean accuracy with Act3D’s views (52.2%), compared with 49.5% using views from the SFT initialization and 49.8% using views collected by Gemini 3.6 Flash itself. Act3D’s views improve accuracy over SFT views on all six benchmarks. With the answerer held fixed, these gains show that the observations collected by the RL policy provide more useful evidence for answering.

Table R6. QA accuracy (%) with Gemini 3.6 Flash as the fixed answerer on a 300-question-per-benchmark subset.

Explorer (views from)ScanQASQA3DSPB MVSPB SIMP3DARKitAvg
Act3D (RL)36.355.043.434.463.081.052.2
SFT init35.053.035.731.562.379.749.5
Base Qwen3.5-9B31.347.732.926.659.073.045.1
Gemini 3.6 Flash35.351.734.636.462.778.349.8
Random walk31.046.029.231.760.374.045.4
Fixed turn-in-place sweep35.051.732.933.960.379.348.9

The passive baselines in Table 1 of our paper already use the same maximum budget, message and scoring, with all passive models given 12 frames from systematic viewpoint sampling and all active models using at most 12 frames collected through their own exploration.

AuthorsOpenReview ↗

Rebuttal for Reviewer Zrae 2/3

>>> W2 & Q2 The link between the POMDP formulation (Sec. 3.1) and SFT/RL (Secs. 3.2–3.3) is unclear. How is the interaction history built and managed, how do model outputs become camera actions, does "implicit belief" have an operational role, and could a step-by-step algorithm be added?

>>> A2 The interaction history in Sec. 3.1 is implemented as an ordered sequence of multimodal messages. We will add the following rollout procedure to clarify how the task definition maps to the implementation.

  1. Initialize the history. The model receives the system prompt, the question, and the image rendered from the initial camera pose.
  2. Generate an action or answer. Given the accumulated history, the model generates its reasoning followed by either a structured render_view_tool call or a final answer. A tool call specifies a relative motion primitive and, optionally, its magnitude. The environment parses the call and applies the corresponding translation or rotation to the current camera pose.
  3. Append the observation. The environment renders an RGB image at the updated pose and returns it along with the camera position and look-at point. The model’s reasoning and tool call, followed by the environment’s response, are appended to the history. The next model turn conditions on this full sequence, including earlier images and actions. A final answer ends the episode.

During SFT, we construct this sequence offline from recorded demonstrations and use teacher forcing on the reasoning, action, and answer tokens. Prompt and environment-observation tokens provide context and are masked from the loss. During RL, the model and environment construct the history online as the rollout unfolds, and GRPO applies the trajectory-level advantage to the model-generated tokens. Evaluation uses the same online interaction loop with the learned policy.

The implicit belief is a conceptual interpretation of how the accumulated observations inform the model about the partially observed scene. In Sec. 3.1, bt(M)=Pr⁡(M∣ht)b_t(M)=\Pr(M\mid h_t) denotes the posterior over possible scenes MM given the interaction history hth_t. This captures how a new view can resolve ambiguity left by earlier observations. For instance, moving the camera can reveal an initially occluded object and give the model enough evidence to answer. In the implementation, the VLM conditions directly on hth_t. Appending a new image and pose updates the context for the next decision, while the posterior serves solely as a theoretical interpretation of the information in that context. The environment explicitly tracks the camera pose to execute actions and render observations.

AuthorsOpenReview ↗

Rebuttal for Reviewer Zrae 3/3

>>> Q3a

Why is ARKit out-of-domain when generated ARKit questions are used in RL?

>>> A3a

Thank you for pointing this typo out. ARKitScenes is indeed in-domain for the mixed-data model. We will correct its label in Table 1 and update the corresponding claims. Although excluding ARKit reduces the average gain in the submitted results from 7.2% to 5.0%, we argue that this remains a substantial improvement over the SFT init and the base model, with gains on all three remaining OOD benchmarks and an 11.3% gain on SPBench-MV. The updated results in Table R1 show the same pattern, with an increase of 6.0% in-domain and 5.0% across the three held-out QA benchmarks, compared to SFT init. On the other hand, prior 3D MLLMs do not generalize. Ross3D leads on ScanQA and SQA3D, both based on ScanNet, and scores about 32% on both SPBench splits. SpatialLadder performs well on SPBench, its training domain, and scores 29.3% on ScanQA and 40.4% on SQA3D. We further illustrate this with the ScanQA-only setting, where Act3D is trained on ScanQA alone and its RL improves over SFT on four of the five other benchmarks, with a mean gain of 2.1%. The largest gain is on MP3D (+6.0%), which is a different dataset with completely different scenes.

Table R1a. By benchmark in-domain vs. out-of-domain gains of Act3D trained on mixed data.

ScanQAMP3DARKitIn-domain avgSQA3DSPB MVSPB SIOut-of-domain avg
Qwen3.5-9B (base)36.730.940.135.933.521.324.726.5
SFT init39.361.865.955.747.439.237.741.4
Act3D (ours mixed RL)42.562.979.661.747.750.541.246.5
Δ (mixed RL − SFT)+3.2+1.1+13.7+6.0+0.3+11.3+3.5+5.0
Δ (mixed RL − base)+5.8+32.0+39.5+25.8+14.2+29.2+16.5+20.0

Table R1b. Average accuracy gain (%) over Qwen3.5-9B on in-domain vs. out-of-domain benchmarks. Table 1 in the paper marks the full list of in-domain and out-of-domain benchmarks for each model.

In-domain benchmarksIn-domain avg performance gain from Qwen3.5-9BOut-of-domain avg performance gain from Qwen3.5-9B
Act3D (ours ScanQA only)ScanQA+8.8+21.5
Act3D (ours mixed RL)ScanQA, MP3D, ARKit+25.8+20.0
Ross3D (+ depth, pose, BEV)ScanQA, SQA3D+21.6+9.0
SpatialLadder-3BSPB MV, SPB SI+30.2+8.1

>>> Q3b

How are training and evaluation scenes and questions separated?

>>> A3b Training and evaluation are disjoint at the scene level for every benchmark, and no evaluation question appears in training. For ScanQA, SQA3D and SPBench we use the official evaluation splits: ScanQA val, SQA3D test, and SPBench-MV/SI as released. Our training data excludes every ScanNet scene that appears in any of these benchmarks (228 scenes). For Matterport3D (MMScan QA) and ARKitScenes, we build evaluation sets from rooms/scenes held out from all training data: 1,000 questions over 60 MP3D rooms and 226 questions over 17 ARKitScenes scenes. We verified 0 overlapping scenes and 0 duplicated questions between training and every evaluation set (the MP3D evaluation is disjoint by room, not by building since the dataset only includes 7 buildings).

Reviewer_DyX3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Combined with supervised fine-tuning, the authors demonstrate that reinforcement learning like GRPO can improve a VLM's joint exploration and question-answering behavior using primarily outcome-based supervision. This is especially true when it's fine-tuned on mixed data. However, I see several concerns regarding methodological novelty, the causal contribution of exploration, and the fairness of evaluation.

Critical strengths

The authors compare the RL policy with its corresponding SFT initialization, helping isolate the effect of reinforcement learning. The evaluation covers different indoor 3D environments and question types. The authors also examine interaction counts, accuracy across different tool-use levels, and qualitative trajectories showing searches for occluded objects and multiview comparisons. These analyses make the work more informative than a paper reporting only final benchmark accuracy.

Critical weaknesses
  1. The methodological novelty of the proposed framework could be better established. Although the paper demonstrates promising results, the approach primarily combines trajectory-supervised fine-tuning, standard GRPO, and an interactive 3D rendering environment. Individually, these techniques are well established.

  2. In Main Result4, the authors claim that the policy adapts how it explores to the question. Although the qualitative results in Figure 6 show improved question-answering accuracy, evidence that the model learns an effective information-acquisition strategy remains limited. Quantitative Comparisons against random exploration, systematic viewpoint sampling, and observation-budget-matched passive baselines would help verify the contribution of learned, question-conditioned exploration.

  3. The evaluation relies heavily on Qwen3-30B-A3B-Instruct. It would be valuable to report agreement with human annotations or assess whether the results hold up with alternative VLM-based judges or evaluation prompts.

Questions for authors

None

6 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer DyX3 1/3

We thank the reviewer for the thoughtful review. We appreciate the recognition that comparing the RL policy with its SFT initialization helps isolate the effect of reinforcement learning, that the evaluation spans diverse indoor environments and question types, and that the interaction-count, tool-use-level and qualitative analyses make the work more informative than final accuracy alone. We address each point below.

>>> W1

The novelty is limited: the method combines trajectory SFT, standard GRPO and a 3D rendering environment.

>>> A1 While we agree that supervised fine-tuning and GRPO are established techniques, our contribution centers on a fundamentally different question. How can a VLM learn to acquire the visual evidence it needs to answer a spatial question when that evidence is missing from its initial view? Prior work applying SFT and GRPO to visual reasoning typically assumes that the provided images contain the evidence needed to answer. VLMs receive one or more static images and reason either in text or directly over the images using tools such as cropping and zooming, as in thinking with images. In these settings, the model learns to reason over the evidence contained in the provided views.

We argue that our novelity lies not in proposing a new method for optimizing the parameters of the models, but rather a new paradigm: reformulating 3D question answering as a sequential decision problem under partial observability, in which the 3D scene serves as an external, interactive environment, instead of being encoded and input to the model like prior 3D LLM methods. The VLM starts from a single egocentric image and acquires subsequent views by moving and turning the camera. Its actions therefore determine what evidence becomes available for subsequent reasoning. This formulation makes deciding where to look, how to integrate observations, and when to answer part of the task the model learns. It enables the model to explore and reason in 3D space, a capability relevant to embodied AI and robotics. We use established training techniques to test this formulation. Optimizing full interaction trajectories with RL improves end-to-end accuracy on all 3D benchmarks, with a averaged improvement of 5.0% out of domain (and 6.0% in domain) compared to trajectory-SFT initialization and 20.0% out of domain and 25.8% in domain compared with the base model.

AuthorsOpenReview ↗

Rebuttal for Reviewer DyX3 2/3

>>> W2

Evidence for question-conditioned exploration is limited. Compare against random exploration, systematic viewpoint sampling, and budget-matched passive baselines.

>>> A2

Thanks for the constructive question. To answer this question, we first measure whether the policy finds relevant objects using ScanQA target exposure, the percentage of questions for which at least 5% of the annotated target object is visible and unoccluded in any observed view during the model's inference rollout.

In a new "6-view budget (mixed)" setup, we train Act3D in the mixed-data setting with a six-view limit during RL, which forces the model to explore within only 6 interactions before answering the question. We apply the same limit during evaluation. The other policies use the standard 12-interaction protocol. In Table R3, the six-view Act3D policy achieves 52.6% averaged accuracy and 87.0% ScanQA target exposure using 4.73 views per question. This policy outperforms most of the compared methods and the SFT init on both metrics while using fewer views, and approaches the two Gemini baselines on both metrics with 39–44% fewer views on average. This illustrates that Act3D can learn to acquire useful and relevant observations both effectively and efficiently.

Table R3. Answer accuracy and target exposure on the full benchmark pools. #Views counts the initial view and all views returned by interactions, averaged across the six benchmarks. The initial view alone exposes the target in 50.6% of ScanQA questions.

Model (active)ScanQASQA3DSPB MVSPB SIMP3DARKitAvg#ViewsScanQA Target Exposed (%)
Qwen3.5-9B (base)36.733.521.324.730.940.131.25.462.9
Qwen3.5-27B27.540.233.231.433.833.833.37.076.6
InternVL3.5-38B30.941.543.241.650.155.843.92.155.1
GLM-4.6V30.642.143.639.540.445.840.35.367.5
Gemini 3.6 Flash39.153.845.544.453.473.051.57.789.5
Gemini 3.1 Pro40.752.750.343.964.472.254.08.589.7
SFT init (mixed)39.347.439.237.761.865.948.65.078.7
Act3D, 6-view budget (mixed)43.748.046.341.860.974.952.64.7387.0
Act3D (mixed, ours)42.547.750.541.262.979.654.111.095.1

Additionally, we hold the answerer fixed and vary only the explorer that collects the views. On a 300-question-per-benchmark subset, we give Gemini 3.6 Flash the question and the view sequence collected by each explorer. Each explorer retains its original trajectory length.

As shown in Table R6, the fixed answerer achieves its highest mean accuracy with Act3D’s views (52.2%), compared with 49.5% using views from the SFT initialization and 49.8% using views collected by Gemini 3.6 Flash itself. Act3D’s views improve accuracy over SFT views on all six benchmarks. With the answerer held fixed, these gains show that the observations collected by the RL policy provide more useful evidence for answering.

Table R6. QA accuracy (%) with Gemini 3.6 Flash as the fixed answerer on a 300-question-per-benchmark subset.

Explorer (views from)ScanQASQA3DSPB MVSPB SIMP3DARKitAvg
Act3D (RL)36.355.043.434.463.081.052.2
SFT init35.053.035.731.562.379.749.5
Base Qwen3.5-9B31.347.732.926.659.073.045.1
Gemini 3.6 Flash35.351.734.636.462.778.349.8
Random-exploration31.046.029.231.760.374.045.4
Fixed turning exploration35.051.732.933.960.379.348.9

The passive baselines in Table 1 of our paper already use the same maximum budget, message and scoring, with all passive models given 12 frames from systematic viewpoint sampling and all active models using at most 12 frames collected through their own exploration.

AuthorsOpenReview ↗

Rebuttal for Reviewer DyX3 3/3

>>> W3

The evaluation relies heavily on Qwen3-30B-A3B-Instruct. It would be valuable to report agreement with human annotations or assess whether the results hold up with alternative VLM-based judges or evaluation prompts.

>>> A3 Thanks for your suggestions. Following your advice, we evaluated agreement across LLM judges and agreement between the LLM judges and a human annotator.

Agreement across LLM judges. We compared the paper’s primary judge, Qwen3-30B-A3B-Instruct, with Gemini 3.6 Flash and Qwen2.5-VL-72B-Instruct. All three judges independently scored Act3D’s answers to 100 randomly sampled questions per benchmark (600 total) using the same evaluation-harness prompt. For this agreement analysis, the judges assigned binary correctness verdicts to all answers, including SPBench numeric answers, which the paper scores using MRA. As shown in Table X, all three judges gave the same verdict on 96.5% of answers (Fleiss’ κ = 0.953). Unanimous agreement was 86.0% on ScanQA and at least 95.0% on each of the other five benchmarks. The primary judge agreed with the majority vote of the three judges on 99.5% of answers.

Table X. Agreement across three LLM judges on 600 Act3D answers.

BenchmarkTypeAll agree (%)Fleiss' κQwen3-30B matches majority (%)
ScanQAOpen-ended86.00.80397.0
SQA3DOpen-ended95.00.933100.0
SPBench-MVMCQ + numeric99.00.982100.0
SPBench-SIMCQ + numeric99.00.976100.0
MP3DOpen-ended100.01.000100.0
ARKitOpen-ended100.01.000100.0
All–96.50.95399.5

Human agreement. We asked a human annotator to label the correctness of 250 Act3D answers without seeing the LLM judges’ verdicts. The sample included 50 answers each from ScanQA, SQA3D, MP3D, and ARKit, plus 50 SPBench multiple-choice answers. SPBench numeric answers were excluded because the paper scores them using MRA. We compared the human labels with the verdicts of each LLM judge, their majority vote, and a string-match scorer.

As shown in Table Y, the paper’s primary judge agreed with the human annotator on 98.0% of answers (Cohen’s κ = 0.96). Agreement ranged from 97.6% to 98.8% across the three individual LLM judges. The majority vote agreed with the annotator on 98.4% of answers, and the string-match scorer agreed on 96.8%.

Table Y. Agreement with human correctness labels on 250 Act3D answers.

Qwen3-30BQwen2.5-VL-72BGemini 3.6 FlashMajorityRule
Agreement with human98.0 / 0.9697.6 / 0.9598.8 / 0.9898.4 / 0.9796.8 / 0.94
Reviewer_NScyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies active 3D question answering. A vision-language model starts from a single egocentric rendering of a reconstructed scene and moves the camera to gather observations before answering. The policy is initialized with teacher-generated interaction trajectories and then trained with GRPO, using rewards for answer correctness, tool use, answer-object visibility, and output format. On six reported benchmarks, the mixed-data policy improves the average score from 48.6 for its SFT initialization to 54.1. The paper also examines the choice of observation source, supervised initialization, tool rewards, initial inputs, and action parameterization.

Critical strengths
  1. I appreciate that the paper learns observation acquisition and answering within the same interaction process and reports a comparison between the SFT initialization and the subsequent RL policy. In Table 1, the mixed-data model’s average score increases from 48.6 to 54.1, with improvements on all six benchmarks. This does not establish how much comes from better exploration versus better answering, but it shows that the second training stage improves the system’s overall performance rather than merely changing its tool-use behavior.

  2. The comparison between rendering and nearest captured photographs in Sec. 6.1 is useful. The two runs share the base model, data fraction, training schedule, tool, and reward, and the rendered-view run performs better at every shared validation checkpoint. Figure 7 also illustrates the positional and angular mismatch between a requested view and its nearest photograph. For a task that lets the model move the camera freely, whether the returned observation faithfully reflects the action is a practical concern. This experiment gives a concrete reason for using the rendering environment.

Critical weaknesses
  1. I have concerns about the interpretation of the out-of-domain results. Figure 3 includes 7,534 ARKit questions in the RL corpus, and Sec. 4.1 explicitly states that ARKit is used for training. Nevertheless, the main results and Table 2 count ARKit as OOD. The training-domain marking in Table 1 also omits it. The paper needs to distinguish unseen scenes from unseen data domains. This affects a central claim in the abstract: ARKit contributes the largest of the four reported OOD gains, and excluding it gives an average improvement of approximately 5.0 percentage points across the remaining three benchmarks, rather than 7.2 across four. The remaining result is still positive, but the current wording overstates the evidence for generalization beyond the training domains.

  2. I am not yet convinced that the performance gains demonstrate more effective exploration specifically. The grounding reward in Eq. 7 uses annotated 3D object locations to reward whether the answer object appears in the trajectory, but neither its coverage nor an ablation removing it is reported. This does not contradict the absence of prescribed intermediate actions or viewpoints, but it leaves open how much the exploration behavior depends on this auxiliary supervision. Moreover, tool calls on correct trajectories are themselves rewarded, so the increase in calls in Figure 4 is not sufficient evidence of better information gathering. The stratified results in Figure 5 and examples in Figure 6 also do not separate improvements in exploration from improvements in answering. A grounding ablation and a direct measure of evidence acquisition would make the main interpretation more convincing.

Questions for authors
  1. How is OOD defined? Why is ARKit labeled OOD despite being used in RL? Please explain whether the distinction concerns scenes, question types, or another distributional difference, and whether the claim of four OOD benchmarks should be revised. Please also clarify the training–evaluation scene separation across datasets sharing ScanNet scenes.

  2. How much does the grounding reward contribute? What fraction of training questions has annotated answer objects, and are results available with the grounding weight set to zero? Is there evidence that RL increases the rate of finding relevant objects or collects more useful observations under the same observation budget? These results would directly affect my second concern. Object-visibility statistics or a matched-budget policy comparison are possible ways to address it; I am not asking for one particular experimental design.

  3. Please clarify the completeness and scoring of the evaluation. Why is Real-3DQA listed in Sec. 4.1 but absent from the results? Were all active and passive predictions in Table 1 re-scored with the same semantic judge? In particular, is Ross3D’s ScanQA score of 50.7 judged accuracy? Since Appendix C reports exactly the same value for ROUGE-L, I would appreciate confirmation of the metric.

  4. How are episodes scored after the budget is exhausted? Sec. 3.1 states that an episode without an answer within the budget is counted as incorrect, while Appendix D.1 provides an evaluation-only message instructing the model to stop using tools and answer. Does this permit an additional answer turn after all twelve tool calls have been used? Are those answers counted in Table 1, and does the same rule apply to every active baseline?

4 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer NScy 1/3

We thank the reviewer for the constructive review. We appreciate the reviewer for finding our paper effective, with ablations to support the claim. We address the reviewer's concerns below.

>>> W1 & Q1 ARKit is in the RL corpus (7,534 questions, Fig. 3) but is counted as out-of-domain, and Table 1 doesn't mark it as a training domain. Excluding ARKit, the out-of-domain gain is about 5.0 rather than 7.2. How is out-of-domain defined, and are training and evaluation scenes separated across datasets that share ScanNet scenes?

>>> A1 Thank you for pointing this typo out. ARKitScenes is indeed in-domain for the mixed-data model. We will correct its label in Table 1 and update the corresponding claims. Although excluding ARKit reduces the average gain in the submitted results from 7.2% to 5.0%, we argue that this remains a substantial improvement over the SFT init and the base model, with gains on all three remaining OOD benchmarks and an 11.3% gain on SPBench-MV. The updated results in Table R1 show the same pattern, with an increase of 6.0% in-domain and 5.0% across the three held-out QA benchmarks, compared to SFT init. On the other hand, prior 3D MLLMs do not generalize. Ross3D leads on ScanQA and SQA3D, both based on ScanNet, and scores about 32% on both SPBench splits. SpatialLadder performs well on SPBench, its training domain, and scores 29.3% on ScanQA and 40.4% on SQA3D. We further illustrate this with the ScanQA-only setting, where Act3D is trained on ScanQA alone and its RL improves over SFT on four of the five other benchmarks, with a mean gain of 2.1%. The largest gain is on MP3D (+6.0%), which is a different dataset with completely different scenes.

Table R1a. By benchmark in-domain vs. out-of-domain gains of Act3D trained on mixed data.

ScanQAMP3DARKitIn-domain avgSQA3DSPB MVSPB SIOut-of-domain avg
Qwen3.5-9B (base)36.730.940.135.933.521.324.726.5
SFT init39.361.865.955.747.439.237.741.4
Act3D (ours mixed RL)42.562.979.661.747.750.541.246.5
Δ (mixed RL − SFT)+3.2+1.1+13.7+6.0+0.3+11.3+3.5+5.0
Δ (mixed RL − base)+5.8+32.0+39.5+25.8+14.2+29.2+16.5+20.0

Table R1b. Average accuracy gain (%) over Qwen3.5-9B on in-domain vs. out-of-domain benchmarks.

In-domain benchmarksIn-domain avg performance gain from Qwen3.5-9BOut-of-domain avg performance gain from Qwen3.5-9B
Act3D (ours ScanQA only)ScanQA+8.8+21.5
Act3D (ours mixed RL)ScanQA, MP3D, ARKit+25.8+20.0
Ross3D (+ depth, pose, BEV)ScanQA, SQA3D+21.6+9.0
SpatialLadder-3BSPB MV, SPB SI+30.2+8.1

>>> Q3 Why is Real-3DQA listed in Sec. 4.1 but missing from the results? Were all Table 1 predictions re-scored with the same judge? Is Ross3D's ScanQA 50.7 judged accuracy, given that Appendix C reports the same value for ROUGE-L?

>>> A3 Thanks for pointing this out. The inclusion of Real-3DQA in Sec. 4.1 is a typo. We excluded it because recurring rendering errors prevented a reliable evaluation. We will revise the paper and remove the reference.

Yes, we rescored all predictions in Table 1 using the same judge and evaluation setup. All comparisons are fair. Ross3D’s 50.7% on ScanQA in Table 1 is judged accuracy, and the 50.7 in Appendix C is ROUGE-L. We verified both values, which coincide at the reported precision. We also repeated the Ross3D evaluation three times and observed a run-to-run standard deviation of 0.4% in judged accuracy.

>>> Q4 How are episodes scored once the budget is exhausted? Sec. 3.1 says an unanswered episode is incorrect, but Appendix D.1 has an evaluation-only "stop and answer" message. Is there an extra answer turn after 12 calls, is it counted in Table 1, and does the same rule apply to all active baselines?

>>> A4 Sec. 3.1 and App. D.1 describe two steps of the same rule we followed.

(i) After the 12th call, the harness sends the D.1 "stop and answer" message once and prefills <answer>. This turn renders no new view and does not count toward #Views. The same turn is used when a model gives two invalid outputs in a row.

(ii) The forced answer is scored like any other answer. An episode is scored incorrect, as in Sec. 3.1, only if it still has no answer after this turn.

All models evaluated in our harness use the same maximum budget, message and scoring, with all passive models given 12 frames and all active models using at most 12 frames.

AuthorsOpenReview ↗

Rebuttal for Reviewer NScy 2/3

>>> W2 & Q2 How much does the grounding reward (Eq. 7) contribute? What fraction of questions have annotated answer objects, and what happens with the grounding weight at 0? Tool calls on correct trajectories are rewarded, so more calls (Fig. 4) don't by themselves show better information gathering. Figs. 5–6 don't separate exploration from answering. Is there evidence that the policy finds relevant objects more often, or collects more useful observations, under the same budget?

>>> A2 Thanks for the question and the constructive suggestion. We address each point in turn below.

How much does the grounding reward (Eq. 7) contribute to the final performance?

To quantify the contribution of the grounding reward in Eq. 7, we compare the full-reward policy with two ablations in the mixed-data setting. The first removes the grounding condition while retaining the interaction bonus. The second removes both. We report answer accuracy, the average number of views, and how often the policy brings the annotated target into view. We evaluate all policies on the full benchmark pools (11,522 questions) using the native harness, a budget of 12 interactions, and a sampling temperature of 0.7.

As shown in Table R2, removing the grounding reward reduces averaged accuracy from 54.1% to 53.2% and ScanQA target exposure from 95.1% to 91.6%, with both policies averaging 11.0 views per question across the six benchmarks. With both rewards removed, RL still improves accuracy from 48.6% to 53.3% and target exposure from 78.7% to 84.7% over the SFT initialization, with similar average view counts (5.0 vs. 5.3).

Table R2. Grounding-reward ablation in the mixed-data setting. #Views includes the initial view and is averaged across the six benchmarks. ScanQA Target Exposed is the percentage of ScanQA questions for which at least 5% of the target object is visible and unoccluded in any observed view.

ScanQASQA3DSPB MVSPB SIMP3DARKitAvg#ViewsScanQA Target Exposed (%)
SFT init39.347.439.237.761.865.948.65.078.7
Act3D, full reward (ours)42.547.750.541.262.979.654.111.095.1
Act3D, no grounding reward41.847.845.440.364.779.453.211.091.6
Act3D, no grounding reward, no tool bonus38.947.248.139.164.781.653.35.384.7

What fraction of questions have annotated answer objects?

An RL training question is eligible for the grounding reward if its answer object has an annotated 3D location.

As shown in Table R2b, approximately 75.6% of the RL corpus meets this criterion, corresponding to about 38,981 of 51,543 questions. The number of eligible ARKit questions is inferred from the overall coverage estimate.

Table R2b. Grounding-reward coverage by RL training source.

RL sourceQuestionsEligible for grounding reward
ScanQA (ScanNet)25,56325,563 (100%)
Curated ScanNet QA8,5918,591 (100%)
Curated ARKit QA7,5344,827 (64.1%)
MP3D9,8550
Total51,54338,981 (75.6%)

What happens with the grounding weight at 0?

When the answer object has no annotated 3D location, the grounding reward cannot be computed, so its weight stays at 0 for that question and only the remaining reward terms apply. These questions still produce a normal gradient as GRPO normalizes rewards within each group of rollouts for the same question, so a grounding term that is 0 for every rollout cancels out of the advantage, and the update is driven by differences in those remaining terms, mainly answer correctness.

(A2 continues in next comment)

AuthorsOpenReview ↗

Rebuttal for Reviewer NScy 3/3

Tool calls on correct trajectories are rewarded, so more calls (Fig. 4) don't by themselves show better information gathering. Is there evidence that the policy finds relevant objects more often, or collects more useful observations, under the same budget?

Thanks for the constructive question. To answer this question, we first measure whether the policy finds relevant objects using ScanQA target exposure, the percentage of questions for which at least 5% of the annotated target object is visible and unoccluded in any observed view during the model's inference rollout. We also report end-to-end QA accuracy as a proxy for how useful these observations are.

To answer this question, we train Act3D in the mixed-data setting with a six-view limit during RL, which forces the model to explore within only 6 interactions before answering the question. We apply the same limit during evaluation. The other policies use the standard 12-interaction protocol.

In Table R3, the six-view Act3D policy achieves 52.6% averaged accuracy and 87.0% ScanQA target exposure using 4.73 views per question. This policy outperforms most of the compared methods and the SFT init on both metrics while using fewer views, and approaches the two Gemini baselines on both metrics with 39–44% fewer views on average. This illustrates that Act3D can learn to acquire useful and relevant observations both effectively and efficiently.

Table R3. Answer accuracy and target exposure on the full benchmark pools. #Views counts the initial view and all views returned by interactions, averaged across the six benchmarks. The initial view alone exposes the target in 50.6% of ScanQA questions.

Model (active)ScanQASQA3DSPB MVSPB SIMP3DARKitAvg#ViewsScanQA Target Exposed (%)
Qwen3.5-9B (base)36.733.521.324.730.940.131.25.462.9
Qwen3.5-27B27.540.233.231.433.833.833.37.076.6
InternVL3.5-38B30.941.543.241.650.155.843.92.155.1
GLM-4.6V30.642.143.639.540.445.840.35.367.5
Gemini 3.6 Flash39.153.845.544.453.473.051.57.789.5
Gemini 3.1 Pro40.752.750.343.964.472.254.08.589.7
SFT init (mixed)39.347.439.237.761.865.948.65.078.7
Act3D, 6-view budget (mixed)43.748.046.341.860.974.952.64.7387.0
Act3D (mixed, ours)42.547.750.541.262.979.654.111.095.1

Figs. 5–6 don't separate exploration from answering.

Following your suggestion, we explicitly separate exploration from answering. Specifically, we hold the answerer fixed and vary only the explorer that collects the views. On a 300-question subset per benchmark, we give Gemini 3.6 Flash the question and the view sequence collected by each explorer. Each explorer retains its original trajectory length.

As shown in Table R6, the fixed answerer achieves its highest mean accuracy with Act3D’s views (52.2%), compared with 49.5% using views from the SFT initialization and 49.8% using views collected by Gemini 3.6 Flash itself. Act3D’s views improve accuracy over SFT views on all six benchmarks. With the answerer held fixed, these gains show that the observations collected by the RL policy provide more useful evidence for answering.

Table R6. QA accuracy (%) with Gemini 3.6 Flash as the fixed answerer on a 300-question-per-benchmark subset.

Explorer (views from)ScanQASQA3DSPB MVSPB SIMP3DARKitAvg
Act3D (RL)36.355.043.434.463.081.052.2
SFT init35.053.035.731.562.379.749.5
Base Qwen3.5-9B31.347.732.926.659.073.045.1
Gemini 3.6 Flash35.351.734.636.462.778.349.8
Random walk31.046.029.231.760.374.045.4
Fixed turn-in-place sweep35.051.732.933.960.379.348.9

Dear Reviewer NScy,

Thank you again for your constructive review.

We have responded to all the questions and concerns in above comments. As the discussion closes in a few days, we would be grateful to know whether these results resolve your concerns, or whether any specific issue still requires clarification. If the new evidence changes your assessment, we would sincerely appreciate your reflecting that in your final score. Thank you again for the constructive discussion.

Best,

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of rebuttal and changes for AC

Dear Area Chair,

We thank the reviewers for the constructive reviews. We appreciate the recognition that the paper addresses a meaningful problem, and performance gain compared to baseline and proprietary models. We summarize our rebuttal and changes below.

1. Novelty (DyX3, iPE7). We argue that our novelty lies not in proposing a new method for optimizing the parameters of the models, but rather a new paradigm: reformulating 3D question answering as a sequential decision problem under partial observability. The key distinction between CoV & Think3D approaches and ours lies in their assumption that complete 3D information is provided as input to the model. Think3D-RL selects from a finite set of overhead viewpoints at the corners of the scene (in 72.0% of the tasks we tested below, it chooses the same BEV-like view: 0° azimuth, looking down at 60°), whereas Act3D supports a continuous range of ego-centric free exploration at human height. CoV similarly has access to prerecorded video frames and a real bird’s-eye view of the scene before refining the camera position and orientation. These initial inputs expose 99.7% of the target objects in the ScanQA validation set. Furthermore, CoV is a training-free, prompt-based framework and therefore relies on the underlying model’s existing ability to explore and reason in 3D. As a result, CoV cannot equip the model with an intrinsic capability to explore and search in 3D space. Under matched conditions, Act3D achieved the highest accuracy on all six spatial benchmarks, averaging 54.1%, compared with 41.4% for Think3D-RL and 41.9% for CoV.

2. Exploration vs. answering (all reviewers). We hold the last-turn answerer fixed and vary only the explorer that collects the views. The fixed answerer achieves its highest mean accuracy with Act3D's views (52.2%), compared with 49.5% using views from the SFT initialization and 49.8% using views collected by second best Gemini 3.6 Flash. To examine less view budget at training time, we retrain 6-view Act3D policy that achieves 52.6% averaged accuracy and 87.0% ScanQA target exposure using 4.73 views per question. Reviewer Zrae acknowledged our new results.

3. Rewards ablation (NScy, iPE7). Removing the grounding reward reduces averaged accuracy from 54.1% to 53.2%, ScanQA target exposure from 95.1% to 91.6%, and requires more training steps to converge. With both rewards removed, RL still improves accuracy from 48.6% to 53.3% and target exposure from 78.7% to 84.7% over the SFT initialization. Both results confirm the gain of our paradiam centers on natural navigation out of restricted view for accuracy improvement.

4. Out-of-domain (NScy, Zrae, iPE7). ARKitScenes scene is in-domain for the mixed-data model. We will correct its label in Tables 1 and 2 and update the corresponding claims. The updated accuracy increase, averaged over all three remaining OOD benchmarks, is 5.0%, which remains a substantial improvement over the SFT init and the base model and yields the biggest OOD-to-in-domain gain ratio from base among similar methods. In the version of Act3D trained on ScanQA alone, the acquisition of exploring ability allows it to gain 21.5% over the base model across the five other out-of-domain benchmarks.

5. Judge (DyX3, iPE7). We evaluated agreement across LLM judges and agreement between the LLM judges and a human annotator. All three judges gave the same verdict on 96.5% of answers (Fleiss' κ = 0.953). The paper's primary judge agreed with the human annotator on 98.0% of answers (Cohen's κ = 0.96).

We will include these results in the revised paper. We will add rollout details, revise ARKitScenes label and stale reference to Real-3DQA.

As the discussion period has only a few days remaining, we would really appreciate it if you could help encourage further discussion around our responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.