acceptodds
Under review as a conference paper at ICLR 2027

Thinking with Cameras: Active Visual Reasoning via Dynamic Viewpoint Control for Surveillance Video Understanding

Abstract

Large vision-language models (LVLMs) have recently achieved remarkable progress in general-purpose video understanding. However, their application to real-world surveillance remains challenging due to the lack of large-scale domain-specific datasets and the limitation of passive observation from fixed viewpoints. In surveillance scenarios, critical visual evidence can be easily missed when targets are distant, small, occluded, or move beyond the current camera view. In this work, we introduce CamVLM, a new framework for Thinking with Cameras, which enables LVLMs to actively acquire visual evidence in real-world surveillance by continuously controlling camera viewpoints. We first construct CCTV-Anomaly, a large-scale surveillance video understanding dataset containing 14,133 videos across 10 anomaly categories, with detailed captions and event annotations. We further formulate viewpoint control as an active visual perception problem and build CamTrack-53K, an object-centric viewpoint trajectory dataset for learning camera actions. Moreover, we propose a reinforcement learning based viewpoint policy optimization framework, which models camera control as a sequential decision-making process and learns long-horizon observation strategies beyond supervised trajectory imitation. Extensive experiments demonstrate that CamVLM achieves state-of-the-art performance under both passive observation and dynamic viewpoint settings, validating the effectiveness of active camera-based reasoning for surveillance video understanding. We will release the datasets, models, and code to facilitate future research.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Surveillance Video Understanding, Large Vision-Language Models, Active Perception

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2ne9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates active visual perception for surveillance video understanding and proposes CamVLM to address the limitations of conventional video understanding methods that cannot actively adjust their observation viewpoints. The method simulates camera panning and zooming by dynamically cropping prerecorded videos, allowing the model to actively adjust its field of view and acquire task-relevant visual information. The authors construct two datasets, CCTV-Anomaly and CamTrack-53K, for surveillance video understanding and viewpoint control policy learning, respectively. During training, the authors combine supervised fine-tuning (SFT) and GRPO, using task completion and target-region coverage rewards to optimize the viewpoint control policy. Experiments on CCTV-Anomaly and UDVideoQA* show that the proposed method achieves favorable performance compared with the evaluated baselines.

Critical strengths
  1. The paper addresses a practically relevant problem, focusing on the potential loss of critical visual information due to limited fields of view in surveillance videos.

  2. The method incorporates dynamic viewpoint control into the reasoning process of vision-language models and optimizes the viewpoint selection policy through reinforcement learning. The overall design is reasonable.

  3. The authors construct two datasets for surveillance video understanding and viewpoint control policy learning, which could provide useful resources for future research.

Critical weaknesses
  1. The paper simulates camera viewpoint control through dynamic cropping of prerecorded videos. Given that the proposed active visual perception setting is a major contribution, the lack of supplementary evaluation on real-world limited-field-of-view or multi-view surveillance videos leaves the representativeness of the simulated environment unclear. Additional evaluations or analyses of the gap between simulated and real-world settings would strengthen the paper. Incorporating real-world limited-field-of-view or multi-view surveillance data would further enhance the research value of the proposed benchmark.

  2. Table 6 only compares different training configurations and lacks separate ablations of the reward components. I suggest comparing GRPO with only the task reward, only the visual evidence reward, and the complete reward function to clarify the individual contributions of these rewards and their combined effect.

  3. I suggest including simple heuristic viewpoint control strategies, such as directly zooming out or scanning in fixed directions, as baselines to assess the necessity of the learned viewpoint control policy.

If the authors adequately address these three concerns, I would be willing to increase my score.

Questions for authors

Could the authors address the concerns raised in the weaknesses above?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 2ne9 (Part 1)

We sincerely thank the reviewer for the constructive comments. We have conducted additional experiments and revised the discussion to better clarify the scope and limitations of our simulated viewpoint-control setting.


1. Representativeness of the simulated viewpoint-control setting

Response: Our current framework simulates active viewpoint control by dynamically cropping prerecorded video frames. We agree that this setup does not fully reproduce physical PTZ cameras or independent multi-camera surveillance systems. In particular, cropping cannot recover visual information outside the original frame or reproduce physical factors such as camera motion, occlusion, and actuation latency. We have therefore clarified that our experiments evaluate camera-like viewpoint selection in a virtualized observation space, rather than physical camera control. To further examine the effect of restricted and multiple viewpoints, we conducted two additional analyses:

(1) Fixed-viewpoint evaluation. As reported in the Passive setting of the ablation study (Section 5.3, Table 4), the model observes only a fixed central region covering 1/9 of the original frame, without dynamically adjusting its viewpoint. This provides a controlled simulation of restricted-field-of-view observation, although it is not equivalent to evaluation on videos natively captured by limited-FOV cameras.

(2) Multi-viewpoint simulation. We further evaluated the model with 2 and 3 initial viewpoints. Specifically, the initial viewpoints are placed at the top-left and center regions for the 2-view setting, and at the top-left, center, and bottom-right regions for the 3-view setting. The same dynamic viewpoint-control strategy is applied to all viewpoints. Importantly, the total visual-token budget is kept fixed across all settings, ensuring that the performance gains are not simply attributable to increased visual input or computational budgets. Instead, the available visual-token budget is distributed across different viewpoints to provide complementary spatial observations. The results are shown below.

Number of viewpointsBUAtrERRRCIAvg
170.629.446.264.772.757.0
271.031.147.064.973.257.7
376.132.849.165.574.059.6

The results show consistent performance improvements as the number of viewpoints increases, with the average score improving from 57.0 to 59.6 under the same visual-token budget. This suggests that allocating a fixed visual-token budget across multiple viewpoints enables the model to exploit complementary spatial evidence more effectively than observing a single viewpoint. We have added these results and clarified the experimental settings in the revised manuscript. We acknowledge that this simulation does not fully capture the challenges of real-world multi-camera systems. Evaluating viewpoint control on independently captured limited-FOV and multi-camera surveillance videos, where camera calibration, temporal synchronization, and cross-camera identity association become relevant, remains an important direction for future work.


2. Ablation of reward components

Response: We conduct additional experiments to isolate the contributions of the task reward and visual evidence reward in GRPO. The results are reported below.

Reward settingBUAtrERRRCIAvg
Task reward only (RtaskR_{\mathrm{task}})68.526.544.653.871.153.2
Visual reward only (RvisualR_{\mathrm{visual}})65.425.543.652.170.751.8
Full reward70.629.446.264.772.757.0

The task reward provides direct supervision for the final question-answering objective, while the visual evidence reward encourages the policy to acquire informative visual observations. Neither reward alone achieves the performance of the full configuration. Their combination yields consistent improvements across all five evaluation categories, indicating complementary contributions to viewpoint-control policy learning. We have included this ablation in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 2ne9 (Part 2)

3. Comparison with heuristic viewpoint-control strategies

Response: To assess whether the performance gain comes from the learned control policy rather than simple viewpoint movements, we added two heuristic baselines:

  • Zoom-out: directly expands the current viewpoint to observe a larger image region.
  • Horizontal scanning: moves the viewpoint horizontally with a fixed step size of 1/9 of the original frame width.

The results are shown below.

MethodBUAtrERRRCIAvg
Zoom-out65.722.643.347.068.249.6
Horizontal scanning64.421.541.744.263.047.0
Ours70.629.446.264.772.757.0

Our method outperforms both heuristic strategies, particularly in reasoning-related categories such as RR. These results suggest that a fixed viewpoint transformation or scanning pattern is insufficient to capture the task-dependent visual evidence required by surveillance video understanding. In contrast, our learned policy dynamically selects viewpoint adjustments according to the ongoing reasoning process. We have added these comparisons to the revised manuscript.

AuthorsOpenReview ↗

Hoping for a Discussion on Our Rebuttal for Paper 21122

Dear Reviewer,

I hope this message finds you well. We have carefully addressed the questions and concerns you previously raised, and we would greatly appreciate any further feedback you might be able to provide. Your insights are highly valuable to us, and we remain fully available to clarify or refine any remaining points.

As the discussion deadline is approaching, if possible, we would be grateful for your response at your earliest convenience, as it would greatly help us move the discussion forward.

Thank you sincerely for your time and effort in reviewing our paper.

Best regards,

The Authors

Reviewer_ST7qOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work, the authors study the problem of surveillance video understanding under both active and passive visual settings. They propose two datasets: CCTV-Anomaly, which focuses on passive visual outlier detection, and CamTrack-53K, an object-centric active video understanding dataset that consists of viewpoint selection trajectories for answering object-specific queries. In addition to these datasets, the authors also propose a training framework, CamVLM, that, among other things, utilizes GRPO-based post-training to improve the model's performance on active viewpoint selection.

Critical strengths
  1. Framing the active perception problem as view/patch selection in a larger image is innovative
  2. The dataset contribution (provided it is made public upon acceptance) is valuable for the community
Critical weaknesses
  1. Differences from prior works are unclear: While the view selection task in the context of surveillance understanding is new, the broader set of skills necessary to perform this task has been explored in prior work on tool-based grounding and video understanding [1], as well as some other works the authors have already cited. I am not entirely convinced by the authors' arguments that these works operate only on fixed visual streams. I believe that the key difference is that the VLM only gets access to the active view in a turn, which is not the case with prior work. The authors should emphasize that; otherwise, the difference from prior works seems unclear.
    1. Related to the above, it is also unclear how their training framework differs from the approaches applied by prior works. If anything there proposed approach seems tailor-made to fit the problem they set out to tackle, and apply it to the narrow domain of surveillance videos. Findings like SFT followed by RL being necessary for better performance are known generally in LLM-related literature.
  2. Unclear prompt for zero-shot baselines: Looking at Figure 7 and Figure 9, the control interface available to zero-shot models is not well-described in the prompts. The model is unaware of the offset range it has available to it (which will change based on where the current view is located); it is not told how much an offset of 1 moves the viewpoint, etc. This is not an issue when training a model with an unclear prompt, as the model will learn to interpret it during training, but for zero-shot baselines, the prompt should be made more explicit.

[1] Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation. Yuan et al., https://arxiv.org/abs/2605.18740

Questions for authors
  1. Outline differences from prior works: See #1 in Critical Weaknesses. Can the authors clearly outline how their task differs from prior work on tool-based visual understanding?
  2. Prompt justification: Is there any justification that the authors can offer for critical weakness #2?
  3. Why UDVideoQA?: The authors focus on UDVideoQA for their analysis, but the type of questions in UDVideoQA do not seem tailored to the kind of anomaly detection questions that the authors claim to be motivated by. So I am unclear on the choice of the evaluation dataset.
  4. Preventing reward hacking in GRPO: A degenerate solution to obtain higher rewards in R_visual (Eq. 9) would be to never zoom in to the image. Is there a strategy adopted to prevent that?
  5. Data Release: Do the authors promise to publicly release the dataset with appropriate de-identification of human subjects and a license for academic research? The authors mention that they will release the dataset, but they have not discussed a release strategy. Given the potentially sensitive nature of the dataset, I wanted to flag this concern.
6 comments
AuthorsOpenReview ↗

Response to Reviewer ST7q (Part 1)

We sincerely thank the reviewer for the constructive comments and for identifying several important aspects that require clarification. We have revised the manuscript to better distinguish our problem formulation from prior work, improve the zero-shot evaluation protocol, clarify the choice of evaluation datasets, provide additional analyses of reward design, and specify our data-release considerations. Our detailed responses are provided below.


1. Differences from prior works and the motivation for the training framework

Response: We agree that the key distinction between CamVLM and prior tool-based visual reasoning methods should be made more explicit. We have revised the Introduction, Method, and Related Work sections to clarify that the main difference lies not simply in the use of iterative visual interaction, but in the active-view-only observation protocol. In CamVLM, the VLM receives only the currently selected viewpoint at each interaction step. It cannot directly access other regions of the original frame unless it explicitly performs a viewpoint-control action. The action changes the observation state and determines which visual evidence becomes available in the subsequent step. This makes visual evidence acquisition an explicit sequential decision-making problem. In contrast, methods such as Vision-OPD and other tool-based visual reasoning approaches support interactive visual operations, including retrieving, cropping, or inspecting task-relevant image or video content. However, their observation interfaces do not impose the same active-view-only constraint, under which the model must control its viewpoint to acquire subsequent observations. We have revised our discussion to focus on this difference without implying that prior methods do not support iterative visual interaction.

Regarding the training framework, our contribution lies in adapting trajectory-level policy learning to the active viewpoint-control problem. Specifically, SFT provides initial supervision for camera-action generation, while GRPO further optimizes sequential viewpoint-control policies using complementary task-level and visual-evidence rewards. This formulation directly connects viewpoint selection with downstream surveillance understanding, rather than optimizing visual interaction independently of the final task. We have revised the manuscript to clarify that the methodological contribution is the formulation and optimization of task-oriented active viewpoint-control trajectories, rather than the introduction of a new general-purpose SFT or RL algorithm.


2. Explicit control instructions for zero-shot baselines

Response: We agree that the original prompts did not sufficiently describe the control interface available to zero-shot models. We have revised the prompts to explicitly specify the action semantics, movement ranges, and boundary constraints.

The modifications include:

  • Offset units: Horizontal and vertical offsets are normalized by the source-frame width and height, respectively. For example, a horizontal offset of 0.1 represents a displacement of 10% of the source-frame width, subject to boundary constraints.
  • Per-step control states: The current window coordinates and the maximum allowable offset in each direction are provided at every step, accounting for position-dependent movement limits.
  • Scaling and boundary handling: We clarify how scaling affects both window dimensions and provide examples of scale factors, window-size constraints, and out-of-bounds handling.
  • Sequential action execution: Multiple actions are executed sequentially, with the viewpoint updated and boundary constraints applied after each action.

We re-evaluate the zero-shot baseline Qwen3-VL-8B using the revised prompts under the dynamic viewpoint setting on CCTV-Anomaly and UDVideoQA*. The results are presented below.

MethodSettingCCTV-AnomalyUDVideoQA*
Cls AvgCap AvgBUAtrERRRCIAvg
Qwen3-VL-8BDynamic45.458.867.422.545.653.159.749.6
CamVLM-8BDynamic70.569.770.629.446.264.772.757.0

With the more explicit control instructions, Qwen3-VL-8B achieves an average score of 49.6 on UDVideoQA*, while CamVLM-8B achieves 57.0 under the same dynamic viewpoint setting. This comparison provides a more informative evaluation of zero-shot control when the available action interface is clearly specified. We have included the revised prompts and the corresponding evaluation results in the supplementary material and revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ST7q (Part 2)

3. Rationale for using UDVideoQA

Response: We clarify that our work focuses on general surveillance video understanding, rather than anomaly detection alone. CCTV-Anomaly is used to evaluate anomaly-related surveillance understanding, whereas UDVideoQA provides a complementary video question-answering benchmark based on traffic surveillance scenes. Although UDVideoQA is not specifically designed for anomaly detection, its questions cover several aspects relevant to surveillance understanding, including object understanding, spatiotemporal relationships, and event reasoning. These tasks require the model to identify relevant visual entities, inspect their spatial and temporal interactions, and acquire sufficient visual evidence to answer questions. We therefore use UDVideoQA* to evaluate the transferability of the learned viewpoint-control policy to a complementary surveillance video QA benchmark. We have revised the dataset description and experimental discussion to better explain the complementary roles of CCTV-Anomaly and UDVideoQA* in our evaluation.


4. Preventing reward hacking in GRPO

Response: We agree that optimizing the visual reward alone could encourage the policy to maintain unnecessarily large viewpoints, since larger views may increase target coverage. To mitigate this issue, our final reward combines task-level and visual-evidence rewards:

R=Rtask+0.6Rvisual,R=R_{\mathrm{task}}+0.6R_{\mathrm{visual}},

where the task reward receives a weight of 1.0 and the visual reward receives a weight of 0.6. The task reward directly evaluates downstream task performance, while the visual reward provides additional feedback on the acquisition of relevant visual evidence. We further conduct reward ablation experiments to examine the effects of these two components.

Reward settingBUAtrERRRCIAvg
RtaskR_{\mathrm{task}} only68.526.544.653.871.153.2
RvisualR_{\mathrm{visual}} only65.425.543.652.170.751.8
Full70.629.446.264.772.757.0

The combined reward achieves the best overall performance, suggesting that the two reward components provide complementary supervision.

We additionally evaluate an explicit viewpoint-size penalty:

Psize=1T∑t=1TSt,P_{\mathrm{size}}=\frac{1}{T}\sum_{t=1}^{T}S_t,

where StS_t is the area of the selected viewpoint normalized by the area of the original frame, and TT denotes the number of observation steps.

Reward settingBUAtrERRRCIAvg
Rtask+0.6RvisualR_{\mathrm{task}}+0.6R_{\mathrm{visual}}70.629.446.264.772.757.0
Rtask+0.6Rvisual−0.2PsizeR_{\mathrm{task}}+0.6R_{\mathrm{visual}}-0.2P_{\mathrm{size}}71.026.545.665.172.356.4

The results indicate that this penalty does not improve performance.

We also analyze the average viewpoint-size ratios under different reward configurations.

Reward settingAverage viewpoint-size ratio
RtaskR_{\mathrm{task}}20.5%
RvisualR_{\mathrm{visual}}90.7%
Rtask+0.6RvisualR_{\mathrm{task}}+0.6R_{\mathrm{visual}}23.1%
Rtask+0.6Rvisual−0.2PsizeR_{\mathrm{task}}+0.6R_{\mathrm{visual}}-0.2P_{\mathrm{size}}19.4%

The visual-reward-only policy tends to select substantially larger viewpoints, whereas the task-reward-only policy favors more localized observations. Combining the two rewards substantially reduces the average viewpoint size compared with using the visual reward alone, while achieving the best QA performance. Although introducing a viewpoint size penalty further reduces the average viewpoint size, it does not improve overall QA performance. We also analyze the viewpoint-selection behavior of the learned policy. Specifically, 48.7% of the selected views cover no more than 25% of the full frame, 19.9% cover 25%–50%, and 31.4% cover 50%–75%. Zoom-out actions account for only 13.5% of all actions.

Taken together, these results provide empirical evidence that the learned policy does not simply maximize target coverage by consistently selecting full-frame or excessively large viewpoints. We acknowledge that the reward formulation does not mathematically eliminate all possible forms of reward hacking. Rather, the combined reward design and additional analyses provide empirical evidence that the learned policy exhibits non-degenerate viewpoint-selection behavior in our current experimental setting. We have added these reward ablations and viewpoint-size analyses to the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ST7q (Part 3)

5. Dataset release and privacy protection

Response: We agree that responsible release of surveillance data requires explicit consideration of privacy protection, data authorization, and redistribution rights. All newly collected data are subject to the corresponding collection authorizations and consent procedures. Before public release, we will apply de-identification measures to the newly collected videos, including blurring identifiable faces and license plates where applicable. For videos obtained from existing public datasets, we will follow their original licenses and redistribution restrictions. Only data for which redistribution is explicitly permitted will be included in the public release, other data will be excluded or made available only under the corresponding access conditions. Upon acceptance, we plan to release the eligible dataset components, annotations, and associated metadata on Hugging Face under clearly specified usage terms, including a CC BY-NC license where applicable. We will document the data sources, authorization procedures, de-identification measures, and access restrictions to support responsible academic use. We have revised the data availability and ethical considerations statements to clarify the proposed release strategy.

Reviewer_hXSCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CamVLM for surveillance video understanding. It simulates camera control through pan and zoom operations on crops of prerecorded frames. The model learns from surveillance annotations and object-based crop trajectories, followed by GRPO with answer and target-coverage rewards. The paper also introduces CCTV-Anomaly and CamTrack-53K and reports gains in event classification, captioning, and question answering.

Critical strengths
  • The motivation is clear. Small targets and incomplete observations are practical problems in surveillance, and adaptive visual attention is a reasonable direction.

  • The new datasets could be useful for surveillance understanding and viewpoint selection. The captions and object-based trajectories provide richer supervision than event labels alone.

  • The results show that dynamic crops help the trained model. On UDVideoQA, CamVLM improves from 47.4 with a fixed crop to 57.0 with dynamic crops, and SFT followed by RL improves over SFT alone.

Critical weaknesses
  • The scope of the camera-control claim exceeds the experimental setup. Moving a crop within recorded frames cannot reveal occluded content or observations outside the original frame, so the evidence supports adaptive cropping rather than real camera control. The distinction from existing visual search and zoom tools needs to be clearer.

  • The visual reward has a basic weakness, it measures target coverage without penalizing viewpoint size, so the full frame maximizes this reward whenever the targets are in frame. The action statistics do not establish that the policy learns an efficient detail-versus-context trade-off, and a reward comparison with area or resolution constraints is missing.

  • The fixed central crop is restrictive, while simple zoom-out, detector/tracker-based crops, and full-frame observation under the same visual-token and inference budget are needed to establish the benefit beyond recovering deliberately hidden evidence.

  • The RL improves the aggregate score over SFT, but the paper does not test delayed rewards, distractors, target reacquisition, or performance across trajectory lengths, and it reports no uncertainty or end-to-end latency.

  • The random CCTV-Anomaly split does not establish separation by source video or camera, and the caption score rewards the presence of five semantic components without a clear penalty for incorrect extra details. Source-disjoint tests and event-level human evaluation would make the gains more convincing.

Questions for authors

No additional questions, I hope the rebuttal can address the weaknesses above, especially the reward design, matched-budget baselines, and the scope of the camera-control claim.

4 comments
AuthorsOpenReview ↗

Response to Reviewer hXSC (Part 1)

We sincerely thank the reviewer for the constructive comments. We appreciate the reviewer’s recognition of the potential of active visual reasoning and agree that several aspects of the current experimental setup and evaluation require clearer explanations and additional evidence. We have revised the manuscript and conducted additional experiments to address these concerns, as detailed below.


1. Scope of camera control and distinction from existing visual search methods

Response: We agree that dynamically moving a crop within a prerecorded frame cannot fully reproduce the sensing capabilities of a physical PTZ camera. In particular, our current simulation cannot recover content occluded in the original recording or acquire information outside the recorded field of view. Therefore, our experiments primarily validate active viewpoint selection and camera-like control policies in a virtualized observation space, rather than physical camera control itself. Our action space is designed to reflect the basic operations of PTZ cameras. Specifically, horizontal and vertical movements correspond to relative pan and tilt commands, zoom actions correspond to relative zoom commands, and the ``None'' action maintains the current viewpoint. These actions can be mapped to standard camera-control interfaces, such as ONVIF, subject to the hardware parameters and physical constraints of the target camera. However, this correspondence establishes the potential transferability of the learned policy, rather than demonstrating equivalence between virtual and physical camera interactions.

The distinction from existing tool-based visual search methods lies primarily in the observation protocol. In CamVLM, the model receives only the current active view at each interaction step. Accessing another region requires an explicit viewpoint-control action, and the resulting action determines the visual observation available at the next step. By contrast, existing image- and video-based reasoning tools typically retrieve, crop, or inspect task-relevant visual evidence from the available image or video content. Thus, CamVLM formulates visual evidence acquisition as a sequential observation-control problem, in which the model must jointly determine what to observe and how to act to obtain subsequent visual evidence. We have revised the Introduction, Method, and Related Work sections to clarify the scope of our simulation and the distinction between active viewpoint control and tool-based visual search. We also acknowledge that validation with physical PTZ cameras, including real-world occlusion, sensing constraints, and motion latency, remains an important direction for future work.


2. Viewpoint-size penalty and the trade-off between target coverage and visual detail

Response: We agree that the original visual reward does not explicitly penalize large viewpoints. To investigate this issue, we introduce an additional viewpoint-size penalty:

Psize=1T∑t=1TSt,P_{\mathrm{size}}=\frac{1}{T}\sum_{t=1}^{T}S_t,

where StS_t denotes the normalized area of the viewpoint at time step tt, and TT is the number of observation steps. We evaluate the effect of this penalty by adding it to the joint task and visual rewards. The results on UDVideoQA* are reported below.

Reward settingBUAtrERRRCIAvg
Rtask+0.6RvisualR_{\mathrm{task}}+0.6R_{\mathrm{visual}}70.629.446.264.772.757.0
Rtask+0.6Rvisual−0.2PsizeR_{\mathrm{task}}+0.6R_{\mathrm{visual}}-0.2P_{\mathrm{size}}71.026.545.665.172.356.4

We further compare the average viewpoint-size ratios under four reward configurations.

Reward settingAverage viewpoint-size ratio
RtaskR_{\mathrm{task}}20.5%
RvisualR_{\mathrm{visual}}90.7%
Rtask+0.6RvisualR_{\mathrm{task}}+0.6R_{\mathrm{visual}}23.1%
Rtask+0.6Rvisual−0.2PsizeR_{\mathrm{task}}+0.6R_{\mathrm{visual}}-0.2P_{\mathrm{size}}19.4%

These results reveal different effects of the reward components. The visual reward alone tends to favor larger viewpoints, whereas task reward alone encourages more localized observations. Combining the two produces a policy with a substantially smaller average viewpoint than the visual-reward-only setting, while achieving the best overall QA performance. Adding an explicit size penalty further reduces the viewpoint area but does not improve the overall QA score. We therefore use the joint task and visual rewards to balance task performance and visual evidence acquisition, rather than claiming that the visual reward alone guarantees an optimal detail-versus-context trade-off. The additional analysis and experiments have been included in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer hXSC (Part 2)

3. Comparison with alternative viewpoint-selection strategies

Response: To examine whether the performance gain comes from sequential viewpoint control rather than simply observing larger or more informative image regions, we introduce three additional baselines: Zoom-out, SAM 2-based tracking, and Full-Frame observation. All methods are evaluated under the same visual-token budget and the same inference settings. The results on UDVideoQA* are shown below.

MethodBUAtrERRRCIAvg
Zoom-out65.722.643.347.068.249.6
Tracker (SAM 2)67.723.544.849.269.151.1
Full-Frame68.126.444.647.970.451.7
Ours70.629.446.264.772.757.0

CamVLM consistently outperforms these alternative strategies, with a 5.3-point improvement over Full-Frame observation in average performance. This suggests that the improvement does not simply arise from expanding the observation region. Instead, learning a task-conditioned sequence of viewpoint adjustments enables the model to acquire more relevant visual evidence for downstream reasoning. We have added the baseline descriptions, implementation details, and comparative results to the revised manuscript.


4. Reward supervision, action intervals, and inference latency

Response: We conduct additional experiments to investigate the contribution of delayed rewards, performance across different trajectory lengths, and the latency of viewpoint-control decisions.

Reward components

To investigate the role of visual feedback under task-level supervision, we compare policies trained using task rewards alone, visual rewards alone, and their combination.

Reward settingBUAtrERRRCIAvg
RtaskR_{\mathrm{task}} only68.526.544.653.871.153.2
RvisualR_{\mathrm{visual}} only65.425.543.652.170.751.8
Full70.629.446.264.772.757.0

The combined reward achieves the best performance, indicating that task-level feedback and visual evidence feedback provide complementary supervision signals. In particular, the visual reward provides additional guidance for viewpoint selection, while the task reward directly reflects downstream question-answering performance.

Sensitivity to action intervals

We further evaluate performance across different trajectory lengths by varying the time intervals between consecutive action predictions.

Action intervalBUAtrERRRCIAvg
1 s70.629.446.264.772.757.0
2 s66.828.240.863.872.355.4
3 s62.625.638.663.168.152.0

As the interval between consecutive actions increases, the policy receives fewer opportunities to update its viewpoint within a given observation period, leading to performance degradation. This suggests that more frequent viewpoint updates help the model respond to changes in visual evidence and capture fine-grained details.

Action-generation latency

We measure the time required to generate viewpoint-control decisions on the UDVideoQA* test set. The average duration of one control round is 319 ms on a single NVIDIA RTX 4090 GPU. A control round corresponds to the model's decision-making process after each 1-s observation interval and may generate either a single action or multiple actions. When the total duration of each control round is averaged over all generated actions, the resulting average latency is 274 ms per action. We have added these experiments and clarified their evaluation scope in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer hXSC (Part 3)

5. CCTV-Anomaly data separation and caption evaluation

Response: We clarify the data-splitting protocol and provide additional event-level caption analysis.

Data separation

In the original data-splitting process, we partitioned the dataset at the camera level, ensuring that the training and test sets contain videos from different camera sources. Thus, no camera is shared between the two sets. We additionally verified that clips originating from the same source video do not appear across the training and test sets. Therefore, the split is disjoint at both the camera and source-video levels. We have clarified this protocol in the revised manuscript. In addition, we conduct event-level manual assessments on 100 randomly sampled test videos. By comparing generated captions with the corresponding video content, we assess whether the descriptions correctly capture the event subjects, actions, and outcomes. We have clarified the sampling procedure and evaluation criteria in the revised manuscript.

Caption factuality and unsupported details

We acknowledge that the original caption score primarily measures the presence of reference semantic components and does not explicitly penalize unsupported additional details. To further examine this limitation, we analyze the generated captions and find that 67.1% omit at least one reference fact, while 17.2% contain incorrect additional details. These findings indicate that omission is a more frequent error type in the current evaluation, but unsupported details remain an important limitation of the scoring mechanism. We have clarified this limitation in the revised manuscript and will investigate more explicit factuality-aware evaluation criteria that distinguish missing information from hallucinated details.

AuthorsOpenReview ↗

Hoping for a Discussion on Our Rebuttal for Paper 21122

Dear Reviewer,

I hope this message finds you well. We have carefully addressed the questions and concerns you previously raised, and we would greatly appreciate any further feedback you might be able to provide. Your insights are highly valuable to us, and we remain fully available to clarify or refine any remaining points.

As the discussion deadline is approaching, if possible, we would be grateful for your response at your earliest convenience, as it would greatly help us move the discussion forward.

Thank you sincerely for your time and effort in reviewing our paper.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.