- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates active visual perception for surveillance video understanding and proposes CamVLM to address the limitations of conventional video understanding methods that cannot actively adjust their observation viewpoints. The method simulates camera panning and zooming by dynamically cropping prerecorded videos, allowing the model to actively adjust its field of view and acquire task-relevant visual information. The authors construct two datasets, CCTV-Anomaly and CamTrack-53K, for surveillance video understanding and viewpoint control policy learning, respectively. During training, the authors combine supervised fine-tuning (SFT) and GRPO, using task completion and target-region coverage rewards to optimize the viewpoint control policy. Experiments on CCTV-Anomaly and UDVideoQA* show that the proposed method achieves favorable performance compared with the evaluated baselines.
Critical strengths
-
The paper addresses a practically relevant problem, focusing on the potential loss of critical visual information due to limited fields of view in surveillance videos.
-
The method incorporates dynamic viewpoint control into the reasoning process of vision-language models and optimizes the viewpoint selection policy through reinforcement learning. The overall design is reasonable.
-
The authors construct two datasets for surveillance video understanding and viewpoint control policy learning, which could provide useful resources for future research.
Critical weaknesses
-
The paper simulates camera viewpoint control through dynamic cropping of prerecorded videos. Given that the proposed active visual perception setting is a major contribution, the lack of supplementary evaluation on real-world limited-field-of-view or multi-view surveillance videos leaves the representativeness of the simulated environment unclear. Additional evaluations or analyses of the gap between simulated and real-world settings would strengthen the paper. Incorporating real-world limited-field-of-view or multi-view surveillance data would further enhance the research value of the proposed benchmark.
-
Table 6 only compares different training configurations and lacks separate ablations of the reward components. I suggest comparing GRPO with only the task reward, only the visual evidence reward, and the complete reward function to clarify the individual contributions of these rewards and their combined effect.
-
I suggest including simple heuristic viewpoint control strategies, such as directly zooming out or scanning in fixed directions, as baselines to assess the necessity of the learned viewpoint control policy.
If the authors adequately address these three concerns, I would be willing to increase my score.
Questions for authors
Could the authors address the concerns raised in the weaknesses above?
5 comments
Response to Reviewer 2ne9 (Part 2)
3. Comparison with heuristic viewpoint-control strategies
Response: To assess whether the performance gain comes from the learned control policy rather than simple viewpoint movements, we added two heuristic baselines:
- Zoom-out: directly expands the current viewpoint to observe a larger image region.
- Horizontal scanning: moves the viewpoint horizontally with a fixed step size of 1/9 of the original frame width.
The results are shown below.
| Method | BU | Atr | ER | RR | CI | Avg |
|---|---|---|---|---|---|---|
| Zoom-out | 65.7 | 22.6 | 43.3 | 47.0 | 68.2 | 49.6 |
| Horizontal scanning | 64.4 | 21.5 | 41.7 | 44.2 | 63.0 | 47.0 |
| Ours | 70.6 | 29.4 | 46.2 | 64.7 | 72.7 | 57.0 |
Our method outperforms both heuristic strategies, particularly in reasoning-related categories such as RR. These results suggest that a fixed viewpoint transformation or scanning pattern is insufficient to capture the task-dependent visual evidence required by surveillance video understanding. In contrast, our learned policy dynamically selects viewpoint adjustments according to the ongoing reasoning process. We have added these comparisons to the revised manuscript.
Hoping for a Discussion on Our Rebuttal for Paper 21122
Dear Reviewer,
I hope this message finds you well. We have carefully addressed the questions and concerns you previously raised, and we would greatly appreciate any further feedback you might be able to provide. Your insights are highly valuable to us, and we remain fully available to clarify or refine any remaining points.
As the discussion deadline is approaching, if possible, we would be grateful for your response at your earliest convenience, as it would greatly help us move the discussion forward.
Thank you sincerely for your time and effort in reviewing our paper.
Best regards,
The Authors
Response to Reviewer 2ne9 (Part 1)
We sincerely thank the reviewer for the constructive comments. We have conducted additional experiments and revised the discussion to better clarify the scope and limitations of our simulated viewpoint-control setting.
1. Representativeness of the simulated viewpoint-control setting
Response: Our current framework simulates active viewpoint control by dynamically cropping prerecorded video frames. We agree that this setup does not fully reproduce physical PTZ cameras or independent multi-camera surveillance systems. In particular, cropping cannot recover visual information outside the original frame or reproduce physical factors such as camera motion, occlusion, and actuation latency. We have therefore clarified that our experiments evaluate camera-like viewpoint selection in a virtualized observation space, rather than physical camera control. To further examine the effect of restricted and multiple viewpoints, we conducted two additional analyses:
(1) Fixed-viewpoint evaluation. As reported in the Passive setting of the ablation study (Section 5.3, Table 4), the model observes only a fixed central region covering 1/9 of the original frame, without dynamically adjusting its viewpoint. This provides a controlled simulation of restricted-field-of-view observation, although it is not equivalent to evaluation on videos natively captured by limited-FOV cameras.
(2) Multi-viewpoint simulation. We further evaluated the model with 2 and 3 initial viewpoints. Specifically, the initial viewpoints are placed at the top-left and center regions for the 2-view setting, and at the top-left, center, and bottom-right regions for the 3-view setting. The same dynamic viewpoint-control strategy is applied to all viewpoints. Importantly, the total visual-token budget is kept fixed across all settings, ensuring that the performance gains are not simply attributable to increased visual input or computational budgets. Instead, the available visual-token budget is distributed across different viewpoints to provide complementary spatial observations. The results are shown below.
The results show consistent performance improvements as the number of viewpoints increases, with the average score improving from 57.0 to 59.6 under the same visual-token budget. This suggests that allocating a fixed visual-token budget across multiple viewpoints enables the model to exploit complementary spatial evidence more effectively than observing a single viewpoint. We have added these results and clarified the experimental settings in the revised manuscript. We acknowledge that this simulation does not fully capture the challenges of real-world multi-camera systems. Evaluating viewpoint control on independently captured limited-FOV and multi-camera surveillance videos, where camera calibration, temporal synchronization, and cross-camera identity association become relevant, remains an important direction for future work.
2. Ablation of reward components
Response: We conduct additional experiments to isolate the contributions of the task reward and visual evidence reward in GRPO. The results are reported below.
The task reward provides direct supervision for the final question-answering objective, while the visual evidence reward encourages the policy to acquire informative visual observations. Neither reward alone achieves the performance of the full configuration. Their combination yields consistent improvements across all five evaluation categories, indicating complementary contributions to viewpoint-control policy learning. We have included this ablation in the revised manuscript.