- Rating
- 4
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies camera-motion understanding in vision–language models and proposes CameraVLM, built on Qwen3.5-VL-9B. The method introduces pose-supervised camera tokens and trajectory-supervised flow tokens to jointly represent camera motion and image motion using a shared visual encoder and multimodal backbone. A two-stage training procedure retains the language QA objective throughout, aligning continuous motion states with question semantics. On CameraBench, CineTechBench, and ShotBench, CameraVLM achieves a macro-average accuracy of 82.52%, substantially outperforming its backbone and the open-source models included in the comparison.
Critical strengths
-
Well-motivated integration of geometric motion states and language interpretation. The paper identifies concrete ambiguities in camera-motion understanding, including camera versus subject motion, pan versus truck, and zoom versus dolly. Its geometric analysis provides a coherent motivation for combining camera states with scene-wide trajectory states. These representations are integrated into a shared multimodal backbone and learned with language supervision throughout training, connecting continuous motion modeling with question answering. The resulting model requires only video and questions at inference.
-
Strong empirical performance supported by complementary evaluations. CameraVLM substantially outperforms its backbone and the compared open-source baselines across multiple camera-motion benchmarks. The comparison covers general-purpose, geometry-aware, camera-motion-specific, and commercial models, providing broad evidence of the method’s effectiveness. Pose estimation, point tracking, component analyses, qualitative examples, and human evaluation further help assess the learned representations and their practical capabilities. Together, these results provide substantive empirical support for the proposed approach.
Critical weaknesses
Major Weaknesses
-
The central claim of query-dependent motion understanding lacks direct evaluation. The paper formulates a query as Q = (a, b, F, τ), emphasizing that the same dynamic state can support different descriptions depending on the target, reference entity, reference frame, and temporal interval. Figure 2 and Appendix A further discuss situations involving opposing viewpoints and mirror reflections. However, the reported results are primarily aggregated by camera-motion primitive or category. The paper does not separately evaluate whether the model produces geometrically consistent changes in its answers when these query elements are varied for the same video. Improvements on the existing benchmarks support better camera-motion recognition, but do not independently establish the claimed ability to reinterpret a continuous state under different queries. A targeted paired-query evaluation would strengthen this claim, with separate results for reference changes, temporal-interval changes, and reflection-related cases. If these situations are already covered by the existing benchmarks, their sample counts and corresponding results should be reported explicitly.
-
The ablations of the two-stage training strategy and component contributions remain confounded. In Table 4, the two-stage variants are trained for 10K iterations per stage, whereas One-stage Training runs only the initial 10K-iteration state-interface stage with the language backbone frozen. Consequently, the difference between 69.0% and 60.8% reflects both the additional training budget and the unfreezing of the language backbone, and cannot be attributed specifically to the ordering of interface adaptation followed by joint training. A budget-matched single-stage baseline that jointly optimizes the model from the beginning would provide a more informative comparison. The authors should also clarify whether “w/o Camera & Flow” removes the supervision, tokens, prediction heads, or the entire corresponding pathways. In addition, a flow-only training variant is missing. The “w/o Flow” variant achieves 72.20% on ShotBench, exceeding the full ablation model’s 70.79%, so the benefit of combining the two state types is not uniform across benchmarks. Although the average results support their complementarity, their individual contributions and the conditions under which they help require clearer characterization.
-
Direct controlled comparisons with closely related trajectory representations and geometry–language interfaces are missing. Table 1 includes a broad set of baselines, but some closely related approaches cited in Section 2 are not evaluated. TrajTok (Zheng et al., CVPR 2026; arXiv:2602.22779) represents trajectory information as tokens for video understanding, and its TrajVLM training includes CameraBench QA. Geometry-Guided Camera Motion Understanding in VideoLLMs (Feng et al., CVPR Workshops 2026; arXiv:2603.13119) directly investigates injecting geometric camera-motion cues into VideoLLMs and also explores feature distillation. These approaches are relevant to the proposed trajectory representation and geometry-to-language interface, respectively. Their original backbones, training data, and evaluation protocols differ, so their published numbers should not be compared directly with the present results. Where implementations or checkpoints are available, compatible evaluations would help establish the specific advantages of CameraVLM. Alternatively, trajectory-token or structured-motion-prompt controls using the same backbone and training budget would be informative. If such comparisons are infeasible, the authors should explain the concrete constraints and more precisely distinguish the contribution from these already cited approaches.
Minor Weaknesses
-
Several implementation details needed for reproducibility remain unspecified. Section 3.5 and Appendix B do not explicitly provide the values of λc and λf, the pose coordinate convention and scale normalization, or the sampling proportions of video-only, pose-only, and video+pose inputs. Table 5 also includes Kubric without language annotations and 3DThinker without camera annotations. The authors should explain how the corresponding losses are activated or omitted when supervision is unavailable, and whether the two training stages use the same data-mixture proportions.
-
The DPT module lacks a citation to its original reference. The flow pathway in Appendix B uses a DPT decoder, but the reference list does not appear to include Ranftl, Bochkovskiy, and Koltun, “Vision Transformers for Dense Prediction,” ICCV 2021. The citation should be added at the first occurrence of this component. Most baselines in the main result tables already have source citations; this comment concerns a specific omission in the implementation description.
-
Some annotations and experimental settings in the supplementary material require clarification. The asterisk attached to CoWTracker in Table 5 is not explained. Table 7 varies the number of flow tokens, spatial grid, temporal window, and refinement iterations, but does not specify whether these variants are retrained models or inference-time modifications of the same checkpoint. Stating these details would make the interpretation of the sensitivity analysis more precise.
Questions for authors
-
Query-dependent evaluation: Do the existing evaluation sets contain paired questions that change the target, reference entity, reference frame, or temporal interval for the same video? If so, please report their coverage and performance separately, including reflection-related cases where available. This would clarify how directly the current results support the query-dependent formulation.
-
Training and component controls: Are results available for a single-stage model jointly trained for the same total 20K iterations as the two-stage ablation model? Please also clarify exactly what is removed in “w/o Camera & Flow” and whether a flow-only training result is available. These controls would help distinguish the effects of training duration, backbone unfreezing, and complementary state supervision.
-
Comparison with closely related approaches: How does the proposed flow-token and state–language alignment design differ operationally from TrajTok/TrajVLM (Zheng et al., 2026; arXiv:2602.22779) and Geometry-Guided Camera Motion Understanding in VideoLLMs (Feng et al., 2026; arXiv:2603.13119), both cited in Section 2? Are compatible comparisons or matched-backbone controls available? If such comparisons are infeasible, please clarify the implementation, backbone, or evaluation constraints that prevent them.
2 comments
W3 & Q3: Comparison with TrajTok/TrajVLM and Geometry-Guided Methods
A: TrajTok has only released its Segmenter and TrajViT-v2 checkpoints without the full TrajVLM checkpoint, and Geometry-Guided has not released its full inference pipeline. We therefore evaluate the released TrajViT-v2 in Table R3, compare on Geometry-Guided's CameraMotionVQA benchmark in Table R4, and run matched-backbone structured and VGGT-guided prompting controls on Qwen3.5-VL in Table R5.
Operational differences: TrajTok compresses video into sparse trajectory tokens for efficient tokenization without modeling camera ego-motion. Geometry-Guided injects external VGGT cues via text prompting or one-way feature distillation without explicit scene-flow tokens. In contrast, CameraVLM jointly learns 3D camera poses and 256-grid full-scene trajectories directly from native VLM features, pooling trajectories into 64 flow tokens that interact with camera and video tokens inside the LLM via two-stage state–language alignment.
Empirical findings: CameraVLM substantially outperforms TrajViT-v2 in Table R3 and all baselines on CameraMotionVQA in Table R4, including the 397B Kling-MLLM. Under the matched Qwen3.5-VL backbone in Table R5, external VGGT-guided prompting brings only modest gains, leaving a large gap to CameraVLM. We will add these comparisons and discussions to the revision.
Table R3. Released TrajViT-v2 Evaluation (Accuracy %)
| Model | CameraBench | CineTech | ShotBench | Avg. |
|---|---|---|---|---|
| TrajViT-v2 | 47.2 | 31.0 | 31.0 | 36.4 |
| CameraVLM | 88.2 | 77.9 | 81.5 | 82.5 |
Table R4. CameraMotionVQA from Geometry-Guided (1,228 Questions, Accuracy %)
| Model | Accuracy |
|---|---|
| Qwen3.5-VL-9B | 27.8 |
| Cambrian-P | 27.3 |
| TrajViT-v2 | 25.6 |
| Kling-MLLM-35B-A3B | 42.0 |
| Kling-MLLM-27B | 47.2 |
| Kling-MLLM-397B | 54.7 |
| CameraVLM | 59.5 |
Table R5. Structured and Geometry-Guided Prompting Controls (Accuracy %)
| Model | Prompt | CameraBench | CineTech | ShotBench | Avg. |
|---|---|---|---|---|---|
| Qwen3.5-VL | Standard QA | 66.0 | 57.5 | 52.6 | 58.7 |
| Qwen3.5-VL | Motion Instruction Prompt | 69.2 | 58.4 | 50.9 | 59.5 |
| Qwen3.5-VL | VGGT-guided Prompt | 70.4 | 61.3 | 62.8 | 64.8 |
| CameraVLM | Standard QA | 87.8 | 78.8 | 81.5 | 82.7 |
| CameraVLM | Motion Instruction Prompt | 86.9 | 77.9 | 80.4 | 81.7 |
W4: Implementation Details
A: We activate or ignore each loss per sample using binary validity masks before averaging over valid batch elements: QA Loss ignores unlabeled text tokens via the cross-entropy ignore index on Kubric, Camera Loss zeros out pose gradients on 3DThinker, and Flow Loss masks out missing or invalid trajectories and frames.
Both stages share the identical data mixture, with input modality sampling ratios following the natural distribution of the combined datasets. We set in Stage 1 and in Stage 2, and represent camera poses in the OpenCV world-to-camera convention anchored at the first frame with translations normalized by mean scene depth. We will include these details in the revision.
W5: DPT Citation
A: Thank you for pointing out the missing citation. We will cite Ranftl, Bochkovskiy, and Koltun, Vision Transformers for Dense Prediction (ICCV 2021), at the first mention of the DPT decoder.
W6: Additional Clarifications
A: The asterisk next to CoWTracker in Table 5 was mistakenly added during formatting. All variants in Table 7 are inference-time modifications using the same trained checkpoint, without retraining. We will remove the asterisk and clarify the experimental settings in the revision.
We thank the reviewer for recognizing the importance of jointly modeling camera and scene motion, the integration of geometric states with language understanding, and the strong empirical results. Below, we address each of the reviewer’s concerns in detail.
A: To directly test whether the model dynamically reinterprets identical videos under changing queries, we evaluated 77 same-video paired queries varying reference frames, mirror reflections, and temporal intervals/order. As shown in Table R1, CameraVLM achieves 96.10% overall, directly confirming that our continuous geometric states support query-dependent motion reinterpretation beyond static classification.
Table R1. Query-Dependent Motion Reasoning (Accuracy %)
A: Table R2 adds a budget-matched One-stage Joint baseline and a Flow-only variant under the same 20K training steps. Here, w/o Camera and w/o Flow remove the entire corresponding pathway, including state tokens, prediction heads, and supervision losses.
Staged alignment: At the matched 20K-step budget, Two-stage Full outperforms One-stage Joint by +5.89 pp, confirming that adapting the state interface before joint tuning prevents optimization conflicts between geometric representations and the LLM.
Complementary conditions: Pose models global camera motion and suffices when scenes are dominated by pure camera movement, which is why Pose-only slightly edges out the full model on camera-centric ShotBench without extra local flow noise. In contrast, Flow is required whenever moving objects entangle with camera motion, driving substantial gains on CameraBench and CineTech.
Table R2. Two-stage Training and Component Ablations (Accuracy %)