acceptodds
Under review as a conference paper at ICLR 2027

CamRM: Towards Reliable Camera-Motion Control in Video Generation with Motion-Grounded Rating Models

Abstract

Test-time scaling provides a practical way to improve camera-motion control in video generation: sample a clip, rate its alignment with the request, and revise the prompt if needed. This process requires a reliable rating model (RM) to determine when to stop. Geometry-based methods can struggle when object motion dominates the view, while fine-tuned visual language models (VLMs) often fail to adapt to synthetic videos. Our benchmark of human-labelled real and synthetic videos shows that these methods struggle to judge camera-motion alignment. Strong performance on real video does not ensure reliable judgment on generated clips: a model tuned only on real footage reaches % balanced accuracy on real video but drops to % on Omni clips. We introduce CamRM, which combines video frames with motion cues to judge camera-motion alignment. We develop CamRM on both the proprietary Gemini model and the open-source Qwen model, with consistent improvements of over % across both. We also develop a training-free agentic model (CamRM-A), showing that our approach requires no fine-tuning. Our CamRM achieves % accuracy and improves balanced accuracy over the strongest baseline by %. In the motion-guided refinement loop, using CamRM as RM improves human-judged alignment by % for Veo 3.1 and % for Omni Flash at the generation cost, reaching % alignment in the Omni Flash T2V case. At generations, the loop reaches % pooled candidate-set oracle alignment versus % for i.i.d. resampling, and matches its -generation oracle rate with three generations, without generator training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Camera Control, Video Generation, Test-time Scaling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_QqiROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CamRM combines video frames, a 3×3 spatial optical-flow summary, and a global image-motion estimate in a vision-language model to judge whether a clip matches a requested camera motion. The paper presents a tuned Gemini model, CamRM-G; a tuned Qwen model, CamRM-Q; and a tool-using variant without parameter updates, CamRM-A. These evaluators are used in a prompt-rewriting and regeneration loop. The test set contains 1,064 clips: 571 from Veo 3.1 or Omni and 493 from CameraBench. Table 1 reports 79.21% weighted average balanced accuracy for CamRM-G, 6.60 percentage points above the strongest non-author baseline at 72.61%.

Critical strengths

A simple design supported across model families. Providing inexpensive motion measurements as text to a VLM is a practical design choice. In Table 2, the untuned model improves from 68.67% balanced accuracy in the vanilla setting to 74.33% with the full cue configuration, so the gains cannot be attributed entirely to fine-tuning. In the matched Qwen SFT ablation, combined cues also outperform flow or global motion alone.

Critical weaknesses

Major concern: central results need a reconciled evaluation protocol. Figure 3(a) reports 53.8% post-loop alignment on 75 Veo I2V items, whereas Figure 3(b) and Table 3 report 61.3% for CamRM-G on the same page. This might reflect averaging across raters versus majority voting per item, but that distinction is not clearly explained. A more direct issue is that Appendix A.6 says the two pass@k strategies share their first candidate, yet Figure 4 shows different k=1 success rates: 54.5% versus 54.2% pooled, 40.6% versus 41.2% on Veo, and 68.9% versus 67.7% on Omni. With identical candidates and evaluation labels, these starting points should agree. The paper needs a unified account of the items, label aggregation, grading rounds, and item-level pairing, including the binary outcomes used for the McNemar tests. My concern is that the results cannot yet be reconciled, rather than that the gains necessarily disappear.

Questions for authors

Q1: Please reconcile the 53.8% in Figure 3(a) with the 61.3% in Table 3/Figure 3(b), and the differing k=1 rates in Figure 4 despite the shared first candidate. Do these results use the same items, grading rounds, and label-aggregation rules? Please provide item-level paired outcomes and update the significance tests where necessary. If the main gains hold after reconciliation, this would directly improve my assessment.

Q2: How exactly are geometric estimates converted into motion categories and binary alignment judgments? Please provide per-model parsing/reconstruction coverage and comparisons that handle failures consistently and use identical mappable subsets. If CamRM retains a clear advantage under this common protocol, it would address my main concern about the baseline comparison.

5 comments
AuthorsOpenReview ↗

Response to Reviewer QqiR (1/2)

Thank you for the detailed review and for stating clearly what would change your assessment. Below we reconcile every number the reviewer raised, add a common-grader evaluation, and update the tests. The main gains hold in every case.

Major concern and Q1 (a). 53.8% in Fig. 3(a) versus 61.3% in Table 3 / Fig. 3(b)

The two numbers come from two separate experiments. Fig. 3(a) measures the full system across four generated sets, and Table 3 compares seven rating models on Veo I2V. Each experiment was graded in its own round by its own group of graders, who focused on that experiment's comparison. The same 75 Veo I2V videos and the same before-loop baseline therefore received different absolute rates in the two experiments; every comparison inside an experiment uses one grader group throughout. The gap does not come from averaging versus majority voting: in both experiments, the after-loop rate is the mean of the graders' labels. Sec. 4 now describes this grading design.

To show that the conclusions do not depend on the grader group, we also evaluated both experiments with one common group of graders, who labelled the returned clips of every configuration in both experiments (App. A.16). Under this common protocol, the two experiments give the same value for CamRM-G on Veo I2V (56.7%). The loop improves every set:

Common gradersVeo I2VVeo T2VOmni I2VOmni T2V
Before the loop (%)37.346.355.466.3
After the loop (%)56.764.277.084.2
Paired sign-flip testp = 0.002p < 0.001p < 0.001p < 0.001

In the rating-model comparison, CamRM-A and CamRM-G are again the two best rating models in the loop:

Rating model in the loop (common graders)Alignment (%)Generations / item
No loop37.3—
CamRM-G, no rewriter40.02.20
Gemini 3.5 Flash46.01.63
CamRM-G, CoT withheld53.32.39
CamRM-Q54.02.28
VGGT55.33.07
CamRM-G56.72.16
CamRM-A60.02.19

The gains are also significant under the paper's own grading: in Fig. 3(a), every set improves after the loop (paired sign-flip tests, p = 0.008, 0.048, 0.002 and < 0.001 for Veo I2V, Veo T2V, Omni I2V and Omni T2V).

Q1 (b). Fig. 4: different k = 1 rates despite the shared first candidate

At k = 1 both strategies show the same first generation, the identical file. The labels are not shared. Each strategy's eight candidates are graded together as one shuffled set, so that graders cannot infer anything from the order, and the shared video is therefore judged once inside each set. A few of these two judgments disagree, which moves the k = 1 rates by at most about 1% (54.2% vs. 54.5% pooled, 40.6% vs. 41.2% on Veo, 68.9% vs. 67.7% on Omni). That shift is well within the 95% intervals of Fig. 4 and an order of magnitude smaller than the k = 8 gap (88.3% vs. 78.4% pooled; McNemar 43 : 10, p < 10⁻⁵). Both strategies are graded in exactly the same way. With one label for the shared video, the loop still reaches 97.6% vs. 90.9% at k = 8 on Omni, and 81.2% vs. at most 66.5% on the Veo 3.1 study of App. A.6; p ≤ 0.001 for both. App. A.6 now states this explicitly.

Q1 (c). A unified account of items, label aggregation, grading rounds and item-level pairing

New App. A.16 adds one protocol table that covers every human evaluation in the paper: the items, what is graded, how labels are combined and which paired test is used. In short, each experiment is graded in its own round by its own group of graders, and every comparison inside an experiment is paired by item. For the loop experiments the per-item outcome is the label of the returned clip; for Fig. 4 it is whether an aligned clip appears among the first k candidates.

Q1 (d). Item-level paired outcomes and updated tests

Every test is paired by item. App. A.16 lists, for every test, the paired counts it uses, i.e. the items that one configuration gets right and the other does not; below are the ones quoted in the text. Table 3 rows use the paper's grading; common-grader rows give the counts of the two graders separately (grader 1; grader 2).

ComparisonΔPaired countsp
Table 3: CamRM-G vs. no rewriter+17.3%17 : 40.007
Table 3: CamRM-G vs. Gemini 3.5 Flash+13.3%13 : 30.021
Fig. 4, pooled, k = 8: loop vs. resampling+9.9%43 : 10< 10⁻⁵
Delivered@8 (App. A.20), Veo 3.1: loop vs. resampling+11.8%33 : 130.005
Delivered@8 (App. A.20), Omni: loop vs. resampling+10.4%29 : 120.012
Common graders: CamRM-A vs. Gemini 3.5 Flash+14.0%19 : 3; 13 : 80.015
Common graders: CamRM-A vs. no rewriter+20.0%21 : 2; 16 : 5< 0.001
AuthorsOpenReview ↗

Response to Reviewer QqiR (2/2)

Q2. How geometric estimates become categories and binary judgments; coverage; consistent failures; identical subsets

All geometry baselines share one rule-based decoder with no learned parameters, so they differ only in the camera poses they estimate, and each method sees the same frames as the VLM raters. The decoder turns the estimated trajectory into net rotation, depth-normalised translation and focal change. It then scores each of the 20 requested motions along its canonical axis with fixed geometric rules; compound motions such as orbits combine axes. A clip is accepted when the requested motion scores at least as high as every competitor, with one threshold fixed in advance (τ = 0) and never fitted on test data. Every RM answers the same request, so pan/truck and zoom/dolly stay distinct for all models, and VGGT's per-frame intrinsics give it the cue that separates a zoom from a dolly (App. A.18).

On coverage, CamRM-G and CamRM-A return a verdict for every clip. The VLM baselines parse at least 98% of clips, except Qwen3.6-27B (about 88%), and the Gemini and Claude raters return a verdict for at least 99.5% of the generated test clips. VGGT, CUT3R and MegaSAM reconstruct nearly every clip, and COLMAP about 75%. FilmOps has no tilt, pedestal or dolly-zoom class and is scored on the clips it can map (marked * in Table 1).

Because our raters have no missing verdicts, any common failure rule leaves their scores unchanged. Counting every missing verdict as an error, which is the rule Table 1 uses for geometry, can only lower the baselines. Dropping missing verdicts instead mainly helps COLMAP, the only geometry baseline with many failures, and even on the clips it reconstructs COLMAP stays far below every CamRM variant. On identical subsets, CamRM-G reaches 78.2% (Veo) and 71.2% (Omni) bAcc on FilmOps's mappable subset, versus 66.4% and 58.9% for FilmOps. CamRM therefore keeps a clear advantage under a common protocol. App. A.18 gives the full description and a per-model coverage table.

Presentation

We added a protocol table (App. A.16) that gives the items, generator version, grader setting, label aggregation and pairing of every human evaluation, and the Fig. 4 caption now states the generator version and how the shared first candidate is graded. Every headline number is scoped to its experiment.

Dear Reviewer QqiR,

Thank you again for your detailed review and for stating so clearly what would inform your assessment. As the discussion period draws to a close, we would like to briefly note how our responses and revised paper (changes in blue) address your points. We reconcile the 53.8% and 61.3% figures and the k = 1 rates in Fig. 4. We also re-grade both loop experiments with one common grader group. Under that group, the loop improves every set (p ≤ 0.002), and CamRM-A and CamRM-G remain the two best rating models. A new protocol table gives the items, aggregation, pairing and paired counts of every test. For the baselines, we document the geometry decoder and per-model coverage, and show that CamRM keeps a clear lead under the same failure rule and on identical subsets.

We would be glad to hear any further thoughts and are happy to address any remaining questions. Thank you once more for your valuable comments and your time.

Reviewer_Z69GOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Summary: A Stronger RM Improves Closed-Loop Camera Control, but Reproducibility and Unseen-Generator Evidence Are Limited

This paper improves the rating model (RM), not the video generator, in a test-time loop. After rejection, a rewriter uses RM reasoning to revise the prompt. CamRM uses frames, a 3x3 optical-flow summary, and global-motion estimates. CamRM-G tunes Gemini 3.5 Flash with GRPO-style RL; CamRM-Q updates Qwen3.5-9B LoRA parameters with SFT and SAPO; CamRM-A keeps the model frozen and calls motion tools.

Critical strengths

The controlled RM swap shows that rating quality affects delivered videos. Related feedback and test-time generation methods exist, while CamRM focuses on camera-motion alignment.

Critical weaknesses

Training and testing use Veo 3.1 and Omni, leaving unseen-generator transfer untested. The loop is not a standard harness. Its budget and stopping rule are given, but complete system prompts are missing. No code is linked; release is promised after publication. This matters because prompting raises bAcc from 68.67% to 72.14%, and giving RM reasoning to the rewriter raises alignment from 49.3% to 61.3%. CamRM-G also depends on a commercial API, while CamRM-A lacks full-loop and cost results.

Questions for authors

Can the authors provide a reproduction package, test an unseen generator, and report CamRM-A's full-loop accuracy and cost?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Z69G (1/2)

We thank the reviewer for the clear summary and the concrete requests on our paper.

W1 and Q. Transfer to an unseen generator

We tested CamRM on Wan 2.2, an open-weight generator family that no CamRM variant saw in training. We generated 306 clips (156 T2V, 150 I2V) from our test prompts, kept every clip, and had two graders label each one. Every rater runs with its Table 1 configuration on the same clips:

Rater (Wan 2.2, bAcc %)T2VI2VAll
CamRM-G (ours)90.289.689.9
Optical-flow rule78.184.481.2
Gemini 3.6 Flash83.073.678.4
Gemini 3.1 Pro77.479.278.3
ACaM Qwen3-VL-8B81.574.878.2
VGGT71.880.476.0
CUT3R70.380.075.0
Gemini 3.5 Flash (base)74.674.074.3
MegaSAM73.074.873.9
Claude Opus-4.874.471.673.0
Qwen3.6-27B70.473.471.9
gemma-4-31B-it72.268.070.1
Claude Opus-4.566.170.068.0
CameraBench-Qwen2.5-VL-7B64.464.464.4
Claude Sonnet-567.658.863.3
Qwen3.5-9B62.054.858.5

CamRM-G has the highest bAcc on both tasks. It leads the strongest baseline by 8.8% (paired bootstrap 95% CI: +3.7% to +15.5%) and its own untuned base by 15.6%. Only 19% of Wan clips are aligned, so this set tests whether a rater rejects wrong clips: CamRM-G rejects 79.8% of them while accepting all 58 aligned clips. The result is now in the main paper (new Sec. 4.7, Table 5).

We also ran a leave-one-generator-out test (App. A.19). CamRM-G trained without any Veo clip reaches 73.7% on Veo, 5.8% above its untuned base and on par with the strongest zero-shot rater (Gemini 3.6 Flash). Its five-set average (77.5%) stays far above every baseline in Table 1.

W2. The loop is a custom harness; complete system prompts are missing

The revision makes the loop fully specified. New App. A.22 gives the complete prompt of every rating model in Tables 1–3, with the cue blocks and the configuration of each variant, the complete rewriter prompt, and the instructions shown to human graders. New App. A.17 gives what the rewriter receives in each round, the two ablation variants (CoT withheld and the VGGT geometry report), the scene-change detector with its thresholds, and Algorithm 1, which specifies the budget accounting, the accept and return rules, scene-change redraws and the no-rewriter mode.

In short, the loop rates the initial clip and accepts a clip only when the RM judges it aligned and no scene cut is detected. Otherwise the rewriter writes a new prompt from the target motion and its definition, the current prompt, the RM's reasoning, the earlier attempts and the failed clip. After three refinement rounds the loop returns the last clip. Every arm in Table 3 runs this same loop and changes only the component named in its row.

W3 and Q. Reproduction package; no code during review

Upon publication we will release the full loop harness and the training and evaluation code, including motion-cue extraction, together with the open CamRM-Q weights, the dataset with its human annotations, and the complete prompts. Until then, the revised paper specifies every component needed to re-implement the method: complete prompts and Algorithm 1 (App. A.22 and A.17), the cue definitions (Sec. 3 and App. A.3), how each baseline reaches a verdict (App. A.18), and a protocol table for every human evaluation (App. A.16).

W4. Results depend on prompts (68.67% → 72.14%; RM reasoning given to the rewriter)

We agree that the prompts matter, which is why the revision publishes them in full (App. A.22): every prompt behind the reported numbers is now given verbatim. Both contrasts the reviewer cites are designed components of the method, and each is isolated by its own ablation. These are the structured rating prompt (Table 2, Vanilla → V1) and the use of the RM's reasoning by the rewriter (Table 3, CoT withheld); the RM's diagnosis is part of how the loop works, not an incidental prompt choice. Most of CamRM's gain comes from the motion cues and training rather than from the prompt. On top of the prompt, the cues alone lift untuned Gemini from 72.1% to 74.3% bAcc, and with training CamRM-G reaches 79.2%. The same recipe on an open Qwen model gains 19.6% over its base, so the benefit is not tied to one model family.

AuthorsOpenReview ↗

Response to Reviewer Z69G (2/2)

W5. CamRM-G depends on a commercial API

CamRM-Q is built on the open Qwen3.5-9B, and we will release its weights. It reaches 75.7% bAcc, above the best open baseline (Qwen3.6-27B, 69.8%) and above every camera-motion specialist (e.g., ACaM-8B, 68.8%). In the loop it lifts alignment from 37.3% to 57.3% (Table 3, paper grading), within 4% of CamRM-G (61.3%). CamRM-A, in turn, needs no training at all, and we report its full-loop results below.

W6 and Q. CamRM-A's full-loop accuracy and cost

We ran CamRM-A as the rating model in the Table 3 protocol, with the same 75 Veo I2V items, rewriter, cap and scene-change check. In the loop, CamRM-A calls its two motion-cue tools, optical flow and global-motion estimation, since the loop's own check already handles scene cuts. All rating models in this comparison are graded by the same common grader group (App. A.16, A.21):

Rating model in the loop (common graders)Alignment (%)Generations / item
Gemini 3.5 Flash (vanilla)46.01.63
CamRM-Q54.02.28
CamRM-G56.72.16
CamRM-A (training-free)60.02.19

CamRM-A gives the highest alignment and the highest fix rate (48.9%) of all rating models. CamRM-G stays the most accurate rater on the benchmark (79.2% vs. 78.0% bAcc), and CamRM-A shows that the same motion cues bring a frozen model to the best loop alignment with no training at all. It improves on the vanilla RM by 14.0% (p = 0.015) and on the loop without a rewriter by 20.0% (p < 0.001). Its generation cost matches CamRM-G's: per item it uses 2.19 generations, 2.08 rating calls and 1.08 rewrites, against 2.16, 2.12 and 1.12. To measure the cost of a rating, we re-ran CamRM-A's loop configuration on the 156 clips the loop rated:

CamRM-A costPer rating callPer item (2.08 calls)
Tool calls2.14.4
Model turns2.75.6
Tokens (input / thinking / output)11.7k / 1.7k / 0.3k24.3k / 3.5k / 0.6k
Wall-clock time27 s55 s

The added cost is a few tool calls per rating: CamRM-A needs no training and uses about as many generations as CamRM-G.

Dear Reviewer Z69G,

Thank you again for your clear and constructive review. As the discussion period draws to a close, we would like to briefly note how our responses and revised paper (changes in blue) address your requests. On the unseen generator Wan 2.2, CamRM-G reaches 89.9% bAcc, 8.8% above the strongest baseline. The revision now specifies the loop in full, with every prompt, the rewriter inputs, the scene-change check and Algorithm 1. In the full loop, the training-free CamRM-A reaches the highest alignment of all rating models (60.0%). It uses about as many generations as CamRM-G and adds only about two tool calls per rating.

We would be glad to hear any further thoughts and are happy to address any remaining questions. Thank you once more for your valuable comments and your time.

Reviewer_SQnYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces CamRM, which combines video frames with textual optical-flow and global-motion summaries to assess camera-motion alignment. It presents a 1,064-video human-labeled benchmark covering real and generated clips, alongside fine-tuned and training-free model variants. Experiments report improved alignment assessment and, through a prompt-refinement loop, better human-judged camera-motion alignment on Veo 3.1 and Omni Flash without retraining the generators.

Critical strengths
  • A consequential evaluation gap: The 1,064-video benchmark shows that strong camera-motion assessment on real footage does not reliably transfer to generated clips. This provides useful evidence that existing evaluation practices can overlook failures relevant to video generation.

  • Controlled evidence of downstream benefit: With the same rewriter and generation cap, replacing the vanilla rating model with CamRM-G improves human-judged alignment from 48.0% to 61.3% on the Veo I2V subset. This connects better assessment to improvements in delivered videos, strengthening the paper’s practical significance.

Critical weaknesses
  • Limited evidence of generalization: The tuned models are trained and tested on the same two generator families, with ambiguous and multi-shot clips filtered out. Robustness to unseen generators remains unestablished, which matters because reliable assessment under synthetic-video distribution shifts is a central motivation.

  • Incomplete validation of practical scaling efficiency: The strongest scaling result—three generations matching eight independent samples—uses candidate-set oracle success, assuming perfect selection. It therefore does not establish equivalent savings for successfully delivered videos, limiting the practical significance of the efficiency claim.

Questions for authors
  • Appendix A.6 states that both strategies share the first candidate, but Figure 4 reports different k=1k=1 alignment rates. Were candidate identities and human labels shared, and what explains this discrepancy?

  • At Figure 4’s matched generation budgets, how do the strategies compare in delivered alignment using the same deployable selection rule instead of oracle selection? This would determine whether the search gains translate into practical output improvements.

3 comments
AuthorsOpenReview ↗

Response to Reviewer SQnY (1/2)

We sincerely appreciate your positive assessment and the precise questions.

W1. Generalization to unseen generators; ambiguous and multi-shot clips are filtered

We tested CamRM on a generator family it has never seen. We generated 306 clips with Wan 2.2 (open weights, 720p; 156 T2V and 150 I2V) from our test prompts and kept every generated clip. While labelling, the graders also checked every clip for cuts, and the final test set contains no multi-shot clip. Two graders labelled every clip (Cohen's κ 0.80 for T2V and 0.68 for I2V), and a clip is aligned when the mean score exceeds the benchmark threshold. No CamRM variant saw Wan data in training. We compare against every family of Table 1, each run with its Table 1 configuration on the same clips:

Rater (Wan 2.2, bAcc %)T2VI2VAll
CamRM-G (ours)90.289.689.9
Optical-flow rule78.184.481.2
Gemini 3.6 Flash83.073.678.4
Gemini 3.1 Pro77.479.278.3
ACaM Qwen3-VL-8B (camera-motion specialist)81.574.878.2
VGGT (geometry)71.880.476.0
CUT3R (geometry)70.380.075.0
Gemini 3.5 Flash (CamRM-G's base)74.674.074.3
MegaSAM (geometry)73.074.873.9
Claude Opus-4.874.471.673.0
Qwen3.6-27B70.473.471.9
gemma-4-31B-it72.268.070.1
Claude Opus-4.566.170.068.0
CameraBench-Qwen2.5-VL-7B (camera-motion specialist)64.464.464.4
Claude Sonnet-567.658.863.3
Qwen3.5-9B62.054.858.5

CamRM-G has the highest bAcc on both tasks. It leads the strongest baseline by 8.8% (paired bootstrap 95% CI on pooled bAcc: +3.7% to +15.5%), the strongest zero-shot VLM by 11.5%, the strongest geometry method by 13.9%, and its own untuned base by 15.6%. Wan follows camera-motion requests far less often than Veo or Omni: only 19% of its clips are aligned, versus 49–71% on the same prompts. This set therefore directly tests whether a rater rejects wrong clips, and CamRM-G rejects 79.8% of them while accepting all 58 aligned clips. We added this result to the main paper as new Sec. 4.7 and Table 5 (with Acc, F1 and bAcc per task), and App. A.19 describes the benchmark.

We also ran a leave-one-generator-out test (App. A.19). We trained the CamRM-G recipe with every Veo clip removed and chose the checkpoint by its Omni accuracy, so Veo is entirely unseen. On Veo it reaches 73.7% bAcc: 5.8% above its untuned base, on par with the strongest zero-shot rater, and above every other baseline in Table 1 (next best on Veo: Gemini 3.1 Pro, 70.7%). It also keeps its strength on the other sets, and its average over all five benchmark sets (77.5%) stays far above every baseline:

bAcc (%)Veo (unseen by the masked model)OmniCameraBenchAverage
Gemini 3.5 Flash (base, zero-shot)68.063.671.468.7
Gemini 3.6 Flash (strongest zero-shot)73.763.875.672.6
Qwen3.6-27B (strongest open VLM)63.665.476.269.8
ACaM-8B (strongest specialist)69.852.175.168.8
VGGT, 24 fps (strongest geometry)64.063.174.368.6
CamRM-G trained without Veo73.767.184.777.5
CamRM-G (all training data)78.269.584.179.2

On filtering: the benchmark asks whether one requested camera motion is present in a continuous shot. A clip with a cut breaks the camera trajectory, so it has no single motion to label, and clips whose annotators gave no clear majority have no reliable ground truth. We therefore exclude both from the benchmark, so that every test label is well defined. In deployment, multi-shot outputs are handled by the loop's scene-change check: a clip with a cut is never accepted and never counts as aligned, which matches the human rubric.

AuthorsOpenReview ↗

Response to Reviewer SQnY (2/2)

W2 and Q2. Scaling efficiency rests on oracle selection; delivered alignment at matched budgets

We now measure delivered alignment with the same rating model for both strategies. CamRM-G judges every candidate of both strategies with the same configuration. For a budget of k generations, each strategy returns the first candidate that CamRM-G accepts and that has no scene cut, or the k-th candidate if none is accepted. Delivered alignment is the human label of the returned clip. We run this on the two scaling studies with human labels for all eight candidates of both strategies: Veo 3.1 (170 items, App. A.6) and Omni (164 items, Fig. 4).

Budget kVeo 3.1, delivered (resampling / loop)Omni, delivered (resampling / loop)Generations used (resampling / loop)
146.5 / 47.6 (same video)67.7 / 68.9 (same video)1.00 / 1.00
4 (the loop's cap in Table 3)47.1 / 57.6 (p = 0.010)68.9 / 78.7 (p = 0.020)2.73 / 2.64
8 (the budget of Fig. 4)49.4 / 61.2 (p = 0.005)68.3 / 78.7 (p = 0.012)4.09 / 3.79

p is the exact McNemar test, paired by item (App. A.16 gives the paired counts); generations are averaged over both generators.

When CamRM-G picks the returned clip, resampling stays nearly flat on both generators, because it can only choose among draws of the same prompt. The loop uses the RM's feedback to change the prompt and keeps improving. From k = 3 on (2.2 generations on average), it delivers more aligned clips than resampling at any budget, on Veo 3.1 and on Omni alike; pooled over both, it delivers 69.8% vs. 58.7% at k = 8 (p < 0.001). The search gains therefore carry over to delivered outputs, with fewer generations. App. A.20 plots delivered alignment for every budget from 1 to 8 in the style of Fig. 4. We have revised the abstract, the introduction and Sec. 4.5 so that the "three generations match eight" result is labelled as a candidate-set (oracle) result, with the delivered result stated next to it.

Q1. Fig. 4 reports different k = 1 rates although both strategies share the first candidate

Yes, the candidate is shared: at k = 1 both strategies show the same first generation, the identical file. The labels are not shared. Each strategy's eight candidates are graded together as one shuffled set, so that graders cannot infer anything from the order, and the shared video is therefore judged once inside each set. A few of these two judgments disagree, which moves the k = 1 rates by at most about 1% (54.2% vs. 54.5% pooled). That shift is well within the 95% intervals of Fig. 4 and an order of magnitude smaller than the gap at k = 8 (88.3% vs. 78.4%), so it does not affect any conclusion. Both strategies are graded in exactly the same way. With one label for the shared video, the loop still reaches 97.6% vs. 90.9% at k = 8 on Omni and 81.2% vs. at most 66.5% on Veo 3.1 (App. A.6); p ≤ 0.001 for both. App. A.6 now states this explicitly.

Dear Reviewer SQnY,

Thank you again for your positive and constructive review. As the discussion period draws to a close, we would like to briefly note that our responses and revised paper (changes in blue) address both of your main points. On an unseen generator, Wan 2.2 (306 clips, no filtering), CamRM-G reaches 89.9% bAcc, 8.8% above the strongest baseline. With CamRM-G picking the returned video for both strategies, the loop delivers more aligned videos than resampling at every budget from three generations on (61.2% vs. 49.4% on Veo 3.1 and 78.7% vs. 68.3% on Omni at k = 8), while using fewer generations. We also clarify the k = 1 rates in Fig. 4.

We would be glad to hear any further thoughts and are happy to address any remaining questions. Thank you once more for your valuable comments and your time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their careful reading and concrete suggestions. The reviews ask for five things, and we have revised the paper and added new results for each. All the major changes in the revised paper are highlighted in blue.

  1. Transfer to an unseen generator (SQnY, Z69G).

    • We collected 306 new clips from Wan 2.2, an open-weight generator family that none of our rating models saw in training, and labelled every clip by hand.
    • CamRM-G reaches 89.9% bAcc. That is 8.8% above the strongest baseline and 11.5% above the strongest zero-shot VLM (Gemini 3.6 Flash, 78.4%).
    • Sec. 4.7 and Table 5 give the full comparison against 15 raters (details in App. A.19).
    • In a leave-one-generator-out test (App. A.19), CamRM-G trained without any Veo clip reaches 73.7% on Veo. That is 5.8% above its untuned base and on par with the strongest zero-shot rater.
  2. Delivered, not oracle, loop gains (SQnY). We let the same rating model, CamRM-G, pick the returned clip for both strategies in our test-time scaling studies: Veo 3.1 (App. A.6) and Omni (Fig. 4).

    • On both Veo 3.1 and Omni Flash, from a budget of three generations on, the loop delivers more aligned clips than resampling does at any budget. At k = 8 it delivers 61.2% vs. 49.4% on Veo 3.1 (p = 0.005) and 78.7% vs. 68.3% on Omni (p = 0.012).
    • It also uses fewer generations than resampling (3.79 vs. 4.09 per item at k = 8); App. A.20 plots every budget from 1 to 8.
    • The abstract, introduction and Sec. 4.5 now state the oracle and the delivered results separately.
  3. Differences between human-evaluation results (QqiR, SQnY).

    • Why the numbers differ. Fig. 3(a) and Table 3 are separate experiments, each graded in its own round by its own group of graders. Fig. 3(a) measures the full system across four sets; Table 3 compares rating models on one set. The same 75 Veo I2V videos therefore received different absolute rates in the two experiments; the gap does not come from averaging versus majority voting. Sec. 4 now describes this grading design.
    • A common grader group. We also report both experiments with one common group of graders, who labelled every configuration in both (App. A.16). Under this common protocol the two experiments give the same value for CamRM-G (56.7%). The loop lifts alignment in every set: from 37.3% to 56.7% on Veo I2V, 46.3% to 64.2% on Veo T2V, 55.4% to 77.0% on Omni I2V and 66.3% to 84.2% on Omni T2V. CamRM-A (60.0%) and CamRM-G (56.7%) remain the two best rating models in the loop.
    • Fig. 4 at k = 1. Both strategies start from the identical first video. We grade each strategy's eight candidates together on one page in random order, so graders cannot tell which round a clip comes from. The shared video therefore appears once on each strategy's page and is judged on both. A few of these two judgments differ, which moves the k = 1 rates by at most about 1% (54.2% vs. 54.5% pooled). This is an order of magnitude smaller than the gap at k = 8 (88.3% vs. 78.4%), so it changes no comparison. Both strategies are graded in exactly the same way. With one label for the shared video, the loop still reaches 97.6% vs. 90.9% at k = 8 on Omni and 81.2% vs. at most 66.5% on Veo 3.1 (App. A.6); p ≤ 0.001 for both. App. A.6 now states how the shared video is graded.
    • Protocol and tests. App. A.16 lists the items, label aggregation, pairing and test of every human evaluation, with the paired counts of every test.
  4. A reproducible loop (Z69G).

    • We agree that reproducibility and openness are critical for the field. We will release the full training and evaluation dataset and the open rating model CamRM-Q. We will also release CamRM-A with its tool harness, which needs no tuning. The release of data, code and model weights is under internal review; in the new revision, we now give every prompt used in this work in full.
    • App. A.22 gives the complete prompts of every rating model, the rewriter prompt and the instructions shown to human graders.
    • App. A.17 gives what the rewriter receives in each round, the scene-change check, and the loop as pseudo-code (Algorithm 1).
  5. CamRM-A in the full loop, and a common protocol for baselines (Z69G, QqiR).

    • As the rating model in the Table 3 protocol, graded by the common grader group, the training-free CamRM-A reaches the highest alignment (60.0%) and the highest fix rate (48.9%) among all rating models, at 2.19 generations per item. Each rating takes about 2 tool calls, 13.6k tokens and 27 s (App. A.21). Together with CamRM-G, the most accurate rater on the benchmark (79.2% bAcc), this shows that our motion cues work both with and without training.
    • App. A.18 explains how every baseline reaches a verdict. It reports per-model coverage and shows that CamRM keeps its advantage under one failure rule and on identical subsets.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.