acceptodds
Withdrawn from ICLR 2027

SCOPE: Auditable Score-Isolated Agentic Adaptation for Video World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video world models are increasingly used as simulators for planning and em- bodied reasoning, but inference-time adaptation creates an evaluation challenge: jointly evolving controls obscure causal attribution and can entangle adaptation with held-out feedback. We introduce SCOPE (Score-Isolated Control Orchestra- tion with Provenance-Bound Evidence), which makes the inference harness an explicit typed state over text, sampling, verification, and selection. Development- supported edits update one axis at a time; the final policy and exact Base fallback are frozen before held-out scoring, with every registered transition bound to its evi- dence. On Physics-IQ, SCOPE achieves 34.94, improving on single-sample Base by +14.24 (95% CI [+8.10,+21.23]), including candidate expansion. A secondary comparison over the same candidate pool yields +7.53 over uniform selection (95% CI [+4.04,+11.52]). Matched ablations identify useful text, sampling, and learned-selection controls. Comparisons with the strongest agentic baseline and equal-budget random updates remain statistically unresolved, while prospective tests identify calibration under task and backbone shift as the next deployment chal- lenge. Together, these results establish an auditable basis for developing inference controls and separating candidate quality from deployment choice.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agentic Adaptation, Video World Models

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.40 over 5 reviews, above 9% of the others.
Reviewer_8TsxOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCOPE, a framework for tuning the inference pipeline around a frozen video model (Wan2.2-5B and CogVideoX). The pipeline is split into text, sampling, verification and selection, an agent edits one of them at a time, and an edit is kept only if it helps on development data and passes some guard checks. Everything is frozen before testing and logged with a hash chain. In practice the system rewrites prompts with GPT-5.4 (physics cards), changes CFG/steps a bit, and picks one of ~3 candidates with a small ridge regressor. On Physics-IQ (Wan2.2, 40 scenes) it goes from 20.70 to 34.94.

Critical strengths

The motivation makes sense, test-time pipelines usually change many things at once and it's hard to tell what actually helped. The authors are also quite honest, they report a lot of null and negative results, and they separate the gain from having more candidates vs. better selection in Table 1.

Critical weaknesses
  1. Most of the improvement comes from the prompt. Just adding the GPT-5.4 physics card takes Wan from 20.70 to 30.04, and after that the CIs in Fig. 2 mostly overlap.
  2. SCOPE doesn't significantly beat the strongest baseline (Qwen-style agent on the same candidate pool).
  3. More importantly, I think what the paper performed is mostly prompt rewriting, a CFG/steps tweak, and best-of-3 with a learned scorer. These are all standard test-time tricks.
Questions for authors

N/A

Reviewer_FwRGOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SCOPE, a framework for adapting and auditing inference-time controls around frozen video generators. It represents text conditioning, sampling, verification, and selection as a persistent typed state, permits development-supported single-axis updates, freezes the resulting policy before held-out scoring, and records accepted and rejected attempts through evidence-bound transition records.

The evaluation includes Physics-IQ, P-AI, and OpenS2V-Eval, together with additional prospective and applicability tests. On the 40-scene Physics-IQ protocol, SCOPE improves over single-output Base and uniform selection from a shared candidate pool. Component experiments support useful text, sampling, and selection controls. However, advantages over the strongest matched agentic comparator and random updates remain statistically unresolved, and prospective evaluations reveal limitations in transfer, routing calibration, and proposal coverage.

Critical strengths

1.Useful separation of candidate construction and selection. The paper distinguishes the full-system improvement over single-output Base from selection among identical candidates. On Wan, the +7.53 Physics-IQ improvement over uniform selection from the shared pool, with a positive confidence interval, provides substantive evidence that the selection procedure improves the returned output in this setting. The scene-level paired analysis also avoids treating nested views as independent samples.

2.Transparent reporting of evidence boundaries and negative results. The paper explicitly states the assumptions behind score isolation and auditability, records rejected attempts, and reports unresolved comparisons, confirmation reversals, and pre-score stopping outcomes. This transparency makes the scope and limitations of the empirical findings easier to assess.

Critical weaknesses

1.The incremental advantage over strong selectors and complete harnesses is insufficiently established. Uniform selection is a useful diagnostic reference, but outperforming it does not establish competitiveness with a well-configured alternative selector. In the dedicated OOF experiment, learned selection improves over uniform Bo16 by +3.33 points, but its advantage over self-score is only +0.66 with a 95% CI of [−1.52, +2.92]. In the main experiment, SCOPE’s advantages over the strongest Qwen-style comparator are +2.07 [−0.27, +6.14] on Wan and +0.52 [−0.39, +1.65] on CogVideoX. These unresolved comparisons do not establish equivalence, but they leave the additional benefit uncertain.The baseline scope is also important: WMReward and VIGOR are evaluated as reward-only variants, while the Qwen and LingBot comparisons use style-based or proxy implementations. These comparisons do not establish superiority over the corresponding complete optimization systems. Closely related workflow and skill-evolution approaches, including COMFYCLAW and GenEvolve, are discussed without empirical comparisons, and the paper acknowledges that a matched NEWTON reproduction is missing. Their differing tasks and training assumptions require care, but the paper still needs either stronger applicable end-to-end baselines or more convincing matched tests of the relevant mechanisms. As presented, the results establish useful individual controls more clearly than a distinctive advantage of the complete SCOPE framework.

2.The quality–cost trade-off is not adequately evaluated. Candidate expansion increases generation work, and the full harness additionally uses controller, verification, selection, and potentially tool operations. The reported A100 profile measures only the generator: approximately 44.45 seconds per video, or 133.34 seconds for three serial generations under the profiled setting. It explicitly excludes VLM/tool execution, selection, and encoding, so it is not an end-to-end SCOPE latency measurement.The paper does not provide a sufficiently complete comparison of end-to-end latency, GPU time, input/output tokens, model and tool calls, or estimated monetary cost across the competing methods. Sharing a candidate pool controls generation expenditure for selector comparisons, but does not necessarily equalize controller and scoring overhead. Consequently, it remains unclear whether SCOPE offers better output quality at a fixed test-time budget, comparable quality at lower cost, or simply a different allocation of additional computation. This is consequential because optimizing the surrounding inference workflow is the central practical contribution.

Questions for authors

1.Baseline implementation and fairness — clarification. For WMReward and VIGOR, which official implementations, weights, preprocessing procedures, and selection rules were used, and which parts of the original optimization methods were omitted? What precisely distinguishes SCOPE from the joint-self-score and Qwen-style comparators on the shared pool? Were competing selectors given comparable development data and calibration opportunities? Explaining why both reward-only baselines score below uniform selection would help assess the comparison.

2.End-to-end resource use — measurement request. Can you provide a per-method breakdown of generated candidates, generation GPU time, end-to-end wall-clock latency, controller/evaluator input and output tokens, model/tool calls, and estimated deployment cost? Please specify hardware, parallelism, caching, and model assumptions, and separate one-time development/training expenditure from per-instance deployment cost. Which of these quantities can be recovered from the existing execution logs?

3.Incremental benefit at matched budgets — analysis of existing results. Can you report direct paired confidence intervals against the fixed-card, fixed-rule, and joint-self-score controls, in addition to the existing Qwen-style comparison? What evidence supports a quality advantage under comparable total test-time resources, rather than only comparable candidate counts?

4.Complete-harness comparison — clarification of contribution. What existing evidence distinguishes SCOPE’s complete workflow from an appropriately matched tool-planning or workflow-evolution baseline, particularly NEWTON? For related systems such as COMFYCLAW and GenEvolve, which mechanisms are applicable to this setting and which are not? If no matched comparison is available, please clarify how narrowly the claimed empirical advantage should be interpreted.

Reviewer_FUszOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

SCOPE organizes the process of improving video generation through inference-time controls while keeping the generator weights fixed. It stores prompt instructions, sampling settings, verification rules, and output selection in a configuration file. During development, proposed changes are made to one component at a time. The system retains a change when it improves the development objective and satisfies predefined checks. It then records the evidence supporting each decision. It saves the resulting procedure before held-out evaluation and retains the original generator output as a fallback.

The authors evaluate SCOPE on two video generators using Physics-IQ and additional task benchmarks. They compare SCOPE with methods that use fixed prompts, choose among generated videos, or use an agent to guide generation. Component studies examine which changes improve performance. Additional evaluations test how the selected procedures transfer to new tasks.

Critical strengths
  1. The breadth of the baseline comparisons is a clear strength. Table 1 includes the original generator, fixed prompt changes, alternative reward-based selectors, and agentic procedures. The comparison with uniform selection from the same three-candidate pool is particularly useful because it holds the generated videos fixed. On Wan, SCOPE improves by 7.53 Physics-IQ points over that reference, which is significant. The component studies also provide strong support for using scene descriptions and sampling settings to improve generation.

  2. The paper makes the separation between development and held-out evaluation explicit and records the evidence supporting registered updates. It reports uncertainty and acknowledges unresolved comparisons and failures under transfer. These help the reader assess the scope of the results, including cases where a proposed change does not improve performance.

Critical weaknesses
  1. The experiments provide stronger evidence for the prompts and selection methods than for the proposed update framework. The headline comparison starts from a single video generated with the original settings. SCOPE changes the conditioning and selects among multiple candidates. This comparison does not show how much of the improvement is attributable to the update framework. Table 1 helps separate some of these effects but leaves the framework's added value unclear.

I see the comparison with random updates as the most direct test of the proposed update procedure. Both methods receive the same budget. SCOPE scores 0.316 points lower, but the confidence interval is too wide to conclude that either method performs better. The results therefore do not really show which procedure performs better. Table 1 compares SCOPE with a stronger agentic alternative and shows that SCOPE exceeds the Qwen-style baseline by 2.07 points on Wan and 0.52 points on CogVideoX. Both confidence intervals include zero, so these comparisons also leave its advantage uncertain.

Figure 2 helps explain where the gains come from much more clearly. Much of the improvement occurs when physics-oriented instructions are added to the prompt. What does the proposed update procedure add to ordinary prompt tuning and output selection?

Recording updates and keeping test scores out of development may provide a separate benefit. But the paper should explain how it evaluates that benefit, because improved video quality by itself does not demonstrate it.

The baseline coverage is useful, but several rows are reward-only adaptations or proxies for named systems. The authors should explain which capabilities they preserve and omit. Sharing candidate videos controls generation cost, while model calls and development effort can still differ. Those costs are needed to assess the fairness of the comparisons.

  1. The presentation makes the actual procedure quite difficult to reconstruct. After carefully reading the abstract and introduction, I could not clearly identify the research question or technical contribution. Section 3.1 introduces abstract terminology before explaining precisely what the system does. Figure 1 repeats this terminology. A worked example would really help show what the configuration contains and how one change affects generation and selection. Table 1 is also difficult to read at normal print size, and the labels in Figure 2 crowd the plotted markers and error bars.

The implementation description leaves the division between automation and human involvement entirely unclear. Who proposes or revises the prompt? Who inspects the outputs? Who decides whether to keep an update? These details affect practicality. The generator-only profile reports ~45 seconds per video and ~133 seconds for three sequential videos; this does not even include other steps such as model/tool execution, selection, and encoding outside the generator. The paper needs the total elapsed time and compute for a complete update, including rejected attempts and any human effort. Development costs should be distinguished from the cost of producing an output after the procedure is frozen.

Questions for authors
  1. What evidence shows that the update framework improves on ordinary prompt tuning and output selection? What benefit does tracking the updates provide, and how do you measure it?

  2. Can you walk through one complete update, from proposing a change to accepting or rejecting it? Which steps are automated, and where does a person participate?

  3. How much time and compute does an update require? Does this include video generation and any human involvement? What is the total development cost? Including rejected changes, what does it cost to produce an output for a new scene?

  4. How do the adapted baselines differ from the original methods? Were they given comparable development and inference budgets?

Reviewer_1giBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces SCOPE, a framework for the inference-time adaptation of video world models, focus on ensuring clean and auditable evaluation. The author says there is a problem with current practices: an improvement in the final score could stem from better descriptions, a larger pool of candidates, or more effective selection, held-out scores are often used in the process of selecting the method itself.

Critical strengths

1.The problem seems to be relevant and meaningful. 2.The experimental design is all right, and negative results are reported honestly.

Critical weaknesses
  1. The method shows no significant advantage over the strongest baseline. its average performance is driven almost entirely by a single scenario.
  2. It fails to generalize across different backbones or benchmarks, and it performs poorly, essentially failing, in prospective (forward-looking) tests.
  3. The qualitative visualizations in the appendix are generated via post-processing, but they are labeled as "OURS," which is potentially misleading.
  4. Novelty is somewhat limited. The SCOPE approach encompasses four-axis control states, one-axis updates, commit gates for development sets, freezing policies, a base fallback mechanism etc. However, which one of these constitutes an irreplaceable research innovation?
  5. The paper is a little bit difficult to follow. The main text is not sufficiently self-contained, some key information has been deferred to the appendix.
Questions for authors
  1. Is there significant gains over strong baselines?
  2. To my reading, core premise remain unverified. "Typed" updates perform no better than random updates. This is critical, as "typed, constrained updates" constitute the very heart of the proposed method.
  3. Is the method able to generalization? The method shows virtually no effect on P-AI, in fact, applying it to CogVideoX with a "shared global" component results in a performance drop of 2.41. On OpenS2V, it performs no better than the proxy baseline.
  4. Prospective deployment largely unsuccessful. Check it. The method triggered safety guards on PAI physics32, was aborted on PhyGround due to insufficient coverage, and showed performance reversal on the CausalVerse validation set.
  5. Can you increase the scale? The primary results are based on only 40 scenarios, leading to very wide confidence intervals.

I am open for discussion and if the authors can address these concerns satisfactorily, I am willing to raise my rating.

Reviewer_6r5oOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes SCOPE, a framework for adapting prompting, sampling, verification, and candidate selection around frozen video world models using development data, with the resulting procedure fixed before held-out evaluation. It records the evidence supporting each update and uses controlled comparisons to separate improvements in candidate generation from improvements in selection.

Critical strengths
  1. The paper provides controlled evidence that demonstrates selection helps. Selecting from the same candidate pool improves Physics-IQ performance over uniform selection, with a positive 95% confidence interval/
  2. The experimental reporting is quite transparent. The paper separates candidate-generation and selection effects, reports paired uncertainty, and acknowledges unsuccessful interventions and confirmation failures.
Critical weaknesses
  1. The writing of the paper significantly obscures the executable method, and the introduction/abstract is very difficult to parse. Strong assumptions are made with the reader's familiarity of paper-specific jargon, in a way where the opening first couple of pages only start to make sense after completing the full paper (and I am still not completely sure if I was able to understand the methodology yet). Section 3 formalizes generic operations such as "proposing", "applying", and "accepting changes" before explaining concretely how proposals are produced. Terms such as SceneLang, Director cards, shared global, and "retirement routing" appear without clear definitions. Even Table 7 mostly maps component names to roles. This makes it difficult to identify precisely what was implemented and what is new, and simply makes the paper difficult to follow. This is largely a writing/expression artefact.

  2. The novelty remains insufficiently established. Reward-guided inference and candidate selection already appear in Yuan et al., Inference-time Physics Alignment (WMReward, 2026); physics-aware tool orchestration and iterative verification appear in Feng et al., NEWTON (2026). Both are cited by SCOPE. Its distinctive contribution therefore appears to be the organization and auditing of updates, whose added value needs clearer justification.

Questions for authors

What measurable benefit comes specifically from the typed update mechanism? Please keep proposals, candidates, development data, and compute fixed, then compare against a simple development-tuned prompt-and-selector pipeline. If the contribution is principally auditability, demonstrate an error the mechanism detects that ordinary experiment tracking misses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.