acceptodds
Withdrawn from ICLR 2027

REEL: Learning to Retrieve, Evaluate, and Evolve Visual Memory for Any Long-Horizon Multi-Shot Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Memory-based long-video generation aims to combine user control with long-range visual consistency. We organize this process into three functional components: a planner, an adapter, and a generator. Representative multi-shot systems couple memory to generator-specific interfaces or use predefined agent workflows, rather than independently training a memory policy for cross-shot objectives. We introduce REEL, a general-purpose visual-memory planner. Through a unified tool interface, REEL learns to retrieve references, create missing visual evidence, and update persistent memory. Backend-specific adapters translate its evidence plans into supported conditioning inputs, allowing the same policy to guide different black-box video generators. To train REEL, we construct two high-quality datasets, REEL-SFT-4k and REEL-RL-1k, covering diverse interaction patterns and prompt types. Training proceeds in three stages: supervised fine-tuning, single-shot Group Relative Policy Optimization (GRPO), and multi-shot GRPO. Multi-shot GRPO carries policy-generated memory across consecutive shots, so later outcomes guide earlier reference selection and memory updates. REEL-Bench tests identity preservation and intended state changes under text-only requests and timed reference-image uploads. On JoyAI-Echo 1.5 and MiniMax-H3, REEL improves cross-shot consistency on ST-Bench by 15.1% and 20.7%, and Core-100 on our proposed REEL-Bench by 2.41 and 8.32 points, respectively, over the no-memory baseline.

Primary area:
generative models
Keywords:
Video Generation, Memory management

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aWtpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents REEL, a trainable visual-memory planner for long-horizon multi-shot video generation. A Qwen3.5-9B planner uses a unified tool interface to retrieve references, create missing images, review candidates, and update memory, while per-backend adapters map its evidence plans onto frozen black-box generators, and training combines SFT with single-shot and multi-shot GRPO that carries policy-generated memory across consecutive shots. The paper contributes REEL-SFT-4k, REEL-RL-1k, and REEL-Bench (300 stories, 2,700 shots) and reports Core-100 gains on REEL-Bench, cross-shot consistency gains over the no-memory baseline on ST-Bench, and transfer to four unseen backends through adapters.

Critical strengths

A generator-agnostic planning interface demonstrated on unseen backends The paper targets the consistency bottleneck of long-horizon multi-shot generation and decouples memory planning from synthesis through a unified tool interface with per-backend adapters. The same frozen planner drives four unseen backends with no target-backend fine-tuning. This matters because generator-agnostic reuse of one memory policy is the paper's most transferable property and the clearest structural support for its generality claim.

Execution reliability at scale The full REEL checkpoint reaches 100% completion on REEL-Bench (all 2,700 requested shots, a mean of 9.0 successful shots per story) and completion 1.00 on the matched Seedance 2.0 and Kling 3.0 subsets, where the SFT planner completes only 0.54 and 0.59 of requests. Completion is measured by execution status rather than quality scores. This matters because stalled trajectories are a typical failure mode of tool-using generation pipelines, and this execution-level reliability is the paper's most robust result that does not rest on model-judged quality.

Critical weaknesses

Overstated primacy and unclear contribution boundary The concurrent work MVAgent (arXiv:2609.30609, September 2026) trains an agent for multi-shot condition construction that maintains a continuity memory, freezes the generator and judges, and optimizes per-shot rewards, a setting that closely matches REEL's. Earlier systems already cover multi-shot memory and reference selection (StoryMem, GroundShot, CoTriSyGen), and reinforcement learning for memory management is established in the LLM agent literature (Memory-R1, Mem-T). What remains relatively new in REEL is the unified tool interface with backend adapters and the multi-shot GRPO that carries policy-generated memory across shots, which amount to a combination and engineering of existing ideas. The paper presents its learned policy and backend-independent interface as the difference from prior systems, and the learned-policy part overlaps with MVAgent, which the paper does not discuss. The claim to be the first general-purpose trainable visual-memory planner therefore cannot be sustained, and the boundary of the contribution remains unclear.

Evaluation circularity and marginal external gains without significance testing The central claim is that the learned memory policy stably improves generation quality, and the evaluation cannot separate the reported improvements in quality, state compliance, and narrative coherence from in-domain optimization toward the same judge and backends used in training. REEL-Bench is judged by Gemini 3.5 Flash, the same model that scores the single-shot and multi-shot GRPO rewards, and the RL rollouts use JoyAI-Echo 1.5 and MiniMax-H3, the two backends of the main evaluation. On the external ST-Bench, the cross-shot consistency margin over the strongest baseline DirectorAgent is only 0.006 and 0.002, and global prompt following is lower. Neither benchmark reports confidence intervals or significance tests for the main comparisons, and the only analysis with Core-100 intervals excludes DirectorAgent. The abstract reports large relative gains of 15.1% and 20.7% on ST-Bench and 2.41 and 8.32 points on REEL-Bench over the weakest no-memory baseline, and the gaps against the strongest baseline are far smaller. The main conclusion of a stable quality improvement is therefore not supported.

Gains over baselines cannot be attributed to the proposed memory policy The reported gains are attributed to the learned memory planner. Every comparison against heuristic baselines bundles this planner with prompt rewriting, reference image creation, candidate review and revision, and extra generation and planning calls, so the gains over these baselines cannot be assigned to the memory policy alone. Those baselines only fill condition slots by fixed rules, and REEL additionally rewrites the full prompt, creates missing reference images, and reviews and revises candidates. The paper's own ablations show that disabling image creation and candidate review lowers the story score by 2.84 and 3.13 points. On the authors' own REEL-Bench, REEL's cross-shot consistency is lower than simple CLIP-based retrieval and StoryMem-style memory selection, and its Core-100 advantage comes mainly from motion, narrative, and prompt following. A memory-centric method that does not lead on the metric most directly tied to memory leaves the value of the learned policy unconfirmed and the share of the gain from stronger prompts and more calls undetermined, so the methodological contribution remains hard to assess.

Questions for authors

What is the minimal essential difference between REEL and the closest systems, MVAgent and GroundShot, and how much net gain remains if reinforcement learning is removed and only the tool interface and prompt rewriting are kept? Do the main comparisons pass paired significance tests, are the 0.006 and 0.002 consistency gaps on ST-Bench significant, and are all methods scored on a common set of fully completed stories? Has the shared judge for training rewards and evaluation been validated independently, and do the main results hold under an independent judge or human evaluation? What are the measured inference latency, call cost, and storage overhead relative to the general-purpose claim? Why is REEL's cross-shot consistency below CLIP and StoryMem-style baselines on REEL-Bench, and under the fixed Core-100 weights is that gap offset by the motion and narrative dimensions? Please withdraw or strictly qualify the claims of being first and of being general-purpose, and clarify the relationship to MVAgent and the reinforcement learning memory management line. Validate the main results with an independent judge or human evaluation and report agreement rates. Add paired significance tests and equal-capability controls that separate the memory policy, prompt rewriting, and extra calls. Report inference cost, failure statistics, and a limitations section, and explain the trade-off between cross-shot consistency and diversity or motion.

Reviewer_HELpOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces REEL, a tool-using visual-memory planner built on Qwen3.5-9B for long-horizon multi-shot video generation. For each shot request, the planner uses a nine-tool, backend-independent interface to inspect a subject registry and memory bank. It can also synthesize missing reference images through external image generators, submit a semantic evidence plan, review generated video candidates with up to one revision, and write durable evidence back to memory. Backend-specific adapters translate these plans into conditioning inputs for black-box video generators. To train and evaluate the policy, the authors build REEL-SFT-4k (33k shot-level teacher trajectories), REEL-RL-1k, and REEL-Bench (300 stories and 2,700 shots with staged uploads and state changes, judged by Gemini 3.5 Flash). Training runs through SFT, single-shot GRPO, and multi-shot GRPO over 10-shot episodes that carry forward the policy's own memory updates. Experiments on JoyAI-Echo 1.5 and MiniMax-H3, along with zero-shot transfer tests on four other backends, compare REEL against heuristic memory rules and DirectorAgent.

Critical strengths
  1. Practical, backend-agnostic planner-adapter architecture: Decoupling the semantic memory policy from generator-specific conditioning slots via lightweight adapters lets a single 9B model drive multiple commercial black-box video generators without modifying their internal architectures. On both main backends, this 9B planner gets higher aggregate Core-100 scores than DirectorAgent (which relies on Claude Opus 4.5) and transfers zero-shot across four additional backends.

  2. Thorough protocol engineering and multi-shot RL budget control: The execution environment enforces strict temporal access controls for mid-story uploads, paired first-attempt random seeds, and verifiable manifest hashes. The training-budget control in Tab. 12 compares 1,000 single-shot updates against 400 single-shot + 600 multi-shot updates on a matched 213-story intersection. Carrying memory across multi-shot rollouts raises task completion and improves shot-level quality with fewer video calls.

Critical weaknesses
  1. Severe numerical anomalies in Table 1 (exact constant offsets between JoyAI-Echo 1.5 and MiniMax-H3): In Tab. 1, multiple MiniMax-H3 rows are exact constant shifts of the corresponding JoyAI-Echo 1.5 rows across all eight independently judged columns: REEL is +2.72 on every single metric (55.48-52.76, 52.91-50.19, 43.82-41.10, 77.53-74.81, 66.33-63.61, 51.89-49.17, 41.89-39.17, 47.72-45.00); No Generated Memory is -3.19 across all columns (within 0.01 rounding); DirectorAgent is -0.32 across all columns; and StoryMem-style Memory is -0.28 across all columns (within 0.01 rounding). It is statistically and physically implausible for two distinct video generators to differ by an identical constant offset across all eight shot- and story-level dimensions.

  2. Unisolated memory mechanism and lower cross-shot consistency on REEL-Bench: On REEL-Bench (Tab. 1), REEL's Cross-Shot Consistency (Cons.) is lower than FIFO, CLIP-based Retrieval, StoryMem-style Memory, and DirectorAgent on JoyAI-Echo 1.5 (49.17 vs. 50.00-55.00), and lower than every memory baseline on MiniMax-H3 (51.89 vs. 52.54-54.72). REEL wins on Core-100 through Prompt Following, Visual Quality, Motion, and Narrative. However, these gains are confounded by capabilities granted to REEL and denied to the heuristic baselines: REEL can author complete replacement prompts, call external image generators (Qwen-Image 2.1, JoyAI-Image-Edit) to create missing references, and review or revise video candidates.

  3. Circular reward-evaluator setup, survival bias in quality metrics, and missing cost/significance data: Gemini 3.5 Flash is used at the same time as the data filter, the GRPO reward model, and the sole judge for REEL-Bench under nearly identical criteria. This setup risks reward hacking and judge overfitting, and the authors only check it with an informal visual inspection. Quality metrics exclude incomplete stories (Sec. 4.1), which introduces survival bias when completion rates differ across variants. Scored-story counts are also omitted from the main tables (Tabs. 1, 2, and 3), particularly for the MiniMax-H3 runs, ST-Bench, and the ablation variants in Tab. 3 where completion rates clearly vary.

Questions for authors
  1. Why are the MiniMax-H3 rows in Tab. 1 exact constant offsets (+2.72, -3.19, -0.32, -0.28) from the corresponding JoyAI-Echo 1.5 rows across all eight metrics?
  2. Why does REEL score lower on Cross-Shot Consistency than the heuristic memory baselines on REEL-Bench (Tab. 1)? Do the heuristic baselines receive a second video candidate or any prompt enhancement beyond the adapter (App. E.2)? How does REEL perform when prompt authoring, external image creation, and candidate revision are disabled so that only memory retrieval and writeback are compared against CLIP and StoryMem-style heuristics?
  3. Can you evaluate the existing REEL-Bench outputs using an independent VLM judge from a different model family (or a formal blinded human study) to verify that the ranking over DirectorAgent and CLIP retrieval holds?
  4. How many stories were completed and scored for each row in Tabs. 1, 2, and 3?
Reviewer_CiEEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents REEL, an agent framework for multi-shot video generation. The idea is to have a planner that decides which reference images to retrieve, generate, or save to memory across shots, and talks to the video model through a backend-independent tool interface. The planner (Qwen3.5-9B) is trained with SFT and then GRPO, including a multi-shot version where memory carries over between shots. Thanks to backend-specific adapters, the same planner can drive different video generators. They also build a new benchmark, REEL-Bench, and show gains over simple memory baselines and a prompted agent.

Critical strengths
  1. The planner/adapter split is a good design. The same planner can plug into different video models, even ones it never saw during training, and still works.
  2. Training with memory carried across shots makes sense, since many memory decisions only pay off later. The training-budget experiment backs this up well.
  3. The experiments are carefully set up, and the paper is upfront about the details.
  4. The new benchmark covers cases other benchmarks don’t really test, like uploads in the middle of a story and characters changing state.
Critical weaknesses
  1. The improvements are not convincing. REEL beats DirectorAgent by only about 1 point on Core-100 with JoyAI-Echo 1.5 (Table 1), with no confidence interval or significance test to show this isn’t noise. For a memory paper, it also doesn’t win on cross-shot consistency on its own benchmark, where CLIP retrieval and StoryMem-style both score higher (Table 1), so most gains seem to come from better prompts rather than better memory. The qualitative results don’t help much either: in Figure 5 it’s hard to see a clear advantage over the baselines, and the main comparison methods (No Generated Memory and DirectorAgent) aren’t shown.
  2. It’s not clear what the RL is really learning. The base model completes only 0.8 shots per story (Table 3a), so a lot of the improvement might just be learning to use the tools correctly, and there’s no analysis of whether memory decisions actually get better. On top of that, quality is only scored on completed stories while completion rates vary a lot between variants, which makes the stage-by-stage comparison hard to interpret.
Questions for authors
  1. What happens if a strong prompted model (e.g., the Claude planner from DirectorAgent) runs on exactly the same tool interface as REEL? That would show how much of the gain comes from training versus the interface itself.
  2. Is the VLM used for the RL reward the same as the benchmark judge? If so, is there any evaluation with an independent judge?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.