acceptodds
Under review as a conference paper at ICLR 2027

StoryRelay: Reference-Addressable Autoregressive Visual Storytelling

Abstract

Reference-conditioned visual storytelling must maintain subject identity and visual continuity across a sequence of shots while adapting to changing actions, viewpoints, and compositions. Generated history can improve cross-shot continuity, but it may also introduce irrelevant or incorrect visual state into later generations. We introduce **StoryRelay**, a reference-addressable autoregressive storytelling framework that jointly leverages persistent references and evolving visual history across shots. StoryRelay maintains story-level reference bindings and allows shot descriptions to address subject references and optional scene references through textual identifiers. For autoregressive story learning, *Autoregressive Story Modeling* (ASM) post-trains a unified multimodal model on interleaved reference-text-image stories, predicting each target keyframe from reference images and preceding ground-truth history. To strengthen identity resolution across multiple visual sources, *Shot-Aware Identity Grounding* (SIG) associates requested subjects with identity-specific evidence from references and historical appearances. At inference, a *feedback-controlled autoregressive rollout* performs history selection, generation, verification, and bounded shot-local correction within the same model. On ViStoryBench, StoryRelay improves cross-reference identity similarity from 0.45 to 0.52 and character-count matching from 65.57 to 68.46 over the original BAGEL.

Primary area:
generative models
Keywords:
Visual storytelling, Image generation, Multimodal models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AHWBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes StoryRelay, a reference-conditioned autoregressive visual storytelling framework that combines persistent references with evolving visual history. The method introduces Autoregressive Story Modeling (ASM) for next-keyframe prediction, Shot-Aware Identity Grounding (SIG) for identity preservation, and a feedback-controlled rollout for history selection and generation correction. Experiments demonstrate improvements in identity consistency, character-count matching, and prompt alignment over BAGEL.

Critical strengths
  • The paper writing is clear, and the motivation of combining persistent references with autoregressive visual history is reasonable.
  • The proposed identity grounding mechanism is clean, providing spatial supervision across references and historical frames without requiring masks at inference.
  • The experimental results are promising, showing improvements in identity preservation and prompt alignment, with ablations and qualitative examples.
Critical weaknesses
  • Fig 5 is overly complicated, making the figure hard to read and understand. Refining the figure to be more simple and highlight the core designs would enhance the paper.
  • The inference efficiency analysis is missing. The rollout allows up to four generation attempts per shot with additional VLM calls, but the paper lacks comparisons of inference latency and computational overhead.
  • The evaluation could be strengthened with more controlled comparisons and analysis of performance over longer sequences, given that the training stories contain 3.67 shots on average.
  • The related work section could be refined to further position the paper with respect to recent reference-conditioned generation methods, such as Scaling Zero-Shot Reference-to-Video Generation (CVPR 2026).
Questions for authors

Please see weakness above.

Reviewer_jJvpOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes StoryRelay, a reference-conditioned visual storytelling framework built on BAGEL that generates an image sequence following individual shot descriptions while maintaining subject identity. It combines three components, autoregressive story modeling (ASM), shot-aware identity grounding (SIG), and feedback-controlled generation that selectively uses the generation history. The model is trained on a video-derived corpus and compared against BAGEL and image editing methods.

Critical strengths

The paper addresses a relevant challenge in visual storytelling, namely maintaining subject identity and visual continuity across an image sequence while following individual shot descriptions. Its design of selectively using the generation history is a reasonable direction, since errors in earlier images can otherwise propagate to subsequent shots. This is the main consideration in favor of the paper, as identity consistency across shots is a central requirement for practical storytelling applications.

Critical weaknesses
  1. The experiments do not isolate the sources of improvement. BAGEL appears to be used without training on the proposed corpus, and the ablation only adds components cumulatively (BAGEL, +ASM, +ASM+SIG, full model). Since L_ASM is BAGEL's own latent flow-matching objective, the gain from +ASM may largely reflect fine-tuning on the new corpus rather than a new learning objective. There is no SIG-only configuration and no controlled comparison of history selection and feedback, and the additional inference cost of up to four generations is not reported. The paper also motivates history selection by error propagation but does not show that error accumulation is reduced. This is consequential because the paper's claims rest on the necessity of each component.

  2. The methodological distinctiveness is unclear. The comparisons focus largely on image editing methods, but the paper does not explain how image editing relates to visual storytelling, what differs between the two tasks, and why this comparison is appropriate. Without this, it is difficult to judge how novel the proposed losses are, particularly since L_ASM is adopted from BAGEL, and the incomplete formulation in Equation (1) and the difficulty of reconstructing the full pipeline further hinder this assessment. Given that the method is a modest variation of existing components, an analysis of failure cases could have provided insight into where and why it helps, but none is presented. As a result, while the paper shows a useful application of a unified multimodal model, its methodological contribution beyond BAGEL and prior work remains unclear.

Questions for authors
  1. Was BAGEL fine-tuned on the proposed corpus in any comparison? A comparison with BAGEL fine-tuned on the same corpus without the proposed components would clarify how much of the gain comes from the data.
  2. How does L_ASM differ from BAGEL's latent flow-matching objective, and what part of ASM is new?
  3. Why are image editing methods the appropriate comparison for visual storytelling, and what differences between the two tasks motivate the proposed losses?
  4. Equation (1) appears incomplete. Could the authors provide the complete formulation and a step-by-step description of the training and inference pipeline?
  5. What are the sources of the video-derived corpus, how accurate is the identity matching, and how were the annotations validated?
  6. What is the inference cost relative to BAGEL given up to four generations, and could the authors show failure cases, including cases where feedback fails to correct earlier errors?
Reviewer_HFHmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

StoryRelay builds on BAGEL-7B-MoT to improve cross-shot continuity in reference-guided autoregressive visual storytelling while reducing identity confusion and error accumulation. It combines an automated dataset of about 15K stories, autoregressive training with shot-aware identity grounding, and feedback-controlled inference that selects relevant history frames and validates outputs before adding them to memory. Only generation-branch LoRA parameters are trained. On ViStoryBench, it improves identity similarity, character-count matching, and prompt alignment over pretrained BAGEL.

Critical strengths
  1. The manuscript is well-motivated by a key challenge: visual history supports continuity but can also propagate errors. Figures 2–3 clearly illustrate identity swaps and character duplication from naively reusing previous frames. Combining spatial alignment during training with controlled history selection and admission at inference offers a coherent and practical solution.

  2. The proposed SIG is well suited to multi-shot storytelling. It gives each source equal weight, preventing close-up history frames from having too much influence. It also teaches the model to suppress characters that should not appear in the current shot.

  3. The data pipeline and implementation are clearly explained. Edited reference images provide clean model inputs, while original video crops support more reliable mask matching. The appendix also gives detailed instructions on RoPE positions, CFG dropout, and loss calculation across GPUs, making the method easier to reproduce.

Critical weaknesses
  1. Most gains come from extra generation attempts. ASM and SIG provide small improvements, while the inference loop accounts for over 70% of the gains in several metrics. Since it allows up to four attempts plus VLM checks, comparisons should give BAGEL the same budget. Average attempts, failure rates, and runtime should also be reported.

  2. Training and testing use different inputs. Training includes all character references with fixed IDs, while inference selects references and changes IDs for each shot. ViStoryBench has no scene reference images, so this part of the method remains untested. The large Scene PA gain also needs analysis to separate history filtering from repeated sampling.

  3. The examples do not clearly show correction through retries. The highlighted shots in Figs. 2–3 succeed on the first attempt, so history selection and different random seeds may explain the improvements. In the documented retry, the system removes all history and still misses the requested action. This provides limited evidence that it preserves visual changes across shots.

  4. The overall design is overly complex. It combines a six-stage data pipeline, a custom training loss, history filtering, VLM selection and checking, retries, and repair. This makes the system harder to reproduce and use, while the lack of separate ablations leaves it unclear which components are necessary.

Questions for authors
  1. How much of the gain comes from history filtering, and how much comes from extra attempts? Please test ASM+SIG with filtering alone and give BAGEL the same retry budget. Also report average attempts, success and fallback rates, and runtime.

  2. Why does Scene PA rise from 42.38 to 63.99? Is removing unsuitable history enough, or are repeated attempts needed? Please also test scene SIG with actual scene reference images, which ViStoryBench does not provide.

  3. Why use all references and fixed character IDs during training, but selected references and changing IDs during inference? Would keeping references and IDs fixed improve quality or speed by allowing KV-cache reuse?

  4. Why is the method tested only on BAGEL? Is it also suitable for other models? Results on another model would help show whether the gains extend beyond BAGEL.

Comments on the paper

1 comment
Area_Chair_4pPxOpenReview ↗

Discussion

Dear Reviewers, The authors have posted their rebuttal. Please review their responses and engage in the discussion.

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.