- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Self-Correction Optimization (SCO), a training-free test-time guidance method for interleaved image-text generation and video generation. At each denoising step up to tau_end = 0.5, instead of only predicting the unconditional and full conditional velocities (v_u and v_f) as in standard CFG, SCO predicts two additional branches in the latent velocity space: an event-off counterfactual branch v_counter and a history-off branch v_event. By taking the differences a_new = v_f - v_counter and a_state = v_f - v_event, the method isolates an explainable new-event direction and a state-preserving direction, and solves a 2-variable bounded quadratic program (Section 3.4) to fuse them into the CFG update. Experiments across modular/unified image-text generators (OpenING and ISG-Bench) and video generators (Video-CraftBench and Interleaved-VLA) demonstrate consistent improvements in visual quality, subject preservation, and background consistency.
Critical strengths
- Intuitive, explainable, and effective test-time guidance Isolating the new-event direction (
v_f - v_counter) and the state-preserving direction (v_f - v_event) in the latent velocity space and fusing them into the CFG update is both intuitive and explainable. As shown in Figure 4 (as well as Figures 2, 6, 9, and 10), background consistency and subject appearance are preserved very well across sequential editing, object removal, and embodied video generation tasks. - Plug-and-play improvements across diverse backbones. Because SCO operates purely at test time without parameter updates, it can be directly plugged into a wide variety of modular and unified image-text models (e.g., MM-Interleaved, Show-o2, BAGEL, SenseNova-U1, DuoGen) and video models (VideoWorld2, Cosmos-2.5, Cosmos-3) with consistent quantitative gains.
Critical weaknesses
- Computationally heavy at inference time with unreported latency/memory overhead. Up to
tau_end = 0.5, SCO predicts four velocities (v_u,v_f,v_counter,v_event) instead of two at each denoising step, which doubles the per-step forward passes during the first half of sampling. Despite this noticeable extra computation, the paper does not report actual wall-clock inference latency or GPU memory overhead compared to standard CFG. - Missing prompt-rewriting details in the text and appendix. Figure 7 in the appendix shows that external LLMs are used to rewrite the text prompts to construct the counterfactual (event-off) and event-only (history-off) branches. However, this prompt-rewriting step is not explained in Sections 3.2-3.3 of the main paper, and although the Reproducibility Statement (Page 10) claims that "the detailed process of text prompt rewriting is displayed in the appendix," the actual prompt templates and rewriting procedure are missing from the appendix text.
- Lack of a fixed-weight fusion baseline in the ablation study (Table 3). While Table 3 evaluates adding
+a_state,+a_new, and+Both, it does not include a simple fixed-weight fusion baseline (similar to standard composable CFG, wherea_newanda_stateare combined with constant scalar weights) to isolate how much improvement specifically comes from the 2D QP solver in Section 3.4.
Questions for authors
- Wall-clock latency and memory overhead: Since predicting four velocities instead of two at each denoising step (up to
tau_end = 0.5) doubles the per-step forward passes, could you report the actual wall-clock inference latency and peak GPU memory overhead of SCO compared to standard CFG? - Details of LLM prompt rewriting: Could you provide the missing prompt-rewriting details promised in the Reproducibility Statement—specifically, which LLM and prompt templates are used to generate the counterfactual and event-only prompts shown in Figure 7?
- Comparison with fixed-weight fusion: In Table 3, how does the 2D QP solver in Section 3.4 compare against simply fusing
a_newanda_statewith fixed constant weights (as in standard composable CFG)?
3 comments
Response to Reviewer 1uJB (2/2)
W3: Lack of a fixed-weight fusion baseline in the ablation study.
Fixed-weight fusion is equivalent to SCO with constant coefficients: , where are held fixed across all sampling steps and all interleaved generation tasks, whereas the bounded quadratic program in Sec. 3.4 adapts them at each step. We compare the two on OpenING, using SenseNova-U1 as the generator. All variants share the same four branches, schedule (), and hyper-parameters, and differ only in how are obtained. For the fixed-weight baseline, the coefficients are set to the mean or the median of the coefficients produced by SCO over all sampling steps and samples, so that the overall correction strength matches that of SCO. Table R1 reveals that SCO reduces FID from 67.2 to 55.9, whereas the mean fixed weights reduce it only to 64.7. Moreover, with mean weights, CLIP-T, GPT-J1, , and GPT-J2 fall below the baseline, and with median weights, all metrics fall below the baseline. We attribute this to the constant correction being applied even at steps where the CFG update already satisfies both constraints, which over-corrects the update.
We additionally include a sequential variant that solves the two constraints one after the other, ignoring the cross term . As shown in Table R1, both orders are clearly inferior to SCO (FID 65.1/60.4 vs. 55.9). The new state variant attains a marginally higher (0.36 vs. 0.35), but falls below the baseline on CLIP-T and GPT-J1. Because the two directions are not orthogonal, the second correction changes the projection onto the first one, so a sequential scheme cannot reliably satisfy both constraints. This confirms the analysis in Sec. 3.4 and motivates solving the two constraints jointly.
Table R1: Necessity of joint optimization, evaluated with SenseNova-U1 on OpenING. All variants use the same four branches and settings, and differ only in how are obtained. Fixed: the median or mean of the SCO coefficients over all steps and samples. Sequential: the two constraints are solved one after the other in the indicated order.
| Method | FID ↓ | CLIP-I ↑ | CLIP-T ↑ | GPT-J1 ↑ | ↑ | ↑ | GPT-J2 ↑ |
|---|---|---|---|---|---|---|---|
| SenseNova-U1 (Diao et al., 2026) | 67.2 | 0.325 | 0.471 | 0.658 | 0.29 | 0.37 | 0.574 |
| + Fixed | 72.1 | 0.307 | 0.459 | 0.625 | 0.27 | 0.33 | 0.530 |
| + Fixed | 64.7 | 0.330 | 0.463 | 0.651 | 0.30 | 0.35 | 0.561 |
| + Sequential (new state) | 65.1 | 0.337 | 0.455 | 0.618 | 0.36 | 0.38 | 0.587 |
| + Sequential (state new) | 60.4 | 0.358 | 0.503 | 0.679 | 0.26 | 0.35 | 0.563 |
| + SCO (joint optimization) | 55.9 | 0.376 | 0.507 | 0.694 | 0.35 | 0.40 | 0.602 |
Thanks for detailed responses
I think responses have fully resolved my concerns. Specially, I agree with author's claim that the inference cost is accepatble given the quality improved. Also, the additional ablations strengthen the importance of joint optimization. I raised my score :).
Response to Reviewer 1uJB (1/2)
We thank the reviewer for the positive assessment and also for the constructive comments.
W1: Inference costs.
This concern is also shared by Reviewers fPE9 and ufvw. The reviewer is correct that SCO doubles the forward passes in the first half of sampling, which amounts to 1.5× the denoising passes of standard CFG overall. Table R3 reports the measured cost of three backbones on two NVIDIA H100 GPUs. Including the rewriter calls, SCO increases the per-sample latency by 25.0–46.0% and the peak GPU memory by 7.7–13.3 GB across MM-Interleaved, SenseNova-U1, and BAGEL. The memory increase arises because the additional branches are evaluated in the same batch as the CFG branches, and can be avoided by evaluating them sequentially at the cost of higher latency. The rewriter Qwen3.6-Plus is accessed through an API and uses no local GPU memory. Given that SCO is training-free and consistently improves generation quality, we consider this overhead acceptable.
Table R3: Inference cost with and without SCO. Latency includes text generation, image generation, and the rewriter calls (Qwen3.6-Plus).
W2: Missing prompt-rewriting details.
This concern is also shared by Reviewers fPE9 and ufvw. In the revision, we describe the prompt rewriting in detail in Sec. 3.2–3.3 and add an appendix subsection (Appendix A.4) with the procedure and complete template. The added content is summarized below.
Rewriting procedure. Besides the full condition cfull of standard CFG, SCO requires an event-off condition ccounter, which differs from cfull only in the current event (Eq. (4)), and a history-off condition cevent, which differs from cfull only in the absence of the inherited history (Eq. (6)). The rewriter is Qwen3.6-Plus, a commercial LLM queried through an API. As illustrated in Fig. 7, for the m-th image slot, the rewriter takes the cached history Hm (all previous text segments and images) and the full prompt as input, e.g., “A woman wearing a red skirt is standing in a kitchen beside a wooden table. She is holding a whole apple and cuts it into two pieces with a knife.”, and returns two prompts in one JSON object:
Because the event-off prompt is a minimal edit of the full prompt, anew=vfull−vcounter captures the event rather than wording differences. The full and event-off branches keep the history images as visual context, the history-off branch removes both history text and history images, and the unconditional branch uses an empty prompt. The rewriter produces only text conditions and never accesses the generator's latents or parameters.