acceptodds
Under review as a conference paper at ICLR 2027

AdaShare: Adaptive Cross-Frame Sharing for Training-Free Consistent Image Generation

Abstract

Consistent visual story generation requires maintaining subject identity and visual style across images under different scene prompts. Existing training-free methods improve cross-frame consistency by sharing reference features across frames. However, unreliable feature sharing can lead to incomplete subject generation and degraded image quality. To address these issues, we propose a training-free framework for adaptive cross-frame feature sharing in consistent text-to-image generation, termed AdaShare. To improve subject completeness and consistency, AdaShare extracts reliable subject masks from cross-attention maps and activates subject feature sharing after subject regions stabilize. We further introduce Adaptive-Capacity Optimal Transport Attention (ACOT Attention), which formulates Extended Attention as an optimal transport problem. By imposing adaptive capacity constraints derived from the areas of the subject masks, ACOT Attention prevents excessive attention concentration on reference tokens caused by token imbalance, enabling more effective utilization of cross-frame features. Furthermore, to maintain style consistency, we introduce Covariance Value Alignment, which applies whitening and coloring transformation to the low-frequency components of value representations while leaving high-frequency structures unchanged. Extensive experiments on ConsiStory+ demonstrate that AdaShare achieves superior performance over training-free baselines in core consistency metrics and human evaluations, while maintaining prompt fidelity and image quality. The source code will be released upon acceptance.

Primary area:
generative models
Keywords:
consistent T2I image generation, diffusion model, training-free

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ddcHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AdaShare is a training-free framework for generating images with consistent subject identity and visual style across different prompts. It combines dispersion-based filtering of cross-attention maps and mask-dependent sharing schedules, Adaptive-Capacity Optimal Transport Attention (ACOT) with reference-token capacity bounds determined by subject-mask areas, and frequency-selective whitening and coloring of attention value features. The paper evaluates the framework on ConsiStory+ through baseline comparisons, ablations, and a user study.

Critical strengths

AdaShare achieves DreamSim and VGG-Style scores of 0.1692 and 1.0588, respectively, outperforming the listed methods while retaining competitive text alignment. These results suggest a practically useful trade-off among identity consistency, style consistency, and prompt fidelity.

Critical weaknesses
  1. Adaptive mask extraction improves IoU only from 61.07% to 62.22% over the baseline in Table 4, with similarly modest CLIP-I and DreamSim gains in Table 3. The qualitative examples do not establish a general improvement in subject completeness. For the style component, low-frequency-only WCT with preservation of high-frequency structure was already explored in WCT², while SADis applies WCT in diffusion generation and addresses high-frequency texture degradation. Applying these ideas to attention values is a concrete distinction, but the manuscript does not sufficiently establish the new technical insight enabled by this adaptation. Thus, despite attractive overall performance, the methodological contribution remains insufficiently compelling.

  2. ACOT differs meaningfully from CoDi’s fixed subject-feature transport by constraining reference-column capacities in dynamic attention. However, at γ=1.25 in Table 7, ACOT underperforms ordinary Extended Attention on CLIP-I and DreamSim, despite improvements in other metrics; this comparison also does not use the final full default configuration. The paper does not separately establish the benefits of mask-area-dependent versus fixed capacities, or adaptive versus fixed scheduling. Consequently, it remains unclear whether the overall gains arise specifically from the proposed mechanisms rather than other components or changes in sharing strength. This uncertainty materially weakens the evidence for the significance of the central contribution.

Questions for authors
  1. Among ACOT’s differences from CoDi, is the principal contribution the reference-column inequality constraint, online updating, or mask-area-dependent capacity? Which existing results separately substantiate these claims?

  2. Under the final default configuration, how do results change when only ACOT is replaced with ordinary Extended Attention or a fixed-capacity variant, keeping masks, CVA, scheduling, and random seeds identical? Can repeated runs or paired confidence intervals establish a reliable benefit from the proposed mechanisms?

  3. Does the Baseline Mask comparison in Table 3 also change the sharing schedule? Can the contributions of mask filtering and adaptive scheduling be separated, and can the authors establish whether these changes meaningfully reduce subject incompleteness rather than merely producing small improvements in average consistency metrics?

2 comments

We thank the reviewer for the thorough evaluation and for recognizing AdaShare's strong performance on ConsiStory+ and its practical balance across identity consistency, style consistency, and prompt fidelity.

1. Core Differences from CoDi, Primary Contribution, and Experimental Support (Weakness 2, Question 1)

Primary Contribution. Compared with CoDi's fixed feature transport, the core contribution of ACOT Attention lies in jointly formulating the reference-column inequality constraint and the mask-area-dependent adaptive capacity bound τ=∣Mi∣/∣M0∣\tau = \vert{}\mathcal{M}_i\vert{} / \vert{}\mathcal{M}_0\vert{} (Eqs. 6 and 33), while online Sinkhorn updating (Algorithm 1) serves as an efficient solver for this constrained optimization.

Supporting Evidence in the Paper:

  • Inequality Constraint & Adaptive Capacity (Figs. 4 and 9): Fig. 4 (right) evaluates the Excess Reference Attention Mass (ERAM) exceeding the adaptive bound τ\tau across the entire ConsiStory+ benchmark, showing that ACOT cuts ERAM from Extended Attention's baseline down to 2.6%. The RAMV heatmaps in Fig. 4 (middle) confirm a more balanced spatial distribution across reference subject tokens, eliminating the missing zebra ears/heads and missing hands shown in Fig. 4 (left) and Fig. 9.
  • Online Updating Ablation (Table 8): Fixing all other hyperparameters, Table 8 isolates the online Sinkhorn iteration count L∈{1,2,3,4}L \in \{1, 2, 3, 4\}, showing consistent improvements across VGG-Style, CLIP-T, CLIP-I, and FID when increasing LL from 1 to 2.
  • Advantages Over CoDi (Tables 1, 2, 4, and 10): Because CoDi extracts masks at the final diffusion step, it requires 19.57 s for mask extraction (Table 4) and 99.29 s for total inference (Table 10), whereas AdaShare requires 3.86 s for mask extraction (IoU 62.22% vs. 61.87%) and reduces total inference time. Across Table 1, AdaShare outperforms CoDi on every metric (vs. CoDi's DreamSim 0.2723, CLIP-I 0.8967, VGG-Style 6.5864, and FID 55.9575) and achieves more than double the human preference rate in Table 2 (39.0% vs. 18.6%).

Manuscript revision: We have updated Section 2 "Related Work" accordingly (marked in blue).

2. Controlled Comparison of ACOT vs. Extended Attention and γ=1.25\gamma = 1.25 Analysis (Weakness 2, Question 2)

We have compared ACOT and Extended Attention in the Appendix Table 7 section.

Controlled Experimental Setup. In Table 7, the adaptive subject masks, CVA module, scheduling strategy, and random seeds are held strictly identical across rows; the sole difference is replacing ACOT Attention with standard Extended Attention.

Performance at γ=1.25\gamma = 1.25. Standard Extended Attention leaves a large fraction of attention mass over-concentrated on a few high-response reference tokens. At a moderately high affinity (γ=1.25\gamma = 1.25), this unconstrained attention collapses onto salient local reference patches. While such local hard-copying slightly favors globally pooled similarity metrics, it starves other semantic parts of the subject and causes structural incompleteness (Figs. 4 and 9). Under the exact same controlled setting at γ=1.25\gamma = 1.25, ACOT Attention outperforms Extended Attention in prompt fidelity (CLIP-T), style consistency (VGG-Style), and overall generation quality (FID: 46.5232 vs. 46.7600).

3. Mask Filtering vs. Adaptive Scheduling and Subject Completeness (Weakness 1, Question 3)

Interaction Between Mask Filtering and Scheduling. In Table 3, the Baseline Mask setting only replaces our spatial dispersion filter (Dtl<τs=170D_t^l < \tau_s = 170, Eqs. 4–5) with standard uniform averaging across candidate layers (Eq. 3), while keeping CVA, ACOT Attention, and the stabilization criterion unchanged. However, as illustrated in Fig. 3 (right), unfiltered cross-attention maps exhibit severe spatial dispersion and background noise across steps 2 to 15. Because consecutive noisy masks fluctuate heavily, the baseline rarely satisfies the IoU≥0.85\text{IoU} \ge 0.85 convergence condition early and is forced to lock a noisy mask at the upper step bound T2=15T_2 = 15. By filtering out near-uniform diffuse layers (where the theoretical uniform variance on a 32×3232 \times 32 grid is 170.5, Appendix B.1 Eqs. 14–19), our Adaptive Mask Extraction stabilizes cleanly at step 11 (green box in Fig. 3 right), preventing subject incompleteness such as the missing hand in Fig. 3 (left) and reducing background leakage in Fig. 8. Manuscript revision: We have updated Section 3.2.1 "Adaptive Mask Extraction" accordingly (marked in blue).

4. Technical Insight of CVA (FS-WCT) in Self-Attention Value Space (Weakness 1)

While frequency separation has been studied in image-level style transfer (WCT²) and single-image stylization, introducing FS-WCT into early self-attention value features ViV_i for multi-frame story generation resolves a distinct conflict: aligning cross-frame global style without overriding prompt-driven pose and scene structure in the current frame. In diffusion self-attention, early value features jointly encode global color/texture distributions in low frequencies and scene layout/pose geometry in high frequencies. As shown in Fig. 5, applying full-spectrum WCT directly to ViV_i introduces up to 12.18% high-frequency spectral deviation, which forces subsequent frames to replicate the reference frame's rigid portrait pose rather than following action prompts such as "walking with a cane". By restricting WCT to low-frequency components of ViV_i (Eqs. 10–11), CVA aligns cross-frame artistic style while preserving the high-frequency structural cues needed for prompt-faithful pose and scene generation. Manuscript revision: We have updated Section 3.1 "Problem Formulation" accordingly (marked in blue).

Reviewer_mE8VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AdaShare, a training-free framework for consistent text-to-image generation built on SDXL. It introduces three components: Adaptive Mask Extraction, which filters spatially diffuse cross-attention maps and activates cross-frame sharing once the subject mask stabilizes; Adaptive-Capacity Optimal Transport Attention (ACOT Attention), which reformulates Extended Attention as an entropy-regularized optimal transport problem with a capacity constraint on each reference token; and Covariance Value Alignment (CVA), which applies a Frequency-Selective WCT (FS-WCT) to the low-frequency components of self-attention value features to align style with a reference frame. The method is evaluated on ConsiStory+ against eight baselines using CLIP-T, CLIP-I, DreamSim, DINO, VGG-Style, and FID, together with a 32-participant user study, ablations, and an extension to FLUX.

Critical strengths
  • Each component is constructed to address a specific problem identified in the paper (noisy subject masks, attention over-concentration on reference tokens, and high-frequency perturbation by WCT), and the problems are illustrated with dedicated analyses (Figs. 3–5). This coherence between the problem definition and the method is the main reason I consider the approach well motivated.

  • The overall results are competitive for a training-free method: AdaShare achieves the best CLIP-I, DreamSim, and VGG-Style among the compared methods in Table 1 and the highest preference in the user study.

Critical weaknesses
  • The ablation studies do not clearly establish the individual contributions of the proposed components. The qualitative results in Fig. 7 show only subtle differences, while Table 3 lacks a baseline without any proposed components. The improvements from Adaptive Mask Extraction are small, and ACOT Attention performs worse than Extended Attention in CLIP-I (0.9295 vs. 0.9318) and DreamSim (0.1682 vs. 0.1622) at γ = 1.25 (Table 7). Although ACOT improves these metrics at γ = 1.5, this is not the default setting. Since the main contribution lies in these components, the current ablations make it difficult to assess their individual effectiveness.

  • Image quality is not evaluated consistently across all baselines. FID is missing for UNO and Infinite-Story in Table 1, possibly because it is measured against SDXL-generated images. However, the additional backbone-independent quality metrics in Table 9 are also limited to SDXL-based methods. This makes it difficult to compare generation quality across different backbones and fully assess the claimed balance between consistency and image quality.

Questions for authors
  • (Clarification) What configuration does each row of Table 3 correspond to (e.g., does "w/ CVA" include any subject feature sharing)?

  • (Additional result) Could the authors report results for a baseline without any of the proposed components and for removing one component at a time, so that each component's contribution can be assessed?

  • (Clarification) At γ = 1.25, ACOT Attention performs worse than Extended Attention in CLIP-I and DreamSim (Table 7). Could the authors explain this result and clarify whether it also holds under the final default configuration with L = 2?

  • (Additional result) Could the authors report image-quality metrics for UNO and Infinite-Story, such as the backbone-independent metrics in Table 9, so that quality can be compared across all baselines?

  • (Clarification) Appendix D.3 describes the inference speed as competitive, but in Table 10 AdaShare (85.01 s) is slower than all other methods in Table 10 except CoDi (99.29 s), and more than twice as slow as StoryDiffusion (37.48 s). Could the authors clarify this claim?

2 comments

We thank the reviewer for recognizing the logical alignment between our problem analysis (Figs. 3–5) and method design, as well as our strong performance in quantitative benchmarks (Table 1) and human evaluations (Table 2).

1. Clarification of Table 3 Configurations and Leave-One-Out Ablations (Weakness 1, Questions 1 & 2)

We apologize for the compact notation in Table 3. The rows in Table 3 correspond to isolating individual modules or removing one component at a time:

  • w/ CVA (Full Model w/o ACOT & Adaptive Mask): Enables only CVA (with FS-WCT) in self-attention for low-frequency style alignment, without any cross-frame subject feature sharing.
  • w/ ACOT Attn (Full Model w/o CVA): Enables cross-frame subject sharing (Adaptive Mask Extraction and ACOT Attention) while completely removing CVA style alignment.
  • w/ Baseline Mask (Full Model w/o Adaptive Mask Extraction): Retains CVA and ACOT Attention but replaces Adaptive Mask Extraction with unfiltered layer averaging.

To show each component's individual contribution, the table below compiles the base model and leave-one-out ablations from Tables 1, 3, 6, and 7:

ConfigurationVGG-Style ↓CLIP-T ↑CLIP-I ↑DreamSim ↓
Vanilla SDXL4.38900.91360.87750.3364
w/ ACOT Attn2.39550.90830.90840.2304
w/ CVA1.31700.90350.91350.2220
w/ Baseline Mask1.06490.90210.92880.1728
w/ Standard WCT0.90250.89750.93290.1597
w/ Extended Attn1.04080.89980.93180.1622
Full Model1.05880.90120.92980.1692

The corresponding FID scores are 42.4033 (w/ ACOT Attn), 43.3768 (w/ CVA), 47.7851 (w/ Baseline Mask), 47.5041 (w/ Standard WCT, i.e., w/o FS-WCT), 46.7600 (w/ Extended Attn, i.e., w/o ACOT), and 45.7914 (Full Model). Removing any single component degrades either consistency, prompt alignment, or overall image quality.

2. Analysis of ACOT vs. Extended Attention at γ=1.25\gamma = 1.25 and Default L=2L = 2 (Weakness 1, Question 3)

Local Hard-Copying Effect. As shown in Sec. 3.1 (Problem 1) and Fig. 4, standard Extended Attention lacks column-wise capacity constraints on reference tokens, leaving 17.7% of reference attention mass in an over-concentrated state (ERAM), whereas ACOT Attention reduces this ratio to 2.6% (an 85.4% reduction). At a moderately high affinity strength (γ=1.25\gamma = 1.25), Extended Attention collapses current-frame queries onto a few maximally activated reference patches. While this unconstrained local copying slightly inflates pooled feature similarity scores (CLIP-I and DreamSim), it breaks subject structural completeness, lowers prompt fidelity, and harms image quality.

Behavior Under Default L=2L = 2 (Table 8). Table 7 fixes a single Sinkhorn iteration (L=1L = 1) with rc=0.25r_c = 0.25. As shown in Table 8, increasing the iteration count to our default L=2L = 2 further improves CLIP-I to 0.9301, CLIP-T to 0.9009, VGG-Style to 1.0309, and FID to 46.2409 (with DreamSim at 0.1697), achieving a better overall trade-off among consistency, text alignment, and image quality (Fig. 1).

3. Backbone-Independent Image Quality Metrics for UNO and Infinite-Story (Weakness 2, Question 4)

We evaluate UNO and Infinite-Story using the four backbone-independent quality metrics from Table 9:

MetricUNOInfinite-StoryOurs
Aesthetic Score ↑\uparrow6.24956.08026.2296
VQAScore ↑\uparrow0.69590.71850.7502
PickScore ↑\uparrow23.037023.408623.2264
MPS ↑\uparrow14.821115.535015.0347

AdaShare achieves the highest VQAScore and remains highly competitive with both FLUX- and Infinity-based models across Aesthetic Score, PickScore, and MPS.

4. Clarification on Inference Speed in Appendix D.3 / Table 10 (Question 5)

We agree that describing our runtime as broadly "competitive" in Appendix D.3 was imprecise, and we will revise this phrasing. AdaShare builds directly upon ConsiStory's cross-frame extended attention architecture. In Table 10 (generating five 1024×10241024 \times 1024 images on an RTX 4090), ConsiStory requires 83.88 s (46.48 GB VRAM), while AdaShare takes 85.01 s (42.36 GB VRAM). Thus, our three adaptive modules add only 1.13 s per five-image set (0.23 s per image, a 1.3% increase) while reducing peak GPU memory by 4.12 GB. With this minimal overhead over ConsiStory, AdaShare substantially improves DreamSim (from 0.2561), VGG-Style (from 4.8126), and FID (from 53.3715). We discussed the computational overhead of our adaptive modules in Appendix F.

Reviewer_pvhyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AdaShare, a training-free framework for consistent text-to-image generation, consisting of three components: adaptive subject mask extraction, Adaptive-Capacity Optimal Transport Attention (ACOT), and Covariance Value Alignment (CVA). Experimental results demonstrate improved subject and style consistency over existing training-free methods while maintaining text alignment and image quality.

Critical strengths
  1. The adaptive subject mask extraction is well-motivated and effectively filters noisy attention maps, yielding more reliable subject masks (Figure 3).

    1. The method achieves strong quantitative results in subject consistency, style consistency, and prompt fidelity compared with existing training-free baselines.
Critical weaknesses

1 The motivation behind ACOT Attention is insufficiently justified. High attention concentration on certain reference tokens may reflect valid semantic correspondences, particularly for identity-critical features such as facial details. Imposing uniform capacity constraints may therefore suppress meaningful attention. Moreover, Table 7 shows only marginal improvements, with even worse DreamSim performance .

 2 The motivation for enforcing style consistency is questionable, as it is not necessarily a key requirement for subject-consistent generation, where diverse visual styles may be desirable. Moreover, existing methods do not appear to exhibit significant style inconsistencies. The VGG-Style metric may also be confounded by subject consistency, as images with more consistent subjects may naturally achieve better style scores.

 3 How does your method perform in terms of pose diversity? The qualitative results appear to show highly similar subject poses and layouts, as well as consistent background color tones across different scenes. Could the proposed method affect not only subject consistency but also background appearance? If so, the reported subject consistency scores may not objectively reflect improvements in identity preservation.
Questions for authors

Please refer to the Critical Weaknesses

1 comment

We thank the reviewer for recognizing the motivation and denoising effectiveness of Adaptive Subject Mask Extraction, as well as AdaShare's quantitative performance across subject consistency, style consistency, and prompt fidelity.

1. Motivation of ACOT Attention, Capacity Constraints, and Table 7 Analysis (Weakness 1)

Design Motivation. Standard Extended Attention applies row-wise softmax independently to each query without global column-wise constraints on reference keys K0K_0. When specific reference tokens strongly match current subject queries, many queries simultaneously collapse onto a few high-response tokens, severely skewing the Reference Attention Mass Vector (RAMV). This imbalanced feature transfer causes incomplete subject geometry, such as missing zebra ears or heads (Fig. 4 left, Fig. 9 top-left) and missing hand details (Fig. 9 top-right). Manuscript revision: We have updated Section 3.1 "Problem Formulation" accordingly (marked in blue).

Inequality Capacity Constraint. ACOT Attention does not force uniform weights across reference tokens; instead, it imposes an inequality upper bound τ\tau on reference columns (Eqs. 6 and 20). By complementary slackness (Eqs. 7 and 29), reference tokens with attention mass at or below τ\tau remain unmodified (vk=1v_k = 1), while only over-concentrated tokens exceeding τ\tau are rescaled.

Table 7 Performance. Table 7 evaluates ACOT across cross-frame affinity strengths γ\gamma (Eq. 1, Appendix C.2). At lower affinities, Extended Attention shows a slight edge in DreamSim because unconstrained hard-copying of high-response local patches inflates pooled similarity while introducing visual artifacts. In contrast, ACOT Attention outperforms Extended Attention in CLIP-T, VGG-Style, and FID at our default affinity, and consistently achieves higher ImageReward across all tested settings:

MethodγImageReward ↑
Extended Attention1.000.9475
ACOT Attention1.000.9486
Extended Attention1.250.8863
ACOT Attention1.250.9064
Extended Attention1.500.8456
ACOT Attention1.500.8522

ACOT Attention consistently improves perceptual preference and prompt alignment across all affinity levels.

2. Style Consistency Motivation, Baseline Limitations, and VGG-Style Decoupling (Weakness 2)

Task Motivation. Following Infinite-Story (Park et al., 2026, AAAI 2026), our work targets consistent visual story generation, where preserving uniform brushstrokes, color palettes, and rendering media across frames is essential alongside subject identity. For applications requiring diverse styles for the same subject, AdaShare's modular design allows users to disable style alignment by setting the blending weight α=0\alpha = 0 in Eq. 12.

Style Drift in Existing Methods. Existing baselines exhibit noticeable cross-frame style inconsistencies. In Fig. 6 (right, gnome sequence), CoDi shifts from a dark illustration style in the first two frames to a light watercolor sketch in the third frame, while IP-Adapter and 1Prompt1Story show clear jumps in color saturation and texture across both the fox and gnome sequences.

Decoupling of VGG-Style and Subject Consistency. VGG-Style measures low-level color distributions and local textures, which are decoupled from subject identity. In Table 1, IP-Adapter and StoryDiffusion achieve strong subject consistency, yet their VGG-Style scores differ substantially and both trail 1Prompt1Story despite comparable identity preservation. Similarly, in Table 3, using CVA alone and using ACOT alone yield very close subject consistency, whereas adding CVA sharply reduces VGG-Style, confirming that VGG-Style reflects style alignment rather than a byproduct of subject consistency. Manuscript revision: We have updated Section 4.2 "Experimental Results Extraction" accordingly (marked in blue).

3. Objective Subject Consistency Evaluation and Pose Diversity (Weakness 3)

Background-Free Evaluation Protocol. As detailed in Sec. 4.1, we follow the standard DreamSim evaluation protocol by removing backgrounds from all generated images using CarveKit and replacing them with random noise prior to metric computation. Consequently, the DreamSim scores reported in Table 1 strictly measure foreground subject identity preservation without interference from background color tones.

Pose Diversity. AdaShare preserves pose flexibility via stage-wise adaptive scheduling (Appendix A.1, Fig. 2b), spatial mask isolation, and frequency-decoupled early style alignment. Substantial variations in subject pose, camera viewpoint, and scene layout are demonstrated across Figs. 5, 6, 9, 10, 11, and 12. Manuscript revision: We have updated Section 3.3 "Covariance Value Alignment Extraction" accordingly (marked in blue).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.