acceptodds
Under review as a conference paper at ICLR 2027

INTRA: Interleaved Non-contiguous Token spaRse Attention

Abstract

Transformers achieve strong performance across modalities but are bottlenecked by the quadratic cost of attention. Sparse attention can reduce this cost, but effective sparsity must preserve useful long-range communication and map well to GPU hardware. We introduce **INTRA**–**I**nterleaved **N**on-contiguous **T**oken spa**R**se **A**ttention–a hardware-aware static sparse attention design based on a relay-communication view under Computational Query Set (CQS) constraints. Rather than directly covering all token-to-token interactions within each sparse layer, INTRA uses complementary sparse groups to provide short mediated communication paths across layers. Scatter forms relay groups that sample broadly across local neighborhoods, while Gather reconnects these groups in subsequent layers. Under a partition model, this structure guides CQS-compatible sparse patterns with global reachability. Our CQS-aware FlashAttention-style kernel computes structured indices at runtime, loads selected non-contiguous tokens, and packs them densely for computation. On an A100, INTRA achieves a speedup in `FLUX.1-dev` generation, with better measured image quality than evaluated CLEAR configurations after LoRA self-distillation. On LLaMA-3.1-8B, it improves long-context prefill efficiency with strong teacher-forced retrieval performance under matched LoRA adaptation. INTRA achieves significant speedups over MoBA and density-constrained NSA in terms of attention computation.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Static Sparse Attention

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WdcrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces INTRA, which establishes cross-layer information propagation paths by alternating between "Scatter" and "Gather" operations on statically sparse groups. It also features a FlashAttention-style kernel designed to handle non-contiguous token access. The authors analyze coverage and connectivity within this grouped framework and, by employing LoRA adaptation, evaluate the approach’s quality and efficiency across image generation, language tasks, and video generation.

Critical strengths
  1. Static linking avoids input-dependent pattern search, while structured indexing and token loading further realize the sparse pattern into an executable GPU kernel. The paper considers both communication architecture and practical runtime overhead.

  2. In the 2048×2048 FLUX.1-dev experiments, generation time was reduced from 66.78 seconds to 43.72 seconds, an improvement of approximately 1.53×, and outperformed the image quality metrics of the compared CLEAR configuration. This supports the practical value of the method in the tested settings.

  3. Ablation of Scatter/Gather combination, number of groups, switching interval, and global layer configuration provides support for design choices. The authors also explicitly distinguish between theoretical connectivity and task quality and disclose the differences between retrieval and timing configurations.

Critical weaknesses
  1. The authors explicitly state that the block-level retrieval configuration in Table 4 differs from the token-level timing configuration. While this disclosure is both necessary and valuable, it cannot substitute for performance benchmarking based on the actual retrieval configuration used. Consequently, the existing experiments separately support the claims that "one configuration offers superior retrieval capability" and "another configuration yields lower attention latency," yet no evidence has been established to show that both hold true simultaneously. Furthermore, the operator latency and SCROLLS scores presented in Table 3 require a more direct alignment of configurations and a joint evaluation. This represents a significant gap in the evidence supporting the conclusions regarding the efficiency-quality trade-offs in the language domain.

  2. In Table 8, the computational costs of the various methods differ substantially. For instance, at a 16K context length, the reported computational costs for INTRA, XAttention, and MInference are approximately 12.5%, 28%, and 58% of the dense baseline, respectively. While varying computational budgets do not inherently constitute an unfair comparison, the fact that INTRA underperforms in five out of the six SCROLLS tasks makes it difficult to discern whether the observed outcomes stem from superior methodological design or simply from selecting an operating point that requires less computation at the cost of lower quality. A comparison that controls for quality, or an analysis across a range of computational budgets, is necessary to determine the true advantages of the proposed trade-offs.

Questions for authors
  1. How much speedup does the actual configuration that generates the paper quality score achieve? The authors have stated that NIAH retrieval and timing use different configurations. Please directly report the actual latency of the block-level configuration that achieved the 0.923 NIAH score; also clarify whether the SCROLLS scores in Table 3 correspond to the configuration used in the speed measurement in the table.

  2. Were the MInference and XAttention latency in Table 3 obtained independently under the same conditions as INTRA? Please clarify whether these data were directly measured or converted from literature results, and explain whether all three used the same input shape, precision, and timing range, including their respective necessary online overhead. This directly determines whether "14.94×, 2.23×" constitute a valid direct comparison.

4 comments

Response to Reviewer

We thank the reviewer for their careful evaluation and constructive feedback. We address each of the reviewer's concerns below.

W1 / Q1 — Latency of the quality-evaluated configuration

Reviewer — Critical Weakness 1

The authors explicitly state that the block-level retrieval configuration in Table 4 differs from the token-level timing configuration. While this disclosure is both necessary and valuable, it cannot substitute for performance benchmarking based on the actual retrieval configuration used. Consequently, the existing experiments separately support the claims that “one configuration offers superior retrieval capability” and “another configuration yields lower attention latency,” yet no evidence has been established to show that both hold true simultaneously. Furthermore, the operator latency and SCROLLS scores presented in Table 3 require a more direct alignment of configurations and a joint evaluation. This represents a significant gap in the evidence supporting the conclusions regarding the efficiency-quality trade-offs in the language domain.

Reviewer — Question 1

How much speedup does the actual configuration that generates the paper quality score achieve? The authors have stated that NIAH retrieval and timing use different configurations. Please directly report the actual latency of the block-level configuration that achieved the 0.923 NIAH score; also clarify whether the SCROLLS scores in Table 3 correspond to the configuration used in the speed measurement in the table.

Thank you for pointing this out. We did not clearly distinguish the NIAH, SCROLLS, and token-level kernel settings. NIAH uses four block-level Scatter layers (block size 256, G=2) and 28 window layers. This is the configuration that obtains 0.923 mean NIAH accuracy in Table 4(a). Table 4(b) reports the sum of 32 attention-layer latencies for this schedule and we also measured full-model prefill speedup with the NIAH configuration on A100:

Context16K32K64K128K
prefill speedup1.214×1.453×1.909×2.785×

Thus, the original NIAH configuration has both a measured quality result at 4K–32K and a measured prefill speedup within that range.

SCROLLS uses block-level Scatter with G=8, rather than NIAH’s G=2. Table 3(a) originally timed a token-level G=8 operator. The two settings have the same nominal 1/8 group sparsity, but different attention masks. We now additionally measure the SCROLLS block-level G=8 operator with our kernel and it matches with the table reported speendup. We will state these distinctions clearly in the revised tables and text.

Single attention operation (ms)8K16K32K
Table 3(a): token-level G=80.5101.7506.500
SCROLLS: block-level G=80.5031.8086.62

W2 — Computation budget versus quality

Reviewer — Critical Weakness 2

In Table 8, the computational costs of the various methods differ substantially. For instance, at a 16K context length, the reported computational costs for INTRA, XAttention, and MInference are approximately 12.5%, 28%, and 58% of the dense baseline, respectively. While varying computational budgets do not inherently constitute an unfair comparison, the fact that INTRA underperforms in five out of the six SCROLLS tasks makes it difficult to discern whether the observed outcomes stem from superior methodological design or simply from selecting an operating point that requires less computation at the cost of lower quality. A comparison that controls for quality, or an analysis across a range of computational budgets, is necessary to determine the true advantages of the proposed trade-offs.

We agree that Table 3 does not prove a quality-matched SCROLLS advantage over XAttention or MInference. But the submitted ablations show that connection placement matters beyond the number of retained entries:

SourceAttention allocationOverall retained densityMean NIAH
Table 5Windows in four selected layers0.0690.028
Table 5Four scatter layers0.1230.923
Table 5Dense attention in four selected layers0.1851.000
Table 16Wider local windows0.2340.222

Table 5 fixes the four selected layer positions and adaptation budget. Table 16 compares different allocations. These are not matched-FLOPs experiments. A new 16K panel provides an additional matched-training-budget control:

Method (16K, 460 examples)Score
INTRA Aligned density0.6699
MoBA0.2560

The two settings use the same reported density (0.1255), rank-16 LoRA, learning rate, 1,200 steps, data, and evaluator. This supports the value of the tested connectivity allocation. It is not a new SCROLLS result or proof of matched-quality latency dominance.

For images, submitted Table 2 offers a directly measured quality–latency comparison:

2048×2048 FLUX.1-devGeneration time (s)FID ↓
CLEAR (r=8)47.8132.46
INTRA43.7215.67

Q2 — Provenance and correction of Table 3(a)

Reviewer — Question 2

Were the MInference and XAttention latency in Table 3 obtained independently under the same conditions as INTRA? Please clarify whether these data were directly measured or converted from literature results, and explain whether all three used the same input shape, precision, and timing range, including their respective necessary online overhead. This directly determines whether “14.94×, 2.23×” constitute a valid direct comparison.

The XAttention and MInference latencies in Table 3(a) were literature-derived estimates. Specifically, we divided our measured FlashAttention-2 latency by the normalized speedup ratios reported in Figure 4 of XAttention. For example, at 8K, the estimated latencies were 2.86/1.5 = 1.91 ms for XAttention and 2.86/0.2 = 14.30 ms for MInference.

We have now independently benchmarked Dense, XAttention, MInference, and INTRA on the same A100. For transparency, we report the absolute latency of every method, including both INTRA's token-level G=8 operator and the newly measured block-level G=8 operator used for SCROLLS. Both have nominal 1/8 grouped attention density, but use different masks.

The submitted INTRA operator used token-level Scatter with G=8, while SCROLLS quality used block-level Scatter with G=8 and 256-token blocks. Both have nominal 1/8 grouped attention density. We have now measured the actual SCROLLS block-level G=8 operator using our kernel.

Attention-operator latency (ms)8K16K32K
Dense5.3721.0082.58
XAttention (S=16)3.578.4226.64
MInference (v0.1.6)39.5452.2772.00
INTRA, token-level G=80.511.756.5
INTRA, SCROLLS block-level G=80.501.86.6

Dear Reviewer, thank you for your helpful comments! Please let us know if you have any further questions or concerns that we can address before the discussion period ends.

Reviewer_9Ef3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces INTRA, which uses predefined sparse attention patterns organized through Computational Query Sets (CQS) to reduce attention computation. INTRA uses Scatter and Gather attention in different layers. Having these different attention patterns allows information to pass between tokens that are not in the same CQS in one layer. With these optimizations, INTRA achieves up to a 1.53× end-to-end speedup for image generation compared to the dense-attention baseline.

Critical strengths

S1. The motivation for combining Scatter and Gather attention is clear for image generation, and the ablation results show better image quality when using both patterns together than using either alone.

S2. The paper provides an efficient GPU implementation for non-contiguous sparse attention through CQS-aware indexing and computation. The kernel results show meaningful reductions in attention latency.

Critical weaknesses

W1. The evaluation configurations are not well matched. The speed and retrieval experiments use different configurations, and retrieval is evaluated only up to 32K, while speed is evaluated up to 64K. Longer-context evaluation, such as 128K, is needed to show whether the method maintains quality. Also, comparisons with recent learned sparse attention methods, such as DSA, are missing.

W2. The novelty of the attention pattern seems limited. Combining local attention and strided or grid attention is already known in works such as Sparse Transformer and MaxViT. Although Table 6 supports combining Scatter and Gather over using either alone, it is unclear what additional benefit the proposed pattern provides over these existing designs. Comparisons under matched sparsity and adaptation budgets are needed to better support the contribution of the attention pattern beyond the kernel implementation.

Questions for authors
  1. For LLM retrieval, do the sparse global layers use Scatter or Gather attention, or a different pattern? Could you clarify which pattern is used in each layer and how the sparse global and local-window layers relate to Scatter and Gather?

  2. Does retrieval accuracy remain high for longer contexts, such as 128K? How does the decrease in retrieval accuracy affect downstream task accuracy?

4 comments

Response to Reviewer

We thank the reviewer for their careful evaluation and constructive feedback. We address each of the reviewer's concerns below.

Q1 — Which attention pattern is used for LLM retrieval?

Reviewer — Question 1

For LLM retrieval, do the sparse global layers use Scatter or Gather attention, or a different pattern? Could you clarify which pattern is used in each layer and how the sparse global and local-window layers relate to Scatter and Gather?

For LLM retrieval, we use a block-level Scatter–Gather design, as described in Section 4.7.

Scatter: Four global layers (0, 8, 16, 24) attend to alternating 256-token blocks with G=2, under causal masking.

Gather: The remaining 28 layers use sliding-window attention (575 tokens with 64 sink tokens). These layers serve the Gather role by mixing neighboring tokens and reconnecting different Scatter groups.

Thus, Scatter provides long-range communication, while sliding-window attention provides local mixing and cross-group reconnection.

W1 / Q2 — Long-context retrieval, downstream tasks, and DSA

Reviewer — Critical Weakness W1

W1. The evaluation configurations are not well matched. The speed and retrieval experiments use different configurations, and retrieval is evaluated only up to 32K, while speed is evaluated up to 64K. Longer-context evaluation, such as 128K, is needed to show whether the method maintains quality. Also, comparisons with recent learned sparse attention methods, such as DSA, are missing.

Reviewer — Question 2

Does retrieval accuracy remain high for longer contexts, such as 128K? How does the decrease in retrieval accuracy affect downstream task accuracy?

For image generation, Table 2 already provides a matched sparse-method comparison at 2K: INTRA is 43.72 s, FID 15.67 versus CLEAR(r=8) at 47.81 s, FID 32.46. Relative to the original dense model’s 66.78 s, the generation speedup is 1.53×.

We clarify the language settings. NIAH uses block-level G=2 and achieves 0.923 mean retrieval accuracy over 4K–32K. SCROLLS uses block-level G=8. SCROLLS uses block-level Scatter with G=8, rather than NIAH’s G=2. Table 3(a) originally timed a token-level G=8 operator. The two settings have the same nominal 1/8 group sparsity, but different attention masks. We now additionally measure the SCROLLS block-level G=8 operator with our kernel and it matches with the table reported speendup. We will state these distinctions clearly in the revised tables and text.

Single attention operation (ms)8K16K32K
Table 3(a): token-level G=80.5101.7506.5
SCROLLS: block-level G=80.5031.8086.62

For longer context, we evaluate on HELMET 128k dataset on a different mask setting that has 12 scatter layers. The overall score is close. For this 128k configuration, A100 full-model prefill speedup reaches 2.100× at 128K. This is a measured quality–speed trade-off, not dense-equivalent retrieval. We use a different configuration at 128K due to our limited GPU training budget. Training with 128K sequences was not feasible within our available resources. We believe longer-context adaptation of SCROLL setting is worth investigating.

On DSA. DSA selects tokens with a seperate index key cache. It has a secondary cache part and it is trained from scratch. No existing work transformed a dense model to DSA model. It requires change of entire model architecture and retrainning. It’s not possible for us to train the whole model to have the additional cache. Also, DSA only works on language models, our INTRA can be applied to both language and image models. We do not claim superiority over DSA.

W2 — Novelty relative to Sparse Transformer and MaxViT

Reviewer — Critical Weakness W2

W2. The novelty of the attention pattern seems limited. Combining local attention and strided or grid attention is already known in works such as Sparse Transformer and MaxViT. Although Table 6 supports combining Scatter and Gather over using either alone, it is unclear what additional benefit the proposed pattern provides over these existing designs. Comparisons under matched sparsity and adaptation budgets are needed to better support the contribution of the attention pattern beyond the kernel implementation.

Our key methodological contribution is the hardware-constrained formulation of sparse communication. Sparse attention should not be designed based on theoretical connectivity or FLOPs alone, as irregular memory access and inefficient GPU block utilization can limit practical speedups.

We do not claim that Scatter/Gather interleaving is our main novelty. As stated in the Introduction (lines 71–75), our first contribution is to formulate neighborhood coverage and relay-channel overlap objectives within ISPD under CQS constraints. These constraints incorporate structured index mappings and GPU block alignment into the design process. Appendix D proves the corresponding max–min properties, and Section 5 connects the formulation to efficient GPU execution.

We additionally conducted a direct comparison with MaxViT-style Block/Grid attention on FLUX.1-dev. We kept the same backbone, layer schedule, text-attention strategy, CLEAR 10K training data, and LoRA adaptation budget (2,000 steps, effective batch size 64, Prodigy lr=1.0, seed 42). Only the sparse attention patterns were changed.

We evaluated two MaxViT-style configurations:

  • M-BUDGET: Grid attention with stride (4,2) and Block attention with P=16. This matches INTRA's 512/256 keys per query in global/local layers.
  • M-FIXED: Fixed-grid attention with G=8 and Block attention with P=8, following MaxViT's fixed-size grid design.
MethodGen. Time (s)SpeedupFID ↓LPIPS ↓CLIP-I ↑
FLUX.1-dev (dense)13.651.00×———
INTRA12.011.14×11.7449.460.9045
MaxViT-style M-BUDGET16.660.82×12.8352.050.9019
MaxViT-style M-FIXED16.660.82×23.70761.340.8589

At matched attention budgets, INTRA achieves better image quality than the MaxViT-style M-BUDGET baseline, with lower FID (11.74 vs. 12.83), lower LPIPS (49.46 vs. 52.05), and higher CLIP-I (0.9045 vs. 0.9019). It is also faster in the evaluated implementation (12.01 s vs. 16.66 s).

Lower FLOPs do not necessarily imply lower latency. Table 1 provides another example: at 16K, INTRA achieves 2.056 ms versus CLEAR's 4.625 ms, despite higher reported computation costs (345.5 vs. 246.2 GFLOPs).

These results support our main contribution: jointly designing sparse communication objectives and hardware execution constraints, rather than simply combining existing local and global attention patterns.

Dear Reviewer, thank you for your helpful comments! Please let us know if you have any further questions or concerns that we can address before the discussion period ends.

Reviewer_wfmAOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes INTRA, a hardware-efficient static sparse attention framework designed to improve long-context language modeling and high-resolution image generation. The core idea is to structure sparse attention through Computational Query Sets (CQSs), enabling FlashAttention-compatible execution while preserving effective non-local communication via complementary Scatter/Gather relay patterns. The authors also study how pretrained dense models can be adapted to these sparse patterns using lightweight LoRA tuning. Experiments on FLUX.1-dev, LLaMA-3.1-8B, and Qwen3-8B show that INTRA achieves favorable quality–latency trade-offs, often outperforming prior sparse baselines in efficiency while maintaining strong downstream performance. Overall, the paper makes a practical contribution at the intersection of sparse attention design, GPU-aware implementation, and pretrained-model adaptation.

Critical strengths

1)The main originality lies in the CQS/ISPD formulation, which turns hardware constraints into part of the sparse-attention design space itself. Rather than proposing another heuristic sparse mask, the paper provides a structured abstraction that jointly specifies query grouping, index mapping, and block alignment for FlashAttention-style execution. 2)A second contribution is the Scatter/Gather relay design with explicit coverage/overlap objectives. This gives a principled way to build static long-range communication paths under structured sparsity, instead of relying on ad hoc sparse patterns, and is one of the paper’s clearest methodological innovations.

Critical weaknesses

1)The conceptual novelty is incremental, with substantial overlap with established factorized and local–global sparse attention. The main originality lies in hardware-aware implementation and pretrained-model adaptation; comparisons with the closest static designs remain limited. 2)The theoretical analysis establishes connectivity and coverage properties, but their connection to information preservation and downstream quality remains insufficiently demonstrated. 3)The quality–efficiency evaluation is not fully aligned across configurations. Kernel-level speedups alone do not establish end-to-end advantages at comparable model quality. 4)Quality preservation is task-dependent, and the evaluation does not fully establish generalization to free-generation long-context tasks or more demanding generation settings. Broader validation and uncertainty estimates would strengthen the empirical claims.

Questions for authors

1)Could you clarify what is methodologically new beyond existing factorized and local–global attention, and how the benefit of the proposed pattern can be separated from kernel optimization and adaptation? 2)How do the theoretical connectivity and coverage properties translate into information preservation and downstream quality in the implemented patterns? 3)Could you clarify whether quality and timing evaluations use the same configurations, and whether the efficiency advantage persists in end-to-end comparisons at comparable quality? 4)To what extent does retrieval performance carry over to free generation, and is generation quality preserved in the settings used to demonstrate acceleration? How stable are the quality results across runs?

5 comments

Response to Reviewer

We thank the reviewer for their careful evaluation and constructive feedback. We address each of the reviewer's concerns below.

W1 / Q1 — Novelty and separation of contributions

Reviewer — Critical Weakness 1

The conceptual novelty is incremental, with substantial overlap with established factorized and local–global sparse attention. The main originality lies in hardware-aware implementation and pretrained-model adaptation; comparisons with the closest static designs remain limited.

Reviewer — Question 1

Could you clarify what is methodologically new beyond existing factorized and local–global attention, and how the benefit of the proposed pattern can be separated from kernel optimization and adaptation?

Our key methodological contribution is the hardware-constrained formulation of sparse communication, rather than optimizing theoretical connectivity or FLOPs alone. A sparse pattern with fewer FLOPs may still perform poorly on GPUs due to irregular memory access and partially utilized computation blocks.

We do not claim that Scatter/Gather interleaving is our main novelty. As explicitly stated in the Introduction (lines 71–75), our first contribution is to formulate coverage and overlap objectives within the ISPD principle under CQS constraints. CQS specifies query coverage, structured index mappings, and GPU block alignment. Appendix D establishes the corresponding max–min coverage and overlap properties, while Section 5 connects these design criteria to structured runtime indexing and dense-tile GPU execution.

We evaluate these contributions from different aspects: Table 6 examines pattern choices, Appendix E analyzes kernel execution, and Table 2 reports end-to-end quality and latency after adaptation. Together, they demonstrate the practical value of jointly considering sparse communication and hardware efficiency.

W2 / Q2 — Connectivity versus information preservation

Reviewer — Critical Weakness 2

The theoretical analysis establishes connectivity and coverage properties, but their connection to information preservation and downstream quality remains insufficiently demonstrated.

Reviewer — Question 2

How do the theoretical connectivity and coverage properties translate into information preservation and downstream quality in the implemented patterns?

We evaluated causal reachability as table below. At 128K, our interleaving design achieves full reachability within two layers, while a window-only configuration reaches only 24.1% after 32 layers. This validates the intended structural connectivity of our design.

ContextInterleave Scatter and GatherFirst full coverageWindow-only coverage after 32 layers
16K100%Layer 2100%
32K100%Layer 276.34%
64K100%Layer 244.84%
128K100%Layer 224.10%

W3 / Q3 — Configuration alignment and end-to-end speed

Reviewer — Critical Weakness 3

The quality–efficiency evaluation is not fully aligned across configurations. Kernel-level speedups alone do not establish end-to-end advantages at comparable model quality.

Reviewer — Question 3

Could you clarify whether quality and timing evaluations use the same configurations, and whether the efficiency advantage persists in end-to-end comparisons at comparable quality?

Thanks for pointing this out. We clarify the language evaluation settings and provide new full-model prefill measurements using the same attention configurations as the quality evaluations.

For NIAH, we use block-level Scatter with G=2, achieving 0.923 mean retrieval accuracy over 4K–32K. With the same configuration, we measure 1.214× and 1.453× full-model prefill speedups at 16K and 32K, respectively.

For SCROLLS, we use block-level Scatter with G=8. Table 3(a) originally measured token-level G=8 attention. Both have nominal 1/8 grouped sparsity but different masks. We have since measured the actual block-level G=8 operator, obtaining similar operator latency. More importantly, we now benchmark full-model prefill using the SCROLLS configuration on A100:

Method8K (ms)Speedup16K (ms)Speedup32K (ms)Speedup
Dense (FA2)675.61.00×1513.51.00×3782.61.00×
INTRA (SCROLLS G=8)592.01.14×1160.41.30×2337.01.62×
XAttention (S=16)673.21.00×1389.41.09×3126.51.21×
MInference1838.90.37×2898.80.52×5008.80.76×

These results show that INTRA's prefill advantage persists at the full-model level under the SCROLLS quality configuration, rather than only in isolated kernel benchmarks. Table 3(b) reports comparable aggregate SCROLLS quality: INTRA scores 36.01, versus 36.88 for XAttention and 36.44 for MInference, although INTRA trails them on five of six tasks.

For image generation, Table 2 already provides matched end-to-end quality and latency results. At 2048×2048, INTRA achieves 1.53× end-to-end speedup over dense FLUX.1-dev (43.72 s vs. 66.78 s), with better FID than CLEAR (15.67 vs. 32.46).

We will revise the tables and captions to clearly distinguish single-operator latency, full-model prefill latency, and end-to-end generation, and explicitly link each timing result to its quality configuration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.