acceptodds
Under review as a conference paper at ICLR 2027

ThunderWorld: Efficient World Model Generation with Selective FP4 Refinement via Native GEMMs

12 views

Abstract

We study whether world models can recover high-quality generation through dense four-bit floating-point (FP4) matrix computation without sparse matrix instructions or token/channel reordering. ThunderWorld is a training-free framework that corrects four-bit weight-and-activation (W4A4) error using additional dense FP4 products formed from quantization residuals. Block-aligned supports jointly govern residual storage, online preparation, and dense execution under a compute budget, preserving every base attention block. Controlled attention diagnostics support structure-informed selection over random and window policies at matched repair computation. On Cosmos3 Edge and Nano across RBench and PAI-Bench-G, a configuration capped at twice base W4A4 matrix work keeps mean task-score gaps within 0.08 RBench points and 0.37 PAI-Bench-G percentage points of BF16. It reduces LPIPS, a perceptual distance to same-input, same-seed BF16 generations, by 3.3–7.8% versus ARCQuant (2.0), with 1.17–1.88× its throughput on RTX PRO 6000 and Jetson AGX Thor. These results provide a numerical and systems basis for world-model inference on devices with native FP4 Tensor Cores.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
World Models, FP4 Quantization, Low-Precision Inference, Video Generation, Mixed-Precision Refinement, Tensor Cores, Efficient Inference, GPU Systems

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hKJkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to make 4-bit (W4A4, NVFP4) inference of Cosmos 3 world models more accurate without sparse kernels or token reordering. The core idea is to quantize the leftover quantization error into extra FP4 matrices and add a few FP4 products that correct the base result. Applying these corrections everywhere is costly. The method therefore repairs only selected tiles under a fixed compute budget, guided by attention patterns in Cosmos 3 and by offline calibration. The same selection also decides which residuals are stored and packed, which reduces memory traffic. The evaluation uses Cosmos 3 Edge (4B) and Nano (16B) on RBench and PAI-Bench-G. At the 2× budget, ThunderWorld gives 3.3 to 7.8% lower LPIPS than ARCQuant (2.0) and 1.17 to 1.88× higher throughput on RTX PRO 6000 and Jetson AGX Thor, with task scores close to BF16.

Critical strengths

S1. Careful and honest evaluation. The evaluation uses about 1,700 scenes per model with paired seeds, bootstrap intervals, and multiple-comparison correction. The controlled study in Appendix A.2 shows that a few extra FP4 products can match native FP8 quality, which is a useful finding on its own. The paper also reports its negative results openly, which makes its claims trustworthy.

S2. Real end-to-end speed measurements. The paper reports full-request latency, kernel-time breakdowns, and weight memory on both a server GPU and an edge device, rather than GEMM speedups alone. It shows that the data preparation around GEMMs, rather than the GEMMs themselves, dominates the cost of existing FP4 correction methods. This is useful for deploying world models on edge hardware.

Critical weaknesses

W1. The method gives back the speed advantage of FP4. Each residual plane recovers part of the precision that FP4 drops, so bilateral repair essentially emulates a higher-precision product using extra FP4 work. At the 2× budget, the matrix work is roughly equal to native FP8, since FP4 has nominally about twice the peak throughput of FP8. Yet FP8 stays more accurate in every setting. The paper never explains why “W4A4 plus repair” is a better design point than simply using FP8, or than mixed designs such as FP4 weights with FP8 activations, which keep most of the memory savings. Without this justification, the core premise of the paper is unclear.

W2. FP8 latency is missing. Native FP8 latency is not reported in Table D.1 or Figure 6, even though FP8 is more accurate than every FP4 method in Table 2. This measurement is needed to judge whether ThunderWorld offers a better accuracy-speed trade-off than FP8.

W3. The baselines may be unoptimized. The speedup over ARCQuant and SharQ comes mostly from data preparation overhead in the authors’ own adaptations of these LLM methods. ARCQuant (1.5) adds only 0.5× extra linear work, yet it runs 1.46 to 2.10× slower than plain W4A4. This points to an implementation issue rather than an inherent cost of the baseline methods.

W4. The speedup is shown only in a narrow setting. All tests use a low 320×192 resolution with batch size one, a setting where the overheads around GEMMs are large. The gain is already smallest when GEMMs dominate (1.17× for Nano on RTX PRO 6000), so it may shrink at realistic resolutions.

W5. The attention analysis is not tested in full generation. Section 3 is the main motivation for tile selection, but it is evaluated only with a local attention-error diagnostic. That diagnostic also uses a different selector from the one deployed in the final method.

W6. Ablations are missing. No full-generation experiment separates the effects of tile selection, attention repair, and offline weight calibration. It is therefore unclear which part of the method produces the gains.

W7. The accuracy gain over ARCQuant is small. At the 2× budget, the LPIPS gain over ARCQuant (2.0) is 3.3 to 7.8%, and it is not statistically significant on Edge RBench after correction. At the 1.5× budget, ThunderWorld is significantly worse than ARCQuant (1.5) in all four settings.

W8. Task scores are missing for the main baselines. Task scores are reported only for BF16, FP8, W4A4, and ThunderWorld. Without scores for ARCQuant and SharQ, it is unknown whether the LPIPS gain matters in practice.

Questions for authors

Q1. What are the latencies of native FP8, BF16, and NVFP4-AWQ under the Table D.1 protocol on both devices?

Q2. Do the W4A4, ARCQuant, and SharQ runs use the same CUDA Graphs, fused kernels, and tuning as ThunderWorld? Why is ARCQuant (1.5) 1.46 to 2.10× slower than W4A4?

Q3. At the 2× budget, how does full-generation quality change if you (a) replace your selector with random or current-mass-only selection, (b) remove attention repair, or (c) remove offline weight calibration?

Q4. Can you report task scores for ARCQuant (2.0), SharQ, and NVFP4-AWQ? Is the 8B judge sensitive enough, given that it cannot separate your 1.5× and 2× settings?

Q5. Do the results hold when development and calibration scenes are excluded from the test set?

Q6. Why is SVDQuant, a W4A4 method for diffusion transformers, not included as a baseline?

Q7. Does the speedup hold at higher resolution or larger batch sizes?

12 comments
AuthorsOpenReview ↗

Response to Reviewer hKJk (1/5)

Thank you for the detailed review. W1, W2 and W3 were correct: at this scale FP8 is the better design point for latency, the submission did not report FP8 latency, and our baselines did not run under the same execution conditions as ThunderWorld. All three change what the revised paper claims. We answer in your order; Appendix F of the revised PDF repeats these answers next to your comments.

W1. Repair gives back FP4's speed advantage; why W4A4 plus repair rather than FP8 or W4A8?

At this scale, FP8 is faster than every repaired configuration and more faithful than our 1.5× and 2× configurations, so for latency FP8 is the better choice, and the paper now says so. We measured why the repaired path is slower and fixed most of it (Section 5.4, Appendix D.6).

Table D.9 and Appendix D.6. Per-GEMM traces of one request on an exclusive B200, before the fixes.

In-situ measurementDense FP4 GEMM (2-SM tile)Repaired FP4 GEMM (1-SM tile)
Time vs FP8 GEMM, Super shapes0.51–0.65×—
Time vs dense FP4, Super1×2.9–3.5× for 1.38–2.2× the MMA work
Share of peak31–40% of FP4 peak (FP8: 34–41% of its peak)14–19% of FP4 peak
Tile2-SM 256×128×2561-SM 128×128×128, 1.23–1.39× slower
Cost per K128 stagebase stageabout 3× a base stage

Takeaway: unrepaired FP4 with BF16 attention is 0.76–0.95× FP8 latency. The gap comes from the repair kernel, whose 1-SM tile waited on residual scale loads, not from the FP4 tensor path. It is largest where requests are GEMM-bound: repair adds 61% to unrepaired FP4 on Cosmos3-Super, against 30% on Edge at 320×192.

Appendix D.6. Kernel and runtime changes, end-to-end latency on an exclusive B200 at 600 W.

ChangeBitwise identicalLatency effectKept
TMA-loaded residual scalesyesHybrid 1.5× −2.0% to −11.9%yes
One-pass residual packingyesHybrid 1.5× −0.7% to −3.4%yes
CFG-branch batching, all methodsFP8/BF16 yes; FP4 no (LPIPS within 0.01)every method 2–10% faster at 320×192yes
K256 repair tileyes+3.0% Super 320, +2.7% Edge 320, neutral Nano and Super 640, about −1% Edge 1280no
Selector folded into packingyes+0.4% to +0.8%no

Takeaway: the first three changes are kept. The K256 tile does not help because activation-repair masks are scattered, so most K256 pairs carry a zero-scaled half. A 2-SM repair collective was not attempted.

AuthorsOpenReview ↗

Response to Reviewer hKJk (2/5)

W1 (continued).

Table D.10. Exclusive B200 at 600 W, n = 18 per arm (Super 640: n = 12). Latency is a ratio to FP8 in the same campaign; best FP8 is the faster of batched and unbatched FP8.

WorkloadHybrid W4A4 / FP8Hybrid 1.5× before fixesHybrid 1.5× fixed / unbatched FP8Hybrid 1.5× fixed / best FP8Hybrid 2× fixed / best FP8
Edge 320×1920.9151.1780.9661.0561.099
Nano 320×1920.8721.2261.0081.0891.190
Super 320×1920.7761.2621.0171.0411.185
Super 640×3840.818—1.0611.061—
Edge 1280×7040.9451.0521.0191.0241.047

Takeaway: after the fixes, repaired FP4 is 2–9% slower than the best FP8. The unbatched column gives CFG batching to ThunderWorld only, so it does not show ThunderWorld to be faster.

Table C.10. LPIPS to BF16 (lower is better) on 64 scenes × 2 seeds per benchmark, paired bootstrap with family-wise 95% intervals against FP8.

ConfigEdge RBenchEdge PAINano RBenchNano PAI
Native FP8.127.088.108.081
ThunderWorld 1.5×.203 (worse).167 (worse).151 (worse).126 (worse)
ThunderWorld 2×.178 (worse).141 (worse).127 (worse).102 (worse)
ThunderWorld 2.5×.140 (n.s.).104 (worse).114 (n.s.).086 (n.s.)
Full three-term.092 (better).059 (better).092 (better).069 (better)

Takeaway: using FP4 MMAs only, three-term repair beats FP8 in all four settings and 2.5× matches it in three. Both are slower than FP8.

What the design offers is memory (static weights 31–34% below FP8; peak device memory 5–9% below FP8 with matched fusion) and the fidelity range above, built from FP4 products only.

Table D.12. Device memory on an exclusive B200 at 320×192 (GiB). Quantized linears are the 168 generation-tower linears, including codes, scales, residual banks and metadata. Request peak is peak allocated memory per request, as in Table 4.

Component (GiB)BF16 EdgeFP8 EdgeHybrid 1.5× EdgeFP8 NanoHybrid 1.5× Nano
Quantized linears2.6251.3130.8646.4694.255
Whole transformer6.2854.9734.52421.82819.614
VAE1.3341.3341.3341.3341.334
Request peak7.8566.5436.14223.53521.410
Peak reserved9.0518.0357.56624.96122.973

Takeaway: the quantized linears are 34% smaller than with FP8 (67% smaller than with BF16). The request peak falls only 6–9% vs FP8, diluted by the unquantized understanding tower, output head and embeddings (3.7 GiB Edge, 15.4 GiB Nano) and the 1.33 GiB VAE, which no method quantizes, FP8 included. On Super the linears shrink 34.3% and the request peak 10.7% (Super table), so the saving grows with model size.

W4A8. Our runtime has no native packed W4A8 kernel, and the available stand-ins either give up the weight-storage saving or use BF16 GEMMs, so we report no W4A8 number (Appendix D.4).

AuthorsOpenReview ↗

Response to Reviewer hKJk (3/5)

W2. FP8 latency is missing.

Now reported, and FP8 is faster. We give FP8 and BF16 the peripheral fusions that our Edge recipes carry, which the submission omitted. Tables D.2 and D.5 add unfused FP8 and BF16, W1 gives the post-fix ratios, and the submission's Figure 6 is now Figure D.3. We use no timing recorded on a shared GPU.

Table 4. Median latency per complete request at 320×192, batch 1, n = 30 per method. B200 is an exclusive GPU at the site's 600 W cap; Thor is measured on the whole board. FP8 and BF16 get the same peripheral fusions as our Edge recipes.

MethodB200 Edge (s)B200 Nano (s)Thor Edge (s)Thor Nano (s)
BF16 + fusion0.8442.6305.97420.615
FP8 + fusion0.8492.1595.49613.849
ThunderWorld 1.5×1.1532.9027.20421.195
ThunderWorld 2×1.2243.3367.876not run
Hybrid 1.5×1.0002.6476.31519.004
Hybrid 2×1.0763.0776.89622.022

Takeaway: FP8 is fastest everywhere. Hybrid 1.5× (FP4 repaired projections, BF16 attention) is 15–37% slower; all-FP4 1.5× is 31–53% slower.

W3. The baselines may be unoptimized.

Correct; they were not execution-matched. Our ARCQuant and SharQ adapters ran eagerly, because the official ARCQuant path needs a host-side scalar that our adapter supports only eagerly. They also ran without peripheral fusion, while ThunderWorld replayed CUDA Graphs (Table D.7).

  • We withdraw the 1.17–1.88× throughput advantage and its attribution to repair organization.
  • We did not run an execution-matched comparison: our server GPUs were shared, and porting the baselines to B200 was outside the scope of the discussion period.
  • The paper makes no latency claim against ARCQuant or SharQ and instead compares against FP8 and BF16 under matched execution.

W4. Speedup shown only at 320×192 and batch 1.

Fidelity claims are now limited to 320×192 and batch 1. Against FP8, the gap narrows as GEMMs grow but does not close. We did not measure batch sizes above 1.

Table D.3. Cosmos3 Edge on an exclusive B200, before the fixes; latency as a ratio to fused FP8. The 320×192 plans are reused at the larger sizes.

Configuration320×192640×3841280×704
FP8 + fusion (s)0.8493.58520.188
Hybrid W4A40.910.920.95
Hybrid 1.5×1.181.101.05
Hybrid 2×1.271.191.10

Takeaway: the gap narrows with resolution. After the fixes, Hybrid 1.5× is 1.024× the best FP8 at 1280×704 (Table D.10).

Cosmos3-Super at 320×192 on an exclusive B200 at 600 W. Latency ratios are to FP8 of the same campaign (no fusion recipe). Before the fixes: FP8 6.625 s, n = 30 (Table D.4). After the fixes: n = 18, or n = 12 at 640×384 (Table D.10). Memory is request-time (Table D.12). LPIPS is against B200 BF16 on 6 videos.

ArmBefore fixes / FP8Fixed / best FP8, 320 / 640Quant. linears (GiB)Request peak (GiB)LPIPS
FP81.0001.000 / 1.00029.0692.70.074
BF161.318—58.13121.8—
Hybrid W4A40.786—18.1982.00.131
Hybrid 1.5×1.2621.041 / 1.06119.1082.80.099
Hybrid 2×1.5241.185 / —20.9284.70.077

Takeaway: on the largest model, unrepaired FP4 is faster than FP8. The fixed Hybrid 1.5× is 4–6% slower than the best FP8 (1.017× / 1.061× unbatched FP8). Its quantized linears are 34.3% smaller and its request peak 10.7% smaller than FP8 (32.0% below BF16). Our on-device preparation stages the BF16 checkpoint, so every arm briefly reaches about 121 GiB at load time; this is excluded above. The earlier PRO 6000 smoke figures did not include the full model and are withdrawn.

Reviewer_X9ipOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ThunderWorld is a training-free method for recovering quality lost during W4A4 world-model inference. It recursively splits weights and activations into base and residual FP4 components, then adds dense FP4 GEMMs to correct the base result. To control the cost, correction is limited to selected matrix tiles. Attention selection combines AR–DM directionality, layer, head, and denoising phase with current attention mass and value-tile magnitude. Linear and FFN selection uses activation-residual energy weighted by offline channel sensitivity. The selected tiles also determine residual storage, online packing, and execution. On Cosmos3 Edge and Nano, the configuration capped at twice the base W4A4 matrix work reduces LPIPS by 3.3–7.8% and delivers 1.17–1.88× the throughput of ARCQuant (2.0), with mean task scores close to BF16.

Critical strengths
  1. The quality–cost tradeoff looks good. At the same nominal 2× matrix-work budget, ThunderWorld improves fidelity and throughput over ARCQuant and SharQ. Static weight storage is also 31–34% lower than FP8, while mean task scores stay close to BF16.
  2. The implementation choices are practical for hardware with native FP4 Tensor Cores. The method keeps the full base computation and adds regular dense FP4 tiles, without parameter training, sparse matrix instructions, N masks, or token/channel reordering.
Critical weaknesses
  1. The missing BF16 and native-FP8 latency results make the practical speedup hard to judge. Their generation quality is reported, but end-to-end timings under the same hardware and generation settings are not. The results show an advantage over other FP4 refinement methods; whether the 2× FP4 configuration beats BF16 or native FP8 remains unclear.
  2. The offline calibration setup needs more detail. Policies are calibrated using BF16 development trajectories, but the final data, sampling procedure, and per-model cost are not clearly reported. The full benchmark also includes scenes used during development, so the extent of calibration/evaluation overlap and its effect on the results are unclear.
Questions for authors
  1. What exact calibration data produced the final 1.5× and 2× policies? Please report scenes and trajectories per model, dataset and task composition, seeds, sampled denoising steps or operator outputs, calibration cost, and overlap with the final benchmarks.
  2. The current FP4 attention implementation is slower than the BF16 attention used by the baselines. Where does most of the time go: FP4 MMA utilization, quantization and packing, or tile selection and scheduling?
4 comments
AuthorsOpenReview ↗

Response to Reviewer X9ip (1/3)

Thank you for the review. We answer in your order; the general response summarizes the revision, and Appendix F of the revised PDF repeats these answers next to your comments.

W1. No BF16 or native-FP8 latency; does 2× FP4 beat BF16 or FP8?

No. With matched execution, FP8 and BF16 are both faster than ThunderWorld 2×, and FP8 is faster than every repaired configuration. In the submission, FP8 and BF16 lacked the peripheral fusions that our Edge recipes carry; Table 4 now gives them the same fusions.

Table 4. Median latency per complete request at 320×192, batch 1, n = 30 per method. B200 is an exclusive GPU at the site's 600 W cap; Thor is measured on the whole board. FP8 and BF16 get the same peripheral fusions as our Edge recipes.

MethodB200 Edge (s)B200 Nano (s)Thor Edge (s)Thor Nano (s)
BF16 + fusion0.8442.6305.97420.615
FP8 + fusion0.8492.1595.49613.849
ThunderWorld 1.5×1.1532.9027.20421.195
ThunderWorld 2×1.2243.3367.876not run
Hybrid 1.5×1.0002.6476.31519.004
Hybrid 2×1.0763.0776.89622.022

Takeaway: FP8 is fastest everywhere. Hybrid 1.5× (FP4 repaired projections, BF16 attention) is 15–37% slower; all-FP4 1.5× is 31–53% slower.

Table D.10. Exclusive B200 at 600 W, n = 18 per arm (Super 640: n = 12). Latency is a ratio to FP8 in the same campaign; best FP8 is the faster of batched and unbatched FP8.

WorkloadHybrid W4A4 / FP8Hybrid 1.5× before fixesHybrid 1.5× fixed / unbatched FP8Hybrid 1.5× fixed / best FP8Hybrid 2× fixed / best FP8
Edge 320×1920.9151.1780.9661.0561.099
Nano 320×1920.8721.2261.0081.0891.190
Super 320×1920.7761.2621.0171.0411.185
Super 640×3840.818—1.0611.061—
Edge 1280×7040.9451.0521.0191.0241.047

Takeaway: after the fixes, repaired FP4 is 2–9% slower than the best FP8. The unbatched column gives CFG batching to ThunderWorld only, so it does not show ThunderWorld to be faster.

ThunderWorld's measured benefit is memory: peak device memory is 5–9% below FP8 and 21–29% below BF16, and static weights are 31–34% below FP8. We use no timing recorded on a shared GPU.

Changed in the PDF: abstract, contributions, Section 5.4 with Table 4, Appendices D.2, D.3 and D.6, and the conclusion.

AuthorsOpenReview ↗

Response to Reviewer X9ip (2/3)

W2. Calibration setup and calibration/evaluation overlap.

Both points are correct. New Appendix C.4 describes the calibration data (Q1) and the overlap. 137 of the 1,694 benchmark scenes (72 RBench, 65 PAI-Bench-G) had a prior calibration, development or selection role, counting decoded-pixel-identical aliases.

Table C.7. 1,557 scenes after removing the 137 with any prior role. LPIPS, ThunderWorld minus ARCQuant at the same budget, Bonferroni-12.

Setting2× − ARCQuant (2.0)Significant?1.5× − ARCQuant (1.5)
Edge RBench−0.0051no+0.0004, n.s.
Edge PAI−0.0096yes+0.0004, n.s.
Nano RBench−0.0067yes+0.0039, n.s. after correction
Nano PAI−0.0104yes+0.0116, worse

Takeaway: the conclusions match the full set. This is a retrospective exclusion, not a held-out test.

The 64-scene subsets behind Tables C.10, C.11 and C.8 are also drawn from the benchmarks.

Q1. What calibration data produced the 1.5× and 2× policies?

Edge and Nano are calibrated independently on the same 16 scenes, with one BF16 trajectory per scene (Appendix C.4). The scenes are the calibration split of a frozen cohort (16 calibration / 16 development / 32 validation / 72 final per model).

Appendix C.4. Calibration lineage per model.

ItemEdgeNano
Scenes16 (9 RBench + 7 PAI-Bench-G)same 16
Trajectories / seeds16 / 301001–301016same
Teacher panels2,688 (168 modules × 16)4,032 (252 × 16)
Joint-search modules166249
Prescale exponents0.5 ×100, 0.75 ×62, 1.0 ×4, identity ×20.5 ×143, 0.75 ×105, 0.25 ×1, 1.0 ×1, identity ×2

Takeaway: every scene is I2V at 320×192, 61 frames, 35 steps, CFG 6. Each trajectory gives one capture per operator at one (phase, CFG branch) pair, and each of the 8 pairs appears twice.

Appendix C.4. Reproduction of the Edge 1.5× calibration from scratch, with the unchanged tools and the original 16-scene cohort, on one exclusive B200.

StageTime
Model load30 s
RTN asset bank7 s
16 calibration trajectories390 s
Plan compile (CPU)29 s
Total7.6 min (0.13 GPU-hours), 10.5 GB peak

Takeaway: the reproduction matches the shipped linear plan exactly (all 1,344 entries, the same 4,000 W1 blocks, bitwise-identical quantized weights, scales and residual banks). 2% of attention head assignments swap between half and full repair at identical compute. The historical cost was not recorded, so this is a measured reproduction cost, not the original cost.

The calibration scenes are among the 137 excluded under W2. In the full-generation ablations, the calibrated prescale has no significant effect on either model (Table C.11), and we reduced the claims about it.

AuthorsOpenReview ↗

Response to Reviewer X9ip (3/3)

Q2. Where does the time go in FP4 attention?

Mostly in quantization and packing around softmax, then in repair and tile selection. We did not measure FP4 MMA utilization inside the attention kernel. In a traced Edge request, FP4 attention takes 233 ms against 84 ms with BF16 attention (Table D.8).

Table D.11. Attention kernels on an exclusive B200 at 600 W: 48 real inputs per model, one CUDA graph per method (including packing, selection, softmax, P quantization and combine), median.

Attention pathEdge ms (×BF16)Nano ms (×BF16)NMSE Edge / Nano
BF16 flash0.0408 (1.00)0.0649 (1.00)0 / 0
FP4, unrepaired0.0667 (1.64)0.1062 (1.64)0.0227 / 0.0377
FP4, full three-term0.1016 (2.49)0.1642 (2.53)0.00033 / 0.00067
FP4, full four-term0.1022 (2.51)0.1667 (2.57)0.00015 / 0.00052
FP4, deployed 1.5×0.1163 (2.85)0.1804 (2.78)0.00083 / 0.00123

Takeaway: FP4 attention costs latency in every configuration. Packing and P quantization already cost 1.64×, repair raises it to about 2.5×, and tile selection to 2.8–2.9×. We therefore recommend BF16 attention (the hybrid).

For the repaired projections, where the remaining gap to FP8 lies, we did measure utilization and the time split. Repair-specific activation preparation adds 75 ms per Edge request at 320×192 (Table D.8).

Table D.9 and Appendix D.6. Per-GEMM traces of one request on an exclusive B200, before the fixes.

In-situ measurementDense FP4 GEMM (2-SM tile)Repaired FP4 GEMM (1-SM tile)
Time vs FP8 GEMM, Super shapes0.51–0.65×—
Time vs dense FP4, Super1×2.9–3.5× for 1.38–2.2× the MMA work
Share of peak31–40% of FP4 peak (FP8: 34–41% of its peak)14–19% of FP4 peak
Tile2-SM 256×128×2561-SM 128×128×128, 1.23–1.39× slower
Cost per K128 stagebase stageabout 3× a base stage

Takeaway: unrepaired FP4 with BF16 attention is 0.76–0.95× FP8 latency. The gap comes from the repair kernel, whose 1-SM tile waited on residual scale loads, not from the FP4 tensor path. It is largest where requests are GEMM-bound: repair adds 61% to unrepaired FP4 on Cosmos3-Super, against 30% on Edge at 320×192.

What we changed.

Appendix D.6. Kernel and runtime changes, end-to-end latency on an exclusive B200 at 600 W.

ChangeBitwise identicalLatency effectKept
TMA-loaded residual scalesyesHybrid 1.5× −2.0% to −11.9%yes
One-pass residual packingyesHybrid 1.5× −0.7% to −3.4%yes
CFG-branch batching, all methodsFP8/BF16 yes; FP4 no (LPIPS within 0.01)every method 2–10% faster at 320×192yes
K256 repair tileyes+3.0% Super 320, +2.7% Edge 320, neutral Nano and Super 640, about −1% Edge 1280no
Selector folded into packingyes+0.4% to +0.8%no

Takeaway: the first three changes are kept. The K256 tile does not help because activation-repair masks are scattered, so most K256 pairs carry a zero-scaled half. A 2-SM repair collective was not attempted.

Tables C.8 and 4. Hybrid = the same FP4 plans with BF16 attention. LPIPS uses 64 scenes × 2 seeds; family-wise intervals.

HybridLinear MMA, Edge / Nano (×W4A4)LPIPS Edge RB / PAILPIPS Nano RB / PAIvs all-FP4 parentvs FP8
1.5×1.42 / 1.43.195 / .166.148 / .119n.s.worse
2×1.84 / 1.90.174 / .138.124 / .097n.s.worse
2.5×2.42 / 2.45.139 / .102.113 / .080Nano PAI bettern.s. in all four

Takeaway: hybrids are at least as faithful as their parents (all 12 differences favor the hybrid, one significantly) and 8–13% faster than them in Table 4.

With all kept fixes, Hybrid 1.5× is 1.024–1.089× the best FP8 (Table D.10, shown under W1).

Changed in the PDF: Section 5.4, Appendices C.5 and D.6, Tables C.8 and D.8–D.11.

Thx for the response, it resolves my concern and I hold my score unchanged.

Reviewer_khWoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies whether video world models can recover BF16-level generation quality using only dense native FP4 GEMMs. ThunderWorld corrects W4A4 quantization error with additional FP4 products formed from weight and activation residuals, but applies them only to selected matrix tiles under a fixed compute budget. Tile selection is guided by structural properties of Cosmos3 attention (directional modality blocks and depth-dependent AR context), and the selected tiles jointly determine residual storage, activation preparation, and dense execution. On Cosmos3 Edge and Nano, the 2x-budget configuration reduces LPIPS by 3.3–7.8% relative to ARCQuant with 1.17–1.88x its throughput, and uses 31–34% less weight storage than FP8.

Critical strengths
  1. The paper shows that low-order FP4 residual products recover most of the W4A4 fidelity loss. Uniform bilateral repair even exceeds native FP8 fidelity in a controlled study, which is a useful and non-obvious result for FP4 inference.
  2. Repair selection is well motivated and empirically validated. Structure-informed tile selection reduces attention-output error by 44.6–66.4% relative to random and 15.8–51.7% relative to window selection at matched compute, approaching an attention-mass oracle. Aligning selection with storage, packing, and execution is a sound system design.
Critical weaknesses
  1. The central efficiency claim is not established against the most relevant alternative. Under the actual 2x budget, native FP8 retains clearly better fidelity, e.g., LPIPS 0.119 vs. 0.172 on Edge RBench, yet no FP8 or BF16 latency is reported. It is therefore unclear when ThunderWorld should be preferred over FP8.
  2. The comparison and scope are limited. The baselines (ARCQuant [1], SharQ [2]) are LLM quantization methods adapted by the authors, while SVDQuant [3], the most directly relevant 4-bit diffusion method, is discussed but not compared. All experiments use a single model family Cosmos3 at low resolution 320x192, and the method relies on its specific AR–DM attention structure, so generality remains unclear.

[1] Haoqian Meng, Yilun Luo, Yafei Zhao, Wenyuan Liu, Peng Zhang, and Xindian Ma. ARCQuant: Boosting NVFP4 quantization with augmented residual channels for LLMs. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL), 2026.

[2] Haoqian Meng, Yilun Luo, Yafei Zhao, Wenyuan Liu, Huaqing Zheng, Xindian Ma, and Peng Zhang. SharQ: Bridging activation sparsity and FP4 quantization for LLM inference. arXiv:2606.26587, 2026.

[3] Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, and Song Han. SVDQuant: Absorbing outliers by low-rank component for 4-bit diffusion models. In International Conference on Learning Representations (ICLR), 2025.

Questions for authors

Could the authors extend the fidelity–latency comparison in Figure 1(b) into a Pareto comparison that also includes native FP8, BF16, and SVDQuant? In the same setting Cosmos3 Edge on RTX PRO 6000, it would be helpful to compare ThunderWorld at several budgets, e.g., 1.5x, 2x, 2.5x, 3x against native FP8, ARCQuant, SharQ, and SVDQuant along three axes: generation quality (LPIPS / task score), end-to-end speedup over BF16, and memory (including peak memory, not only static weight storage). This would clarify whether ThunderWorld offers a better quality–speed–memory trade-off than FP8 and existing 4-bit methods, or mainly a memory advantage.

4 comments
AuthorsOpenReview ↗

Response to Reviewer khWo (1/3)

Thank you for the review. We answer in your order; the general response summarizes the revision, and Appendix F of the revised PDF repeats these answers next to your comments.

W1. Efficiency is not established against FP8; no FP8 or BF16 latency.

We agree. With matched execution, FP8 is faster than every repaired ThunderWorld configuration and more faithful than our 1.5× and 2× configurations. At this scale, ThunderWorld should not be preferred over FP8 for latency; its measured advantage is memory. We now give FP8 and BF16 the peripheral fusions that our Edge recipes carry, which the submission omitted.

Table 4. Median latency per complete request at 320×192, batch 1, n = 30 per method. B200 is an exclusive GPU at the site's 600 W cap; Thor is measured on the whole board. FP8 and BF16 get the same peripheral fusions as our Edge recipes.

MethodB200 Edge (s)B200 Nano (s)Thor Edge (s)Thor Nano (s)
BF16 + fusion0.8442.6305.97420.615
FP8 + fusion0.8492.1595.49613.849
ThunderWorld 1.5×1.1532.9027.20421.195
ThunderWorld 2×1.2243.3367.876not run
Hybrid 1.5×1.0002.6476.31519.004
Hybrid 2×1.0763.0776.89622.022

Takeaway: FP8 is fastest everywhere. Hybrid 1.5× (FP4 repaired projections, BF16 attention) is 15–37% slower; all-FP4 1.5× is 31–53% slower.

Table 4, same runs. Peak allocated device memory.

MethodB200 Edge (GiB)B200 Nano (GiB)Thor Edge (GiB)Thor Nano (GiB)
BF16 + fusion7.8630.007.8630.00
FP8 + fusion6.5423.536.5423.53
ThunderWorld 1.5×6.1421.386.1521.88
Hybrid 1.5×6.1421.396.1421.88

Takeaway: peak device memory is 5–9% below FP8 and 21–29% below BF16. Static weights are 31–34% below FP8.

Why repaired FP4 was slower, and the fixes (Section 5.4, Appendix D.6)

Table D.9 and Appendix D.6. Per-GEMM traces of one request on an exclusive B200, before the fixes.

In-situ measurementDense FP4 GEMM (2-SM tile)Repaired FP4 GEMM (1-SM tile)
Time vs FP8 GEMM, Super shapes0.51–0.65×—
Time vs dense FP4, Super1×2.9–3.5× for 1.38–2.2× the MMA work
Share of peak31–40% of FP4 peak (FP8: 34–41% of its peak)14–19% of FP4 peak
Tile2-SM 256×128×2561-SM 128×128×128, 1.23–1.39× slower
Cost per K128 stagebase stageabout 3× a base stage

Takeaway: unrepaired FP4 with BF16 attention is 0.76–0.95× FP8 latency. The gap comes from the repair kernel, whose 1-SM tile waited on residual scale loads, not from the FP4 tensor path. It is largest where requests are GEMM-bound: repair adds 61% to unrepaired FP4 on Cosmos3-Super, against 30% on Edge at 320×192.

Table D.10. Exclusive B200 at 600 W, n = 18 per arm (Super 640: n = 12). Latency is a ratio to FP8 in the same campaign; best FP8 is the faster of batched and unbatched FP8.

WorkloadHybrid W4A4 / FP8Hybrid 1.5× before fixesHybrid 1.5× fixed / unbatched FP8Hybrid 1.5× fixed / best FP8Hybrid 2× fixed / best FP8
Edge 320×1920.9151.1780.9661.0561.099
Nano 320×1920.8721.2261.0081.0891.190
Super 320×1920.7761.2621.0171.0411.185
Super 640×3840.818—1.0611.061—
Edge 1280×7040.9451.0521.0191.0241.047

Takeaway: after the fixes, repaired FP4 is 2–9% slower than the best FP8. The unbatched column gives CFG batching to ThunderWorld only, so it does not show ThunderWorld to be faster.

Table D.6. Thor whole-board memory in use, one interleaved round, n = 6. The fix releases host copies of the FP4 weight planes after warm-up; outputs are byte-identical.

ModelFP8 + fusion (GiB)Hybrid 1.5×, fixed (GiB)Change
Edge14.2913.74−3.8%
Nano31.4328.89−8.1%

Takeaway: with the fix, whole-board memory is below FP8. Table 4 was measured before the fix.

AuthorsOpenReview ↗

Response to Reviewer khWo (2/3)

W1 (continued). Fidelity with FP4 MMAs only.

Table C.10. LPIPS to BF16 (lower is better) on 64 scenes × 2 seeds per benchmark, paired bootstrap with family-wise 95% intervals against FP8.

ConfigEdge RBenchEdge PAINano RBenchNano PAI
Native FP8.127.088.108.081
ThunderWorld 1.5×.203 (worse).167 (worse).151 (worse).126 (worse)
ThunderWorld 2×.178 (worse).141 (worse).127 (worse).102 (worse)
ThunderWorld 2.5×.140 (n.s.).104 (worse).114 (n.s.).086 (n.s.)
Full three-term.092 (better).059 (better).092 (better).069 (better)

Takeaway: using FP4 MMAs only, three-term repair beats FP8 in all four settings and 2.5× matches it in three. Both are slower than FP8.

Changed in the PDF for W1: abstract, contributions, Section 5.4 with Table 4, Appendices D.2, D.3 and D.6 (Tables D.8–D.10), and the conclusion.

W2. Adapted baselines, no SVDQuant, one model family at 320×192.

  • Baselines. You are right that the comparison was not on equal terms. Our ARCQuant and SharQ adapters ran eagerly and without peripheral fusion, while ThunderWorld replayed CUDA Graphs (Table D.7), so we withdraw the throughput claim over them. We did not run an execution-matched comparison: our server GPUs were shared, and porting the baselines to B200 was outside the scope of the discussion period. The paper makes no latency claim against them and instead compares against FP8 and BF16 under matched execution.
  • SVDQuant is orthogonal to our method: it absorbs outliers with a low-rank higher-precision branch, whereas we add FP4 residual products. We did not reproduce it on Cosmos3 within the discussion period, and the paper says so (Section 6, Appendix D.8).

Scope. Fidelity claims are now limited to Cosmos3 Edge and Nano at 320×192. Latency now also covers larger resolutions and Cosmos3-Super.

Table D.3. Cosmos3 Edge on an exclusive B200, before the fixes; latency as a ratio to fused FP8. The 320×192 plans are reused at the larger sizes.

Configuration320×192640×3841280×704
FP8 + fusion (s)0.8493.58520.188
Hybrid W4A40.910.920.95
Hybrid 1.5×1.181.101.05
Hybrid 2×1.271.191.10

Takeaway: the gap narrows with resolution. After the fixes, Hybrid 1.5× is 1.024× the best FP8 at 1280×704 (Table D.10).

Cosmos3-Super at 320×192 on an exclusive B200 at 600 W. Latency ratios are to FP8 of the same campaign (no fusion recipe). Before the fixes: FP8 6.625 s, n = 30 (Table D.4). After the fixes: n = 18, or n = 12 at 640×384 (Table D.10). Memory is request-time (Table D.12). LPIPS is against B200 BF16 on 6 videos.

ArmBefore fixes / FP8Fixed / best FP8, 320 / 640Quant. linears (GiB)Request peak (GiB)LPIPS
FP81.0001.000 / 1.00029.0692.70.074
BF161.318—58.13121.8—
Hybrid W4A40.786—18.1982.00.131
Hybrid 1.5×1.2621.041 / 1.06119.1082.80.099
Hybrid 2×1.5241.185 / —20.9284.70.077

Takeaway: on the largest model, unrepaired FP4 is faster than FP8. The fixed Hybrid 1.5× is 4–6% slower than the best FP8 (1.017× / 1.061× unbatched FP8). Its quantized linears are 34.3% smaller and its request peak 10.7% smaller than FP8 (32.0% below BF16). Our on-device preparation stages the BF16 checkpoint, so every arm briefly reaches about 121 GiB at load time; this is excluded above. The earlier PRO 6000 smoke figures did not include the full model and are withdrawn.

A four-scene check on another model family at 960×544 is consistent with the ladder ordering. It has no FP8 reference and no significance test (Appendix D.8).

AuthorsOpenReview ↗

Response to Reviewer khWo (3/3)

Q. A Pareto comparison with FP8, BF16 and SVDQuant along quality, speed and memory.

We report all three axes for 1.5×, 2×, 2.5× and full three-term repair (about 3×) against FP8 and BF16. The answer to your last question is the second option: at this scale, ThunderWorld offers a memory advantage, not a better quality–speed trade-off than FP8.

Cosmos3 Edge on an exclusive B200. LPIPS (RBench / PAI) from Tables C.10 and C.8; latency as a ratio to fused FP8 before the fixes (Tables 4 and D.2); peak memory.

ConfigurationLPIPS RB / PAILatency vs FP8Peak (GiB)
FP8 + fusion.127 / .0881.006.54
BF16 + fusionreference0.997.86
ThunderWorld 1.5×.203 / .1671.366.14
Hybrid 1.5×.195 / .1661.186.14
ThunderWorld 2×.178 / .1411.446.18
Hybrid 2×.174 / .1381.276.18
Hybrid 2.5×.139 / .1021.01*6.43
Full three-term.092 / .0591.47*7.42

Takeaway: no configuration beats FP8 on both quality and speed. * = against unfused FP8 in an earlier campaign (Table D.2); our Edge recipes carry the fusions, so these two ratios favor ThunderWorld. On Nano, three-term takes 1.93× unfused FP8.

Table D.12. Device memory on an exclusive B200 at 320×192 (GiB). Quantized linears are the 168 generation-tower linears, including codes, scales, residual banks and metadata. Request peak is peak allocated memory per request, as in Table 4.

Component (GiB)BF16 EdgeFP8 EdgeHybrid 1.5× EdgeFP8 NanoHybrid 1.5× Nano
Quantized linears2.6251.3130.8646.4694.255
Whole transformer6.2854.9734.52421.82819.614
VAE1.3341.3341.3341.3341.334
Request peak7.8566.5436.14223.53521.410
Peak reserved9.0518.0357.56624.96122.973

Takeaway: the quantized linears are 34% smaller than with FP8 (67% smaller than with BF16). The request peak falls only 6–9% vs FP8, diluted by the unquantized understanding tower, output head and embeddings (3.7 GiB Edge, 15.4 GiB Nano) and the 1.33 GiB VAE, which no method quantizes, FP8 included. On Super the linears shrink 34.3% and the request peak 10.7% (Super table), so the saving grows with model size.

  • After the fixes, Hybrid 1.5× is 1.056× the best FP8 on Edge at 320×192 and 1.024× at 1280×704 (Table D.10).
  • Our RTX PRO 6000 timings were recorded on shared GPUs, so we do not use them. The clean timings come from the exclusive B200 and from Thor, which gives the same ordering (Table D.5).
  • The 2× configuration keeps mean task scores within 0.08 RBench points and 0.37 PAI-Bench-G points of BF16 (Section 5.3).
  • We have no execution-matched ARCQuant or SharQ points and no SVDQuant point (W2), so we do not draw a combined Pareto front with them.

Changed in the PDF: Tables C.10, 4, C.8 and D.2–D.5, and Appendix F.1.

Thank you for the detailed response and the substantial additional experiments. I appreciate that the revised paper states its limitations clearly, including where FP8 is faster or more faithful.

The FP4-only ladder is an interesting result. However, in the new Pareto comparison, no configuration improves on native FP8 on more than one of quality, latency and memory, and the peak-memory saving over FP8 (5–9%) is modest. I still find it hard to see when I would choose this method over FP8 in practice, so I will keep my rating.

Thanks again for the careful work.

Comments on the paper

6 comments
AuthorsOpenReview ↗

General response (1/5)

We thank the reviewers. Several criticisms were correct, and the revision changes what the paper claims. Changed text is blue in the revised PDF, and Appendix F quotes every weakness and question with our answer.

What we concede

  1. Native FP8 is faster than every repaired ThunderWorld configuration. In the submission, FP8 and BF16 lacked the peripheral fusions (RMSNorm, pointwise, rotary tables, codec graph) that our Edge recipes carry. We now give them the same fusions and time them on an exclusive NVIDIA B200 and on Jetson AGX Thor. FP8 is faster on both models and both devices, and it is more faithful than our 1.5× and 2× configurations.
  2. The 1.17–1.88× throughput claim over ARCQuant is withdrawn. Our ARCQuant and SharQ adapters ran eagerly and without these fusions. We did not run an execution-matched comparison: our server GPUs were shared, and porting the baselines to B200 was outside the scope of the discussion period. The paper makes no latency claim against them.
  3. At matched budget, 1.5× is not better than ARCQuant (1.5). By LPIPS it is statistically indistinguishable in three of four settings and worse on Nano PAI-Bench-G (Table C.6).
  4. The benchmarks are not held out. 137 of 1,694 scenes had a prior calibration, development or selection role. Excluding them changes no conclusion (Table C.7).

Table 4. Median latency per complete request at 320×192, batch 1, n = 30 per method. B200 is an exclusive GPU at the site's 600 W cap; Thor is measured on the whole board. FP8 and BF16 get the same peripheral fusions as our Edge recipes.

MethodB200 Edge (s)B200 Nano (s)Thor Edge (s)Thor Nano (s)
BF16 + fusion0.8442.6305.97420.615
FP8 + fusion0.8492.1595.49613.849
ThunderWorld 1.5×1.1532.9027.20421.195
ThunderWorld 2×1.2243.3367.876not run
Hybrid 1.5×1.0002.6476.31519.004
Hybrid 2×1.0763.0776.89622.022

Takeaway: FP8 is fastest everywhere. Hybrid 1.5× (FP4 repaired projections, BF16 attention) is 15–37% slower; all-FP4 1.5× is 31–53% slower.

Table 4, same runs. Peak allocated device memory.

MethodB200 Edge (GiB)B200 Nano (GiB)Thor Edge (GiB)Thor Nano (GiB)
BF16 + fusion7.8630.007.8630.00
FP8 + fusion6.5423.536.5423.53
ThunderWorld 1.5×6.1421.386.1521.88
Hybrid 1.5×6.1421.396.1421.88

Takeaway: peak device memory is 5–9% below FP8 and 21–29% below BF16. Static weights are 31–34% below FP8.

AuthorsOpenReview ↗

General response (2/5)

Why repaired FP4 was slower, and what we fixed (Section 5.4, Appendix D.6)

Table D.9 and Appendix D.6. Per-GEMM traces of one request on an exclusive B200, before the fixes.

In-situ measurementDense FP4 GEMM (2-SM tile)Repaired FP4 GEMM (1-SM tile)
Time vs FP8 GEMM, Super shapes0.51–0.65×—
Time vs dense FP4, Super1×2.9–3.5× for 1.38–2.2× the MMA work
Share of peak31–40% of FP4 peak (FP8: 34–41% of its peak)14–19% of FP4 peak
Tile2-SM 256×128×2561-SM 128×128×128, 1.23–1.39× slower
Cost per K128 stagebase stageabout 3× a base stage

Takeaway: unrepaired FP4 with BF16 attention is 0.76–0.95× FP8 latency. The gap comes from the repair kernel, whose 1-SM tile waited on residual scale loads, not from the FP4 tensor path. It is largest where requests are GEMM-bound: repair adds 61% to unrepaired FP4 on Cosmos3-Super, against 30% on Edge at 320×192.

Appendix D.6. Kernel and runtime changes, end-to-end latency on an exclusive B200 at 600 W.

ChangeBitwise identicalLatency effectKept
TMA-loaded residual scalesyesHybrid 1.5× −2.0% to −11.9%yes
One-pass residual packingyesHybrid 1.5× −0.7% to −3.4%yes
CFG-branch batching, all methodsFP8/BF16 yes; FP4 no (LPIPS within 0.01)every method 2–10% faster at 320×192yes
K256 repair tileyes+3.0% Super 320, +2.7% Edge 320, neutral Nano and Super 640, about −1% Edge 1280no
Selector folded into packingyes+0.4% to +0.8%no

Takeaway: the first three changes are kept. The K256 tile does not help because activation-repair masks are scattered, so most K256 pairs carry a zero-scaled half. A 2-SM repair collective was not attempted.

Table D.10. Exclusive B200 at 600 W, n = 18 per arm (Super 640: n = 12). Latency is a ratio to FP8 in the same campaign; best FP8 is the faster of batched and unbatched FP8.

WorkloadHybrid W4A4 / FP8Hybrid 1.5× before fixesHybrid 1.5× fixed / unbatched FP8Hybrid 1.5× fixed / best FP8Hybrid 2× fixed / best FP8
Edge 320×1920.9151.1780.9661.0561.099
Nano 320×1920.8721.2261.0081.0891.190
Super 320×1920.7761.2621.0171.0411.185
Super 640×3840.818—1.0611.061—
Edge 1280×7040.9451.0521.0191.0241.047

Takeaway: after the fixes, repaired FP4 is 2–9% slower than the best FP8. The unbatched column gives CFG batching to ThunderWorld only, so it does not show ThunderWorld to be faster.

Repaired FP4 is not faster than FP8 in the revised paper; after the fixes the gap is 2–9%.

AuthorsOpenReview ↗

General response (3/5)

What the evidence supports

Table C.10. LPIPS to BF16 (lower is better) on 64 scenes × 2 seeds per benchmark, paired bootstrap with family-wise 95% intervals against FP8.

ConfigEdge RBenchEdge PAINano RBenchNano PAI
Native FP8.127.088.108.081
ThunderWorld 1.5×.203 (worse).167 (worse).151 (worse).126 (worse)
ThunderWorld 2×.178 (worse).141 (worse).127 (worse).102 (worse)
ThunderWorld 2.5×.140 (n.s.).104 (worse).114 (n.s.).086 (n.s.)
Full three-term.092 (better).059 (better).092 (better).069 (better)

Takeaway: using FP4 MMAs only, three-term repair beats FP8 in all four settings and 2.5× matches it in three. Both are slower than FP8.

Table C.11. Deployed / ablated LPIPS (lower is better) on the same subsets. A marked cell (*) is significant after family-wise correction over 32 contrasts (20,000 bootstrap replicates).

AblationEdge RBenchEdge PAINano RBenchNano PAI
2×: FP4 attention unrepaired.178/.272*.141/.222*.127/.330*.102/.378*
2×: attention budget moved to linear.178/.269*.141/.221*.127/.321*.102/.376*
1.5×: uniform head/phase quotas.203/.201.167/.179*.151/.172*.126/.142*
2×: uniform linear quotas.178/.183.141/.151.127/.134.102/.110*
2×: no weight-channel sensitivity.178/.180.141/.145.127/.128.102/.101
2×: identity prescale.178/.171.141/.143.127/.136.102/.104

Takeaway: attention repair dominates. The head/phase schedule helps in 3 of 4 settings and linear allocation in 1 of 4. Weight sensitivity and prescale show no measurable effect, and we removed those claims.

Random-tile and mass-only selector ablations are not available: the first run failed, and a re-run produced no data in time.

Tables C.8 and 4. Hybrid = the same FP4 plans with BF16 attention. LPIPS uses 64 scenes × 2 seeds; family-wise intervals.

HybridLinear MMA, Edge / Nano (×W4A4)LPIPS Edge RB / PAILPIPS Nano RB / PAIvs all-FP4 parentvs FP8
1.5×1.42 / 1.43.195 / .166.148 / .119n.s.worse
2×1.84 / 1.90.174 / .138.124 / .097n.s.worse
2.5×2.42 / 2.45.139 / .102.113 / .080Nano PAI bettern.s. in all four

Takeaway: hybrids are at least as faithful as their parents (all 12 differences favor the hybrid, one significantly) and 8–13% faster than them in Table 4.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.