- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PrePair, a training framework for speculative drafting in Generative UI under component-level semantic acceptance. The authors identify a limitation of conventional token-level speculative decoding, where semantically valid UI components may be rejected because their token sequences differ from those of the target model.
To address this issue, PrePair combines a tree-structured semantic verifier with a preserve-and-repair training strategy. Accepted components are reinforced through a policy-gradient objective, while rejected components receive selective knowledge distillation at divergent token positions identified through chain repair. The two objectives operate on disjoint supervision positions to avoid conflicting training signals.
Experiments with a Qwen3-1.7B drafter and a Qwen3-32B target demonstrate improvements in semantic acceptance from 81.0% to 87.8% over SFT. The complete system achieves a 3.49× end-to-end speedup over direct target decoding, compared with 2.33× for retrained EAGLE-3, while improving the average Macaron-A2UI quality score from 3.65 to 3.74 over SFT.
Critical strengths
- Well-motivated problem formulation and interesting methodology.
The paper identifies a meaningful mismatch between token-level agreement and semantic correctness in structured generation. The observation that many semantically accepted UI components differ from the target's outputs provides clear motivation for reconsidering conventional speculative decoding objectives. The proposed preserve-and-repair framework is intuitive and technically interesting, particularly its separation of reinforcement for accepted components and selective distillation for rejected ones.
- Promising efficiency improvements and clear presentation.
The reported 3.49× end-to-end speedup is substantial, and the comparisons with EAGLE-3 and token-level acceptance demonstrate the practical potential of semantic speculative decoding. The paper also evaluates output quality using independent models and provides useful ablations. Overall, the presentation is clear, and the figures effectively illustrate the proposed verification and training procedures.
Critical weaknesses
- The contribution of disjoint supervision is not sufficiently isolated from other optimization choices.
A central methodological claim of PrePair is that assigning preservation and correction objectives to disjoint positions avoids conflicting training signals and improves drafter performance. However, the "plain sum" baseline in Section 4.4 modifies several optimization choices simultaneously, including the treatment of negative advantages, the reward margin term, and chain-position weight normalization. Consequently, its substantial performance degradation cannot be attributed specifically to overlapping supervision. Although the existing ablations demonstrate the effectiveness of the overall training strategy, they provide limited evidence for the independent contribution of disjoint supervision, which is important for establishing the novelty of the proposed method relative to existing selective distillation approaches.
Suggested experiments: I encourage the authors to conduct a controlled ablation comparing the original PrePair objective with a variant that additionally applies target distillation to accepted components, while keeping the reward formulation, optimization settings, and training budget consistent. Reporting acceptance rates, output quality, and repair efficiency for both variants would directly clarify whether avoiding distillation on semantically accepted components is responsible for the observed improvements.
- The reliability of semantic verification on realistic UI errors remains insufficiently established.
PrePair relies heavily on the semantic verifier to determine which components should be preserved and which require correction. However, Appendix G primarily evaluates three content-related criteria using synthetic defects constructed from 107 accepted content components. While the reported results are encouraging, they provide limited evidence about naturally occurring generation errors, especially those involving dialogue state and interactive functionality. The authors also acknowledge that these context-dependent criteria are less reliable. This raises uncertainty about whether improvements in verifier acceptance consistently reflect better semantic correctness. In particular, although acceptance increases from 81.0% to 87.8%, the L2 task-construction score remains unchanged at 1.95 relative to SFT, suggesting that improvements may be uneven across different aspects of UI quality.
Suggested experiments: I encourage the authors to evaluate the verifier on a representative sample of naturally occurring accepted and rejected components, using independent semantic correctness annotations. Reporting false-accept and false-reject rates, particularly for state continuity and interactivity, would provide stronger evidence of verifier reliability. An additional breakdown of acceptance improvements against task-level quality would help clarify the relationship between verifier judgments and actual UI correctness.
Questions for authors
Q1. Can the authors provide a controlled ablation that isolates the contribution of disjoint supervision from other optimization choices? In particular, how does PrePair compare with a variant that also distills accepted components under otherwise identical training settings?
Q2. How reliable is the semantic verifier on naturally occurring UI generation errors? Could the authors provide false-accept and false-reject statistics for state continuity and interactivity, and clarify why the higher semantic acceptance rate does not lead to an improvement in the L2 task-construction score?