- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MASK-ON, a compact model merging method that enables both AR and diffusion generation while keeping the AR model's parameters unchanged. Experiments on four AR-diffusion model pairs and four benchmarks show that MASK-ON achieves the highest average accuracy among the compared merging methods, supporting both generation modes with modest additional storage.
Critical strengths
-
Extensive experimental evaluation.
- The method is evaluated against various model-merging baselines across multiple models and benchmarks.
-
Analysis of the proposed method.
- Ablation studies examine the impact of individual components on performance.
Critical weaknesses
-
Limited novelty.
- Although MASK-ON introduces modifications to scale fitting and reconstruction-error-based pruning selection, its core design builds on existing methods. The extent to which these extensions constitute novel methodological contributions remains unclear.
-
Insufficient analysis of the relationship between the problem and method design.
- The paper does not sufficiently analyze which AR-to-diffusion-specific challenges motivate each proposed modification. Moreover, comparisons with T-Switch are not conducted under equal storage budgets, leaving the performance--storage trade-off insufficiently examined.
Questions for authors
Please clarify MASK-ON's novelty over existing methods and how its design addresses the challenges of the target problem.
-
What specific limitations does T-Switch have in this setting, and how does each proposed modification address them? What are the distinct methodological contributions of MASK-ON over existing methods?
-
What challenges are specific to the AR-to-diffusion setting, and why are the proposed design choices necessary to address them?
Final Remarks
We sincerely thank the Area Chair and all reviewers for their careful and constructive feedback on our submission, our method for retaining autoregressive (AR) and diffusion generation with an unchanged AR checkpoint and a compact AR-to-diffusion update.
We are grateful that the reviews recognized strengths of our work, including:
We addressed the reviewers’ main concerns with new experiments and revisions to the manuscript:
We also analyzed the structure of the ternary mask (Reviewer UywZ Q4). With fitted scales, even random positions score within about one point of Mask-On, which indicates that the fitted scales, the part that Mask-On adds, carry much of the recovery (Appendix E.4).
In summary, the new results show that one unchanged AR checkpoint and a compact update, fitted from calibration data alone, recover diffusion performance on the dense models and let both modes share one backbone at inference. The reviewers’ follow-up responses indicate that the discussion resolved many of their questions and concerns, and we have responded to the points that remain open. We believe the revised submission now states this contribution more clearly and supports it with direct evidence. We are deeply grateful to the Area Chair and reviewers for the feedback that led to these improvements.
Best regards,
Authors