W1. The proposed framework mainly extends existing XLNet and Set Diffusion architectures, with limited methodological novelty.
The core architecture of π-XLNet largely follows the two-stream design of XLNet and a specific configuration of Set Diffusion. Arbitrary-order language modeling and the use of different token prediction orders have already been extensively investigated in previous studies.
The main methodological difference appears to be the introduction of the Mallows distribution to control training-order disorder, together with a reformulation of the relationship between XLNet and Set Diffusion.
However, Set Diffusion has already explored flexible order distributions and the interpolation between autoregressive and diffusion-based generation. The paper does not sufficiently establish why the proposed Mallows distribution offers fundamental advantages over existing approaches.
Therefore, the incremental technical contribution remains somewhat limited. More direct comparisons with existing arbitrary-order modeling approaches, particularly Set Diffusion, would help clarify the novelty and necessity of the proposed framework.
W2. HBD-n is not sufficiently compared with existing test-time scaling and iterative refinement approaches.
The proposed HBD-n algorithm aims to improve generation quality by exploiting local future information through stochastic branching and backtracking. However, this objective is closely related to existing test-time scaling, search-based decoding, and iterative refinement approaches, which also improve generation quality by spending additional inference computation.
The paper mainly compares HBD-n with natural-order autoregressive decoding and official SetDiff decoding, without sufficiently evaluating representative alternatives such as Best-of-N sampling, beam search, self-consistency, or remasking-based iterative refinement.
This limitation is particularly important because HBD-n introduces substantial additional inference overhead. Without comparisons under matched computational budgets, it remains unclear whether the proposed algorithm provides a better quality–efficiency trade-off than existing inference-time scaling techniques.
W3. The claimed training-efficiency advantage is not sufficiently supported by controlled experiments.
The paper estimates that π-XLNet requires approximately 3.1–3.3 times the training cost of an autoregressive baseline to achieve comparable natural-order prediction loss. This is contrasted with the substantially higher training costs reported for masked diffusion models in previous studies.
However, these comparisons are based on different model architectures, training configurations, evaluation objectives, and computational cost measurements. Furthermore, the reported 3.1–3.3× training-cost multiplier is estimated from empirical scaling fits rather than directly measured under matched target-quality conditions.
Consequently, the current results do not convincingly establish that π-XLNet provides a substantial training-efficiency advantage over existing diffusion-based approaches. Direct experiments with representative SetDLM or masked diffusion baselines under comparable training budgets would strengthen this claim.
W4. The evaluation of HBD-n does not fully isolate the contribution of the proposed decoding algorithm.
The reported generation improvements are encouraging, but some experimental comparisons employ different sampling configurations.
For example, in the GSM8K experiment, the official SetDiff baseline uses deterministic decoding, whereas HBD-n employs stochastic sampling with nonzero temperature and top-p truncation. Similarly, the MAUVE comparison uses different top-p values for the autoregressive baseline and HBD-n.
Therefore, the observed improvements cannot be attributed solely to the proposed branching and backtracking mechanism.
A more controlled evaluation using matched sampling parameters, together with quality–computation trade-off curves, would be necessary to establish the effectiveness of HBD-n.
W5. The relationship between the training-order framework and the proposed decoding algorithm is not sufficiently established.
The paper presents Mallows-based training and HBD-n decoding as two major contributions. However, HBD-n does not inherently depend on the proposed Mallows distribution and is also evaluated using publicly available SetDLM models.
The experiments do not sufficiently investigate how different training-order distributions affect HBD-n performance or whether Mallows-based training specifically benefits the proposed decoding strategy.
As a result, the training and inference contributions appear relatively independent rather than forming a tightly integrated framework. Additional experiments examining their interaction would help establish a more coherent methodological contribution.