- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces a continuous-time generative process for variable-length structured data. In addition to the usual diffusion/flow state, an auxiliary length process determines when new elements are inserted. The construction is intended to preserve a tractable training objective: the state dynamics can be trained with a flow-matching-style regression loss, while insertion times and positions are learned from the auxiliary process. The paper also proposes a split sampler that alternates evolution of the continuous state and length components.
The method is evaluated mainly on layout generation. On RICO it obtains FID 2.32, improving over the reported OneFlow result of 3.31 and matching the oracle-length LayoutFlow result of 2.37. Additional experiments cover PubLayNet and content-aware layout generation on CGL, along with comparisons of Euler, direct split, and staggered split samplers. These results suggest that learning length jointly with content is feasible without conditioning on the ground-truth count.
Critical strengths
-
The auxiliary-process construction is a natural way to bring variable cardinality into continuous-time generative modeling. It is more satisfying than predicting the length in a separate first stage, and the paper gives a reasonably unified treatment of state evolution, insertion, and sampling.
-
The RICO result is strong: VL-DE improves substantially over the reported variable-length baseline and reaches the performance of a method given the oracle length. The ablations over the length horizon and the comparison of sampling schemes are also useful rather than purely decorative.
Critical weaknesses
-
Several theoretical steps that justify the training objective need repair or additional assumptions. The conditional-expectation identification in Theorem 3.1 appears to require an explicit richness/identifiability condition on the feature class. More importantly, the factorization argument for Theorem 3.3 treats the length operator as acting only on the length state, whereas the insertion operation in Eq. (13) changes both the sequence and its length. Lemma 3.4 also drops the time-dependent hazard weight; this preserves a pointwise optimum only under an infinite-capacity argument, not the gradients of the shared finite network used in practice. I suspect the construction can be justified, but the current proof does not establish the full equivalence as written.
-
The empirical scope is narrower than the general claim, and the proposed higher-order sampler is not yet convincingly validated. The experiments use bounded-length layouts with one length variable; the Crello text component is generated by an external autoregressive model rather than the proposed insertion process. For sampling, direct splitting costs three function evaluations per step, while the cheaper staggered version loses much of the gain and Euler is competitive or better per function evaluation. There is no order-convergence experiment showing the claimed benefit of splitting. Baseline sampling budgets and training variability are also not fully normalized.
Questions for authors
-
Can the authors provide a corrected proof of Theorems 3.1 and 3.3 that states the required identifiability and regularity assumptions and explicitly handles the fact that an insertion changes both the sequence state and the length? How does removing the hazard weight affect optimization with a finite shared network?
-
What are the exact subflows used by the split integrator, and are they solved exactly or numerically? Please report error versus step size and quality versus function evaluations or wall-clock time; this would clarify whether a higher-order advantage is realized in practice.
-
Can the authors demonstrate the framework on a genuinely non-layout variable-length object, or on multiple interacting length variables, without relying on a separate autoregressive generator? This would materially strengthen the generality claim.
3 comments
Thank you for the thoughtful response. My questions and concerns have been resolved. I will raise my score from 2 to 3.
We are truly grateful for the reviewer’s insightful feedback, which helped us improve the quality of the paper!
We thank the reviewer for the careful reading of our paper and the insightful suggestions. Your feedback has substantially improved the quality of our submission. We have made changes and uploaded them in the revised PDF with changes in blue. Below we address the questions and concerns:
W1 / Q1: Theoretical steps
Regarding Theorem 3.1, we agree that some conditions were missing to completely formalize our result. We have added the assumptions and modified the proof, which can be found in Appendix C.2. In short, we have
Regarding Theorem 3.3, we agree that the proof of this result was not correct and are thankful to the reviewer for noting it. However, the result still holds and we have included a corrected proof that does not split the generator as before. The proof relies on the construction of our variable length process: it shows that a single coordinate is a Markov process and writes down its law and generator. Since the coordinates are independent, they are jointly Markov, which gives us the result.
Regarding Lemma 3.4 (hazard weight). The reviewer is correct that applying Theorem 3.2 to the generator (12) gives the insertion and initialization terms multiplied by a hazard weight, which we did not include in Lemma 3.4. The justification for this does not require an infinite-capacity argument. We have added Remark C.11, which shows that a positively (possibly time-dependent) reweighted Bregman divergence is a Bregman divergence, so that Theorem 3.2 applies.
However, a good question is how different reweightings affect the optimization. Many works have been dedicated to this, for instance [1] and [2]; in particular, [2] found that different reweightings correspond to different variational bounds. In our case, we did not use the hazard weight because it diverges as γt→1, but agree that this is worth checking, since one could avoid the issue by not sampling where γt≈1. To get a better understanding we ran an ablation with 3 possible reweightings (details in Appendix E.3):
Where we find that the hazard rate can hurt the performance due to an increase in scale, but this can be fixed through a simple rescaling.
[1] Kingma, Diederik, and Ruiqi Gao. "Understanding diffusion objectives as the ELBO with simple data augmentation." [2] Shi, Jiaxin, and Michalis K. Titsias. arXiv:2511.19664, 2025.
W2: Empirical scope and the sampler
Since this point was raised by all reviewers, we have included an answer in the general response. In short, we have added two new experiments: molecule generation on QM9 (Section 4.3), and a synthetic arithmetic task with two modalities of different lengths (Section 4.4), both generated by our insertion process without an external autoregressive generator. The results are summarized in the general response.
Regarding our samplers. We have also included experiments for the samplers on QM9 (Appendix E.6, Figure 9), where we found similar results to those for RICO. Per step, both splitting samplers are ahead of Euler at small step counts, and per network evaluation Euler and the staggered split are on par with the direct split trailing behind.
Whenever a subflow admits a closed-form solution we solve it exactly; this is the case, for Euclidean diffusions. For CTMCs (the insertions and the discrete modalities) no closed-form solution is available, so these subflows are solved numerically. As noted by the reviewer, this makes our samplers not be second order and we include the error vs step size in Appendix D.4.
Finally, we agree that the samplers performance might be underwhelming due to the need for many NFEs. However, they can motivate a better design of neural networks for variable length methods so that evaluating one of the flows becomes cheaper, but we leave such an investigation to future work.
Sampling budgets and variability. We agree that this was poorly explained on our end. We tested each baseline on a grid of {50,100,1000} steps and reported the best result for each method, which resulted in this discrepancy; we have adjusted the manuscript to reflect this.