acceptodds
Under review as a conference paper at ICLR 2027

Variable Length Diffuse Everything: Arbitrary Length Generative Modeling on Arbitrary State Spaces

Abstract

Diffusion and flow-based generative models have achieved remarkable results across diverse applications, but most assume data of fixed-dimension/length. Meanwhile, real-world problems such as layout generation often involve variable-length, multimodal data, such as continuous bounding box coordinates paired with discrete types. Existing methods typically address variable length by padding data to a fixed size or by introducing specialized objectives to model lengths explicitly, adding complexity and limiting generality. We propose Variable Length Diffuse Everything (VL-DE), a simple and general framework for variable-length multimodal generation on arbitrary state spaces. Instead of padding, VL-DE handles variable length through an insertion-based mechanism grounded in a principled theoretical framework, yielding a tractable training objective. In addition, by utilizing the fact that VL-DE is based on a generator with an additive form, we develop a higher-order sampler by appropriately handling the time dependence. Extensive experiments on layout generation demonstrate the effectiveness of our approach.

Primary area:
generative models
Keywords:
Diffusion Models, Multimodal, Layout, Variable Length

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iYA8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a continuous-time generative process for variable-length structured data. In addition to the usual diffusion/flow state, an auxiliary length process determines when new elements are inserted. The construction is intended to preserve a tractable training objective: the state dynamics can be trained with a flow-matching-style regression loss, while insertion times and positions are learned from the auxiliary process. The paper also proposes a split sampler that alternates evolution of the continuous state and length components.

The method is evaluated mainly on layout generation. On RICO it obtains FID 2.32, improving over the reported OneFlow result of 3.31 and matching the oracle-length LayoutFlow result of 2.37. Additional experiments cover PubLayNet and content-aware layout generation on CGL, along with comparisons of Euler, direct split, and staggered split samplers. These results suggest that learning length jointly with content is feasible without conditioning on the ground-truth count.

Critical strengths
  1. The auxiliary-process construction is a natural way to bring variable cardinality into continuous-time generative modeling. It is more satisfying than predicting the length in a separate first stage, and the paper gives a reasonably unified treatment of state evolution, insertion, and sampling.

  2. The RICO result is strong: VL-DE improves substantially over the reported variable-length baseline and reaches the performance of a method given the oracle length. The ablations over the length horizon and the comparison of sampling schemes are also useful rather than purely decorative.

Critical weaknesses
  1. Several theoretical steps that justify the training objective need repair or additional assumptions. The conditional-expectation identification in Theorem 3.1 appears to require an explicit richness/identifiability condition on the feature class. More importantly, the factorization argument for Theorem 3.3 treats the length operator as acting only on the length state, whereas the insertion operation in Eq. (13) changes both the sequence and its length. Lemma 3.4 also drops the time-dependent hazard weight; this preserves a pointwise optimum only under an infinite-capacity argument, not the gradients of the shared finite network used in practice. I suspect the construction can be justified, but the current proof does not establish the full equivalence as written.

  2. The empirical scope is narrower than the general claim, and the proposed higher-order sampler is not yet convincingly validated. The experiments use bounded-length layouts with one length variable; the Crello text component is generated by an external autoregressive model rather than the proposed insertion process. For sampling, direct splitting costs three function evaluations per step, while the cheaper staggered version loses much of the gain and Euler is competitive or better per function evaluation. There is no order-convergence experiment showing the claimed benefit of splitting. Baseline sampling budgets and training variability are also not fully normalized.

Questions for authors
  1. Can the authors provide a corrected proof of Theorems 3.1 and 3.3 that states the required identifiability and regularity assumptions and explicitly handles the fact that an insertion changes both the sequence state and the length? How does removing the hazard weight affect optimization with a finite shared network?

  2. What are the exact subflows used by the split integrator, and are they solved exactly or numerically? Please report error versus step size and quality versus function evaluations or wall-clock time; this would clarify whether a higher-order advantage is realized in practice.

  3. Can the authors demonstrate the framework on a genuinely non-layout variable-length object, or on multiple interacting length variables, without relying on a separate autoregressive generator? This would materially strengthen the generality claim.

3 comments

We thank the reviewer for the careful reading of our paper and the insightful suggestions. Your feedback has substantially improved the quality of our submission. We have made changes and uploaded them in the revised PDF with changes in blue. Below we address the questions and concerns:

W1 / Q1: Theoretical steps

Regarding Theorem 3.1, we agree that some conditions were missing to completely formalize our result. We have added the assumptions and modified the proof, which can be found in Appendix C.2. In short, we have

  • Stated the assumptions on the joint process: it is Markov and satisfies the weak Fokker-Planck equation (Assumption C.4).
  • Wrote the required assumptions on the test functions (Assumption C.5).
  • Required an integrability condition for Ft,x1,s1F_{t,x_1,s_1} (Assumption C.6). The theorem now defines Ft(x)F_t(x) directly as a conditional expectation, so no explicit condition on the feature class is needed.
  • Added Remark C.7, which clarifies the assumptions needed for Kt\mathcal{K}_t to recover the probability path, as in generator matching.

Regarding Theorem 3.3, we agree that the proof of this result was not correct and are thankful to the reviewer for noting it. However, the result still holds and we have included a corrected proof that does not split the generator as before. The proof relies on the construction of our variable length process: it shows that a single coordinate is a Markov process and writes down its law and generator. Since the coordinates are independent, they are jointly Markov, which gives us the result.

Regarding Lemma 3.4 (hazard weight). The reviewer is correct that applying Theorem 3.2 to the generator (12) gives the insertion and initialization terms multiplied by a hazard weight, which we did not include in Lemma 3.4. The justification for this does not require an infinite-capacity argument. We have added Remark C.11, which shows that a positively (possibly time-dependent) reweighted Bregman divergence is a Bregman divergence, so that Theorem 3.2 applies.

However, a good question is how different reweightings affect the optimization. Many works have been dedicated to this, for instance [1] and [2]; in particular, [2] found that different reweightings correspond to different variational bounds. In our case, we did not use the hazard weight because it diverges as γt→1\gamma_t\to1, but agree that this is worth checking, since one could avoid the issue by not sampling where γt≈1\gamma_t\approx1. To get a better understanding we ran an ablation with 3 possible reweightings (details in Appendix E.3):

Insertion weightFIDAlignmentOverlap
11 (ours)2.29 ±\pm 0.130.1990.560
min⁡(λt,50)\min(\lambda_t,50)3.14 ±\pm 0.180.3050.640
min⁡(λt,50)/4.22\min(\lambda_t,50)/4.222.31 ±\pm 0.120.2570.536

Where we find that the hazard rate can hurt the performance due to an increase in scale, but this can be fixed through a simple rescaling.

[1] Kingma, Diederik, and Ruiqi Gao. "Understanding diffusion objectives as the ELBO with simple data augmentation." [2] Shi, Jiaxin, and Michalis K. Titsias. arXiv:2511.19664, 2025.

W2: Empirical scope and the sampler

Since this point was raised by all reviewers, we have included an answer in the general response. In short, we have added two new experiments: molecule generation on QM9 (Section 4.3), and a synthetic arithmetic task with two modalities of different lengths (Section 4.4), both generated by our insertion process without an external autoregressive generator. The results are summarized in the general response.

Regarding our samplers. We have also included experiments for the samplers on QM9 (Appendix E.6, Figure 9), where we found similar results to those for RICO. Per step, both splitting samplers are ahead of Euler at small step counts, and per network evaluation Euler and the staggered split are on par with the direct split trailing behind.

Whenever a subflow admits a closed-form solution we solve it exactly; this is the case, for Euclidean diffusions. For CTMCs (the insertions and the discrete modalities) no closed-form solution is available, so these subflows are solved numerically. As noted by the reviewer, this makes our samplers not be second order and we include the error vs step size in Appendix D.4.

Finally, we agree that the samplers performance might be underwhelming due to the need for many NFEs. However, they can motivate a better design of neural networks for variable length methods so that evaluating one of the flows becomes cheaper, but we leave such an investigation to future work.

Sampling budgets and variability. We agree that this was poorly explained on our end. We tested each baseline on a grid of {50,100,1000}\{50,100,1000\} steps and reported the best result for each method, which resulted in this discrepancy; we have adjusted the manuscript to reflect this.

Thank you for the thoughtful response. My questions and concerns have been resolved. I will raise my score from 2 to 3.

We are truly grateful for the reviewer’s insightful feedback, which helped us improve the quality of the paper!

Reviewer_r2pfOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a variable-length diffusion framework and evaluates it through a limited set of experiments. The evidence does not yet substantiate its claims of generality across arbitrary state spaces.

Critical strengths

The proposed approach to variable-length generation has some potential for extension, and the formal development is fairly complete. The visual examples are also useful.

Critical weaknesses

1. The title and claims are much broader than the demonstrated results. Despite the extensive formal development, the method is trained and evaluated only on a relatively narrow layout-generation task. This does not adequately support the claims about arbitrary lengths and arbitrary state spaces.

2. The experimental evidence is too limited relative to the complexity of the framework and its proofs. Although the paper falls within the field of generative models, much of the main text is devoted to complex derivations, while intuitive evidence, visualizations, and experimental results are relatively limited. This makes the paper feel unbalanced. Several results that appear important to assessing the contribution are placed in the appendix. The main paper should stand on its own; reviewers should not have to rely on the appendix to evaluate the central claims. Substantial restructuring would be needed to bring the presentation up to the conference’s standard.

3. The paper contains invalid citations at lines 464 and 1220. These errors further undermine confidence in the care taken with the submission.

Questions for authors

My main concerns are covered above.

The paper’s organization and presentation appear to draw on Diffuse Everything. Could the authors compare VL-DE with relevant methods discussed in that paper on shared benchmarks, such as the Trend and MLE evaluations? This would help clarify the contribution of the variable-length mechanism.

Could the authors report results on additional layout-generation benchmarks, particularly PubLayNet? Appendices 5 and 6 discuss PubLayNet, but do not appear to report VL-DE’s own performance on it. Why are these results absent?

3 comments

We thank the reviewer for the careful reading of our paper and the constructive feedback. We have made changes to the submission and uploaded them in the revised PDF with changes in blue. Below we address the questions and concerns:

W1: Claims broader than the demonstrated results

Since this point was raised by all reviewers, we have included an answer in the general response. In short, we have added two new experiments: molecule generation on QM9 (Section 4.3), where VL-DE is on par with Branching Flows and with models that are given the number of atoms, and a synthetic arithmetic task with two modalities of different lengths (Section 4.4), both generated by our insertion process without an external autoregressive generator.

W2: Balance between derivations and experiments

We understand that the submission devoted more space to our theoretical results than to the experimental results, although the derivations and proofs are in the appendix. We find that our framework can open up many possibilities, and that it is an integral part of our contribution. The auxiliary-process extension of Generator Matching (Theorems 3.1 and 3.2) applies to any state space and gives a simple recipe for designing variable-length processes with a tractable, simulation-free objective. Furthermore, there are other possibilities for auxiliary processes that go beyond the variable length setting. We believe the main text needs to present these results for the method to be understood.

We have also made efforts to rebalance the paper by adding new experiments on other domains: molecule generation on QM9 (Section 4.3) and arithmetic equations with two varying lengths (Section 4.4), in addition to RICO, PubLayNet and CGL, and we have completed the PubLayNet results on all tasks. The central results of every experiment are in the main text; the appendix only holds experimental details, ablations and proofs. We hope that the reviewer can recognize our theoretical contributions and, together with the broader experiments, finds the paper better balanced.

W3: Invalid citations at lines 464 and 1220

We thank the reviewer for catching this. We have corrected a typo in the bibtex.

Q1: Comparison with the methods of Diffuse Everything on shared benchmarks

We must note that the MLE and Trend metrics are for a tabular generation problem that is not variable length. For this reason, the comparison wouldn't be suited there: when every element is inserted at t=0t=0, VL-DE reduces to a fixed-length multimodal diffusion, so these would not test the variable-length mechanism, which is our contribution.

Q2: Missing results on PubLayNet

We thank the reviewer for the chance to clarify this. The submission reported VL-DE on PubLayNet for unconditional generation (Table 1), but not for the conditional tasks GenType, GenTypeSize, and refinement. The reason is that these tasks fix the number of elements, so they are not variable-length tasks and do not test our mechanism. Although the completion task does have a variable length, we decided to focus on the unconditional setting. On RICO, we followed the protocol of LayoutFlow and trained a single model on a mixture of tasks so that it can perform all of them, which is why we reported them there. We agree that this was confusing. To have a complete comparison, we have trained a PubLayNet model with the same recipe as on RICO and included its results on all tasks in the paper (Tables 6 and 7, Appendix E.1). This also improves unconditional generation, lowering the FID from 9.239.23 to 8.458.45, which is now better than LayoutFlow (8.878.87), a model that is given the number of elements. We are thankful to the reviewer for motivating this experiment, which has improved the quality of the submission.

Thanks for your comprehensive response. While some of my concerns are addressed, i still think that this paper needs a lot of modifications from its original version to the accept line. This work can be better with a so-called major revision. As I now recognize its theoretical contribution, I will increase my score to 2.

We thank the reviewer for his time and acknowledgement of our contributions. We have actually already included major revisions, which thanks to ICLRs policy has been updated in the submissions PDF. These have expanded our scope to molecule generation as well as datasets with multiple varying lengths. We appreciate taking these into consideration and any further suggestions on how to improve our submission. Thanks in advance!

Reviewer_UaDGOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Variable Length Diffuse Everything (VL-DE), a diffusion-based framework for generating variable-length, multimodal data across arbitrary state spaces. The authors point out that existing diffusion and flow-based models typically assume fixed-length data and often handle variable-length inputs by padding them to a fixed size. The paper primarily focuses on layout generation, where the number of elements is not known in advance and varies across samples. Experimental results demonstrate that VL-DE achieves competitive performance on layout generation benchmarks.

Critical strengths
  • The idea of handling variable-length data through an insertion-based, variable-length interpolant is well-motivated and well-grounded. The problem the authors tackle is important: for many modalities (e.g., images and videos), we currently resort to fixed-length representations because they are easier to model, even though this is not the most natural choice, as the underlying information is inherently of adaptive length. The proposed approach is a promising candidate for modeling such variable-length representations.

  • The method is theoretically well-grounded: the authors show that the conditional probability path of the joint insertion-and-denoising process factorizes in closed form, which yields a tractable, simulation-free training objective.

  • The framework is evaluated on layout generation against state-of-the-art baselines and shows improvements over them in most settings. This demonstrates that the approach can address a real-world problem rather than only toy datasets or tasks.

Critical weaknesses
  • While I am not an expert in this area, I am concerned about the gap between the paper’s claims and its empirical scope. The title, “Variable Length Diffuse Everything”, and the framing suggest a general solution for variable-length generation, yet all experiments are on layout generation. Experiments on other domains with variable-length multimodal data would better support the claimed generality.

  • Given this limited scope, the performance gains over the strongest baselines are also small. Although the improvements are clearer against other variable-length methods, the advantage over existing layout models remains modest.

Questions for authors

NA

1 comment

We thank the reviewer for the careful reading of our paper and the encouraging comments. We have made changes to the submission and uploaded them in the revised PDF with changes in blue. Below we address the concerns:

W1: Limited empirical scope

Since this point was raised by all reviewers, we have included an answer in the general response. In short, we have added two new experiments: molecule generation on QM9 (Section 4.3), where VL-DE is on par with Branching Flows and with models that are given the number of atoms, and a synthetic arithmetic task with two modalities of different lengths (Section 4.4), both generated by our insertion process without an external autoregressive generator.

W2: Size of the gains over the baselines

We agree that the gains over the strongest layout models may seem small. However, those models (e.g. LayoutFlow) are given the number of elements of each layout, while VL-DE has to generate it. Our key contribution is that the same performance is reached without access to this information. Against methods that also generate the length, we find that VL-DE offers greater performance. The same holds in our new experiment on molecule generation on QM9, where VL-DE matches ADiT, a state-of-the-art method that is given the number of atoms, which shows that the length can be modeled effectively beyond layouts.

Comments on the paper

1 comment

We thank all reviewers for their time and their constructive feedback. We have uploaded a revised PDF with all changes marked in blue. Since the empirical scope was raised by all reviewers, we describe the new experiments here and address the remaining points in the individual responses.

New experiments beyond layouts

Although our framework makes no assumption on the state space, this was brought up by all reviewers, so we understand the importance of testing in domains beyond layout generation. We have made extensive efforts and have now included two new problems in our manuscript: molecule generation on QM9 and a synthetic dataset that demonstrates the case of two varying lengths.

Molecule generation on QM9 (Section 4.3). We follow the evaluation protocol of Branching Flows and train under the settings of Transdimensional Diffusion adapted to our process. VL-DE performs on par with Branching Flows and models with length access like ADiT, while being significantly ahead of Transdimensional Diffusion. This demonstrates that VL-DE effectively learns variable-length data in a new domain and is competitive with the state of the art: compared to Branching Flows it has the higher average 1−1-KS and is ahead on 88 of the 1010 individual distribution metrics, with a comparable JSDm.

ModelLength givenJSDm ↓Avg. 1−1-KS ↑ValidUnique
ADiTyes0.0860.9710.9850.979
Multimodal FMyes0.0630.9770.9890.965
Transdimensional Jumpsno0.1200.8700.9780.986
Branching Flowsno0.0800.9690.9870.973
VL-DE (ours)no0.0820.9750.9700.979

Arithmetic equations: two modalities of different lengths (Section 4.4). To the best of our knowledge there is no standardized benchmark for multiple interacting lengths, and extending our Crello demonstration would require substantial engineering effort. To demonstrate our method's flexibility, we introduce a task with two modalities of different lengths: a sequence of symbols and a sequence of numbers, which are coupled only through the rules of arithmetic. We construct equations whose two sides are equal, with symbols in {(,),+,−,×,=}\{(,),+,-,\times,=\} and numbers in [−10,10][-10,10] (construction in Appendix E.7). The task is challenging: the model has to learn the coupling between the two lengths, and, even when it does, small errors in the numbers can result in large deviations from the equality, since they are amplified by products. VL-DE closely matches the joint distribution of the number of numbers and of parenthesis groups (Figure 4), 94.3%94.3\% of its samples are valid, with half of the valid equations balanced to within 0.50.5.

Summary of the other changes

  • Theory (Appendix C). We have added important assumptions to theorem 3.1. Corrected the proof of Theorem 3.3, and explained why Lemma 3.4 is justified for any positive time reweighting of the loss (Remark C.11).
  • Loss reweighting ablation (Appendix E.3). We compare the loss with different loss reweightings
  • Samplers. We added the comparison of the samplers on QM9 (Appendix E.6, Figure 9) and an analysis of the error versus the step size on RICO and QM9 (Appendix D.4, Figure 5).
  • PubLayNet (Table 1, Appendix E.1). We trained the PubLayNet model with the same multi-task recipe as on RICO and now report all tasks (Tables 6 and 7). The unconditional FID improves from 9.239.23 to 8.458.45, ahead of LayoutFlow (8.878.87), which is given the number of elements.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.