acceptodds
Under review as a conference paper at ICLR 2027

Learning a Mixture of GFlowNets

22 views

Abstract

Learning an ensemble of GFlowNets to sample from a discrete target distribution has become a common approach for achieving better state space exploration and convergence than that of a monolithic sampler. However, these methods often add a substantial runtime overhead to the base model, and their conceptual connection remains elusive. To address this, we first propose a general-purpose theoretical framework for describing a mixture of GFlowNets, which we specialize into continuously (CI) and discretely indexed (DI) collections. On the one hand, we show CI GFlowNets can be interpreted through the lens of a random features expansion, provably boosting the sampler's expressivity in graph-structured tasks and reducing learning instability via spectral shifting. On the other hand, we demonstrate DI GFlowNets encompass prior approaches for GFlowNet training and provide the foundation for the newly proposed Stratum-Conditioned (SC) GFlowNets. This method, which is inspired by the Doob's -transform of Markov chains, decomposes the state space according to a prescribed modular function and restricts each component to sample from a distinct subset of it. Importantly, SC GFlowNets support centralized and component-wise embarrassingly parallel training, and we show both of them significantly speed up learning convergence and mode coverage without introducing any non-negligible extra computation.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
GFlowNets, Sampling, Bayesian

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TZM9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

his paper introduces a general framework for mixtures of GFlowNets, in which a component index is sampled before generating an object, and the resulting mixture is trained to match a reward-proportional target distribution. The framework accommodates both continuously indexed and discretely indexed components through a mixed trajectory balance condition.

For continuous indexing, the authors connect random-feature augmentation to improved expressivity and training stability. For discrete indexing, they propose Stratum-Conditioned (SC) GFlowNets, which restrict individual components to subsets of the terminal space defined through object statistics such as length or distance. Components can share a conditional neural network or be trained independently in parallel, with their learned partition functions determining the mixture weights at inference time. The paper also establishes connections to Doob’s h-transform, Subgraph Asynchronous Learning, and Boosted GFlowNets. Experiments across several discrete sampling tasks show improved convergence and exploration with little additional per-step computation.

Critical strengths

S1. Simple and practically appealing method.

SC GFlowNets provide a straightforward mechanism for encouraging exploration of different regions without requiring an auxiliary exploration network or sequentially trained boosters. The shared-network implementation is particularly attractive because it can retain approximately the computational cost of a single sampler.

S2.Useful probabilistic formulation.

The relationship between component rewards Rk(x)=R(x)q(k∣x)R_k(x) = R(x)q(k|x), component partition functions, and inference-time mixture weights is clearly motivated. This provides a useful framework for understanding how specialized components can collectively recover the original target distribution.

S3. Broad empirical evaluation.

The experiments cover set generation, Hypergrid, graph generation, sequence generation, and variable selection. The paper also examines alternative partitions, the number of components, and parallel training. Reporting runtime and FLOPs, in addition to training iterations, is relevant to the claimed computational benefits.

S4.Informative connection to conditioned stochastic processes.

The Doob’s h-transform interpretation offers a useful conceptual explanation of SC components as generation processes conditioned on terminal events, even though the transform itself is not used directly for training.

Critical weaknesses

W1. Proposition 5.7 does not establish efficient reachability under the stated assumptions

Equation (6) requires finding an extension satisfying both

\quad \text{and} \quad \psi(s \cup s') \in [a_k, b_k).$$ However, the proof in Appendix H.4 reduces this problem to enumerating possible intersection counts and checking whether $$\xi(n + m) \in [a_k, b_k),$$ without retaining the terminal-feasibility constraint $s \cup s' \in \mathcal{X}$. It also does not establish that the proposed extension is reachable through the allowed transitions. Having only polynomially many possible statistic values does not imply that a valid terminal completion realizing each value can be found or recognized efficiently. Additional assumptions on the terminal space and transition structure appear necessary. Since this proposition supports the claimed tractability of constructing SC action masks, this gap affects an important theoretical justification of the method. The particular experimental environments may still permit efficient masking, but this does not follow from the proposition as currently stated. **W2. The claimed positive spectral shift is not implied by Proposition C.7** The expansion $$H_{\mathrm{TB}} =H_0+\frac{\sigma^2}{2} H_{\mathrm{Reg}}+O(\sigma^4)$$ does not establish that random features increase the Hessian eigenvalues or improve its conditioning. The paper does not prove that $H_{\mathrm{Reg}}$ is positive semidefinite. Moreover, eigenvalues of a sum of symmetric matrices are not generally obtained by adding their eigenvalues componentwise, contrary to the interpretation given in Appendix H.12. Consequently, the claimed theoretical explanation that random features stabilize training through an upward spectral shift is not supported by the current argument. The empirical stability result remains interesting, but the theoretical interpretation needs additional assumptions or substantially weaker wording. **W3. The assumptions of Lemma C.6 are insufficient for its remainder bound** The lemma assumes zero-mean random features with covariance $\sigma^2 I$, whereas its proof additionally uses the assertion that all odd moments vanish. Zero mean alone does not imply vanishing third moments. Obtaining the stated $O(\sigma^4)$ expansion requires further distributional, moment, and smoothness assumptions. Similarly, differentiating an $O(\sigma^4)$ remainder twice with respect to model parameters does not automatically preserve that order without suitable uniform derivative bounds. These technical issues may be repairable, but resolving them would not by itself establish the positive spectral-shift claim above. **W4. The action mask in Example 5.4 appears incorrect as written** The forward transition is masked according to $$p_F^{(k)}(s, s + 1) \propto \mathbb{1} \left[s + 1 \in \mathcal{X}_k\right] \exp(\cdots).$$ For the second and third intervals, the successor of the initial state is outside the selected interval, while termination is also forbidden. Thus, the component would have no valid action at initialization. The mask should instead test whether the target interval remains reachable after the transition, rather than whether the immediate successor already belongs to that interval. This may be a formula error rather than an implementation error, but it needs clarification because this example is central to the explanation of SC sampling. **W5. The experiments do not sufficiently isolate the sources of improvement** SC simultaneously introduces component conditioning, hard reachability masks, separate partition-function parameters, and mass-based component sampling. The current experiments do not clearly establish which mechanisms account for the gains. In particular, comparisons between uniform and active component sampling, and between component-conditioned models with and without target specialization, would strengthen the empirical analysis. Accuracy-versus-wall-clock curves on representative tasks would also more directly support the practical speedup claims than iteration-based convergence curves combined with separate per-step timing measurements.
Questions for authors

Q1. Reachability and Proposition 5.7:

How does the proof account for the constraint s∪s′∈Xs \cup s' \in \mathcal{X} in Equation (6), as well as reachability through valid transitions? What additional assumptions on the terminal space and state graph are needed, and do they hold in every experimental domain?

Q2. Spectral regularization:
Can the authors establish conditions under which HRegH_{\mathrm{Reg}} is positive semidefinite and the claimed eigenvalue increase holds? Otherwise, would they revise the spectral-shift claim and distinguish the loss expansion from the empirical observation of improved stability?

Q3. LINES implementation:
How can components k=2,3k = 2, 3 move from the initial state under the mask in Example 5.4? Does the implementation check immediate interval membership or future reachability of the interval?

Q4. Expressivity claim:
Does “any target distribution” in Proposition C.5 require assumptions concerning graph-isomorphism invariance or node-label features? How does the cited random-feature GNN universality result establish universality for the specific pairwise action parameterization in Equation (11)?

Q5. Ablations and practical efficiency:
Could the authors provide ablations of active component sampling and component conditioning, together with convergence-versus-wall-clock comparisons against the single-component and boosted baselines?

Reviewer_Gxb9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a unifying theoretical framework for studying mixtures of GFlowNets.The authors formalize this setting as learning a mixture of components indexed by$\gamma$, trained with a mixed trajectory balance (MTB) objective, Z pM(dγ) pF(γ)(so,τ)  =  R(x) qM(dγ∣x) pB(γ)(x,τ)Z\, p_M(d\gamma)\, p_F^{(\gamma)}(s_o, \tau) \;=\; R(x)\, q_M(d\gamma \mid x)\, p_B^{(\gamma)}(x, \tau) where pMp_M and qMq_M are prior and posterior mixing distributions. The authors study two specialized cases: continuously indexed (CI) GFlowNets where γ\gamma is a Gaussian random feature passed to the policy and discretely indexed (DI) where γ\gamma is a discrete variable. The authors argue that CI version makes GNN-based GFlowNets universal for graph generation and act as a Hessian regularizer that stabilizes training. The authors propose stratum-conditioned (SC) GFlowNets, an instantiation of a DI mixture where the outcome space is partitioned into KK strata, and a shared network conditioned on kk is masked so that component kk can only reach stratum kk. The authors show tha each stratum policy is a Doob hh-transform of the global GFlowNet but computing the reachability masks is NP-complete in general (outside certain special cases). The authors also show how recent work like SAL and Boosted GFlowNets can be formulated as instantiations of their theory. The authors empirically validate the approach on some synthetic settings showing improvements over a vanilla GFlowNet trained with TB.

Critical strengths
  • The proposed Stratum Conditioned GFlowNets is a clear idea and given a kk-embedding and a reachability mask it speeds up convergence on all seven domains, at roughly the same compute cost as a vanilla GFlowNet.
  • The paper is also quite thorough in terms of establishing the theoretical framework and covering the various implications of the framework, e.g. reframing existing approaches within the framework.
  • The authors also include the code to reproduce the results, which is quite easy to run and check.
Critical weaknesses
  • The biggest concern I have with the paper is about the significance of the framework studied in the paper. Specifically, based on my understanding a KK-component mixture can simply be described as a GFlowNet on an augmented state graph whose first action picks kk and Prop. 3.3 and B.1 and Cor. 5.2 follow directly from MTB being exactly TB on that graph. While some of the results are still interesting, I feel the core contribution of a unified theoretical framework is quite weakened.
  • SC GFlowNet depends on access to a reachability mask which is impractical in most practical settings. Even some of the other theoretical results about CI GFlowNets including the regularization effect and universality on graphs are not tested empirically.
  • The other major criticism I have is with the empirical results. Beyond the limitations of just having experiments on small synthetic tasks, the empirical setup on those tasks itself is also quite weak. Only a vanilla TB baseline is used for comparison and all closely related work but comparisons with SAL are missing and Boosted GFlowNets are only considered in the Appendix, and other exploration approaches in the literature are completely omitted. There is also no ablation to separate the the partition from the ϵ\epsilon-uniform sampling of kk, the KK separate log⁡Zk\log Z_k, or the extra conditioning. A random-partition control and a stratified-replay baselines are missing as well as a comparable ϵ\epsilon-random vanilla TB baseline.
Questions for authors
  • Is an SC GFlowNet exactly equivalent to a standard GFlowNet on an augmented DAG whose first action picks the stratum, with a learned pF(so→k)p_F(s_o \to k)? If so, are there specific implications which the MTB view captures that this equivalence misses?
  • Were the baseline hyperparameters (learning rates, ϵ\epsilon, log⁡Z\log Z learning rate, optimizer) tuned separately? Does the improvement survive with Adam instead of Muon?
  • How do you check reachability for strata in constrained spaces such as DAGs or ancestral graphs, where not every subset is admissible?
  • I think [1] is quite closely related to mixture GFlowNets and should be discussed. It focuses on mixtures of trained GFlowNets different from the exact setting in this paper but is closely related.
  • Minor inconsistencies:
    • qq vs. qMq_M
    • "Theorem 5.9" vs. Prop. 5.9
    • BB vs. dγd\gamma in Eq. (5)
    • Citing Muon (Jordan et al.) for "stochastic gradient descent" in Eq. (5) is odd and Madan et al. (SubTB) appeared at ICML 2023, not 2022.
Reviewer_yTCYOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a general purpose theoretical framework that describes how mixtures of GFlowNets can be constructed and learned to guarantee correct sampling from the reward distribution. Authors argue that continuously indexed mixtures based on random features can improve the model's expressivity in graph-structured tasks. Next, authors design a specific way to learn discretely indexed mixtures called Stratum-Conditioned (SC) GFlowNets based on decomposing the target space into a number of parts through modular functions and restricting each GFlowNet in a mixture to sample its own subset. The authors demonstrate improved convergence speed of the proposed SC GFlowNet approach on a number of benchmarks.

Critical strengths

The paper has clear motivation and is dense with theoretically and methodologically interesting ideas. The presented general theoretical construction for learning mixture of GFlowNets is well-crafted and easy to follow. Most of the modern literature on GFlowNet focuses on heuristic-based engineering modifications, while the present paper deepens theoretical and methodological understanding, which holds notable scientific importance and value. The proposed design of a posterior mixing distribution via hand-crafted coverings is intuitive and well-motivated theoretically. Example 5.4 was also very helpful for understanding. I also found the presented connections to Doob's h-transforms interesting. The presented experiments show clear training improvements over base GFlowNets without sacrificing wall-clock time.

Critical weaknesses

In my opinion, the manuscript has major presentational problems. A critical consequence of some of these problems is the fact that the text does not support some of the claimed contributions, while the others hinder readability. I want to preface the following critique by saying that I did not study the whole Appendix of this paper in much detail, which may contain answers to some of my concerns. However, the main text by itself has to be self-contained, clearly presenting all of the paper's claimed contributions with supporting evidence, and it is explicitly stated in ICLR author guidelines that "reviewers are not required to read the appendix".

  1. The authors claim as a contribution that they demonstrate the proposed framework's usefulness in providing an unifying view of prior techniques. "We prove this framework encompasses two prior methods, Subgraph Asynchronous Learning (Silva et al., 2025b) and Boosted GFlowNets (Dall’Antonia et al., 2026b), unifying otherwise disparate algorithms." However, there is no discussion of these works in the main text. In addition, the main text presents no experimental comparison with these works, which is also a significant issue in my opinion since they should be treated as most direct baselines. Some comparisons are given in Appendix, however, I see no problem from the presentational standpoint in including these methods in the plots in Section 6 as baselines.

  2. Both the abstract and introduction claim improved mode discovery/coverage as a contribution, however, I found no experiments tracking any mode discovery metrics in the paper, nor any further discussions on this topic.

  3. The authors separate the presentation of the proposed framework into continuously (CI) and discretely indexed (DI) mixtures of GFlowNets. The abstract claims "On the one hand, we show CI GFlowNets can be interpreted through the lens of a random features expansion, provably boosting the sampler’s expressivity in graph-structured tasks and reducing learning instability via spectral shifting". While the idea of augmenting the GFlowNet model with random features to improve expressivity that the authors discuss is interesting in my opinion, the whole CI GFlowNet discussion gets only 1/4 of a page in the main text. There are also no experiments presented with CI GFlowNets: the whole experimental evaluation in Section 6 focuses only on discretely indexed mixtures.

  4. Related to previous points, the presented set of experiments is rather limited in my opinion, only showing that choosing stratums in a certain way and training a mixture following the proposed SC GFlowNet approach improves upon training a base trajectory-balance GFlowNet. For a theory-leaning paper the choice of benchmarks is perfectly fine in my opinion, but the only information presented in Section 6 is "we work better than the very standard base GFlowNet". As stated above, including prior mixture-based GFlowNet based methods as baselines is crucial. The claims regarding improved mode discovery should either be dropped or supported by new experiments. An ablation studying how the number of models in a mixture affects the results would also be very informative in my opinion.

  5. Compared to the rest of the text, the section "How to design a tractable covering?" is very difficult to read and understand, and required me many re-reading attempts, as well as looking into some proofs in the Appendix to understand what is going on. The main issue blocking the reader is the fact that the paper does not say what it means by "modular function", neither providing a definition nor referencing some prior literature. Second, the whole discussion is heavily based on the assumption that the GFlowNet generation process can be reduced to the process of constructing a multiset of some fixed component set. While this is explained for some example problems considered in the paper, it is very unclear how it can be done for general GFlowNet DAG environments. I think it is worth a separate discussion that either proves some general reduction or explicitly states that the results are limited to some specific class of problems. For example, can the authors explain how classical GFlowNet molecule construction environments (fragment-based and atom-based) can be viewed through this multiset construction lens?

  6. I honestly do not understand the reason for including Section 5.2 as a page-long section in the text. The presented Doob's h-transform connection is informative and interesting in my opinion. However, the stated theoretical results are just a direct application of the classical results on conditioning a Markov chain to hit a subset that is done via a harmonic function. See, e.g., Part 2, Chapter VI, §15 “Killed Markov Processes” of Classical Potential Theory and Its Probabilistic Counterpart, J.L. Doub, 1984. What is even more strange to me is the fact that these results do not appear to impact the proposed SC GFlowNet model in any way, neither in the training procedure nor in parameterisation (authors state that learning h functions is impractical), the theory and methodology behind the whole model is derived beforehand without the use of these results. Because of all this, I would suggest trimming the section to leave Doob's h-transform theoretical connections as a remark with a following discussion.

The paper is dense with very interesting ideas, however, the concerns stated above prevent me from recommending acceptance in its current form. I am willing to raise my score if they are sufficiently addressed.

Questions for authors

See Weaknesses.

Comments on the paper

1 comment

We thank all reviewers for the feedback and for the engagement throughout the review process. We outline below the main changes we made during the rebuttal, which have helped strengthen our work.

  1. We have brought the comparison against Boosted GFlowNets to the main text and included new experiments showing our method, SC GFlowNets, converges faster than SAL (another mixture-based baseline; Sec. 6, F).

  2. We expanded the discussion on Continuously Indexed GFlowNets, providing an empirical illustration of its universal approximation capabilities in graph-structured tasks (Sec. 4, Fig. 2).

  3. We conducted additional ablation analyses, showing the positive effect of active partition sampling (vs. uniform sampling) and stratum-conditioning (vs. feature augmentation alone) on SC GFlowNets’ performance (Sec. 6, G).

  4. We fixed the observed typos and made explicit assumptions that were previously implicit in a few statements (Sec. 5, Sec. C).

  5. We added comparisons to a recent GFlowNet baseline (Local Search), which also implements an exploration-boosting heuristic, underlining our method’s faster learning convergence (Sec. G, Fig. 33).

  6. We included mode-tracking experiments, which highlight SC GFlowNets’ improved exploratory capabilities (Sec. 6).

To summarize, our work develops an abstract framework for learning a mixture of GFlowNets (Sec. 3), connects it to existing methods (e.g., Sec. 5 and D, Props. D.4, E.4, and F.4), and derives practical learning algorithms from it.

In particular, we introduce SC GFlowNets, establish the conditions for its tractable implementation (Prop. 5.7), its limitations (Prop. 5.6), and its connection to Doob’s hh-transform (Remark 5.8, Prop. D.6), and empirically demonstrate it improves upon prior mixture-based approaches in terms of learning convergence, computational cost, and mode discovery (Sec. 6, E, F, G). We also prove that a continuous mixture of GFlowNets is a universal approximator for isomorphism-invariant distributions on edge-additive graph-structured tasks (Sec. 4, Prop. 4.3).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.