- Rating
- 4(+2)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes Mixture of GFlowNets (MoG), which uses a learned classifier to partition the reward distribution across multiple GFlowNets. The authors propose "Mode Bias", a postive feedback loop that leads to mode collapse. It's a very insightful and important diagnosis for GFlownet. Further, the MoGX introduces classifier-guided trajectory exchange to improve exploration. Experiments demonstrate improved mode coverage under a fixed sampling budget.
Critical strengths
The paper identifies and clearly formulates mode bias as an bottleneck in GFlowNet exploration, highlighting why simply increasing parallel sampling may fail to discover new modes. This is a valuable insight with practical application for Science. The proposed framework is simple and well-motivated. The experiments cover multiple reward landscapes and population sizes, and the improvements of MoGX are encouraging.
Critical weaknesses
First, the paper should better discuss recent approaches to mode collapse, for example, RapTB (Wang et al., ICML 2026), which explores credit assignment and diversity-aware replay, and ACE (Dall'Antonia et al., ICML 2026), which studies complementary exploration. Replay buffer can be a vital factor to mitigate mode collapse, such disscussion or experiments will better clarify the contribution of MoGX, especially given the reduced gains with SubTB in appendix.
The paper lacks direct evidence that learned partitioning and classifier-guided exchange are responsible for the improvements. Comparisons with fixed partitions or random exchange, along with an analysis of mode overlap between policies, would strengthen the claims. The mixture-correctness result is valid under exact component convergence, but the joint classifier-policy training dynamics are not discussed. In particular, Appendix C derives classifier self-consistency using partition function priors, whereas the classifier seems to be trained using equally sampled components. The authors should clarify this discrepancy and discuss how imperfect component convergence and partition-function estimation affect mixture correctness.
Most experiments use HyperGrid-style environments. Additional experiments on structured or variable-length generation tasks, such as molecular generation (SMILES), would better demonstrate the generality and practical relevance of the method. Moreover, the task like mouelces generation will also provide additional metrics for evaluating the diversity (fingerprint similarity). I suggest such experiments for it will better support your claims.
Questions for authors
See weakness part. I'm wiling to raise my score if the authors address my concerns above.
5 comments
W2. Evidence that the learned partition and exchange cause the gains. We ran the two ablations that you suggested. They run 4 seeds of MoGX at on the three compositional rewards and ideotype.
- Frozen partition. is a fixed random partition for both shaping and routing. We calibrated it so that every component owns a region. Its normalized entropy (0.5) matches the entropy of the learned partition.
- Random destination. is learned and still selects which trajectories leave a rank. But it sends them to a random other rank.
The frozen partition tests whether the partition must be learned. The random destination tests whether must route the trajectories. We ran 384 new experiments: the two ablations, an unmodified MoGX control, and NoComm, with 4 seeds at on 4 tasks. The control reproduces the MoGX of the paper. The table gives the modes at a common budget of trajectories for , and for on ideotype (mean ± s.d. over 4 seeds).
| Task (budget) | NoComm | MoGX | Frozen partition | Random destination |
|---|---|---|---|---|
| Bitwise XOR, K=64 (M modes) | 11.4 ± 0.4 | 111.4 ± 0.6 | 121.1 ± 2.7 (1.09×) | 121.1 ± 1.0 (1.09×) |
| Mult. Coprime, K=64 (M modes) | 21.3 ± 0.7 | 101.5 ± 0.7 | 101.2 ± 0.2 (1.00×) | 105.0 ± 0.3 (1.04×) |
| Cond. Multiscale, K=64 (M modes) | 18.5 ± 0.2 | 72.0 ± 2.0 | 70.5 ± 0.7 (0.98×) | 76.5 ± 0.6 (1.06×) |
| Ideotype, K=32 (% of optima) | 20.7 ± 0.7 | 29.1 ± 0.8 | 33.7 ± 2.3 (1.16×) | 23.7 ± 2.3 (0.81×) |
| Ideotype, K=64 (% of optima) | 28.9 ± 0.4 | 36.5 ± 1.4 | 41.2 ± 0.6 (1.13×) | 32.3 ± 0.2 (0.89×) |
Partition plus exchange drives the gain. At , all three MoGX variants find 10–11× the modes of NoComm on Bitwise XOR, about 5× on Mult. Coprime, and about 4× on Cond. Multiscale. This holds even when the partition is random and fixed. So the gain comes from the mixture structure of MoGX, and it does not depend on a carefully trained classifier.
A fixed partition is a simpler variant that also works. On 3 of 4 tasks (considering the full sweep), the fixed random partition matches the learned one or does better. However, the fixed partition also has a cost: a random classifier leaves more than half of the 64 components with no states, so we had to calibrate it to a target entropy first. Instead, the learned partition is balanced with no tuning. We added the fixed partition to the paper as a simpler option; thank you for suggesting this.
The learned classifier matters for routing. On HyperGrid, the policies rarely find the same mode ( for every MoGX variant). There, routing has no overlap to remove, and random routing is as good as MoGX (+4% to +9% at ). Ideotype is different, because policies can find the same optima. There, guided routing finds 23% more optima than random routing at and 13% more at . With random routing, the policies overlap more than in any other arm (see below). We added this analysis in the appendix.
Mode overlap. We measured the redundancy from the per-rank logs of every run. This value is the mean number of agents that found each mode (1 = disjoint, 64 = identical). On ideotype (1,024 optima, where some overlap is unavoidable), it is 7.7 for NoComm, 5.0 for MoG, 3.3 for MoGX, and 12 for DDP, without replay. For MoGX, we count each optimum for the agent that sampled it, before the exchange. Because MoG has no exchange, its decrease from 7.4 to 5.0 is direct evidence that the learned partition sends agents to different optima. Moreover, the ablations you suggested show that guided routing also keeps the agents apart. The pairwise Jaccard index between the optima of two agents is 0.11 for MoGX, 0.43 for NoComm, and 0.53 for the random destination (). The frozen partition is in between (Jaccard 0.21). On Bitwise XOR ( modes), even independent agents rarely find the same mode (), so overlap cannot explain the gap. However, the difference is due to learning speed. Each agent samples the same number of trajectories, and its mode hit rate jumps once it learns the parity rule. MoG agents learn it sooner: they reach a hit rate of 0.5 after 2,900 steps at , against 4,600 for independent agents. MoGX components also train on the high-reward trajectories that other components route to them, so they reach a hit rate of 0.5 after just 675 steps.
Appendix C and the classifier prior. Thank you for the careful reading; you are correct. Each rank averages its classifier loss over its own batch, and the ranks then average their gradients. So every rank has the same weight, and the class prior is uniform. For , the Bayes-optimal classifier is . In MoGX, trains on the post-exchange batch, with labels from the receiving rank, so is replaced by that batch's distribution. Mixture correctness does not depend on priors or labels. The proposition holds for any pointwise assignment , because .
The self-consistency argument in App. C changes in light of this. With the old prior, every classifier was a fixed point, so the argument did not predict where training goes. With the uniform prior, a fixed point needs one of two conditions. Either each state belongs to one component, or all the components that share a state have equal . If they do not, the classifier moves the shared mass to the component with the smaller . So at exact convergence, shared states move toward the components with less reward mass. We have updated App. C with this result; thanks for pointing it out.
Imperfect convergence and partition-function estimation. We have measured both.
During training, for any classifier state. Thus, tests whether the learned partition functions keep up with the moving classifier. Across all paper MoG/MoGX runs, from 5k steps on, MoG has nats at every . This is the same range as a single GFlowNet. MoGX has a constant +0.09–0.10 nats.
We also computed each component's exact target by passing all states through the classifier. This is too expensive to repeat along a run, so we did it on 16 final checkpoints (original, , and , 2 seeds). For MoG, each learned matches its target to 0.005 nats RMS. For MoGX, every is about 0.1 nats high, which is the same offset as above. Because all components share this shift, it cancels in the mixture weights , and the weights stay within 0.006 of exact.
The mixture is also robust to imperfect convergence. In every MoGX checkpoint, the median component's KL divergence to its own target is 1.6–22× the value of sampling noise. The mixture's KL to the true target is within 1.35× of that value, because the component errors cancel in the -weighted sum.
We also queued in the compute cluster a stress test: the mixture KL along training, with from to (the paper uses ). We will post the result here.
W3. Structured, variable-length tasks. We agree, and we built such a task. The environment assembles molecules from 72 fragments with ordered attachment instructions (up to six fragments). The reward is RDKit QED. The environment has 201 action IDs, up to 129 legal actions per state, and trajectories of variable length. The runs of MoG, MoGX, NoComm, and DDP are in the compute cluster queue; we will post the results and analysis here as soon as they complete.
To summarize, the ablations show that partition plus exchange drives the gain, even with a fixed random partition. The learned classifier matters for routing: on ideotype, guided routing finds up to 23% more optima than random routing and keeps the agents apart. The results show that the mixture is correct during training and under imperfect convergence. The replay baseline and the molecule task are in the compute cluster queue. We will post each result in this thread when it is complete.
Thank you for the review, and for your openness to a higher score. You raised three concerns; we answer them in the order of your review. Several answers use new analyses of the paper runs. We ran the two ablations that you asked for, and we give the results below. We also implemented a molecule task. Its runs are waiting in the cluster queue, and we will post them here when they are complete.
W1. Related work and replay. Thank you for the references, which we now discuss in Sec. 2. RapTB (Wang et al., ICML 2026) works inside one learner. A rooted-prefix balance term sends credit to early prefixes. Its submodular replay refreshes the buffer for reward and diversity. ACE (Dall'Antonia et al., ICML 2026) trains an explorer GFlowNet together with the main GFlowNet. ACE uses divergent trajectory balance to push the explorer toward high-reward states that the main sampler under-represents.
The main GFlowNet of ACE trains on samples from the two GFlowNets. ACE mixes the two policies by their partition functions, but only to collect training data; the main GFlowNet alone must match the target. In both papers, then, a single sampler is responsible for the full target, while better credit assignment, better replay or one dedicated explorer helps it. MoG instead splits the target itself across K peers that train in parallel, and the Z-weighted mixture is itself the sampler. The ideas are compatible, because each MoG component is an ordinary GFlowNet with its own shaped reward. A component could train with RapTB and refresh its buffer with SubM on autoregressive tasks, or pair with its own ACE explorer that searches the regions it under-samples.
About replay: the paper reports every result in two regimes, with a replay buffer (ρ=0.5) and without one (ρ=0). The methods have the same rank order in the two regimes. Thus, the gain of MoGX does not depend on replay. For a direct test, we added a diversity-aware replay baseline to the comparison. These runs are in the compute cluster queue with the others.
Our new step analysis identifies a specific cause for the smaller gains under SubTB. On Bitwise XOR, SubTB and DB reach a mode hit rate in the range of 0.55–0.82 within 600 steps, but TB needs 2–5k steps, depending on the reward. The gains of MoG are largest when the base objective is slow to take off, as TB is here. We have added this analysis to the appendix.