- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates the application of Generative Flow Networks (GFlowNets) to hierarchical planning environments with cyclic state transitions, complex dependencies, and multiple valid solution paths. Unlike conventional GFlowNets, which typically operate on directed acyclic graphs (DAGs), practical planning environments often involve reversible actions and repeated state visits. Building on the non-acyclic GFlowNet framework of Morozov et al. (2025), the authors implement a Sub-Trajectory Balance (SubTB) GFlowNet with experience replay in the HierarchyCraft environment. Experimental results demonstrate advantages over several reinforcement learning baselines, particularly in long-horizon tasks with sparse rewards.
Critical strengths
-
Meaningful problem setting. Extending GFlowNets to hierarchical planning with cyclic transitions is a worthwhile research direction, although the underlying non-acyclic formulation is largely inherited from prior work.
-
Promising experimental results. The proposed method achieves strong performance on several challenging tasks. In Table 2, GFN achieves a 90.6% success rate on Gold, compared with 4.6% for SoftDQN. In the tuned experiments (Table 8), GFN achieves 54.4% success and 37.7 strategic modes on Ender-eye, compared with 28.3% and 7.7 for SoftDQN. The additional ablations on demonstrations, backward policies, and n-step backups further strengthen the empirical analysis.
Critical weaknesses
-
Limited methodological novelty. The main components are largely adapted from existing work, including the non-acyclic GFlowNet formulation of Morozov et al. (2025), SubTB, and replay-based training. The contribution primarily lies in their integration and empirical evaluation on HierarchyCraft. The paper introduces neither a new flow-matching objective nor a novel learning algorithm or convergence analysis for cyclic environments.
-
Insufficient explanation of Equation (5). Although Equation (5) defines edge flow through the expected number of edge visits, its role in addressing repeated state visits is not sufficiently explained. The authors should clarify the intuition behind this formulation, the interpretation of the expectation over trajectories, and how the resulting expected-visit flow handles cycles.
-
Unclear practical implementation of Equation (5). It remains unclear how the edge flow defined in Equation (5) is implemented or computed in practice. Both TB and SubTB are formulated using state flows rather than explicit edge-flow estimates. The authors should clarify whether Equation (5) is used directly in training or serves only as a theoretical justification for the state-flow parameterization.
-
Insufficient justification for combining TB and SubTB. Equation (6) introduces an additional TB loss on top of SubTB. However, SubTB already incorporates constraints over different trajectory lengths, including full-trajectory balance, with their relative weighting controlled by λ_sub. Although the authors state that the additional TB term anchors the global flow scale, its necessity remain unclear. In my opinion, this is very likely unnecessary.
-
Limited comparison with recent GFlowNet methods. More advanced GFlowNet training approaches, such as Adaptive Teacher, could potentially be incorporated into the proposed framework and yield stronger performance. Comparisons with such methods would provide a more comprehensive evaluation of the proposed approach.
Questions for authors
see weaknesses