- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies sparse feedback in autoregressive RL for combinatorial optimization. It compares several exploration strategies and proposes Construction-Sequence Exploration (CSE), which uses decayed visitation counts of consecutive construction decisions to provide a novelty bonus that is gradually annealed during training. Experiments cover eight combinatorial optimization problems and evaluate final costs under various dense and gated reward settings.
Critical strengths
None
Critical weaknesses
The motivation is questionable. Section 3.1 defines the COP reward such that only a complete globally optimal solution receives nonzero reward. This is not the standard setting in RL4CO or NCO, where the cost of a feasible solution is typically used as a dense terminal objective. If the paper intends to study sparse rewards, it should clearly frame this as an artificially constructed threshold or gated benchmark rather than a general COP setting.
The experimental evaluation is insufficiently justified and documented. The paper repeatedly refers to globally optimal solutions, yet does not clarify whether exact optima are available for all tasks, how they are obtained, or how they are used. Even basic information such as instance sizes is missing, along with important details on instance generation, dataset splits, training settings, and inference procedures. The set of compared methods is also quite limited, making it difficult to assess the significance of the reported results. Moreover, the evaluation mainly reports raw final costs without optimality gaps or other standard metrics, making it difficult to assess solution quality or compare performance across tasks. Finally, some of the related GFlowNet references described as “recent” are in fact relatively old, which raises concerns about the completeness and currency of the related-work discussion.
Questions for authors
I would like the authors to restate their motivation and clearly define the reward function for the COP.