- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes DPaO, which turns multi-objective sequential optimization into a learning-to-rank problem. It builds rankings from Pareto dominance and uses a Plackett–Luce model to train a generative policy directly. Experiments compare DPaO with policy-based and GFlowNet baselines on synthetic sequence tasks, neural architecture search, DNA design, and molecule generation, using hypervolume and other quality and diversity metrics. Ablations examine the effects of tie-breaking, the ranking model, and the entropy coefficient.
Critical strengths
-
The method trains a generative policy directly from Pareto-based rankings, without choosing scalarization weights or a hypervolume reference point for training. This reduces task-specific choices and provides a useful alternative to decomposition-based approaches.
-
The experiments cover several domains with 10 seeds per setting, showing clear gains on Bigram-3D/4D and improved hypervolume on some DNA tasks. The repeated runs and ablations strengthen the evidence that the approach is effective on these tasks.
Critical weaknesses
-
Since ranking-based optimization is already established, the contribution depends on how LTR is adapted to Pareto optimization. However, Eq. 1 introduces preferences between some Pareto-incomparable solutions based on their domination counts, and these preferences can change with the batch composition. Respecting dominance does not justify these additional preferences. The paper does not sufficiently explain why learning them should improve Pareto-set quality, leaving its central motivation only partially supported.
-
The experiments do not establish a clear overall performance advantage. On Unigram-2D, DPaO achieves an HV of 0.16 ± 0.20, compared with 0.46 ± 0.03 for OP-GFN (Table 6). It also trails POCCO on the two-objective DARTS tasks. On molecule generation, its higher HV comes with worse IGD+ and R2 than MOGFN (Table 19). These results limit the evidence for a reliable improvement over existing methods.
Questions for authors
-
Could you clarify how DPaO relates to RaM [1] and Global-Order GFlowNets [2]? RaM uses LTR for offline optimization, while Global-Order GFlowNets address conflicts in local Pareto supervision. What additional challenge does DPaO solve, and how does its ranking construction differ from these approaches?
-
Eq. 1 uses domination counts to assign ranks, but Appendix D mentions non-dominated sorting. These are not generally equivalent. Which one was used in the experiments?
-
I am not sure why different domination counts should imply a preference between two Pareto-incomparable solutions. These preferences can also change depending on the other samples in the batch. What justifies learning them, and have you checked their effect on the resulting solution set?
-
When all samples in a batch are mutually non-dominated, Eq. 5 seems to give a likelihood of one and therefore no ranking gradient. Is this correct? If so, what encourages the policy to maintain coverage in this case?
-
Appendix H says that training uses 32,000 trajectories, while Figures 5 and 6 show roughly three million. Could you clarify how trajectories are counted and what budget was used? Were all methods given the same number of objective evaluations?
[1] Tan et al. Offline Model-Based Optimization by Learning to Rank. ICLR 2025.
[2] Pastor-Pérez et al. Global-Order GFlowNets. ICLR 2025 workshop.
5 comments
2. Relation between non-dominated sorting and dominance count: Our initial draft had a mistake, caused by our goal to specify the learning target in the most compact way. Indeed, we use the Pareto Ranking [1, 2] which can be computed by iteratively finding Pareto fronts from the dataset, always removing the currently found Pareto front from the dataset. We have updated the manuscript (Sec 4) with the correction, and included a pseudocode for computing Pareto Rank.
3. I am not sure why different domination counts should imply a preference between two Pareto-incomparable solutions? As noted above, our initial draft had a mistake; Indeed, we do not assume preference between two Pareto-incomparable solutions, and instead learn a ranking over the partial ordering induced by Pareto dominance.
4.What happens when all solutions in a batch are mutually non-dominated: When all points in a batch are non-dominated, there is no information to perform an effective update. This is an inherent disadvantage for all batch based update rules. While this is theoretically plausible scenario, we empirically demonstrate that this is not an issue in practice. Appendix D performs ablation over the batch size, showing that the number of partitions is always positive. Furthermore, the performance remains consistent with changing batch size.
5. Number of samples: The x-axis label for Figures 5 and 6 contained a typo (10^5 instead of 10^4), which has now been corrected. All methods are given 32 000 samples. We do not perform any caching of values, so each generated solution is removed from the total budget. This is standard practice followed by other baselines (e.g.[3, 4] ).
[1] K. Deb, et al., "A fast and elitist multiobjective genetic algorithm: NSGA-II," in IEEE Transactions on Evolutionary Computation, 2002
[2] Pastor-Pérez et al. Global-Order GFlowNets. ICLR 2025 workshop.
[3] Jain, M. et al. Multi-Objective GFlowNets. ICML 2023
[4] Chen, Y., & Mauch, L. Order-Preserving GFlowNets. ICLR 2024
Weakness 2:
- We have performed additional experiments in molecular generation tasks, considering synthetic accessibility and molecular weight as additional objectives in addition to the LogP and QED objectives to better match the real-world molecule generation (Table 19). In this updated task, we obtain the best performance in HV, IGD+, R2 and -metric.
- We also fixed a mistake in DNA task normalization. After rerunning these experiments (Tables 16, 17, 18), we obtain the best performance in most cases (tied with MOGFN in 3D case for HV, but better in R2)
- In 2D DARTS benchmarks, with near convex solution sets we obtain competitive performance with POCCO, while outperforming it in higher dimensional tasks with non-convex solution sets.
Table 16: DNA (Energy, Inv. Length):
| Method | HV () | R2 () |
|---|---|---|
| POCCO (L) | 0.36$\pm$0.07 | 4.44$\pm$0.65 |
| POCCO (C) | 0.08$\pm$0.06 | 6.24$\pm$0.63 |
| MOReinforce (L) | 0.12$\pm$0.01 | 6.45$\pm$0.36 |
| MOReinforce (C) | 0.13$\pm$0.02 | 6.42$\pm$0.37 |
| MOURex (L) | 0.11$\pm$0.11 | 5.98$\pm$0.85 |
| MOUrex (C) | 0.05$\pm$0.03 | 6.47$\pm$0.17 |
| CAPQL | 0.12$\pm$0.01 | 6.73$\pm$0.28 |
| MOGFN | 0.33$\pm$0.05 | 4.40$\pm$0.58 |
| OP-GFN | 0.41$\pm$0.13 | 3.79$\pm$0.89 |
| DPaO (ours) | 0.60$\pm$0.09 | 2.86$\pm$0.54 |
Table 19: Small molecule optimization (LogP, QED, SA, MW)
| Method | HV () | IGD+ () | R2 () | -metric () | PC entropy () |
|---|---|---|---|---|---|
| POCCO (L) | 0.29$\pm$0.02 | 0.44$\pm$0.02 | 2.96$\pm$0.17 | 0.39$\pm$0.02 | 1.37$\pm$0.20 |
| POCCO (C) | 0.24$\pm$0.02 | 0.50$\pm$0.03 | 3.27$\pm$0.19 | 0.46$\pm$0.03 | 1.14$\pm$0.29 |
| MOReinforce (L) | 0.26$\pm$0.02 | 0.46$\pm$0.02 | 3.09$\pm$0.12 | 0.42$\pm$0.03 | 1.31$\pm$0.34 |
| MOReinforce (C) | 0.27$\pm$0.02 | 0.46$\pm$0.02 | 3.05$\pm$0.16 | 0.42$\pm$0.03 | 1.28$\pm$0.35 |
| MOURex (L) | 0.26$\pm$0.02 | 0.47$\pm$0.03 | 3.12$\pm$0.17 | 0.42$\pm$0.02 | 1.32$\pm$0.13 |
| MOUrex (C) | 0.26$\pm$0.01 | 0.47$\pm$0.02 | 3.09$\pm$0.15 | 0.42$\pm$0.03 | 1.36$\pm$0.12 |
| CAPQL | 0.21$\pm$0.05 | 0.52$\pm$0.05 | 3.43$\pm$0.46 | 0.48$\pm$0.06 | 1.15$\pm$0.30 |
| MOGFN | 0.27$\pm$0.02 | 0.46$\pm$0.03 | 3.05$\pm$0.17 | 0.41$\pm$0.03 | 1.38$\pm$0.18 |
| OP-GFN | 0.22$\pm$0.05 | 0.49$\pm$0.07 | 3.50$\pm$0.38 | 0.46$\pm$0.07 | 0.92$\pm$0.34 |
| DPaO (ours) | 0.30$\pm$0.05 | 0.44$\pm$0.02 | 2.93$\pm$0.41 | 0.39$\pm$0.01 | 1.35$\pm$0.18 |
Comparison to existing methods
GR-GFN comparison in representative environments. Better performing method in each metric is bolded. We can provide full evaluation results for GR-GFN upon request.