- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper investigates whether official finite-K rankings of neural multi-objective combinatorial optimization (MOCO) solvers depend on how final solution sets are extracted from decoded candidate pools. It distinguishes the neural constructor from the extractor and compares official preference-based WS/TCH/PBI selection with Preference-Agnostic Selection (PAS), which applies classical greedy hypervolume (HV) subset selection to the same frozen pool without additional neural decoding or local search. The experiments identify PAS-induced mean-rank reversals for closely performing constructors in certain settings, while most model-pair mean rankings reported in Table 1 remain unchanged under PAS. The paper also introduces Unmoved-then-Align (U2N) as a separate local-inefficiency diagnostic, combining dominance-monotone refinement, preference-guided search restricted to T-unmodified solutions (W|R), and final HV-based selection. The authors recommend reporting official-extractor and PAS HV side by side to distinguish performance under the official preference-based extraction protocol from the HV of a set-level-selected K-archive on the same pool.
Critical strengths
1.It identifies a mismatch in existing reference-based neural multi-objective combinatorial optimization methods regarding solution selection and result evaluation (HV-based evaluation of solution sets). This finding is of great significance for comparing algorithms more objectively. 2. The paper evaluates five established neural constructors across multiple routing and knapsack problem settings. PAS improves HV in 53 of 56 finite-K settings, with three ties, providing evidence of extraction-related performance losses. The authors also report cases where model rankings remain unchanged and appropriately separate NHDE-P's unbounded external archive from fixed-K comparisons. 3. The analyses of PAS, refinement of the official archive, T-only refinement, residual preference-guided search, unrestricted W, and larger preference budgets provide useful insights into the relative contributions of candidate-pool selection and local refinement. These comparisons help distinguish pool-truncation losses from improvements obtained through additional local search.
Critical weaknesses
1.The writing is obscure and hard to follow, containing numerous disjointed and jumpy descriptions, which seriously hinders readers from grasping the intended meaning. Besides, many symbols are ambiguous or not defined in advance. 2.The paper demonstrates that changing the extraction protocol can reverse mean rankings, but the statistical evidence supporting a new superior constructor is less conclusive in some key cases. For example, Table 1 reports that the WE-CA versus GIMF-P mean HV difference on Bi-TSP20 changes from +0.00044 under official extraction to −0.00008 under PAS. However, Table A.39 shows that the 95% bootstrap confidence interval for the PAS difference is [−0.00017, +0.00002], which includes zero. The confidence interval for the change in the difference excludes zero, supporting an extractor effect, but not establishing a statistically reliable new winner. This distinction does not invalidate the observed mean-rank reversal. Nevertheless, the conclusions would be stronger if the paper more explicitly distinguished changes in observed rankings from statistically supported changes in relative superiority. The results also suggest that most larger-gap comparisons preserve their original mean rankings, so the practical scope of the ranking sensitivity should be emphasized. 2. Section 4.2 provides a reasonable motivation for restricting preference-guided search to solutions not improved by the dominance-monotone T operator. However, Table 3 and Table A.46 show that unrestricted Wall achieves higher HV in several settings. For example, on WE-CA Bi-TSP100, Wall obtains 0.70764 compared with 0.70678 for U2N, while both report 101 unique non-dominated solutions. This observation does not necessarily imply that Wall is preferable, since unrestricted search can reduce non-dominated solution counts in other settings and may require additional neighborhood evaluations. However, the current evidence does not fully isolate the benefit of the specific residual restriction. Reporting actual neighborhood evaluation counts and comparing the methods under similar search-effort budgets would provide a clearer justification for this design choice. 3. Although PAS requires no additional neural decoding or local search, it still incurs non-negligible subset-selection costs. Appendix E and Table A.15 report approximately 10–11 minutes of PAS postprocessing per 100 Bi-TSP100 instances, with full U2N requiring approximately 19–23 minutes. The paper appropriately distinguishes these offline CPU costs from the already-incurred GPU decoding budget and acknowledges that the current implementation is not fully optimized. Nevertheless, since the authors recommend PAS as a general reporting companion to official extraction, its practical computational overhead is relevant. The asymptotic complexity discussion does not fully establish the efficiency of an optimized implementation across different candidate-pool sizes and objective dimensions. Additional optimized runtime results or a more systematic cost–quality analysis would strengthen the practical recommendation.
Questions for authors
1.Given that the PAS mean-gap confidence interval for WE-CA versus GIMF-P on Bi-TSP20 includes zero (Table A.39), how should readers interpret the reported mean-rank reversal? Could the authors clarify how the statistical uncertainty affects the interpretation of the reported mean-rank reversals, as opposed to a descriptive change in mean ranking? 2.What is the actual neighborhood-search effort required by U2N compared with unrestricted Wall? Could the authors compare HV, unique non-dominated solution counts, and neighborhood evaluations under similar computational budgets? Such an analysis would help determine whether restricting W to the unmodified residual provides a favorable performance–cost trade-off. 3.Since the current PAS implementation incurs substantial offline processing time, what performance can be expected from an optimized implementation of greedy HV subset selection? Could the authors provide runtime measurements or complexity scaling with respect to candidate-pool size and objective dimensionality to support the proposed dual-track reporting protocol? 4.What does Aoff mean? The full names of PAS and U2N should be restated in the main text. What do M1–M3 stand for? What is N(x) in M2 and J_k(P) in M3? Symbols cannot be directly used without prior definition or citation. This makes the paper unintelligible. What is the intention of the paragraph “A K=2 count...” after Contributions?