acceptodds
Under review as a conference paper at ICLR 2027

Gating Is Attention: A Plug-and-Play Preference-Gated Dual-Attention Module for Neural Multi-Objective Combinatorial Optimization

Abstract

Neural multi-objective combinatorial optimization (MOCO) solves scalarized subproblems conditioned on preference vectors. Existing methods mainly use preferences for early representation fusion, weight-conditioned embedding, or route selection. However, they provide limited control over the attended features that directly guide decisions. We propose Preference-Gated Dual Attention (PGDA), a lightweight plug-and-play module that treats objective preferences as conditional controls over attention outputs. After standard attention, PGDA generates a gate from the context, the preference embedding, and the attention output. The gate then reweights the attended representation for preference-aware decision making. This modulation prioritizes preference-relevant features without altering instance representations. Experiments on multiple MOCO benchmarks show that PGDA provides consistent gains for single-model solvers and strong POCCO-based backbones. PGDA also achieves competitive solution quality with efficient inference, especially in cross-scale and large-scale generalization settings.

Primary area:
optimization
Keywords:
Neural Multi-Objective Combinatorial Optimization, Preference-Gated Attention, Multi-Objective Optimization, Neural Combinatorial Optimization, Preference Conditioning, Attention Mechanism

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_psMrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Preference-Gated Dual Attention (PGDA), which allows the preference vector to further control representations after attention aggregation. It uses a preference FFN and a gate FFN that takes the context, preference, and attention output as inputs, and applies dimension-wise gating to attention outputs in both the encoder and decoder. Experiments cover three MOCO benchmarks, multiple backbones, POCCO integration, cross-scale generalization, and instance augmentation, using normalized HV, Gap, and inference Time as the main metrics.

Critical strengths

None

Critical weaknesses

The novelty is insufficiently established. PGDA essentially applies a preference-conditioned feature gate to the output of standard attention. The paper does not clearly establish a substantive technical distinction from these existing approaches, nor does it provide sufficiently convincing comparisons to demonstrate that PGDA constitutes a meaningful methodological contribution.

The experimental evaluation raises serious concerns about the reliability and interpretability of the reported results. The experimental setup appears to closely follow POCCO, including the benchmark settings and even the presentation style of several figures. However, unlike POCCO, which explicitly defines Gap as the relative difference between a method’s HV and the HV of POCCO-W, this paper does not clearly specify what its reported Gap is measured against. More concerningly, the reported Gap values are not consistent with those in the original POCCO results, while many datasets report a Gap of 0.00% without even providing a clear reference value that could explain how this number is obtained. As a result, it is unclear how the reported Gap values were computed or how the experiments were actually conducted. This lack of a clearly defined and reproducible evaluation protocol makes the reported results difficult to independently verify and raises substantial concerns about the reliability of the experimental data.

Questions for authors

At this stage, I have substantial concerns about both the contribution of this paper and the validity of its experimental evaluation. I would appreciate further clarification from the authors to establish the originality and independence of their work, particularly given its close resemblance to POCCO in both the experimental setup and presentation. More importantly, the authors should provide sufficient details and evidence to clarify how their experiments were conducted and how the reported results were obtained, so that the work can be independently verified and distinguished from simply reproducing or closely imitating POCCO.

7 comments
AuthorsOpenReview ↗

Reponse to CW: The novelty is insufficiently established

Response CW1

Thank you for this comment. PGDA's contribution is not element-wise multiplication itself, but a preference-guided, context- and output-aware post-attention mechanism that directly modulates attended features for neural MOCO.

1. Methodological distinction. As shown in Figure 1 and Section 4.2 (Eqs. (5)–(8)), PGDA generates gates jointly conditioned on the pre-attention context (X), attention output (Y), and preference representation (\mathrm{Pref}):

\widetilde{Y}=g\odot Y.$$ Unlike methods that primarily inject preferences into representations or select computation paths, PGDA modulates attention outputs while retaining the standard QKV attention computation. **Appendix E (Propositions 1–3)** further analyzes its distinction from conventional gating, residual-aware modulation, and preference-conditioned gradient filtering. **2. Experimental evidence.** We provide four complementary validations: - **Preference-only ablation (Figure 4):** Full PGDA outperforms preference-only gating on all five Bi-TSP scales, including **0.7356 vs. 0.7300** HV on Bi-TSP200. - **Parameter-matched control (Appendix N, Tables 14–15):** PGDA outperforms the capacity-matched POCCO-W-WIDE baseline in **10 of 14 settings**, with three ties and one lower result. - **Mechanism analysis (Appendix J, Figures 9–11):** Learned gates modify attended features, policy concentration, and actual next-node decisions under controlled interventions. - **Generality (Appendix G, Figure 6; Appendix K, Tables 9–10):** PGDA works across CNH, WE-CA, and POCCO-based backbones. On euclidAB500 and clusAB500, POCCO-W-PGDA improves HV from **0.7860 to 0.7975** and **0.8517 to 0.8624**, respectively. **Together, these results establish that PGDA offers a distinct preference-conditioned attention modulation mechanism whose benefits cannot be explained solely by increased parameter capacity.** The revised manuscript now provides both theoretical distinctions and targeted experimental controls to substantiate this contribution.
AuthorsOpenReview ↗

Reponse to CW: The experimental evaluation raises serious concerns about the reliability and interpretability of the reported results.

Response CW2

Thank you for this comment. We have clarified the HV Gap calculation, disclosed the reference values used for each benchmark, and documented the evaluation procedure to make the reported results independently verifiable.

1. How are the reported Gap values calculated?

Different neural MOCO studies use different Gap reference conventions. To ensure consistent comparisons, we reconstruct a benchmark-specific zero-Gap reference HV from the published HV and Gap pairs in WE-CA (Chen et al., 2025).

Specifically, we use WS-LKH for Bi-TSP, Tri-TSP, and KroAB; MOEA/D for Bi-CVRP; and WS-DP for Bi-KP. The calculation is:

  • HVref=HVbaseline/(1−Gapbaseline/100)HV_{ref} = HV_{baseline} / (1 - Gap_{baseline} / 100)

  • Gap=(HVref−HVmethod)/HVref×100Gap = (HV_{ref} - HV_{method}) / HV_{ref} × 100\\%

For example, on Bi-TSP20, WE-CA reports WS-LKH with HV = 0.6270 and Gap = 0.02%, yielding a reconstructed zero-Gap reference HV of 0.62713.

Representative reference values are shown below.

BenchmarkReference methodReported HV / GapZero-Gap HV
Bi-TSP20WS-LKH0.6270 / 0.02%0.62713
Bi-TSP100WS-LKH0.7090 / −0.33%0.70665
Bi-CVRP20MOEA/D0.4255 / 1.07%0.43010
Bi-KP100WS-DP0.4532 / 0.02%0.45329

All methods on the same benchmark are evaluated against the same reconstructed reference. Consequently, differences from POCCO's reported Gap values result from different reference conventions, not changes to the underlying HV results. Reported Gap values of 0.00% may also arise from rounding to two decimal places.

The complete reconstruction for all 17 benchmarks is provided in the revised Appendix O, “HV Gap Calculation in the Main Text” (Table 16). We have also clarified the evaluation protocol in Section 5.1. The additional OOD benchmarks in Appendix K explicitly use their own WS-LKH-relative Gap convention.

2. Why do some figures resemble those in POCCO?

We follow visualization conventions widely used in neural MOCO research because they facilitate cross-paper comparisons and make benchmark results easier to interpret.

Importantly, our manuscript also provides PGDA-specific analyses beyond standard performance visualization:

  • Figure 1: PGDA architecture and post-attention gating mechanism.

  • Figure 4: Component and preference-only gating ablations.

  • Figures 6–7: Cross-backbone effectiveness and architectural sensitivity.

  • Figures 9–11: Learned feature modulation, policy entropy, and controlled decoding interventions.

These figures directly examine PGDA's architecture, behavior, and effectiveness rather than reproducing POCCO's analyses.

Overall, the revised manuscript now provides explicit Gap definitions, complete benchmark-specific reference values, and additional PGDA-specific analyses. These revisions make the reported results easier to reproduce, verify, and distinguish from previous work.

AuthorsOpenReview ↗

Response to Question

Response Question

We have strengthened the manuscript to clarify PGDA's independent contribution, experimental procedures, and reproducibility.

1. Originality and independence. Unlike POCCO's conditional computation and pairwise preference learning, PGDA introduces preference-guided post-attention modulation conditioned on the context, attention output, and preference representation (Section 4.2, Figure 1; Appendix E, Propositions 1–3).

Importantly, PGDA also improves CNH and WE-CA independently of POCCO (Appendix G, Figure 6). Further evidence includes the preference-only ablation (Figure 4), parameter-matched WIDE control (Appendix N, Tables 14–15), and controlled gate analyses (Appendix J, Figures 9–11). These experiments distinguish PGDA's contribution from simply increasing model capacity or reproducing POCCO.

2. Experimental reliability and reproducibility. We have clarified the training and evaluation protocols in Sections 4.3 and 5.1 and Appendix B. To eliminate hardware-related runtime inconsistencies, we re-evaluated all POCCO-C/W baselines and their PGDA variants on the same 48-GB vGPU, using single-GPU inference and matched evaluation settings. All corrected inference times are reported in Tables 1–2 and the corresponding appendix results.

We also explicitly define HV Gap and provide the complete benchmark-specific reference values reconstructed from WE-CA in Appendix O (Table 16), enabling independent verification. Appendix P further documents the provenance of existing assets.

3. Experimental presentation. We adopt established neural MOCO visualization conventions to facilitate cross-paper comparisons. The manuscript also includes PGDA-specific architectural illustrations, ablations, feature analyses, and controlled decoding interventions that go beyond standard performance comparisons.

Overall, PGDA is methodologically distinct from POCCO, and the revised manuscript provides targeted experimental controls, standardized runtime measurements, and explicit evaluation references to substantiate its contribution and the reliability of its results.

Reviewer_NpaMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Preference-Gated Dual Attention (PGDA), a plug-and-play module for neural multi-objective combinatorial optimization. The core idea is to modulate the output of standard attention using a preference-conditioned gate generated from the pre-attention context, the preference embedding, and the attention output itself. The same mechanism is applied in both the encoder and decoder, while the underlying training procedures remain those of the corresponding backbones, including REINFORCE-based training and POCCO-style pairwise preference learning.
The method is evaluated on multi-objective TSP, CVRP, and knapsack problems, covering bi-objective and tri-objective settings, multiple neural backbones, large-scale instances, and KroAB benchmark instances. The experiments show that PGDA can improve several preference-conditioned solvers, with more visible gains in some large-scale and out-of-distribution settings, although the improvements are not uniformly positive across all backbones and problem sizes.

Critical strengths
  1. The paper provides relatively broad evidence for the plug-and-play nature of PGDA. The module is evaluated with multiple neural MOCO backbones, including CNH, WE-CA, and POCCO variants, and across TSP, CVRP, and knapsack problems, with additional tri-objective, large-scale, and KroAB evaluations.

  2. The method is simple and easy to integrate into existing solvers. It leaves the underlying attention and training objectives largely unchanged and introduces preference information directly at the attended representation used for decision making. The large-scale and KroAB experiments provide some evidence that this modulation can be particularly useful under scale and distribution shifts.

Critical weaknesses
  1. The main methodological novelty remains somewhat limited and is not fully isolated experimentally. PGDA is essentially a preference-conditioned multiplicative gate applied to the output of standard attention. The current ablations remove individual PGDA inputs or insertion locations, but I did not find a parameter-matched control such as a non-preference gate, FiLM-style modulation, or an ordinary MLP with comparable additional capacity. This is important because PGDA substantially increases the parameter count of some backbones; for example, POCCO-W increases from 2.00M to 3.38M parameters. Therefore, the current evidence does not fully establish that the gains arise specifically from the proposed preference-aware gating rather than from additional model capacity.

  2. The reported gains over strong backbones are often small and not uniformly positive, while the statistical evidence does not clearly account for training randomness. For example, PGDA slightly degrades POCCO-W on Bi-TSP20/50 and provides only modest improvements in many other settings. The paper reports a Wilcoxon signed-rank test over test instances, but does not clearly report results across independent training seeds.

Questions for authors
  1. PGDA increases the parameter count of POCCO-W from 2.00M to 3.38M, while the current ablations mainly remove PGDA inputs or change insertion positions. Could the authors provide a parameter-matched control, such as a non-preference gate, FiLM-style modulation, or an ordinary MLP with comparable capacity, to verify that the gains come specifically from preference-aware gating rather than additional model capacity?

  2. All neural methods are evaluated with weighted-sum scalarization. Since weighted sum may not recover unsupported Pareto-optimal solutions on non-convex fronts, how confident are the authors that the observed benefits of PGDA extend to other decomposition functions such as Tchebycheff or PBI?

  3. The KroAB evaluation is used to support out-of-distribution generalization, but each KroAB size corresponds to a single benchmark instance. Is this evidence sufficient to support the broader OOD-generalization claim, or could the authors provide additional instances or repeated distribution-shift evaluations?

  4. The authors attribute the slight degradation on Bi-TSP20/50 to preference gating reducing solution-set diversity, but this explanation appears speculative. Could this hypothesis be supported by a direct diversity/coverage analysis of the generated Pareto sets or by an analysis of the learned gates?

8 comments
AuthorsOpenReview ↗

Response to CW1: Parameter-Matched Control and PGDA's Methodological Contribution

Response CW1

Thank you for this comment. We have added a parameter-matched control to distinguish PGDA's architectural contribution from increased model capacity. The results show that simply adding parameters does not reproduce PGDA's performance gains.

1. Does increased model capacity explain PGDA's improvements?

No. We introduce POCCO-W-WIDE, which increases the original FFN width without adding PGDA. The resulting parameter counts closely match those of POCCO-W-PGDA.

Table R1. Parameter-matched comparison (Appendix N, Table 14).

ProblemFFN width (original → WIDE)POCCO-WW-WIDEW-PGDA
Bi-TSP512 → 10522.00M3.38M3.38M
Bi-CVRP512 → 10521.98M3.37M3.37M
Bi-KP512 → 10142.08M3.37M3.37M

The WIDE and PGDA variants differ by less than 0.02% in parameter count, enabling a direct comparison of their architectural designs.

2. Does PGDA outperform the parameter-matched control?

Yes, in 10 of 14 evaluated settings, with three ties and one lower result, as reported in Appendix N (Table 15).

Table R2. Non-augmented HV comparison (higher is better).

BenchmarkPOCCO-WW-WIDEW-PGDA
Bi-TSP200.62750.62740.6274
Bi-TSP500.64110.64100.6410
Bi-TSP1000.70550.70520.7060
Bi-TSP1500.70330.70130.7042
Bi-TSP2000.73710.73410.7384
Bi-CVRP200.42940.42940.4295
Bi-CVRP500.41020.41020.4103
Bi-CVRP1000.40840.40820.4085
Bi-KP500.35620.35610.3561
Bi-KP1000.45340.45340.4531
Bi-KP2000.36030.36020.3603
KroAB1000.69810.69740.6991
KroAB1500.69460.69330.6956
KroAB2000.73450.73300.7346

The advantage becomes particularly evident on larger instances. Compared with WIDE, PGDA improves HV by +0.0043 on Bi-TSP200, +0.0023 on KroAB150, and +0.0016 on KroAB200.

3. Does the gating design itself matter?

Yes. The additional preference-only control in Figure 4 further isolates the contribution of context- and output-aware modulation. Full PGDA outperforms preference-only gating across all five Bi-TSP scales, including 0.7018 vs. 0.6969 on Bi-TSP150 and 0.7356 vs. 0.7300 on Bi-TSP200.

Together, these controls show that additional model capacity alone cannot explain PGDA's gains, while jointly conditioning the gate on context, attended features, and preference provides measurable benefits.

We have incorporated these experiments into Appendix N (Tables 14–15) and Figure 4, strengthening the evidence for PGDA's methodological contribution.

AuthorsOpenReview ↗

Response to CW2: Training Robustness and Scale-Dependent Performance

Response CW2

Thank you for this comment. We acknowledge that PGDA does not improve every setting and that instance-level Wilcoxon tests do not directly measure training-seed variability. However, additional independently retrained variants and large-scale experiments provide further evidence of PGDA's robustness and effectiveness.

1. Are the improvements limited to a single trained configuration?

No. In Appendix H Figure 7, we independently retrained six PGDA variants with different gate depths, hidden dimensions, and insertion positions.

Although these variants do not always match full PGDA, all six outperform the original POCCO-W on Bi-TSP100/150/200 and augmented KroAB100/150/200. This shows that the improvements persist across different PGDA designs and independent training runs.

These experiments evaluate different architectural configurations, rather than multiple random seeds of one fixed architecture. They therefore provide evidence of robustness across designs, while the instance-level Wilcoxon tests assess a separate source of variability.

2. Why are some gains small or negative?

POCCO-W already incorporates FiLM-based preference modulation, which may interact with PGDA's additional post-attention gating.

Our mechanism analyses in Appendix J (Figures 9–11) provide supporting evidence:

  • Figures 9–10 reveal different feature-attenuation patterns between POCCO-C-PGDA and POCCO-W-PGDA.
  • Figure 11 demonstrates changes in policy entropy and next-node decisions under controlled gate interventions. POCCO-W-PGDA exhibits stronger policy concentration during some decoding stages.

Figure 9: POCCO-C-PGDA feature modulation | Figure 10: POCCO-W-PGDA feature modulation | Figure 11: Gate effects on decoding

We also provide six GIF animations illustrating how the decoder feature representations change at each decoding step across Bi-TSP20/100/200:

POCCO-C-PGDA: Bi-TSP20 | Bi-TSP100 | Bi-TSP200

POCCO-W-PGDA: Bi-TSP20 | Bi-TSP100 | Bi-TSP200

Together, these analyses visualize the backbone-dependent effects of gating and provide a plausible explanation for the minor degradations on smaller instances.

3. Do the improvements persist on larger problems?

Yes. As shown in Appendix K (Tables 9–10), both PGDA-enhanced backbones improve HV across all four additional 300- and 500-node Euclidean and clustered settings without augmentation.

Table R1. HV improvements from PGDA on additional large-scale instances (higher is better).

DatasetΔHV (POCCO-C-PGDA)ΔHV (POCCO-W-PGDA)
euclidAB300+0.0180+0.0023
euclidAB500+0.0469+0.0115
clusAB300+0.0267+0.0040
clusAB500+0.0014+0.0107

Notably, POCCO-W-PGDA improves HV by +0.0115 on euclidAB500 and +0.0107 on clusAB500, despite its minor degradations on Bi-TSP20/50.

Overall, the independently retrained variants, mechanism analyses, and additional large-scale results provide complementary evidence that PGDA's benefits extend beyond a single trained configuration. The revised manuscript includes these analyses in Appendices H, J, and K and clarifies the backbone- and scale-dependent nature of the improvements.

AuthorsOpenReview ↗

Response to Q1: Parameter-Matched Capacity Control

Response Q1

Thank you for this comment. We have addressed this concern through a parameter-matched control experiment, detailed in our Response C1.

Specifically, we introduce POCCO-W-WIDE, which increases the original FFN width without adding PGDA, resulting in nearly identical parameter counts to POCCO-W-PGDA (differences below 0.02%).

Across 14 non-augmented benchmarks, PGDA outperforms WIDE in 10 settings, ties in three, and performs slightly worse in one. These results show that simply increasing model capacity does not reproduce PGDA's overall performance gains.

The complete parameter counts and HV comparisons are provided in Appendix N (Tables 14–15). Please refer to Response CW1 for the detailed experimental results and analysis.

Reviewer_pYpyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Preference-Gated Dual Attention (PGDA) for multi-objective combinatorial optimization. The method introduces a gating module after standard attention in both the encoder and decoder. The module jointly uses contextual representations, a preference embedding, and the attention output to generate sigmoid gates that reweight the attention output element-wise, allowing preference information to condition both encoding and decoding throughout the model.

Critical strengths
  1. The module has a straightforward design, can be applied to both the encoder and decoder, and is compatible with different training approaches, suggesting potential for reuse.

  2. The experiments cover multiple multi-objective combinatorial optimization tasks and problem sizes, helping assess the method's applicability across different settings.

Critical weaknesses
  1. The performance gains are inconsistent across settings, and some improvements are small, limiting the empirical support for the module's effectiveness. In the non-augmented results in Tables 1 and 2, performance degradation occurs more frequently with POCCO-W than with POCCO-C. Although the authors offer possible explanations for some of these declines, they do not sufficiently explain why degradation occurs more often for POCCO-W in these results.
Questions for authors
  1. In several settings, adding PGDA increases the parameter count but reduces the reported total inference time. For example, replacing POCCO-C with POCCO-C-PGDA reduces the time from 16 s to 10 s on Bi-CVRP20 (Table 1), from 20 s to 16 s on Bi-KP50, and from 36 s to 29 s on Bi-KP100 (Table 2), all without augmentation. Could the authors explain these reductions and clarify whether the baseline and PGDA use identical evaluation budgets and timing procedures on the same hardware?

  2. The introduction suggests that early preference signals may weaken during deeper encoding. Could the authors fix the total encoder depth and decoder configuration, apply PGDA only in the first N layers, and increase N to examine whether the gains already saturate in shallow layers? This differs from varying the gating network's own depth, as studied in Appendix H.

  3. Could the authors include a gate conditioned only on hλh_\lambda as a control to assess the additional contribution of conditioning on the context and attention output?

  4. After preference-conditioned gating, the hidden representations of an instance generally vary with the preference. What, specifically, does “preserving instance representations” mean?

9 comments
AuthorsOpenReview ↗

Response to R1-CW1: Backbone-Dependent Performance and Generalization

Response CW1

Thank you for this comment. We acknowledge that PGDA does not uniformly improve all settings. Our additional analyses reveal backbone-dependent modulation patterns, while larger instances consistently benefit from PGDA.

1. Why does POCCO-W exhibit more degradations?

POCCO-W inherits FiLM-based preference modulation from WE-CA. PGDA's additional post-attention gating may interact with this existing conditioning mechanism, potentially attenuating useful features on smaller instances where the original preference-conditioned representations are already effective.

To investigate this behavior, we conducted the following analyses:

  • Feature modulation (Appendix J, Figures 9–10): We visualize decoder features before gating, learned gate values, and features after gating. The results reveal different feature-attenuation patterns across the two backbones.

    Figure 9: POCCO-C-PGDA feature modulation | Figure 10: POCCO-W-PGDA feature modulation

  • Decoding behavior (Appendix J, Figure 11): Controlled interventions at identical decoding states show that PGDA changes policy entropy and next-node decisions. POCCO-W-PGDA exhibits stronger policy concentration during some decoding stages, providing a plausible explanation for its greater sensitivity to additional gating.

    Figure 11: Gate effects on decoding behavior

We additionally provide six GIF animations to visualize how decoder feature representations evolve at every decoding step on Bi-TSP20/100/200. These animations allow direct observation of the dynamic gating process across different problem scales and backbones:

POCCO-C-PGDA: Bi-TSP20 | Bi-TSP100 | Bi-TSP200

POCCO-W-PGDA: Bi-TSP20 | Bi-TSP100 | Bi-TSP200

Together, the static analyses and step-by-step visualizations illustrate how PGDA modifies decoder features and decisions, supporting a backbone-dependent interpretation of the observed performance differences.

2. Does PGDA remain effective on larger instances?

Yes. We conducted additional evaluations on 300- and 500-node Euclidean and clustered Bi-TSP instances. The results in Appendix K (Tables 9–10) show consistent improvements for both PGDA-enhanced backbones.

Table R1. Non-augmented HV comparison on larger Bi-TSP instances (higher is better).

DatasetPOCCO-CC-PGDAΔHV (C)POCCO-WW-PGDAΔHV (W)
euclidAB3000.73480.7528+0.01800.75970.7620+0.0023
euclidAB5000.71060.7575+0.04690.78600.7975+0.0115
clusAB3000.80130.8280+0.02670.83260.8366+0.0040
clusAB5000.81060.8120+0.00140.85170.8624+0.0107

Both PGDA-enhanced backbones outperform their respective baselines across all four settings. Notably, POCCO-W-PGDA improves HV by +0.0115 on euclidAB500 and +0.0107 on clusAB500, despite its minor degradations on smaller problems.

Overall, PGDA's effectiveness is backbone- and scale-dependent. While additional gating may slightly affect well-established preference representations on smaller instances, it provides more substantial benefits on the tested larger problems. We have incorporated these mechanism analyses and additional experiments into the revised manuscript.

AuthorsOpenReview ↗

Response to Q1: Inference-Time Consistency and Fair Evaluation

Response Q1

Thank you for pointing out this inconsistency. The apparent inference-time reductions were caused by differences in the original hardware environments, rather than an inherent speedup from PGDA. We have therefore re-measured and updated the inference times of all POCCO-C/W baselines and their PGDA-enhanced variants across every evaluated problem instance and augmentation setting.

1. Unified hardware and evaluation protocol.

The original POCCO runtime values were taken from the published paper, which used NVIDIA A100 80-GB GPUs and an AMD EPYC 7742 CPU, whereas our initial PGDA measurements were obtained on a different platform with six 32-GB GPUs and 72 Intel Xeon Platinum 8375C vCPUs.

To eliminate this inconsistency, we re-evaluated all POCCO and POCCO-PGDA variants on the same platform, equipped with a 48-GB vGPU and 40 Intel Xeon Platinum 8470Q vCPUs. All inference experiments were conducted on a single GPU without multi-GPU parallel acceleration. Each baseline–PGDA pair used identical test instances, preference vectors, decoding configurations, augmentation settings, batch sizes, and timing procedures.

To prevent GPU out-of-memory (OOM) errors, we adjusted test_batch_size for specific settings: Bi-TSP50-Aug (20), Bi-TSP100-Aug (5), Bi-TSP200 (50), Bi-KP200 (100), and Bi-CVRP100-Aug (20). These adjustments were applied consistently to the corresponding baseline–PGDA pairs.

2. Corrected inference-time results.

The updated measurements resolve the reported inconsistencies. For example:

DatasetPOCCO-CPOCCO-C-PGDA
Bi-CVRP2010 s10 s
Bi-KP5013 s14 s
Bi-KP10024 s27 s

The corrected results show that PGDA introduces only a modest inference-time overhead, consistent with its additional gating operations. Importantly, these examples are not isolated corrections: all inference-time results for POCCO-C, POCCO-W, POCCO-C-PGDA, and POCCO-W-PGDA have been updated across the complete evaluation.

The revised measurements are reported in Tables 1–2 and the corresponding appendix results. We have also clarified the hardware configuration, single-GPU evaluation protocol, and batch-size adjustments in Section 5.1, ensuring fair and reproducible runtime comparisons.

AuthorsOpenReview ↗

Response to Q2: Encoder-Layer Coverage and Saturation

Response Q2

Thank you for this suggestion. We conducted additional encoder-layer coverage experiments with a fixed six-layer encoder and an unchanged decoder. The results show that shallow-layer PGDA captures most of the performance benefits after retraining, while deeper-layer gating provides additional improvements. This differs from the gating network depth analysis in Appendix H.

1. Does PGDA performance saturate in shallow encoder layers?

We first varied the number of PGDA-enabled encoder layers from N=1N=1 to N=6N=6 by progressively disabling deeper gates in the fully trained model.

As shown in Figure 8 (Appendix I), removing deeper gates substantially reduces HV on Bi-TSP and Bi-CVRP, particularly on larger instances. For example, increasing coverage from N=3N=3 to N=6N=6 improves HV from 0.669725 to 0.706009 on Bi-TSP100 and from 0.679228 to 0.738435 on Bi-TSP200.

Figure 8: Effect of PGDA encoder-layer coverage

These results indicate that the fully trained model relies on deeper gates. However, disabling gates after training may introduce a training–inference mismatch.

2. Can shallow-layer PGDA achieve comparable performance after independent retraining?

To distinguish architectural effectiveness from post-training gate removal, we independently retrained a model with PGDA enabled only in the first three encoder layers, keeping the total encoder depth and decoder unchanged.

Table 8 (Appendix I). HV comparison between independently retrained shallow-layer PGDA and full-layer PGDA, without augmentation.

BenchmarkRetrained N=3Full N=6
Synthetic Bi-TSP
Bi-TSP200.62730.6274
Bi-TSP500.64080.6410
Bi-TSP1000.70570.7060
Bi-TSP1500.70380.7042
Bi-TSP2000.73800.7384
Real-world KroAB
KroAB1000.69870.6991
KroAB1500.69470.6956
KroAB2000.73450.7346

As shown in Table 8, the retrained three-layer variant approaches, but does not fully match, the six-layer PGDA model. The remaining HV differences are approximately 1×10−41\times10^{-4} to 4×10−44\times10^{-4} on synthetic Bi-TSP and 1×10−41\times10^{-4} to 9×10−49\times10^{-4} on KroAB.

Overall, these experiments show that PGDA gains do not fully saturate in shallow layers. The first three layers capture most of the benefits after independent retraining, while deeper preference-guided modulation provides additional improvements.

We have incorporated both experiments into Appendix I (Figure 8 and Table 8) and clarified the Introduction to avoid implying that early preference signals necessarily disappear during deeper encoding.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Revisions and Additional Experimental Evidence

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their constructive feedback. We have uploaded a revised manuscript PDF, with all additions and modifications highlighted in blue for easy identification. We have also provided individual responses to each reviewer. The major revisions are summarized below.

1. Methodological Novelty

We clarified PGDA's distinction from existing preference-conditioning methods (Section 4.2; Appendix E) and added targeted controls:

  • Preference-only ablation (Figure 4): Full PGDA outperforms preference-only gating across all five Bi-TSP scales.
  • Parameter-matched control (Appendix N, Tables 14–15): PGDA outperforms the capacity-matched WIDE baseline in 10/14 settings, with three ties and one lower result.
  • Cross-backbone validation (Appendix G, Figure 6): PGDA is effective beyond POCCO, including CNH and WE-CA.

These experiments distinguish the proposed gating mechanism from simply increasing model capacity.

2. Experimental Reliability and Reproducibility

We addressed two important evaluation concerns:

  • Inference time: All POCCO-C/W baselines and PGDA-enhanced variants were re-evaluated on the same 48-GB vGPU, using single-GPU inference and matched evaluation protocols. The corrected results are reflected in Tables 1–2 and the appendices.
  • HV Gap: We clarified the unified reference calculation and provided complete benchmark-specific reference values in Appendix O (Table 16).

Training and evaluation details have also been clarified in the revised manuscript.

3. Robustness and Generalization

We conducted additional experiments to evaluate PGDA beyond the original settings:

  • Appendix H (Figure 7): Six independently retrained architectural variants demonstrate robustness across PGDA configurations.
  • Appendix I (Figure 8, Table 8): Encoder-layer coverage experiments distinguish shallow-layer effectiveness from deeper-layer contributions.
  • Appendix K (Tables 9–10): Six additional Euclidean/clustered instances at 100/300/500 nodes. PGDA improves both POCCO backbones in all 24 paired comparisons, including augmented evaluations, without retraining.
  • Appendix L (Tables 11–12): Independently trained Tchebycheff-based PGDA achieves higher reported HV than published TCH-trained POCCO-W results on all four compared Bi-TSP benchmarks.

4. Mechanism and Diversity Analyses

We added feature-level and decision-level analyses to explain PGDA's behavior, including its occasional small-instance degradations.

We additionally provide step-by-step decoding GIFs for Bi-TSP20/100/200:

POCCO-C-PGDA: TSP20 | TSP100 | TSP200

POCCO-W-PGDA: TSP20 | TSP100 | TSP200

Complete anonymous visualization repository

Conclusion

The revised manuscript provides stronger methodological controls, standardized runtime evaluations, transparent HV Gap calculations, and broader robustness and mechanism analyses. These additions directly address the reviewers' central concerns regarding novelty, reliability, and generalization.

We respectfully invite the Area Chair and Reviewers to review the uploaded PDF, with all revisions highlighted in blue, alongside our detailed reviewer responses.

Thank you again for your time and consideration.

AuthorsOpenReview ↗

Request for Reviewer Follow-up Before the Discussion Deadline

Dear Area Chair,

Thank you for the time and effort you have devoted to handling our submission and coordinating the review process.

As the rebuttal discussion period is nearing its end, we would be grateful if you could encourage reviewers who have not yet responded to our rebuttal to share whether our responses have addressed their concerns. We would welcome the opportunity to clarify any remaining questions while the discussion is still open.

Thank you again for your time and support.

Best regards, The author

Dear Authors,

Thank you for reaching out.

I have sent a reminder to the reviewers encouraging them to engage with the author response and to reply to the authors before the rebuttal deadline. I will also continue to encourage reviewers to participate in the subsequent reviewer–AC discussion.

I will make my recommendation based on careful consideration of the submission, all review comments, and your responses.

Thank you again for your time and effort.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.