acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Manifold‑Driven Knowledge Transfer via Reinforcement Learning for Multi‑Objective Multi‑Task Optimization

Abstract

Multi‑objective multi‑task optimization (MO‑MTO) improves search performance by reusing cross‑task manifold‑structured knowledge. Existing manifold‑driven MO‑MTO algorithms adopt a fixed knowledge‑transfer probability (KTP), which fails to adapt local transfer frequency according to runtime feedback including population evolution, manifold validity, and offspring survival outcomes. To address this limitation, we propose RL-EMT-MSKT, which formulates task-specific KTP adjustment as a Markov decision process and learns multiplicative KTP updates from runtime observations using proximal policy optimization. Specifically, we design a Cross‑Task Relation Encoder with scaled dot‑product attention to aggregate pairwise task states, and employ Proximal Policy Optimization to learn multiplicative KTP updates. We introduce an adaptive differential evolution (ADE) operator that adjusts its mutation and crossover parameters to strengthen intra-task search. For inter-task transfer, local structure learning (LSL) generates offspring using knowledge of task manifolds. The learned KTP then controls, for each individual, the probability of invoking LSL rather than ADE. Evaluated on the CEC21‑MTMO‑CPLX benchmark with IGD and normalized HV metrics alongside Wilcoxon statistical tests, our proposed method outperforms seven state‑of‑the‑art MO‑MTO competitors on most test instances. Ablation studies validate the individual contributions of the RL‑based KTP controller and ADE component.The source code is available at:https://github.com/tomz18690-cell/RL-EMT-MSKT

Primary area:
optimization
Keywords:
Multi-task optimization, multi-objective optimization, reinforcement Learning, transfer probability Control

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_o2CeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies multi-objective multi-task optimization (MO-MTO) and proposes RL-EMT-MSKT, which controls local manifold transfer at the offspring level. The core idea is to treat the task-specific knowledge-transfer probability (KTP) as a sequential decision variable. A shared PPO-based policy, with a Cross-Task Relation Encoder (CTRE), selects multiplicative KTP updates from a small discrete action set. The updated KTP then decides whether each parent generates a local offspring through Local Structure Learning (LSL) or through an adaptive differential evolution (ADE) fallback. The reward combines population-level SPEA2 fitness improvement with operator-specific survival feedback comparing LSL and ADE offspring. The method is evaluated on the CEC21-MTMO-CPLX benchmark with ten two-task problems, 30 independent runs, seven MO-MTO comparators, IGD and normalized HV metrics, statistical tests, KTP trajectory plots, and ablation variants. The paper claims improved solution quality, statistically significant wins over comparators, and useful contributions from both learned KTP control and ADE.

Critical strengths
  1. Coupling the transfer probability with offspring-level operator selection and feeding survival outcomes back into the policy is a reasonable way to address negative transfer in manifold-based MO-MTO. If the learned policy actually produced task-specific transfer schedules, this would be a meaningful contribution beyond fixed or heuristic transfer probabilities.

  2. The paper reports 20 task-level evaluations, 30 runs per task, seven MO-MTO baselines, two metrics, statistical comparisons, KTP trajectories, and ablation variants. This gives the reader enough material to see where the method helps and where it does not, even though the presentation has serious problems.

  3. The reward design is concrete and reusable. Combining a normalized population-quality term with an operator-survival term is a practical way to credit transfer decisions at the offspring level. This is a better-motivated design than using only final population quality, and it could be adapted to other adaptive transfer operators.

Critical weaknesses
  1. The central claim of adaptive task-specific transfer control is not well supported by the learned behavior. Figure 6 shows KTP trajectories collapsing to the lower bound of 0.05 on essentially all problems, which means LSL is selected only rarely after the early generations. In that regime the policy is mostly learning to suppress LSL and rely on ADE, not to produce a rich task-specific transfer schedule. This matters because the paper’s first two contributions are precisely the learned KTP policy and the cross-task relation encoder. In addition, with two tasks and the self-mask in Eq. (5), the CTRE attention reduces to a deterministic counterpart exchange. So the “relation encoder” is not learning a relation beyond a swap/concatenation. The paper needs to show that the policy is doing more than falling back to ADE.

  2. The policy is trained on 300 augmented two-task instances generated from the same ten CEC21-MTMO-CPLX problems and then evaluated on the unperturbed official versions of those same problems. There is no disjoint problem family, no more-than-two-task setting, and no real-world case. The paper itself notes this limitation, but the main claims are still stated broadly. For a strict venue, the gap between “trained on perturbations of the test family” and “general adaptive transfer control” needs to be narrowed with additional evidence.

  3. The main text describes an 11-dimensional task state, but Appendix Table 4 lists more components and contains typographical errors. Several details needed to reproduce the method are missing or scattered, including the feedback-window length, LPCA validity conditions, normalization constants, and the exact treatment of invalid manifold models. The ablation is reported mainly as average task-level IGD rank, without per-task statistics or a clear fixed-KTP baseline in the main paper. Runtime overhead for policy inference and manifold modeling is also not reported, which matters for an evolutionary algorithm paper.

Questions for authors
  1. Can you quantify the actual LSL selection rate after the first few hundred generations, and provide a fixed-KTP or no-LSL ablation with per-task IGD/HV statistics?

  2. The main text and Appendix E.2 disagree on the number of significant IGD wins against comparators, and Table 3 is incomplete. Can you provide clean full IGD/HV tables with corrected win/loss/tie counts, and state clearly whether the tests are Wilcoxon or Mann–Whitney, with or without Holm correction?

  3. Given Eq. (5) and the two-task self-masked attention, CTRE reduces to a deterministic counterpart exchange. What is learned by CTRE beyond this swap? Can you provide an ablation that removes CTRE or replaces it with simple concatenation of the two task embeddings?

  4. The policy is trained on augmented variants of the same ten official problems and tested on the unperturbed versions. How do you rule out benchmark-specific overfitting? Would the method still hold on a disjoint problem family or a setting with more than two tasks, and what is the wall-clock overhead of policy inference plus manifold modeling?

6 comments
AuthorsOpenReview ↗

1. Declining KTP trajectories and the contribution of LSL [W1 / Q1 (partial)]

We thank the reviewer for the observation regarding the learned KTP trajectories. We agree that declining KTP values alone are insufficient to fully characterize the contribution of local manifold transfer.

W1: KTP approaches 0.05 on many tasks, raising concerns that the learned policy largely suppresses LSL in favor of ADE.

Q1 (partial): What is the actual later-stage LSL selection rate, and what do fixed-KTP or no-LSL per-task IGD/HV ablations show?

To further investigate this issue, we have extended the ablation study in Table 8 by comparing the RL-EMT-MSKT with other versions with fixed KTP values of 0, 0.05, and 0.5, together with a predefined KTP decay schedule.

The results show that the RL-EMT-MSKT achieves the best average IGD rank of 2.75, outperforming fixed KTP = 0.5 (3.65), fixed KTP = 0.05 (4.05), fixed KTP = 0 (5.50), and the predefined decay schedule (3.60).In particular, fixed KTP = 0 completely disables LSL selection while retaining the independent GSL mechanism. Its inferior average IGD rank suggests that allowing LSL to participate in offspring generation contributes positively to the overall performance under the evaluated settings. Furthermore, the full method outperforms both the fixed low-KTP configuration and the predefined decay schedule, indicating that its performance advantage cannot be explained solely by a low final KTP value.

Importantly, KTP represents the probability of selecting LSL, while actual LSL execution also depends on the validity of the required LPCA models. GSL continues to operate independently of KTP. Therefore, the learned KTP trajectories should be interpreted together with the operator execution conditions and controlled ablation results.

These findings provide further empirical support for learned dynamic KTP control and show that neither completely disabling LSL nor maintaining a fixed low transfer probability achieves the same overall performance as the RL-EMT-MSKT.

Table 8. Average IGD ranks of the seven selected variants on CEC21-MTMO-CPLX. Lower ranks indicate better performance.

RankVariantMean IGD Rank ↓
1Ours2.75
2Ours w/o state fusion3.35
3Ours (KTP decay)3.60
4Ours (Fixed KTP = 0.5)3.65
5Ours (Fixed KTP = 0.05)4.05
6Ours w/o ADE5.10
7Ours (Fixed KTP = 0)5.50
AuthorsOpenReview ↗

2. Degeneracy of CTRE and the contribution of cross-task feature fusion [W1 / Q3 (partial)]

The reviewer questions whether self-masked CTRE performs learned task-relation modeling beyond swapping two task representations.

W1 (CTRE aspect): Two-task self-masked attention collapses to deterministic counterpart exchange and does not learn attention weights.

Q3 (partial): What is learned beyond this exchange, and can the module be removed or compared with simple concatenation?

We have further examined the original formulation and confirm that, with only one admissible counterpart for each target task, self-masked attention assigns a fixed weight of one to the counterpart after Softmax normalization. Consequently, the query-key similarity scores do not affect the resulting feature exchange.

Based on this observation, we have reformulated CTRE as Cross-Task Feature Fusion (CTFF) in Section 3.3 and Appendix B.2. The revised formulation directly exchanges counterpart features using a fixed permutation matrix, eliminating redundant query-key computations while retaining the learnable feature projection, residual connections, and nonlinear fusion transformations.

Thus, the counterpart exchange pattern is deterministic in the current two-task setting, while the feature representations and subsequent fusion transformations remain learnable. The resulting representations incorporate runtime information from both tasks and are used by the shared actor to produce task-specific KTP adjustments.

Furthermore, Table 8 includes an ablation that removes CTFF. The RL-EMT-MSKT achieves an average IGD rank of 2.75, compared with 3.35 without cross-task feature fusion. This result provides empirical support for the contribution of the fusion module to overall optimization performance in the evaluated setting.

These revisions clarify the actual computation performed by the cross-task module and more precisely characterize its role within the RL-based KTP controller.

AuthorsOpenReview ↗

3. Training and evaluation independence and policy generalization [W2 / Q4 (generalization)]

The reviewer raises concerns about template overlap between training and testing and benchmark-specific overfitting.

W2: Training and testing use the same underlying CEC21-MTMO-CPLX templates, limiting claims about broad generalization.

Q4 (generalization, partial): What evidence addresses benchmark-specific overfitting, and would the policy transfer to disjoint problem families or more than two tasks?

To address the overlap between the underlying templates used for training and evaluation in the original protocol, we introduced a held-out-template experiment in Section 4.4 and Appendix F.1. Specifically, we train a separate policy using only P1–P5, with 20 augmented instances per problem, yielding 100 training instances. P6–P10 and all their augmented variants are entirely excluded from training. After training, the policy parameters are frozen and evaluated directly on the ten held-out tasks corresponding to P6–P10, with 30 independent evaluation runs per algorithm.

The held-out policy achieves the lowest mean IGD on six of the ten tasks and the highest mean normalized HV on five tasks. Against EMT-MSKT, the Holm-corrected W/L/NS counts are 7/1/2 for both metrics. Furthermore, compared with the remaining six representative MO-MTO algorithms, the held-out policy achieves 6 to 9 significant wins per comparator for each metric. Complete results are reported in Tables 10 and 11.

These results demonstrate that the learned policy remains competitive on previously unseen problem configurations, even when the corresponding templates and their augmented variants are entirely unseen during training. These provide evidence of policy generalization across problem configurations within the evaluated MO-MTO benchmark.

Main-comparison and holdout training and evaluation protocols.

SettingMain comparisonHoldout evaluation
Training problemsP1–P10P1–P5
Instances per training problem3020
Training instances / episodes300 / 300100 / 100
Official test problemsP1–P10P6–P10
Test tasks2010
Evaluation runs per method3030
Policy updates during testingNoneNone
Reviewer_GPosOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes RL-EMT-MSKT, an extension of manifold-based multi-objective multitask optimization that replaces a fixed knowledge-transfer probability (KTP) with a task-specific controller learned using PPO. The controller uses runtime search information from the paired tasks, processed through a Cross-Task Relation Encoder (CTRE), to multiplicatively update KTP. The method is evaluated on the 10 two-task CEC21-MTMO-CPLX problems against seven MO-MTO baselines.

Critical strengths

None

Critical weaknesses

W1. Unclear necessity and independent contribution of the attention-based CTRE.

The proposed CTRE is evaluated only in a two-task setting, where self-masking leaves exactly one admissible counterpart for each target task. As a result, the attention weights are deterministic, i.e., αt,t=0\alpha_{t,t}=0 and αt,tˉ=1\alpha_{t,\bar t}=1, and the module effectively performs counterpart feature exchange rather than nontrivial attention-based relation weighting. Under this setting, it is therefore unclear why an attention mechanism is needed at all, since there is no choice among multiple source tasks and no learned weighting over alternative inter-task relations. Moreover, the paper does not provide an ablation that isolates the contribution of CTRE itself, such as comparisons against direct concatenation, simple feature fusion, or an MLP-based encoder.

W2. Limited independence between training and evaluation

The evaluation protocol uses the unperturbed official instances as test problems, while the training set consists of perturbations generated from those same underlying problem templates. Although the exact training and test instances are not identical, the test instances are structurally very close to the training distribution. A cleaner evaluation would hold out entire problem templates, rather than only holding out the unperturbed version of an otherwise seen template.

W3. Insufficiently specified and limited Fixed-KTP baseline.

The paper includes a “w/o RL-KTP” or fixed-KTP ablation, but the actual fixed KTP value used for this baseline is not clearly specified. Moreover, the evaluation does not compare the learned controller against multiple fixed KTP settings. Since the learned trajectories often decrease substantially from the initial value of 0.5 toward the lower bound, performance can plausibly be sensitive to the chosen fixed KTP. Therefore, comparison against a single, unspecified fixed-KTP baseline is insufficient to establish that dynamic RL-based adaptation is necessary.

Questions for authors

Q1. Regarding the role of CTRE, please clarify what is actually learned by the attention mechanism in the current two-task setting. Since self-masking leaves only one admissible counterpart, the attention weight appears to be fixed to one regardless of the query-key similarity. It would be helpful to explain the role of the attention formulation under this setting and to provide an ablation against simpler alternatives such as direct counterpart concatenation, linear projection, or MLP-based feature fusion. If CTRE is intended to support nontrivial task-relation modeling, results with more than two tasks would also help substantiate this claim.

Q2. Please clarify the intended notion of generalization under the current training/evaluation protocol. Since the training instances are perturbations of the same underlying problem templates used for final evaluation, it is difficult to separate robustness to instance-level perturbations from generalization to unseen problem structures. A held-out-template experiment, where one or more official problem templates are excluded entirely from training and used only for evaluation, would provide a cleaner assessment of whether the learned controller captures transferable runtime behavior rather than template-specific patterns.

Q3. Please specify the exact KTP value used in the “w/o RL-KTP” / fixed-KTP ablation and provide the rationale for this choice. Since the learned KTP trajectories frequently move from the initial value of 0.5 toward substantially smaller values, performance may depend strongly on the chosen fixed baseline. Comparisons with several fixed KTP values, and ideally with a simple non-RL adaptive schedule or heuristic controller, would help establish whether the improvement is due specifically to learned dynamic control rather than to an unfavorable fixed-KTP configuration.

5 comments
AuthorsOpenReview ↗

1. Necessity and independent contribution of CTRE [W1 / Q1 (partial)]

We thank the reviewer for identifying the degeneracy of self-masked attention in the two-task setting. After a deep examination, we confirm that this observation is mathematically correct.

W1: The necessity and independent contribution of attention-based CTRE are unclear because two-task self-masked attention has deterministic weights.

Q1 (partial): What is learned by CTRE in two tasks, and how does it compare with direct concatenation, linear projection, or MLP feature fusion?

The original Cross-Task Relation Encoder (CTRE) employs self-masked scaled dot-product attention to integrate task representations. However, in the two-task setting, each target task has only one admissible counterpart. Consequently, the attention weight assigned to the counterpart is always one after Softmax normalization, and the query-key similarity scores have no influence on the resulting feature exchange. The original CTRE therefore effectively performs deterministic exchange of projected counterpart representations. Based on this observation, we have reformulated CTRE as Cross-Task Feature Fusion (CTFF) in Section 3.3 and Appendix B.2. The revised formulation directly implements counterpart feature exchange using a fixed permutation matrix, eliminating redundant query-key computations while retaining the learnable feature projection, residual connections, and nonlinear fusion transformations. Although the exchange pattern is determined by the two-task structure, the learnable transformations continue to integrate evolutionary state information from both tasks for subsequent KTP decisions.

Furthermore, Table 8 provides an ablation of the cross-task feature fusion module. The RL-EMT-MSKT achieves an average IGD rank of 2.75, compared with 3.35 when CTFF is removed. This result supports the contribution of incorporating cross-task feature fusion into the learned KTP controller. Moreover this proves that the fused task representations are useful for dynamic KTP decisions under the evaluated setting.

Accordingly, we have clarified the actual role and computation of CTFF and more precisely characterized our principal contribution as RL-based dynamic control of local manifold transfer using cross-task evolutionary feedback.

Table 8. Average IGD ranks of the seven selected variants on CEC21-MTMO-CPLX. Lower ranks indicate better performance.

RankVariantMean IGD Rank ↓
1Ours2.75
2Ours w/o state fusion3.35
3Ours (KTP decay)3.60
4Ours (Fixed KTP = 0.5)3.65
5Ours (Fixed KTP = 0.05)4.05
6Ours w/o ADE5.10
7Ours (Fixed KTP = 0)5.50
AuthorsOpenReview ↗

2. Independence between training and evaluation [W2 / Q2]

We thank the reviewer for highlighting the need for a more independent evaluation of policy generalization. We agree that testing on unperturbed instances derived from the same templates used during training provides limited evidence of adaptation to unseen problem configurations.

W2: Training perturbations and official test instances share underlying problem templates, limiting independence.

Q2: Can full problem templates be held out during training to evaluate generalization to unseen configurations?

To address this concern, we introduced a held-out-template evaluation in Section 4.4 and Appendix F of the revised manuscript.

Specifically, we train a separate policy exclusively on P1–P5, using 20 perturbed variants per problem to construct 100 training instances. P6–P10 and all their augmented variants are completely excluded from training. After training, the policy parameters are frozen and evaluated directly on the ten held-out tasks corresponding to P6–P10, with 30 independent evaluation runs per algorithm.

The held-out policy achieves the lowest mean IGD on six of the ten test tasks and the highest mean normalized HV on five tasks. Against EMT-MSKT, the Holm-corrected W/L/NS counts are 7/1/2 for both metrics. Furthermore, compared with the remaining six representative MO-MTO algorithms, the held-out policy achieves 6 to 9 significant wins per comparator for each metric. Complete results are reported in Tables 10 and 11.

These results demonstrate that the learned policy maintains competitive optimization performance on previously unseen problem configurations, even when the P6–P10 problem templates and their augmented instances are entirely excluded from training. This provides further evidence of generalization across problem configurations. Main-comparison and holdout training and evaluation protocols.

SettingMain comparisonHoldout evaluation
Training problemsP1–P10P1–P5
Instances per training problem3020
Training instances / episodes300 / 300100 / 100
Official test problemsP1–P10P6–P10
Test tasks2010
Evaluation runs per method3030
Policy updates during testingNoneNone
AuthorsOpenReview ↗

3. Fixed-KTP baselines and the effectiveness of dynamic control [W3 / Q3]

The reviewer questions whether learned KTP control is better than clearly specified fixed-KTP and simple adaptive alternatives.

W3: The original fixed-KTP baseline is insufficiently specified and only one fixed setting was evaluated.

Q3: Which fixed KTP was used, and how does the learned policy compare with several fixed values and a simple non-RL decay schedule?

The original fixed-KTP ablation used KTP = 0.5, consistent with the initial KTP of the tested RL-EMT-MSKT. Following the reviewer's suggestion, we have extended the ablation study in Table 8 to include versions with fixed KTP = 0.05, fixed KTP = 0, and a predefined KTP decay schedule, in addition to the fixed KTP = 0.5 configuration.

The fixed KTP = 0.05 variant examines whether a consistently low transfer probability can explain the performance advantage of the learned policy. The fixed KTP = 0 variant disables LSL selection while retaining the independent GSL mechanism. The predefined decay baseline starts from KTP = 0.5 and multiplies it by 0.96 at each generation until reaching the lower bound of 0.05, providing a comparison against a simple non-learning adaptive rule.

The results show that the RL-EMT-MSKT achieves the best average IGD rank of 2.75, outperforming fixed KTP = 0.5 (3.65), fixed KTP = 0.05 (4.05), fixed KTP = 0 (5.50), and the predefined decay schedule (3.60).

These findings indicate that the performance advantage of the full method cannot be explained solely by maintaining a low KTP value or following a predefined decay schedule. In particular, although the learned KTP often approaches the lower bound during later search stages, the RL-EMT-MSKT still achieves a better average IGD rank than the fixed KTP = 0.05 and decay variants. This suggests that state-dependent adjustments throughout the evolutionary process indeed contribute to its overall performance.

We sincerely appreciate the reviewer's constructive feedback, which has helped us improve the mathematical clarity of the cross-task fusion module and strengthen the empirical evidence for policy generalization and adaptive transfer control.

Reviewer_KbyvOpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes RL-EMT-MSKT, which adapts the knowledge-transfer probability (KTP) of local structure learning (LSL) in a manifold-driven multi-objective multi-task optimization (MO-MTO) framework. Task-specific KTP adjustment is formulated as a Markov decision process and learned via proximal policy optimization (PPO). A Cross-Task Relation Encoder aggregates paired-task states, while an adaptive differential evolution (ADE) operator serves as the fallback for local offspring generation. The learned KTP decides, for each parent, whether to invoke LSL (when valid) or ADE. On the CEC21-MTMO-CPLX benchmark, the method outperforms seven representative MO-MTO algorithms on most tasks; ablations confirm the contributions of the RL controller and ADE.

Critical strengths

The motivation is clear: fixed KTPs cannot respond to evolving population progress, manifold validity, and offspring survival feedback. Casting task-level KTP control as an MDP that incorporates these signals yields a coherent closed-loop design. The technical components are carefully specified. The 11-dimensional state covers progress, current KTP, operator survival rates, SPEA2-based quality statistics, decision-space diversity, manifold similarity/validity, and stagnation. Multiplicative actions with clipping keep adjustments gradual, and the reward jointly considers population-quality improvement and the relative survival of LSL versus ADE offspring. Experimental practice is solid: 30 independent runs, IGD and normalized HV, Holm-corrected Wilcoxon tests, training on perturbed instances, and testing on the official problems. The method obtains the best mean IGD on 11 of 20 tasks and records substantial numbers of significant wins against each comparator. Ablation ranks further support the value of both the learned KTP controller and ADE. Reproducibility is reasonably well supported by detailed appendices and a code link.

Critical weaknesses

Differentiation from concurrent RL-based transfer control is limited. MetaMTO (Zhan et al.) already uses attention-based cross-task modeling and multi-role RL to decide where, what, and how to transfer; L2T (Wu et al.) also employs PPO for transfer timing and strategy. The present work focuses on the probability of local manifold transfer, yet the Cross-Task Relation Encoder with self-masking reduces, in the two-task setting, to deterministic counterpart exchange (αt,t=0\alpha_{t,t}=0, αt,tˉ=1\alpha_{t,\bar{t}}=1). The practical benefit of the attention mechanism is therefore modest, and a clearer statement of the incremental contribution relative to both EMT-MSKT and these RL approaches would strengthen the paper. Evaluation is confined to two-task instances drawn from a single benchmark family. The authors acknowledge this limitation. Broader coverage (more than two tasks, additional suites, larger distribution shifts, or real-world cases) is needed to support stronger claims of generality. Observed KTP trajectories consistently decline toward the lower bound of 0.05, suggesting that LSL is progressively de-emphasized; the paper correctly notes that trajectories alone do not establish causality, but deeper analysis of when local manifold transfer is beneficial remains limited. Reward design and hyper-parameter choices receive little justification or sensitivity analysis. Equal weighting of the quality and transfer terms, the specific tanh scaling factors (10 and 5), the discrete action set {0.96,0.98,1.00,1.02,1.04}\{0.96, 0.98, 1.00, 1.02, 1.04\}, the feedback-window length, and the PPO update interval can all influence behavior, yet none are systematically examined. Computational overhead is unreported. Manifold modeling (LPCA/GSL/LSL) plus policy inference occurs every generation. Without runtime comparisons against EMT-MSKT and the other baselines, the practical cost of the approach is difficult to assess. Interpretation of results could be more nuanced. Performance is problem-dependent (e.g., MTEA-D-DN is substantially better on P7-T2), and variances are large on several tasks. A clearer discussion of the conditions under which the learned policy helps or hurts would improve the paper.

Questions for authors

Given that CTRE degenerates to deterministic exchange in the two-task case, does the attention mechanism provide measurable gains when more than two tasks are present? Are multi-task experiments planned? What is the wall-clock overhead of RL-EMT-MSKT relative to EMT-MSKT, and what fraction is attributable to policy inference versus manifold construction? How sensitive are the results to the reward weights, tanh coefficients, and action granularity? Have simpler adaptive rules been compared? The consistent decline of KTP toward 0.05 suggests that LSL becomes less useful later in the search. Can state-feature analysis or controlled experiments clarify when and why this occurs?

5 comments
AuthorsOpenReview ↗

1. Differentiation from MetaMTO and L2T [W1 / Q1 (partial)]

We thank the reviewer for this important comment. We agree that the distinctions between our approach and existing RL-based knowledge transfer methods, particularly MetaMTO and L2T, should be more clearly articulated, together with the actual role of the cross-task feature fusion module.

W1: Limited differentiation from MetaMTO/L2T and the deterministic nature of self-masked CTRE attention in two-task optimization.

First, our work differs from these methods in both the optimization setting and the transfer control mechanism. MetaMTO primarily addresses single-objective evolutionary multitasking, including many-task optimization, and employs multi-role reinforcement learning to jointly determine where, what, and how knowledge should be transferred. Similarly, L2T mainly focuses on single-objective evolutionary multitasking by learning transfer timing and strategies. In contrast, our work focuses on multi-objective multitask optimization (MO-MTO), where each task involves multiple conflicting objectives and requires consideration of population convergence, diversity, and Pareto-front approximation quality.

To address this setting, we develop a task-specific dynamic KTP controller that uses PPO to adjust the probability of local structure learning (LSL) based on evolutionary states, manifold information, and offspring origin-retention feedback. This mechanism coordinates cross-task manifold transfer with within-task adaptive differential evolution (ADE),** extending RL-based transfer control to dynamic configuration of local manifold transfer in MO-MTO.**

Second, we have further examined the reviewer's observation regarding the degeneracy of the original Cross-Task Relation Encoder (CTRE). In the original formulation, self-masked attention was employed to integrate task representations. However, in the current two-task setting, each target task has only one admissible counterpart. Consequently, the attention weight is always one after Softmax normalization, and the query-key similarity scores have no influence on the resulting feature exchange. The original CTRE therefore effectively performs deterministic exchange of the projected counterpart representations.

Based on this observation, we have reformulated CTRE as Cross-Task Feature Fusion (CTFF), which directly implements counterpart feature exchange through a fixed permutation matrix. This formulation eliminates redundant query-key computations while retaining the learnable feature projection, residual connections, and nonlinear fusion transformations. CTFF therefore preserves the effective feature exchange and fusion operations of the original CTRE in the two-task setting while providing a more accurate description of the actual computation. Although the counterpart exchange pattern is fixed, the learnable feature transformations continue to provide fused state representations for subsequent KTP decisions.

Finally, we provide comparative and ablation evidence supporting the effectiveness of the proposed approach. Given the differences in optimization settings, we select seven representative MO-MTO algorithms as baselines: EMT-MSKT, MMTO-ETA, MMTO-TGT, KR-MTEA, MTDE-MKTA, MTEA-DCK, and MTEA-D-DN. All methods are evaluated under the same two-task multi-objective benchmark setting to ensure direct comparability. RL-EMT-MSKT achieves the lowest mean IGD on 12 of the 20 benchmark tasks.

Furthermore, the ablation results in Table 8 support the effectiveness of learned dynamic KTP control. The full method achieves an average IGD rank of 2.75, outperforming fixed KTP = 0.5 (3.65), fixed KTP = 0.05 (4.05), and a predefined KTP decay schedule (3.60). These results indicate that the learned controller provides better overall performance than the fixed and predefined adaptive configurations.

Q1 (partial): Does attention offer measurable benefits with more than two tasks, and are experiments on more tasks planned?

The present evaluation covers the two-task setting and does not assess attention benefits with more than two tasks. Extending the framework to additional tasks is identified as a direction for future work.

Overall, our work addresses task-specific control of local manifold transfer in two-task MO-MTO by integrating cross-task state representations, evolutionary feedback, and reinforcement learning. We have clarified its distinctions from existing RL-based transfer methods and provided a more precise formulation of CTFF to accurately characterize the methodological contribution.

Table 8. Average IGD ranks of the seven selected variants on CEC21-MTMO-CPLX. Lower ranks indicate better performance.

RankVariantMean IGD Rank ↓
1Ours2.75
2Ours w/o state fusion3.35
3Ours (KTP decay)3.60
4Ours (Fixed KTP = 0.5)3.65
5Ours (Fixed KTP = 0.05)4.05
6Ours w/o ADE5.10
7Ours (Fixed KTP = 0)5.50
AuthorsOpenReview ↗

2. Evaluation scope and generalization [W2 / Q1 (scope, partial)]

The reviewer questions whether experiments limited to two-task problems from one benchmark support broader generalization claims.

W2: Evaluation is restricted to two-task cases from a single benchmark family; broader test settings are needed to support generality.

Q1 (scope, partial): Would testing with more than two tasks strengthen the evidence for generalization? We agree that evaluation on perturbed versions of the same problem templates provides limited evidence of generalization to unseen problem structures.

In the new version, we introduced a held-out-template experiment in Section 4.4 and Appendix F.1. A separate policy is trained exclusively on P1-P5 using 100 augmented instances. P6-P10 and their perturbations are excluded entirely from training. The frozen policy is then evaluated on the ten held-out tasks, with 30 independent evaluation runs per method.

The experimental results show that the held-out policy achieves the lowest mean IGD on six of the ten test tasks and the highest mean normalized HV on five tasks. Compared to EMT-MSKT, the Holm-corrected W/L/NS counts are 7/1/2 for both metrics. Furthermore, compared with the remaining six representative MO-MTO algorithms, the held-out policy achieves 6 to 9 significant wins in each metric. Complete results are reported in Tables 10 and 11 of Appendix F.1.

These results demonstrate that the learned policy maintains competitive optimization performance on previously unseen problem configurations, even when the P6–P10 and their augmented templates are entirely excluded from training, providing further evidence of generalization across problem configurations. Main-comparison and holdout training and evaluation protocols.

SettingMain comparisonHoldout evaluation
Training problemsP1–P10P1–P5
Instances per training problem3020
Training instances / episodes300 / 300100 / 100
Official test problemsP1–P10P6–P10
Test tasks2010
Evaluation runs per method3030
Policy updates during testingNoneNone
AuthorsOpenReview ↗

3. Declining KTP trajectories and the usefulness of LSL [W3 / Q3 (adaptive rule) / Q4 (partial)]

We thank the reviewer for this observation. We agree that declining KTP trajectories alone are insufficient to fully characterize the contribution of local manifold transfer.

W3: KTP consistently declines toward 0.05, making the benefit of LSL at different search stages unclear.

Q3 (adaptive rule): Have simple non-RL adaptive KTP schedules been compared?

Q4 (partial): Can controlled experiments or state analysis explain when and why LSL becomes less useful?

To further investigate this issue, we have extended the ablation study in Table 8 of Appendix E.3?? by adding a competitor with fixed KTP values of 0, 0.05, and 0.5, together with a predefined KTP decay schedule. The results show that the ours method **achieves an average IGD rank of 2.75, outperforming fixed KTP = 0.5 (3.65), fixed KTP = 0.05 (4.05), fixed KTP = 0 (5.50), and the predefined decay schedule (3.60). **In particular, the fixed KTP = 0 variant completely disables LSL selection while retaining the independent GSL mechanism. Its inferior average rank indicates that local manifold transfer is helpful under the evaluated settings.

These results suggest that RL-EMT-MSKT’s performance advantage cannot be explained solely by the low final KTP values. Dynamic KTP control allows the learned policy to adjust the usage of LSL and ADE according to evolutionary runtime states, enabling more flexible operator coordination throughout the search.

These findings provide further evidence of the effectiveness of RL-based dynamic transfer control in the MO-MTO setting.

Table 8. Average IGD ranks of the seven selected variants on CEC21-MTMO-CPLX. Lower ranks indicate better performance.

RankVariantMean IGD Rank ↓
1Ours2.75
2Ours w/o state fusion3.35
3Ours (KTP decay)3.60
4Ours (Fixed KTP = 0.5)3.65
5Ours (Fixed KTP = 0.05)4.05
6Ours w/o ADE5.10
7Ours (Fixed KTP = 0)5.50

Comments on the paper

1 comment

Dear Reviewers, the authors have submitted their response. Kindly review it and update your review where appropriate.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.