acceptodds
Under review as a conference paper at ICLR 2027

Consensus Accelerates Evolution: Black-Box Many-Task Optimization via Information-Geometric Consensus

Abstract

Natural evolution strategies (NESs) provide a principled information-geometric framework for black-box optimization. However, scaling NESs to many concurrent tasks is hindered by hyperparameter sensitivity and high-variance natural-gradient estimates. With limited sampling per task, noisy distribution updates reduce useful progress and slow convergence. To address these limitations, we propose GemNES, an online meta-black-box optimizer using concurrent many-task optimization trajectories as supervision for data-efficient adaptation through dual-level information-geometric consensus (IGC). Specifically, global IGC pools normalized task-trajectory progress into a consensus signal for natural-gradient meta-policy updates and task-conditioned control. Meanwhile, local IGC aggregates aligned peer natural gradients in standardized tangent coordinates to improve each task update. Together, they encode cross-task agreement through natural-gradient updates on the meta-policy and task-distribution manifolds. Under alignment and regularity conditions, our analysis identifies a sufficient safe-transfer region, quantifies scale reduction relative to unsuppressed transfer, and derives tighter finite-horizon contraction for the evolving rank surrogate. Empirical results on synthetic many-task benchmarks and neural policy search demonstrate faster and more sustained convergence compared with representative baselines.

Primary area:
optimization
Keywords:
Many-task Optimization, Evolution Strategy, Black-Box Policy Search

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dkdVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GemNES, an online meta-black-box optimizer for many-task optimization built on separable NES. Existing NES's high-variance natural gradient estimates hinder the scaling, resulting in slow convergence. Proposed GemNES adopt the dual-level IGC scheme, which encode cross-task agreement through natural gradient updates on meta policy and task-distributions. Experiments cover many-task and brax-MuJoCo benchmarks demonstrate faster and sustained convergence compared with representative NES-based methods and classical RL methods.

Critical strengths
  • I like the idea using concurrent task trajectories as meta training signal and as variance-reduction evidence. I think this is reasonable design and meta-reward reuses evaluated batches, so it costs no extra function evaluations.

  • The paper gives through and candid evaluation results, including per-instance tables with confidence interval, runtime analysis, and multiple ablation and sweeps.

Critical weaknesses
  • The core claimed mechanism (dual-level IGC) contributes little. the gains come mainly from meta-learned control, specifically step-size adaptation. In Table 2, without meta-control setup is significantly worse on synthetic instances and MuJoCo benchmark. Removing cross-task consensus and affinity filter matters on only few instances and tasks, so the evidence thus points to online meta-learned learning rate control helps SNES rather than 'consensus accelerate evolution'. The authors should either reframe the contribution or show settings where consensus yields a large significant benefit over w/o-transfer setup.

  • The comparison with RL baselines in Table 1 is not matched in budgt. The paper states that proposed GemNES attains significantly higher reward than classical RL baselines, PPO and SAC, on 6 and 8 of 10 tasks. But each ES method and RL methods different environment per steps, and GemNES is therefore compared against methods given up to 5-25 times fewer environment interactions. Could you verify the RL setups and either compare at matched environment steps or remove the claims of superiority over classical RL?

Questions for authors
  • How does GemNES compare to (a) a frozen, randomly initialized meta-policy, (b) random control perturbations, and (c) SNES with a tuned learning rate or standard step-size adaptation? This is needed to attribute gains to meta-learning.

  • Can you confirm the PPO/SAC configurations and provide a comparison at matched environment steps?

6 comments

We sincerely thank you for the suggested experiments. We have updated and uploaded the manuscript, with all changes marked in blue.

Weakness 1 & Question 1 (1/2). Global IGC and attribution controls

Weakness 1

The core claimed mechanism (dual-level IGC) contributes little. the gains come mainly from meta-learned control, specifically step-size adaptation. In Table 2, without meta-control setup is significantly worse on synthetic instances and MuJoCo benchmark. Removing cross-task consensus and affinity filter matters on only few instances and tasks, so the evidence thus points to online meta-learned learning rate control helps SNES rather than 'consensus accelerate evolution'. The authors should either reframe the contribution or show settings where consensus yields a large significant benefit over w/o-transfer setup.

Question 1

How does GemNES compare to (a) a frozen, randomly initialized meta-policy, (b) random control perturbations, and (c) SNES with a tuned learning rate or standard step-size adaptation? This is needed to attribute gains to meta-learning.

Meta-learning provides the architecture, while Global IGC specifies how tasks jointly train the controller. In Section 3.1, each candidate meta-policy receives a score pooling normalized progress across tasks (Eq. (4)). This is the global consensus signal, favoring control rules that improve pooled task progress. The score's rank utility drives a natural-gradient update of the shared meta-policy distribution (Eq. (5)), following its Fisher geometry. The policy then maps each task's state to its own step sizes, shaping exponent, and ρ\rho.

Step-size adaptation is therefore one effect of Global IGC. Local IGC forms consensus at the task-distribution level by aggregating affinity-selected peer natural gradients (Eqs. (6)–(7)). Together, they coordinate controls and directions using the same search trajectories.

1. Finer-grained ablation

The original w/o-meta disables learned base controls and Local transfer together, losing significantly on 14 synthetic instances and 6 MuJoCo tasks. We added rho-only to retain adaptive Local transfer with fixed base controls. The original w/o-transfer retains base-control learning and removes Local.

Both rho-only and w/o-meta use GemNES's base learning rates with mean/scale multipliers and shaping exponent fixed to (1,1,1). Rho-only retains affinity and suppression and learns ρ\rho through Global progress. w/o-meta sets ρ=0\rho=0.

We used 16 synthetic instances with 30 runs each. Each has 50 tasks of dimension 50, using 50 samples per update and 50,000 evaluations per task. In both tables, + / - / = counts comparators significantly better / worse / not significantly different from GemNES (two-sided Wilcoxon rank-sum, alpha = 0.05). Average Friedman ranks are within each 4-method comparison. Lower is better.

Method+ / - / = vs GemNESAverage rank
GemNESBase1.69
rho-only (new)2 / 10 / 42.56
w/o-transfer0 / 5 / 112.19
w/o-meta0 / 14 / 23.56

Full results are in Appendix F.4, Table A-XV. GemNES ranks first. Comparisons with rho-only and w/o-transfer assess base-control learning and adaptive Local transfer, respectively. With base controls fixed, rho-only improves mean UOV over w/o-meta on 12/16 instances and ranks better (2.56 vs 3.56), supporting adaptive Local transfer with fixed step sizes and shaping.

In Table 2, freezing step-size, shaping, and transfer-rate adaptation causes 8, 2, and 3 significant losses, respectively. w/o-ada-transfer fixes ρ=0.25\rho=0.25, with 0/3/13 (+ / - / =). These results support all three controls.

2. Frozen, random, and tuned controls

The three requested controls use the same 16 instances and 30 evaluation runs each.

Method+ / - / = vs GemNESAverage rank
GemNESBase1.75
FrozenRandom3 / 10 / 32.56
RandomControl3 / 9 / 42.75
SNES-Tuned2 / 12 / 22.94

Full results are in Appendix F.5, Table A-XVI.

(a) FrozenRandom. Freezing the initial meta-policy weights preserves architecture, state feedback, and Local IGC. The results support online adaptation.

(b) RandomControl. We sample reward-independent outputs from a fixed bank of 256 joint control vectors per task, calibrated to the initial policy distribution. Local IGC is retained throughout. The results support learning from progress feedback.

(c) SNES-Tuned. We tested 9 mean/scale multiplier pairs in 0.5,1,22\\{0.5,1,2\\}^2, using 3 development runs on WCCI22-MaTSO4 and 50,000 evaluations per task. Selection minimized log terminal error averaged across tasks and runs.

Appendix C.2 defines the variants, and Appendix F.5 details tuning. Frozen and random controls support online meta-policy adaptation within Global IGC. SNES-Tuned supports the combined framework against tuned fixed rates.

Weakness 1 & Question 1 (2/2). Local adds gains beyond learned controls

Thank you for asking where Local consensus provides a substantial benefit. w/o-transfer retains base-control learning and switches off Local transfer.

In the original 10-run policy-search experiment, GemNES significantly improves mean reward over w/o-transfer by 11.4% on HalfCheetah and 30.3% on Ant.

VariantTasks+ / - / = vs GemNES
w/o-transfer100 / 3 / 7

Full taskwise reward means (SDs) and significance results appear in Appendix F.4, Table A-XIV. Counts + / - / = indicate that the comparator is significantly better / worse / not significantly different from GemNES (two-sided Wilcoxon rank-sum, alpha = 0.05). Removing affinity filtering also significantly lowers Ant reward from 2658.3 (691.0) to 2250.2 (604.0).

Eq. (12) makes the link between Global and Local explicit. Global's learned step sizes enter the base and transfer directions, and ht=ρtdt\mathbf h_t=\rho_t\mathbf d_t combines its learned rate with Local's filtered, suppressed direction. The following diagnostics test the gains of these realized updates.

To check whether Local helps simply by making updates larger, we added an equal-length comparison. We compared the learned-rate update with zero transfer and with an isolated update rescaled to the full step's Euclidean length in (μ,log⁡σ)(\boldsymbol\mu,\log\boldsymbol\sigma) coordinates. Gradient estimates and learned base controls are identical in these one-step comparisons. In Corollary 2, Eq. (15), χt\chi_t measures the alignment gained by changing direction. Uniform positive rescaling gives χt=0\chi_t=0.

We used CEC19-MaTSO1 (high global similarity, Rosenbrock tasks) and WCCI22-MaTSO10 (low global similarity, mixed-function tasks). With 5 trajectories, 50 tasks, and 20/50/80% checkpoints, we obtain 750 states per instance. To test whether the gains last, we also compared learned ρ\rho with zero transfer over 40 further all-task updates from 20/50% checkpoints. Both branches follow the same Global learning rule, with controls adapting to their own states. One-step and fresh endpoint evaluations use 3 matched antithetic batches of 50 samples.

ComparisonInstanceRelative objective gain (%), mean (SD)P / Z / N
Learned vs zero, one stepCEC19-MaTSO10.0061 (0.0036)700 / 50 / 0
Learned vs zero, one stepWCCI22-MaTSO100.0505 (0.0944)413 / 335 / 2
Full vs equal-length isolatedCEC19-MaTSO10.0059 (0.0035)700 / 50 / 0
Full vs equal-length isolatedWCCI22-MaTSO100.0452 (0.0753)403 / 344 / 3
Learned vs zero, 40 updatesCEC19-MaTSO15.6869 (6.5431)467 / 33 / 0
Learned vs zero, 40 updatesWCCI22-MaTSO108.2545 (7.8960)378 / 114 / 8

Positive gain means a lower objective value than the reference. Entries are mean (SD) across 5 trajectory medians. P/Z/N count intervals above zero, containing zero, or below zero among 750 one-step or 500 endpoint states. These unadjusted pointwise 95% intervals use 1000 paired bootstrap resamples.

To test affinity separately, we compared filtered and unfiltered updates using the same samples and controls. Filtering yields positive 95% intervals for the change in measured sts_t, the alignment term in Theorem 1, Eq. (13), in 87.9% (14.2%) and 56.7% (27.0%) of states. Percentages are mean (SD) across trajectories (Appendix G.1).

After measuring objective gains in ff, we tested the theoretical gain condition using frozen-rank utility Φt\Phi_t at 3 preselected tasks and the 20/50% checkpoints of the same 5 trajectories. We fix Global's learned step sizes and the realized Local direction, retaining each reference batch's ranks and sample-dependent shaping along the path. Corollary 3 tests the utility gain at learned ρt\rho_t relative to zero transfer. A positive slope bound mtm_t certifies gain throughout (0,ρt](0,\rho_t] by Eq. (20). This holds at 13/27 and 20/21 active states, with 95% simultaneous coverage over the tested active subset. The other 12 states have ρ=0\rho=0. Appendix G.2, Table A-XXIV reports counts by checkpoint.

The equal-length tests support Local's directional contribution, and the 40-update tests show short-term gains as Global continues to adapt. Eq. (20) connects this evidence to theory by certifying positive surrogate gains from the combination of learned Global controls and Local directions at the passing states.

Weakness 2 & Question 2. ES/RL interaction budgets and configurations

Weakness 2

The comparison with RL baselines in Table 1 is not matched in budgt. The paper states that proposed GemNES attains significantly higher reward than classical RL baselines, PPO and SAC, on 6 and 8 of 10 tasks. But each ES method and RL methods different environment per steps, and GemNES is therefore compared against methods given up to 5-25 times fewer environment interactions. Could you verify the RL setups and either compare at matched environment steps or remove the claims of superiority over classical RL?

Question 2

Can you confirm the PPO/SAC configurations and provide a comparison at matched environment steps?

Thank you for raising this concern. ES and PPO/SAC use data and computation differently, so fairness depends on which cost is matched. GemNES aggregates returns from parallel policy rollouts, whereas PPO/SAC reuse transitions for policy and value updates. Equal environment steps compare interaction efficiency. Wall-clock time also includes the different update costs.

Prior ES–RL work distinguishes sample efficiency from wall-clock performance (Salimans et al., 2017). LES (Lange et al., 2023) included PPO reference scores from Lu et al. (2022), obtained with default Brax settings, alongside ES learning curves.

GemNES uses a nominal budget of 50,000 policy evaluations per task, each averaging 5 rollouts of up to 1000 steps. Incumbent reevaluations count toward the same budget. GemNES checks the budget after each synchronized all-task generation and stops when it is reached. Early termination shortens the episodes.

Following your suggestion, we removed the general superiority claim over PPO/SAC and their win/loss summaries. Table 1 now compares ES methods. Table A-XII retains PPO/SAC taskwise means and standard deviations as reference results.

The mean end-to-end wall-clock times per run, averaged over 10 runs, are 8.04 h for GemNES, approximately 8.9 h for PPO, and 11.1 h for SAC (Figure A-1(b)). These training times are of a similar order, making terminal rewards useful references under the reported setup.

Both RL implementations use Brax 0.13.0, with the following settings.

SettingPPOSAC
Configured training steps50,000,00010,000,000
Final logged training steps, every run56,197,12010,005,760
Runs per task1010
Policy hidden layers / activation4 × 32 / tanh4 × 32 / tanh
Learning rate / discount3e−4 / 0.993e−4 / 0.99
Training environments / batch size2,048 / 1,024128 / 256
Maximum episode horizon1,0001,000
Recorded evaluation points / evaluation environments50 / 550 / 5
Observation normalization / reward scalingEnabled / 0.1Enabled / 0.1
Evaluation policy setting in runnerDeterministicStochastic

Section 4.1, Appendix F.3, and Table A-V state these configurations and the revised scope. The revised comparison provides performance references at these reported runtimes and method-specific budgets.

Reviewer_JbU6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GemNES, a black-box many-task optimizer combining online meta-learning of search controls with affinity-guided natural-gradient transfer and scale suppression. It provides conditional analyses of transfer benefits and surrogate contraction. Experiments on synthetic benchmarks and neural policy search report improved optimization performance, with ablations assessing both components.

Critical strengths
  1. Integrated search adaptation. GemNES combines online learning of task-specific search controls with affinity-guided gradient transfer and scale suppression. Reusing concurrent optimization trajectories avoids separate offline pretraining and additional objective evaluations for the meta-reward, supporting practical applicability.
  2. Experimental coverage. Evaluations include synthetic many-task benchmarks, neural policy search, and task-count scaling. Comparisons with several evolutionary baselines and component ablations provide evidence of practical benefits from online adaptation and cross-task information sharing.
Critical weaknesses
  1. Limited connection between theory and implementation. The guarantees require positive consensus alignment and suitable transfer rates, but the implemented affinity filter and learned controller do not enforce these conditions. Experiments do not directly establish their satisfaction, limiting the theory’s explanatory value.
  2. Unvalidated cross-task coordinate correspondence. Gradient transfer relies on prescribed coordinate matching, including equal-index neural-network parameters. Such correspondence need not imply functional equivalence across independently optimized policies, and its sensitivity is not evaluated, leaving the transfer mechanism’s applicability insufficiently established.
Questions for authors

Please refer to the Critical Weaknesses.

7 comments

We thank you for your careful feedback. We have updated and uploaded the manuscript, with all changes marked in blue.

Weakness 1. Theory and implementation (1/2)

Limited connection between theory and implementation. The guarantees require positive consensus alignment and suitable transfer rates, but the implemented affinity filter and learned controller do not enforce these conditions. Experiments do not directly establish their satisfaction, limiting the theory’s explanatory value.

We added direct tests of alignment, objective improvement, and a gain condition at the learned rates. These ask whether affinity selects useful directions, whether the resulting updates improve optimization, and which updates obtain a theoretical guarantee. The original Theorem 1 and Corollary 1 identify positive alignment and suitable rates as sufficient conditions for utility gains. Affinity selects peers by sampled directional agreement, while Global IGC learns controls from pooled progress in Eqs. (4)–(5).

Eq. (12) connects the two levels. Global's learned step sizes enter both the base and transfer directions, and its learned rate scales Local's filtered, suppressed direction through ht=ρtdt\mathbf h_t=\rho_t\mathbf d_t. The theory identifies when adding this transfer improves the update under these controls.

The new synthetic diagnostics in Appendix G use CEC19-MaTSO1 (high global similarity, Rosenbrock tasks) and WCCI22-MaTSO10 (low global similarity, mixed-function tasks). We saved 20/50/80% checkpoints across 5 trajectories and 50 tasks, giving 750 states per instance.

1. New alignment and affinity tests. The original Theorem 1 uses Eq. (13), where st=Rt⊤E[ht∣Ft]>0s_t=\mathbf R_t^\top\mathbb E[\mathbf h_t\mid\mathcal F_t]>0 means expected transfer aligns with the surrogate gradient. To test this term directly, we added an experiment in Appendix G.1. We estimate gradient and transfer using 2 independent pools of 20 batches with 50 samples each. Filtered and unfiltered updates share samples and controls. Δst\Delta s_t measures the filter's effect, while χˉt\bar\chi_t compares normalized directions (Eq. (15)).

States with positive 95% intervalCEC19-MaTSO1WCCI22-MaTSO10
Alignment sts_t, Eq. (13)93.3% (14.9%)65.5% (31.1%)
Filtering effect Δst\Delta s_t87.9% (14.2%)56.7% (27.0%)
Mean directional advantage χˉt\bar\chi_t, Eq. (15)93.3% (14.9%)58.7% (27.5%)

Entries are mean (SD) across trajectories. Unadjusted pointwise intervals use 500 bootstrap resamples. Filtering strengthens the alignment term in Eq. (13) at many tested states (new Table A-XXII).

2. New objective-gain tests. To see whether the measured alignment translates into optimization gains, we added one-step and 40-update experiments on the objective ff in Appendix G.2. One-step tests fix gradients and base controls, comparing learned transfer with zero transfer and an isolated update rescaled to the full update's length in (μ,log⁡σ)(\boldsymbol\mu,\log\boldsymbol\sigma) (the original Corollary 2, Eq. (15)). The equal-length comparison holds step length fixed to examine the contribution of direction. To test whether gains persist, we also continued learned-versus-zero branches for 40 all-task updates from the 20/50% checkpoints. During these continuations, both branches continue Global learning from their own states.

Relative objective gain (%), mean (SD)CEC19-MaTSO1WCCI22-MaTSO10
Learned vs zero, one step0.0061 (0.0036)0.0505 (0.0944)
Full vs equal-length isolated0.0059 (0.0035)0.0452 (0.0753)
Learned vs zero, 40 updates5.6869 (6.5431)8.2545 (7.8960)

Entries are mean (SD) across 5 trajectory medians. Evaluation uses 3 matched antithetic batches of 50 samples. Full results and interval counts appear in new Table A-XXIII. The equal-length gains support a contribution from direction, and the 40-update results show short-term gains during continued Global adaptation. These tests measure the objective ff. We next check the theoretical gain condition for the frozen-rank utility Φt\Phi_t.

Weakness 1. Theory and implementation (2/2)

3. New tests of learned rates and sufficient conditions. To connect the objective gains to theory, we evaluate the frozen-rank utility Φt\Phi_t at saved updates. Keeping the reference ranks fixed lets us compare the same utility as the transfer rate varies. In the original Corollary 1, Eq. (14) guarantees positive gain when alignment outweighs curvature cost. We added analytic curvature bounds and confidence bounds in Appendix A.5 to test this condition. For an active state with ρt>0\rho_t>0, we absorb step sizes into the isolated direction pt\mathbf p_t and transfer direction dt\mathbf d_t. The sufficient condition becomes

Ct−=At−−Lt(∣pt∣∣dt∣+ρt∣dt∣2/2)>0.C_t^-=A_t^- -L_t(\\|\mathbf p_t\\|\\|\mathbf d_t\\|+\rho_t\\|\mathbf d_t\\|^2/2)>0.

Here At−A_t^- is a lower bound on alignment Rt⊤dt\mathbf R_t^\top\mathbf d_t, and LtL_t bounds curvature uniformly over the update region. We screened all 1500 states using alignment point estimates and the analytic curvature bound, then applied confidence bounds to the 60-state subset described below. These Eq. (14) screens yield no positive margins with the available uniform bounds. The bound subtracts a worst-case curvature cost over the update region, so a nonpositive margin leaves the actual utility change undetermined.

To assess the learned rates along their actual transfer paths, we added Corollary 3, the transfer-path certificate in Eq. (20). With realized Global controls and the Local direction fixed, it follows the segment from the isolated update to the full update at learned ρt\rho_t. The margin mtm_t is a lower bound on utility slope along this segment, accounting for curvature between sampled nodes. If mt>0m_t>0, adding transfer improves utility throughout the segment, giving

\qquad 0<r\le\rho_t.$$ Appendix A.5 derives both bounds. The certificate fixes Global's learned step sizes and the realized Local direction, then tests the learned $\rho_t$. Each reference batch's ranks and sample-dependent shaping are retained along the path. For this new experiment, we tested tasks 1, 25, and 50 at the 20/50% checkpoints of all 5 trajectories per instance, giving 60 states. Each active state uses 9 nodes and 4096 independent reference batches of 50 samples, increased to 16384 at 16 states with a nonpositive first-stage bound. Density ratios retain the reference ranks and shaping when evaluating the frozen-rank utility. Candidate variances below twice the reference variances ensure finite Gaussian second moments. The confidence bounds cover all tested active states in this 60-state subset, all nodes, and both prespecified sampling stages simultaneously. | States passing Eq. (20) / active states | 20% checkpoint | 50% checkpoint | Total | |---|---:|---:|---:| | CEC19-MaTSO1 | 12/12 | 1/15 | **13/27** | | WCCI22-MaTSO10 | 8/9 | 12/12 | **20/21** | In total, Eq. (20) certifies **33/48 active states**. CEC19 passes **1/15 states at the 50% checkpoint**, where reference-distribution overlap is poorer and bounds are wider. Transfer is inactive at the other 12 states because $\rho=0$. Section 4.2 now states this coverage alongside the Eq. (14) screening result. The new Table A-XXIV in Appendix G.2 gives the path and endpoint counts by checkpoint. To ensure that the certificates apply to the implemented update, Appendix A.5 bounds the numerical differences between directions reconstructed from endpoints and those computed from stored gradients. The corrections $E_{D,t}$ and $E_{\Phi,t}$ are subtracted from the path-slope and endpoint-gain lower bounds. All reported counts include these corrections. The supplementary package `TransferCertificateSupplement` provides the directions computed from saved gradients, learned rates, batch statistics, and all 60 statewise results for reproducing Table A-XXIV. **At the passing states, the new Eq. (20) certifies positive surrogate gain throughout $(0,\rho_t]$ with 95% simultaneous confidence.** The alignment tests connect affinity to $s_t$, and the objective tests measure improvements in $f$. The certificate links the two levels by verifying a gain condition for $\Phi_t$ under the learned Global controls and realized Local directions at those states.

Weakness 2. Cross-task coordinate correspondence

Unvalidated cross-task coordinate correspondence. Gradient transfer relies on prescribed coordinate matching, including equal-index neural-network parameters. Such correspondence need not imply functional equivalence across independently optimized policies, and its sensitivity is not evaluated, leaving the transfer mechanism’s applicability insufficiently established.

Thank you for raising the coordinate-matching concern. Equal-index matching defines an architectural channel for gradient transfer. Corresponding positions can have different learned functions. We added function-preserving neuron-permutation tests to change this channel while preserving starting policy behavior. The original mapping πkj\pi_{kj} in Eqs. (6)–(7) defines this correspondence. We clarified its implementation in Appendix C.1. The first 3200 Brax policy parameters share architectural positions, while variable-sized input/output blocks use heuristic correspondence. Affinity assesses sampled directional agreement within this channel.

In the new experiment in Appendix G.3, we independently permuted the neurons within hidden layers 2 and 3 of each task and adjusted adjacent weights and biases. We aggregated gradients in the permuted coordinates and mapped them back to each target's original coordinates.

We tested Ant and HalfCheetah across 3 independent trajectories in the original 10-task environment, at approximately 10% and 20% of the budget. At each of the 12 states, we compared isolated, original-mapping, and 5 permuted one-step branches, giving 84 branches in total. All branches start from the same search distribution with matched Global controls. Transfer branches recompute affinity and retain suppression. Evaluation uses 3 batches of 50 policies and 5 rollouts per policy, with identical Gaussian perturbations, environment resets, and frozen observation statistics across branches. The 6 states with nonzero ρ\rho are shown below.

Task / active checkpointTrajectoryρ\rhoOriginal Local gain (paired-batch SD)Permuted mean gain [min, max]
Ant / 10%10.0039839.85 (19.48)[12.45, 34.02]
Ant / 10%20.002252.56 (40.57)[−27.77, 2.74]
Ant / 20%30.2353948.32 (44.45)[38.26, 56.73]
HalfCheetah / 10%10.002855.09 (104.58)[−53.46, 2.08]
HalfCheetah / 10%20.004964.86 (40.27)[−14.66, 26.33]
HalfCheetah / 20%30.30499−21.97 (17.39)[−62.66, 10.74]

Both original and permuted gains are return differences from the isolated branch. SD describes the 3 paired evaluation-batch differences. Ant gains from Local on average at the active checkpoint of each trajectory, with Global controls matched within each state. The 3 evaluation batches have mixed gain signs in trajectories 2 and 3. HalfCheetah includes a negative mean, and some permuted mappings yield higher means than the original mapping.

All 30 active-state permutations change Local updates, and 17 give higher mean returns than the isolated update. This shows that the choice of coordinate matching affects transfer even when starting policy behavior is preserved.

At the other 6 states, the learned ρ\rho is zero, so Local is switched off. In Eq. (12), ht=ρtdt=0\mathbf h_t=\rho_t\mathbf d_t=0. All branches then have the same base update and updated distribution, giving identical returns under paired evaluation. These states occur at 20% in trajectories 1–2 and 10% in trajectory 3.

The original 10-run ablation complements this sensitivity test by evaluating Local transfer under the prescribed mapping over the full optimization budget. w/o-transfer keeps base-control learning, and GemNES significantly improves 3 tasks, including 11.4% on HalfCheetah and 30.3% on Ant.

VariantTasks+ / - / = vs GemNES
w/o-transfer100 / 3 / 7

Full taskwise reward means (SDs) and significance results appear in Appendix F.4, Table A-XIV. Counts + / - / = indicate that the comparator is significantly better / worse / not significantly different from GemNES (two-sided Wilcoxon rank-sum, alpha = 0.05).

New Table A-XXV and Figure A-6 in Appendix G.3 report the mapping-sensitivity experiment. The new tests demonstrate one-step mapping sensitivity at early Ant and HalfCheetah checkpoints across 3 trajectories. The original 10-run ablation shows that Local transfer with the prescribed mapping improves complete-training performance on selected tasks.

Reviewer_xEgXOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes using Information-Geometric Consensus to unify global IGC and local IGC using Fisher geometry.

Critical strengths

This paper is clear and easy to understand. However, due to the significant differences between this paper and my familiar field, it is difficult to provide a substantial Strengths analysis.

Critical weaknesses

This paper differs significantly from my familiar field, making it difficult to provide a substantial Weakness analysis.

Questions for authors

Since EMA is generally a Log-normal or Multimodal Distribution, its normalization in Eq. (4) is not a normal distribution. Will tanh result in missing important information, and will it yield characteristics different from the authors' expectations?

Will the choice and composition of the group lead to significantly different results?

The authors expose Global progress information and Local geometric information in a coupled feedback loop. Is there a decoupled ablation study?

3 comments

We sincerely thank you for your careful reading and questions. We have updated and uploaded the manuscript, with all changes marked in blue.

Question 1. EMA normalization and tanh

Since EMA is generally a Log-normal or Multimodal Distribution, its normalization in Eq. (4) is not a normal distribution. Will tanh result in missing important information, and will it yield characteristics different from the authors' expectations?

Thank you for asking about this. EMA in Eq. (4) tracks a running baseline and scale from batch objective means and variances. The Gaussian assumption applies to the NES search distribution.

Tanh preserves the sign and ordering of normalized improvements, but makes very large improvements less distinguishable. We use this compression to reduce the influence of extreme task rewards. Because each reward lies in (−1,1)(-1,1), changing one reward with all other terms fixed changes the pooled score by at most

∣ΔJ^m∣≤2/(KH).|\Delta\widehat{\mathcal J}_m|\le 2/(KH).

Here K is the number of tasks and H the visits per candidate. For a coordination group, the bound is 2/(∣Gg∣H)2/(|\mathcal G_g|H). Compression can change candidate rankings after task rewards are combined. Section 3.1 explains this behavior, and Appendix B.1 gives the grouped bound. Thus, tanh trades resolution at extreme improvements for bounded task influence in Global IGC.

Question 2. Group composition

Will the choice and composition of the group lead to significantly different results?

Thank you for raising this question. To test group composition directly, we added RandomGroups, which randomly assigns tasks to balanced groups. The partition stays fixed within each run.

We compared it with the original contiguous grouping at K = 32, 64, 128, 256, 512, and 1024, using 30 runs per method. Each task kept its original task-identity input ιk\iota_k from the contiguous partition. Task sets, group sizes, D = 20, population size 20, the group cap of 100, and 10,000 evaluations per task were matched.

VariantTask scales+ / - / = vs GemNESp-value range
RandomGroups60 / 0 / 60.355–1.000

Full results for all 6 scales, including UOV means (SDs) and individual p-values, appear in Appendix E, Table A-IX. Counts + / - / = indicate that the comparator is significantly better / worse / not significantly different from GemNES (two-sided Wilcoxon rank-sum, alpha = 0.05).

The original synthetic and policy-search experiments each use 1 group, as do the K = 32/64 checks here. At K = 128–1024, changing membership changes mean UOV by −6.39% to +4.81%, with all p > 0.35. Randomizing balanced group membership caused no detectable performance change on these PKACP task sets.

Question 3. Finer-grained ablation

The authors expose Global progress information and Local geometric information in a coupled feedback loop. Is there a decoupled ablation study?

Thank you for suggesting this experiment. In the original study, w/o-meta fixes the base controls and switches off Local transfer, so it removes both effects together. We added rho-only to separate learning the base controls from using Local transfer with a learned rate.

ConfigurationLearn step sizes and shapingLocal IGC with learned ρ\rhoStatus
GemNESYesEnabledOriginal configuration
rho-onlyNo (fixed controls)EnabledNew ablation
w/o-transferYesDisabledOriginal ablation
w/o-metaNo (fixed controls)DisabledOriginal joint ablation

Rho-only and w/o-meta keep GemNES's base learning rates and fix the mean-step multiplier, scale-step multiplier, and shaping exponent to (1,1,1). Rho-only retains affinity and suppression, with Global IGC learning ρ\rho from task progress. w/o-meta uses ρ=0\rho=0.

We evaluated all 4 configurations on 16 synthetic instances with 30 runs and 50,000 evaluations per task. The same + / - / = convention applies. Average Friedman ranks refer to these 4 methods. Lower is better.

Method+ / - / = vs GemNESAverage rank
GemNESBase1.69
rho-only (new)2 / 10 / 42.56
w/o-transfer0 / 5 / 112.19
w/o-meta0 / 14 / 23.56

Full per-instance UOV means (SDs) and significance results appear in Appendix F.4, Table A-XV.

GemNES versus rho-only tests base-control learning, and GemNES versus w/o-transfer tests adaptive Local transfer. With base controls fixed, rho-only has lower mean UOV than w/o-meta on 12/16 instances and a better rank (2.56 vs 3.56). This supports Local transfer controlled by learned ρ\rho with fixed step sizes and shaping.

Section 4.2 and Appendix C.2 now make this distinction explicit. The finer-grained ablation supports both learned base controls and adaptive Local transfer, with the full method performing best overall.

Thank you again for your valuable comments. We have carefully considered your questions and added clarifications and experiments on EMA normalization and tanh compression, grouping robustness across six PKACP scales, and finer-grained ablation through the new rho-only variant. These additions are included in our responses and revised manuscript, with changes marked in blue. We would greatly appreciate your thoughts on whether they address your questions and would be happy to discuss any remaining concerns.

Thank you again for your time and consideration. If your schedule permits, we would be grateful for your thoughts on whether our clarifications and additional experiments address your concerns. We would be happy to discuss any remaining questions before the discussion period ends.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Rebuttal Updates

Dear Area Chair and Reviewers,

Thank you for your time and valuable feedback. We have carefully considered the reviews and uploaded a revised manuscript with changes marked in blue.

Our rebuttal strengthens the evidence through four additions:

  1. Mechanism attribution. The new rho-only, FrozenRandom, RandomControl, and SNES-Tuned comparisons clarify the roles of adaptive controls and transfer. Full-training ablations show that Local transfer improves HalfCheetah and Ant rewards by 11.4% and 30.3% while retaining base-control learning.
  2. Theory and implementation. New alignment, equal-length update, and short-horizon continuation tests examine how transfer improves optimization. The new pathwise certificate verifies positive surrogate gains at 33/48 active states in a preselected subset, with 95% simultaneous confidence and reproducible statewise results.
  3. Grouping and mapping. RandomGroups tests cover six PKACP scales up to 1024 tasks. Function-preserving neuron permutations across three trajectories assess one-step mapping sensitivity.
  4. Comparison scope. PPO/SAC configurations are documented, and their results are presented as descriptive references under the reported protocols. The main policy-search table focuses on ES comparisons.

Together, these additions clarify the central novelty of GemNES: coupling online control adaptation and cross-task gradient transfer through dual-level information-geometric consensus. Global IGC pools task progress to update a shared meta-policy through natural gradients, while Local IGC aggregates affinity-selected peer natural gradients. Learned controls guide transfer, and the resulting search progress informs subsequent adaptation. This coupling reuses ongoing search trajectories and provides a unified approach to many-task optimization, supported by conditional theoretical guarantees and complementary empirical evidence.

AuthorsOpenReview ↗

Summary of Major Reviewer Concerns and Our Responses

Dear Area Chair and Reviewers,

Thank you for your time and constructive feedback. During the discussion, we responded to each reviewer comment and added experiments, theoretical analysis, and clarifications to the manuscript. Changes are marked in blue.

GemNES introduces a unified information-geometric framework for black-box many-task optimization. Global IGC learns task-conditioned step sizes, rank shaping, and transfer strength from pooled search progress. Local IGC aggregates affinity-selected peer natural gradients. The two levels work together to adapt search controls online and share useful directions across tasks, using the trajectories generated during optimization.

  1. Theory and its connection to the implemented updates. The analysis gives sufficient conditions for same-state gains in frozen-rank utility and, under additional assumptions, tighter finite-horizon contraction. The uniform bound in Eq. (14) did not certify gains in the 1,500-state screen. The path-specific certificate in Eq. (20) verified gains at 33 of 48 active states in the prespecified subset, with 95% simultaneous confidence and uneven coverage across checkpoints. The one-step gains of 0.006–0.05% summarize the broader objective experiment. The manuscript does not report objective gains for the certified states separately. The affinity and equal-length diagnostics support the role of update direction, and the 40-update continuations showed gains of 5.69% and 8.25% on the original objectives. These results support a local mechanism for beneficial transfer. Its contribution to full-run performance remains open.

  2. Empirical performance and component contributions. The synthetic many-task and high-dimensional policy-search experiments show faster convergence against ES baselines. The new rho-only ablation separates base-control learning from adaptive Local transfer, while FrozenRandom, RandomControl, and SNES-Tuned provide further comparisons for online adaptation. With base-control learning retained, enabling Local transfer improved mean full-budget returns by 11.4% on HalfCheetah and 30.3% on Ant.

  3. Coordinate correspondence. The function-preserving neuron-permutation study shows that coordinate choice affects one-step transfer and that useful transfer can persist after partial permutation. At one Ant checkpoint, the original mapping outperformed all five tested permutations. At one HalfCheetah checkpoint, it reduced return relative to the isolated update. The full-budget ablation supports the use of Local transfer under the implemented mapping. Whether equal-index matching offers a consistent advantage over alternative mappings remains open.

  4. Normalization and grouping. We clarified how the EMA estimates the baseline and scale and how the tanh transformation limits the influence of extreme task-level rewards. The RandomGroups experiments cover six PKACP scales up to 1,024 tasks and found no statistically detectable grouping effect in the tested multi-group settings.

  5. RL comparison scope. We documented the gradient-based RL configurations and report their results as descriptive references under the stated budgets and runtimes. The main policy-search table compares ES methods.

Reviewer JbU6’s final follow-up arrived shortly before the discussion closed. We would have welcomed more time to clarify how the certificate and objective experiments relate and what the permutation comparisons establish. The time remaining allowed only a brief reply. We appreciate the reviewer’s careful reading and concrete questions, and we thank the Area Chair and all reviewers for considering the revised manuscript and the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.