acceptodds
Under review as a conference paper at ICLR 2027

Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models

Abstract

Offline multi-objective optimization (MOO) seeks solutions with better objective trade-offs using only a fixed dataset, without querying the objectives. Diffusion models trained on such data have emerged as a promising approach, but their samples are not inherently better than the data and must be steered toward the Pareto front. Existing methods guide or condition every sampling step. We instead act on the initial noise and leave the sampling process unchanged. Across Off-MOO-Bench, we observe that the objectives, as functions of the noise, are sensitive to only a few directions. We estimate these directions once per task via a Recursive Feature Machine using function values alone, and a small cache serves every trade-off, so each candidate costs one noise displacement and one ODE solve. We prove that this displacement increases the learned scalarized objective in expectation, and that sweeping trade-offs recovers the flow's attainable front up to proxy and steering errors. With additional guidance, for which we introduce novel data-adaptive and Pareto-aware operators, our method attains the best average hypervolume rank among generative methods on 47 tasks, at comparable or lower sampling cost. Steering alone outranks the best prior generative method at a fraction of its sampling cost.

Primary area:
generative models
Keywords:
Steering, Offline multi-objective, Guidance, Diffusion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_v1bMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose Active-Subspace Noise Steering (ASNS), which shifts generative guidance in offline multi-objective optimization from the sampling trajectory to the initial noise space. Specifically, the method employs RFM to estimate the average gradient outer product (AGOP) of each objective in the noise space from proxy-evaluated noise–objective pairs, and extracts a small number of dominant sensitivity directions. For different preference weights, the method efficiently constructs the corresponding subspaces using cached cross-objective geometric information. Candidate solutions can then be generated with a single noise shift followed by one unmodified ODE sampling pass. The authors further introduce data-adaptive and Pareto-aware late-stage guidance as complementary mechanisms, and evaluate the proposed method on 47 tasks from Off-MOO-Bench, demonstrating its effectiveness and sampling efficiency.

Critical strengths
  1. The authors propose a novel perspective on generative guidance for offline MOO. By shifting optimization signals from step-wise trajectory guidance to the initial noise space, the paper offers a perspective distinct from existing generative approaches to offline multi-objective optimization. The overall method is structurally simple and exhibits good modularity.
  2. The method supports different trade-off weights through a single precomputation stage and cached cross-objective geometric information, while maintaining relatively low computational overhead.
  3. The paper provides substantial theoretical analysis and relatively systematic investigations of the underlying mechanisms. The experiments cover a broad range of tasks, with an overall reasonably comprehensive evaluation protocol.
Critical weaknesses
  1. Whether the observed spectral decay reflects intrinsic objective geometry.Although the authors show that downstream HV is robust to RFM hyperparameters, it remains unclear whether the observed spectral decay faithfully reflects the underlying noise-space objective rather than the inductive bias of the RFM estimator. I suggest directly comparing the true AGOP with the RFM-estimated AGOP on tasks where exact gradients are computationally accessible. Additionally, experiments examining the degradation of proxy accuracy would be valuable for understanding how optimization performance changes as prediction errors increase.
  2. The AGOP characterizes the average squared gradient magnitude. However, a direction exhibiting high sensitivity may correspond to gradients with opposite signs across different regions of the search space. Consequently, a globally fixed steering direction may not consistently improve the objective values across different samples.
  3. The first-order improvement characterized in Theorem 1 is essentially related to the mean gradient. Therefore, an important baseline would be to directly steer the initial noise along the estimated mean gradient. Such a comparison would help establish whether the AGOP-based active subspace provides meaningful benefits beyond simple mean-gradient steering, and thus clarify the necessity of the proposed active-subspace construction.
  4. Table 2 demonstrates clear sampling-time savings, but the practical importance of this speedup is under-motivated. Since the reported timing excludes training and most methods already generate 256 candidates within seconds to minutes, it would be useful to quantify when sampling becomes a meaningful fraction of the end-to-end cost, or identify applications where repeated low-latency generation is important. Otherwise, the space devoted to this efficiency argument might be better allocated to additional algorithmic analysis and ablation studies.
  5. Although RFM-Cone achieves the best average rank among generative methods, its overall average rank is slightly worse than MM+ICT, and its relative advantage varies considerably across task families. In light of the No Free Lunch (NFL) principle, algorithmic effectiveness is inherently dependent on the characteristics of the problem. Therefore, a more important question may be to identify the specific scenarios in which ASNS is particularly effective, as well as those in which it is less suitable, rather than emphasizing its advantages across all problems.
Questions for authors

Please refer to Weaknesses.

5 comments
AuthorsOpenReview ↗

We thank the reviewer for their helpful comments and questions! We address them all below [1/2].

W1: Is the spectral decay intrinsic?

We ran both suggested experiments (new App. B.2.7–B.2.8). (1) The exact AGOP is already low-rank, so the decay is a property of the noise-space objective and not of the inductive bias of the RFM estimator. (2) HV is stable when the proxies are retrained on 50% and 25% of the data, even where proxy accuracy drops most (see reply to isuK Q4).

Exact vs RFM AGOP. On 3 tasks we computed the exact AGOP by differentiating through the full ODE solve at 5,000 fresh noises (5 seeds). DD is the dimension of the representation the flow is trained on, and the participation ratio is (∑jλj)2/∑jλj2(\sum_j\lambda_j)^2/\sum_j\lambda_j^2.

Task (D)RE36 (4)C-10/MOP1 (31)IN-1K/MOP8 (174)
Participation ratio (exact / RFM)2.8 / 2.46.2 / 4.92.9 / 2.1
Mean-gradient share (exact / RFM / random)1.00 / 1.00 / 0.980.87 / 0.85 / 0.410.92 / 0.91 / 0.08
  • The decay is intrinsic. The exact spectrum has 2.9 effective directions out of 174 on IN-1K/MOP8. The RFM sharpens the decay by 14–28% but does not create it.
  • The RFM recovers the directions that matter. By Thm. 1, steering gains in proportion to the mean gradient its directions contain, and the RFM directions contain as much of it as the exact ones (last row).
  • Beyond these 3 tasks. (i) Within a task family the effective rank (App. B.1, a soft-weight measure) does not grow with DD (10–15 for C-10/MOP over DD = 24–140), following the intrinsic dimension of the data (§3.1). (ii) On 11 tasks (DD up to 10,184), projecting a Stein estimate of the mean gradient, which the RFM never sees, onto the RFM subspace never lowered its accuracy by more than 0.005 and raised it on every NAS task (App. B.3.5). (iii) An RFM inductive bias would produce directions that matter to the RFM but not to the objective. Instead, the top RFM direction raises the true oracle objective, while a trailing or random direction of equal length does not (Fig. 1b).

Proxy degradation (App. B.2.8). To understand how performance changes as prediction errors increase, we first bound it and then test it.

  • Bound. Thm. 1 guarantees ascent on the proxy objective without assuming it is accurate. Proxy error enters additively: it perturbs the learned directions (Cor. 2) and costs at most 2εprox2\varepsilon_{\text{prox}} in scalarized value (Thm. 3), which is the cost of optimizing any proxy, so steering adds no proxy dependence of its own.
  • Test. We retrained the proxies on 50% and 25% of the data and reran RFM-AN, RFM-Cone and ParetoFlow+ on 5 tasks. Held-out Spearman fell most on ZDT4 (0.47→0.30) and MO-Hopper (0.38→0.32), yet normalized HV100 changed by at most 0.03 for every method and task, within seed noise, and our standing relative to ParetoFlow+ did not change. Stronger tests (5% subsets, label noise) will follow in final version.

W2: Opposite signs

We now measure it directly (new App. B.3.4): the single global ASNS step still improves most samples, 73% to 100% of candidates on 6 tasks. We agree that gradients along a single direction can have opposite signs, and Thm. 1 already accounts for it. E.g., for H(z)=z12H(z)=z_1^2 the gradient 2z12z_1 has opposite signs on the two halves of the noise, so the mean gradient is 0 while the AGOP is 4. ASNS handles such directions in two ways.

Fixed step. Alg. 1 gives each direction the sign of its mean derivative dˉj\bar d_j (Pearson sign, §3.3), and in Thm. 1 the gain is set by dˉj\bar d_j, not by λj\lambda_j. A direction whose gradients cancel has dˉj≈0\bar d_j\approx0 and adds nothing (reversing the leading direction lowers the true objective, App. B.2.9, Fig. 9), so the gain comes from the directions with a consistent sign. Cancellation is real: the sign-consistent share cj=(uj⊤E∇Hw)2/λjc_j=(u_j^\top\mathbb E\nabla H_w)^2/\lambda_j is about 10−310^{-3} for the top 10 directions on C-10/MOP1 and IN-1K/MOP8 (it is energy-weighted, so a few large opposing gradients can dominate it). Yet, decoding the same noise with and without the step (5 seeds), the true scalarized objective improves for most candidates, i.e. the step improves samples consistently:

C-10/MOP1C-10/MOP4IN-1K/MOP8RE36DTLZ7ZDT3
Candidates improved73%89%97%87%99.8%100%

Mean gain is positive on every task, and the consistency holds at every step size tested: on IN-1K/MOP8, 79% of candidates already improve at a tenth of the default step.

Sample-dependent steps. Where signs do flip, a sample whose gradient opposes the fixed step is not improved by it. Each sample must follow its own gradient, which our guided variants do (§4). This needs the AGOP: in the example above the mean gradient gives no direction, while the AGOP still identifies z1z_1. RFM-Cone also resolves conflicts between objectives, decreasing none to first order (Thm. 2). See W3, which quantifies how much such steps gain.

AuthorsOpenReview ↗

Rebuttal to Reviewer v1bM continued [2/2]

W3: Mean-gradient baseline

We clarify that the AGOP is not an estimate of the mean gradient. It is the second moment of the gradient and identifies a subspace, of which the mean gradient is one direction. We ran the requested baseline at matched norm and budget: for a fixed step, the simplest instance of ASNS, it matches RFM-AN, as Thm. 1 predicts (App. B.3.1). The active subspace is necessary once the step depends on the sample, as in our guided variants: there it is provably optimal and a single mean-gradient vector cannot be used (new Prop. 2). Full results are in our reply to isuK W2, Remark 1 & Prop. 2 (App. C.3), & App. B.3.

  • Fixed step (RFM-AN). By Thm. 1 the gain of a fixed step is set by the mean gradient, so it is the best fixed direction (Remark 1), and the active subspace contains it: 85–91% on C-10/MOP1 and IN-1K/MOP8, against 8–41% for a random subspace of the same rank (W1).
  • Sample-dependent steps. The mean gradient keeps only what all samples share, a share cw=∥E∇Hw∥2/E∥∇Hw∥2c_w=\Vert\mathbb E\nabla H_w\Vert^2/\mathbb E\Vert\nabla H_w\Vert^2 of the gradient energy, 0.004–0.28 on our tasks. A step that follows each sample's gradient can use the rest: its first-order ceiling is up to 1/cw1/\sqrt{c_w} higher, and among rank-rr subspaces the active subspace maximizes it (Prop. 2). On 120 synthetic objectives with exact gradients (DD = 10–174), the per-sample step in the AGOP subspace beats the Stein shift on 119 of 120, and the AGOP subspace beats a random one on all 120. On 4 benchmark tasks, per-sample steps gain 1.2–2.2× as much as the Stein shift (App. B.3.2–B.3.3).
  • Full method. RFM-APG and RFM-DAMG reshape each sample's own proxy gradient with the AGOP at every guided step, which needs the AGOP as a matrix. Both improve HV over steering alone, in 72% and 65% of task–seed pairs (App. B.4), and lower the average rank from 8.93 (RFM-AN) to 8.36 and 8.49 (Table 1).

W4: When sampling cost matters

Sampling becomes a meaningful share of the cost when a trained model is queried repeatedly. In hardware-aware NAS (our MO-NAS tasks), DC-QFA (arXiv:2604.10170, 2026) trains one supernet and then runs a hardware-constrained search for each target device's latency and memory budget. In MORL (our MO-Hopper and MO-Swimmer tasks), users specify a preference at test time and get the matching policy without retraining (Chen et al., ICLR 2026). Each device or user is a new preference query. There the gap is large: sampling costs as much as training after about 190 queries for ParetoFlow+, but not until about 3,800 for RFM-AN (App. A.4). We agree a single run is dominated by training (about 51 min on MO-Hopper).

  • Each new preference. ParetoFlow+ builds each weight's next state from the offspring of its neighbouring weights, so a new preference reruns its full procedure (16.2 s on MO-Hopper). RFM-AN reuses its cache and answers in 0.8 s.
  • Share of end-to-end cost spent on sampling (MO-Hopper, training included):
QueriesParetoFlow+: share / totalRFM-AN: share / total
10.5% / 51 min0.8% / 51 min
10035% / 1.3 h3% / 52 min
1,00084% / 5.3 h21% / 1.1 h
  • More objectives. A fixed preference grid has 13 weights at m=2m=2 but 6,188 at m=6m=6. On RE61, tracing that front takes ParetoFlow+ 2.2 min and RFM-AN 18 s, precomputation included.

W5: Where ASNS is effective

We agree, and the revision now characterizes this by two properties of the problem rather than by task: whether the data are low-dimensional, and how far the front lies from the data.

  • Most effective: low-dimensional data, best solutions close to the data. The flow contracts the prior onto the data manifold (§3.1), so a few noise directions capture most of the gradient (Prop. 1) and a short step along them improves the objective (Thm. 1). MO-NAS is this case: the best architectures differ from the data by a few of 5–34 categorical choices. There RFM-Cone ranks first (C-10/MOP) and second (IN-1K/MOP) among all 18 methods.
  • Regime 1: the front is far from the data. Reaching it needs a large step, and Thm. 3 shows the error grows with the step, as the flow and proxies leave the data. DTLZ and ZDT are this case, with fronts on the bounds of the design space: on ZDT4 larger steps lower HV by 15% (App. B.2.1). Surrogate-based NSGA-II searches the design space directly and has no such limit (rank 3.3 and 4.9, against 9.8 and 9.5 for RFM-Cone). Guidance closes part of the gap (App. B.4).
  • Regime 2: too few dimensions to reduce. RE has D≤7D\le7 and effective rank 1.4–5.5, so there is little low-rank structure to exploit and steering has no advantage over direct search (rank 9.6, against 5.7 for MM+Tri-Mentor).

In both regimes ASNS trails direct surrogate search but stays best or on par among generative methods (rank 9.6 vs 13.0 for PCD on RE). On MORL the limit is proxy accuracy, not geometry (reply to isuK Q4).

AuthorsOpenReview ↗

Thanks for your review!

Dear Reviewer v1bM,

As the rebuttal period comes to an end, we wanted to thank you for taking the time to read our work. We would also like to follow up on our rebuttal response and ask whether we were able to address all of your questions. Additionally, we have also posted a summary comment for all reviewers that lists where each change appears in the revised PDF. We would be happy to address any remaining concerns!

Reviewer_X4RFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an Active-Subspace Noise Steering (ASNS) method for offline multi-objective optimization. The core idea is to identify low-dimensional directions in the initial noise space that are most relevant to the objectives and to steer generated solutions toward the Pareto front through a single noise displacement, rather than applying gradient guidance at every step of the sampling trajectory.The authors use RFM and AGOP to estimate objective-sensitive directions in noise space, reuse a shared precomputation structure to support efficient steering across multiple preference weights, and further introduce data-adaptive and Pareto-aware guidance operators. Theoretically, the paper shows that appropriate noise steering can improve the scalarized surrogate objective in expectation and analyzes the errors associated with subspace truncation and Pareto-front recovery. Experiments on 47 tasks from Off-MOO-Bench show that the proposed method achieves competitive overall performance among generative multi-objective optimization methods while maintaining relatively low sampling cost. Overall, the main contribution of this work is to exploit objective-relevant geometry in noise space for offline multi-objective optimization and to shift most of the geometry-estimation cost to a one-time precomputation stage, enabling relatively efficient generative optimization.

Critical strengths

1.The paper proposes a practically novel noise-space steering framework. It identifies objective-sensitive low-dimensional directions in the initial noise space and uses a one-time precomputation to support steering across different preference weights. The method retains strong optimization capability while avoiding step-by-step trajectory optimization, leading to relatively efficient sampling. This is one of the paper’s main technical contributions. 2.The theoretical analysis is well aligned with the empirical evaluation. The paper provides theoretical results on improving the scalarized surrogate objective through noise steering, active-subspace truncation, and Pareto-front recovery error, and evaluates the method through systematic experiments and ablation studies on 47 Off-MOO-Bench tasks. The results show competitive performance among generative multi-objective optimization methods and provide empirical support for the effectiveness of the proposed noise-space structure.

Critical weaknesses

1.The computational efficiency comparison is not fully comprehensive. The cost of the RFM geometry precomputation is reported separately and is not included in the main sampling-time comparison, so the claimed efficiency advantage mainly reflects amortized inference cost. For settings where optimization is performed only once per task, the end-to-end computational advantage may be smaller than the reported sampling results suggest. It would be helpful to report total runtime including precomputation and provide an amortization analysis. 2.The statistical evidence in the main experiments is still limited. The comparisons are mainly based on means, standard deviations, and average ranks over five random seeds, while the W/T/L results are reported without significance testing. Since the performance gaps between some methods are relatively small, the current results support an average performance advantage, but provide weaker evidence for a consistent superiority over existing methods.

Questions for authors

1.The theoretical analysis of empirical AGOP consistency appears to treat the RFM surrogate as fixed and assumes that the noise samples used to estimate the AGOP are independent of the surrogate. However, in the actual algorithm, the RFM is trained on the same samples and its gradients are evaluated on those samples. Could the authors clarify how the current theoretical results apply to this data-dependent setting? Would additional sample splitting, cross-fitting, or stability analysis be required? This issue directly affects the consistency between the theoretical guarantees and the implemented algorithm. 2.The main runtime comparison does not include the one-time RFM geometry precomputation cost in the reported sampling time. Could the authors provide the end-to-end runtime including precomputation and clarify how many different preference weights or repeated sampling runs are needed before this cost is effectively amortized? This would materially affect the assessment of the method’s computational efficiency advantage.

4 comments
AuthorsOpenReview ↗

We thank the reviewer for their thoughtful comments! Please see our responses below. We group W1 with Q2, and W2 with Q1, since each pair covers the same topic [1/2]

W1: Computational Efficiency

We now report end-to-end cost: charging the full precomputation to a single run, RFM-AN is still faster than ParetoFlow+ (same flow and proxies) on 5 of 6 tasks and than PCD on 4 of 6, and breaks even within 3 runs otherwise (tables below). We also clarify 2 things. (1) Table 2 already lists the full one-off precomputation in its first row (7.4–22.7 s), and App. B.2.9 (Study 2) separately measures cache construction, per-weight assembly and refitting. Our efficiency claims in the abstract and §5 refer to sampling cost. (2) More importantly, this precomputation is amortized even when optimization is run only once per task. We have added this discussion in App. A.4, and state the claim in §5 as "fastest to sample on every timed task" to make its scope explicit.

Why one run already amortizes. Tracing a front means sweeping many preference weights, 256 in our experiments. Guided methods (ParetoFlow, ParetoFlow+) recompute their guidance at every integration step for every weight, and PCD runs its conditional model with classifier-free guidance for every target. Our precomputation is paid once per trained model and shared by all 256 weights: the metric for any weight is assembled from the cached blocks (Eq. 8) in about 1 ms on ZDT2, versus 0.56 s for refitting the RFM per weight (App. B.2.9, Fig. 10b). Spread over one run it costs 0.03–0.09 s per weight. The table nevertheless charges all of it to that run.

End-to-end time (s) = precomputation + sampling (Table 2) for one full run of 256 weights. Baselines have no precomputation. Training is excluded for all methods, as in Table 2.

MethodDTLZ2ZDT4C-10/MOP1MO-HopperRE21RE61Mean
ParetoFlow7.48.97.910.88.611.89.2
ParetoFlow+12.412.211.116.213.519.314.1
PCD6.87.622.2149.422.625.339.0
RFM-AN8.98.79.529.08.68.712.2
RFM-APG10.911.615.753.710.211.218.9
RFM-DAMG11.510.812.250.010.712.117.9
RFM-Cone14.412.013.554.911.627.522.3

Guided variants. RFM-Cone, RFM-APG and RFM-DAMG add per-step guidance for quality, so their first-run cost exceeds ParetoFlow+(17.9–22.3 vs 14.1 s). They remain 1.7–2.2× faster than PCD end-to-end on average, and once the cache exists they sample faster than ParetoFlow+ on average (7.9–12.3 s vs 14.1 s, Table 2). The efficiency claim rests on RFM-AN, which alone significantly outperforms both ParetoFlow+ and PCD (see W2).

High dimension (MO-Hopper, D = 10,184). This task has the largest precomputation (22.7 s) and still supports the claim. Against PCD, RFM-AN takes 29.0 s versus 149.4 s in a single run, about 5× faster. Against ParetoFlow+ it samples 2.6× faster (6.3 s vs 16.2 s), so it is slower in the first run but ahead from the third.

Break-even. Number of runs N after which RFM-AN's total time, precomputation included, is at most the baseline's. N = 1 means it is already faster in the first run.

RFM-AN vsDTLZ2ZDT4C-10/MOP1MO-HopperRE21RE61
ParetoFlow+111311
PCD221111
ParetoFlow212611

The original ParetoFlow, which uses its own cheaper sampler, is the cheapest baseline to run once. RFM-AN overtakes it within 2 runs on all tasks except MO-Hopper.

Reuse beyond one run. Any additional weight, or a different (e.g. non-linear) scalarization (App. B.2.9), reuses the cache at a few milliseconds plus one ODE solve. This is the natural setting for a decision-maker exploring trade-offs, whereas guided methods pay their full per-step cost again for every new weight. A new task, dataset or retrained model requires a new precomputation, which we count as a new setup.

Q2: 1-time RFM Precompute

One run of 256 weights already amortizes the precomputation against ParetoFlow+ on 5 of 6 tasks and against PCD on 4 of 6, and at most 3 runs are needed in the remaining cases. Please see our reply to W1 for the end-to-end and break-even tables.

  • One run suffices. All 256 weights share one cache, so the precomputation costs 0.03–0.09 s per weight.
  • Remaining cases recover within 2 runs against PCD (DTLZ2, ZDT4) and within 3 against ParetoFlow+ on MO-Hopper. MO-Hopper has the largest setup cost (22.7 s), yet RFM-AN and RFM-Cone are still 5.2× and 2.7× faster end-to-end than PCD, the best prior generative method.
  • New weights reuse the cache at about a millisecond each (App. B.2.9, Fig.10b), and repeated sweeps with the same trained model pay the setup once.

So the sampling advantage of ASNS over per-step guidance on the same flow and proxies translates into a first-run wall-clock advantage on 5 of the 6 timed tasks. This discussion is added to App. A.4.

AuthorsOpenReview ↗

Rebuttal to Reviewer X4RF continued [2/2]

W2: Statistical Evidence

The advantage holds under the significance tests we added (new App. A.2.1). Counting only differences that are significant on a task, RFM-AN has 29 wins and 4 losses against ParetoFlow+ (sign p = 1.1e-5). Against PCD, RFM-Cone wins on three times as many tasks as it loses (22 vs 7, sign p = 0.008). The two methods excel on different task families, so the size of the gain varies from task to task (Wilcoxon p = 0.15). This is what we claim: RFM-Cone matches or beats PCD on 40 of 47 tasks, at about a third of its sampling cost (§5.2, Table 2).

ComparisonW/T/L (Table 1)Sig. W/T/LSign pWilcoxon p
RFM-AN vs ParetoFlow+ (same flow, proxies)37/0/1029/14/41.1e-54.5e-4
RFM-Cone vs ParetoFlow* (better of ParetoFlow, ParetoFlow+ per task)36/0/1127/17/38.4e-61.1e-3
RFM-Cone vs PCD27/2/1822/18/70.0080.15

Tests. All use the raw per-seed HV100 (47 tasks × 5 seeds). A task counts as a win or loss only if the difference is significant on that task (Welch t-test over seeds, BH q < 0.05), and otherwise as a tie. A sign test asks whether significant wins outnumber losses, and a Wilcoxon test asks whether the advantage is consistent across tasks. A Friedman test over the 18 methods rejects equal performance (p = 2.7×10⁻¹⁷).

  • Most raw losses are seed noise. Significance removes a much larger share of our losses than of our wins, raising the win:loss ratio from 3.7:1 to 7.3:1 against ParetoFlow+ and from 3.3:1 to 9:1 against ParetoFlow*. The remaining ties have small gaps or high seed variance (e.g., MORL).  
  • vs guidance. With the flow and proxies fixed, RFM-AN gives a consistent gain, significant both per task and across tasks. RFM-Cone adds a consistent gain over steering alone (Wilcoxon p = 0.01).  
  • vs PCD. RFM-AN alone also wins significantly more often than it loses (20 vs 8, sign p = 0.036).  
  • Robustness. The conclusions are unchanged at HV75 and under a paired t-test, and at HV50 significant wins still outnumber losses against every generative baseline (sign p ≤ 0.004).

Q1: Theoretical analysis of empirical AGOP consistency

Could the authors clarify how the current theoretical results apply to this data-dependent setting? Would additional sample splitting, cross-fitting, or stability analysis be required?

We clarify that no splitting, cross-fitting or stability analysis is needed. Evaluating the fitted RFM on fresh noise satisfies the independence assumption exactly, and a new corollary (App. C.4) bounds the in-sample estimator of Algorithm 1 by its distance to that fresh estimate. We measure this distance to be small (3–4% for D ≤ 174), with no significant change in HV on any task.

What the theory covers. We agree that Prop. 3 (Prop. 2 before revision) assumes the evaluation points are independent of the fitted surrogate, whereas Algorithm 1 evaluates gradients on the points used for fitting. Evaluating the same fitted RFM on n fresh i.i.d. Gaussian points satisfies the assumption while keeping all fitting data, so under Prop. 3's moment condition this estimate converges to the fitted surrogate's population AGOP.

In-sample estimator (new corollary). Conditional on the fit, with probability at least 1−δ1-\delta over the fresh draws,

∥M^win−Mwϕ∥F≤∥M^win−M^wfr∥F+κw/(nδ).\Vert\hat M^{in}_w-M^{\phi}_w\Vert_F \le \Vert\hat M^{in}_w-\hat M^{fr}_w\Vert_F+\sqrt{\kappa_w/(n\delta)}.

This finite-sample bound follows from the second-moment bound in Prop. 3's proof, Markov's inequality and the triangle inequality, so it needs no stability argument. It does not by itself give consistency of the in-sample estimator, which would also require the first term to vanish as n grows. We therefore measure the first term relative to the in-sample AGOP's Frobenius norm, the cosine between the steering displacements of the two estimates, and the paired difference in normalized HV (5 seeds, full sweep):

Task (D)Relative AGOP discrepancySteering cosineΔHV (95% CI)
RE36 (4)3.0%0.80−0.015 [−0.053, 0.023]
ZDT4 (10)3.3%0.93−0.003 [−0.030, 0.025]
C-10/MOP1 (31)4.1%0.97+0.003 [−0.014, 0.021]
IN-1K/MOP8 (174)3.0%0.93−0.001 [−0.004, 0.002]
MO-Hopper (10,184)14%0.95+0.005 [−0.046, 0.056]

The first term is small, the steering directions agree, and every HV interval includes zero, so in practice the implemented estimator matches the one the theory covers. On MO-Hopper the difference is mostly a rescaling (the in-sample AGOP is 9–14% smaller), to which steering is invariant (displacement cosine 0.95). In the revision, §3.2 (p. 6) describes the fresh-sample estimator, for which Prop. 3 holds exactly, and App. B.2.6 reports this comparison.

Thank you for your response. However, I will maintain my original score.

Thank you for reading our response, and for your constructive review and support of the paper! Your suggestions on end-to-end runtime, significance testing and the in-sample estimator made the revision stronger. If any specific concern remains on these points, we would be glad to address it in the remaining discussion period. We have also posted a summary comment for all reviewers that lists where each change appears in the revised PDF.

Reviewer_isuKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Active-Subspace Noise Steering (ASNS) for offline multi-objective optimization (MOO). Instead of guiding the generative model at every sampling step, this paper learns objective-sensitive directions in the initial noise space using Recursive Feature Machines (RFM) and Average Gradient Outer Products (AGOP). The learned geometry is cached and reused across different objective preferences, allowing the method to steer initial noise before generation. The paper also introduces several optional guidance operators to further improve optimization performance. Experiments on 47 Off-MOO-Bench tasks show that the proposed methods achieve competitive results, with RFM-Cone obtaining the best average hypervolume rank among the generative methods evaluated.

Critical strengths
  1. Interesting idea with practical value. I think moving optimization from the sampling trajectory to the initial noise space is a useful direction. Learning reusable objective-sensitive directions and sharing the precomputation across different preference weights is a reasonable way to reduce inference-time overhead. Although the underlying techniques are not entirely new, adapting them to offline multi-objective optimization is a meaningful contribution. I deem these as solid advancement in this specific area.

  2. Extensive experiments with promising results. The evaluation covers 47 tasks from several benchmark families, with multiple baselines and ablation studies. The reported performance and sampling efficiency are encouraging, especially for the noise-steering-only variant.

Critical weaknesses
  1. Incomplete comparisons with relevant methods. While the paper includes a fairly extensive evaluation, I think some closely related methods deserve more attention. In particular, SPREAD (ICLR 2026) and PGD-MOO (NeurIPS 2025) are directly relevant generative approaches to multi-objective optimization, but neither is included in the experimental comparisons. It would be helpful to compare these methods on their overlapping benchmark tasks, or provide a more detailed explanation of the difficulties involved. DOMOO (ICML 2026) is another relevant recent work that would be useful to discuss, and a comparison would be welcome if feasible. These comparisons would help establish the proposed method's advantages more clearly.

  2. The additional benefit of the learned RFM/AGOP geometry is not fully isolated. The current ablations show that steering along the learned directions performs better than random, trailing, or reversed directions. However, since ASNS applies a fixed noise displacement for each preference weight, it is still unclear how much of the improvement comes specifically from the learned AGOP geometry, rather than from a simpler noise-space shift. For example, the mean-gradient direction can be estimated directly from the same cached noise-score pairs using Stein's identity, without fitting an RFM or computing its eigendecomposition. I think a small comparison against this simpler alternative would be useful. This would help explain what the additional RFM/AGOP machinery contributes to the final optimization performance.

Questions for authors
  1. Comparison with related methods. Could the authors consider additional comparisons with SPREAD, PGD-MOO and DOMOO on the tasks supported by both methods? If implementation compatibility is an issue, a more specific explanation would be helpful.

  2. A simpler noise-steering baseline. Would it be possible to compare RFM-AN with a direct mean-gradient or noise-score covariance steering baseline, estimated from the same cached samples? Using the same displacement norm and candidate budget would make this especially informative. A few representative tasks should be enough.

  3. Connection to single-objective optimization and related work. Since the proposed noise-steering approach can also be applied to a single objective, I would appreciate a brief discussion of what additional challenges arise when extending it to multi-objective optimization, and which components specifically address these challenges. In particular, it would help to explain the roles of the cross-objective geometry cache and Pareto-aware guidance.

    Relatedly, the positioning could be improved by briefly discussing more work on diffusion-based black-box optimization and related generative optimization methods. In addition to the papers already cited, the authors may consider the following relevant references:

    These works explore different approaches, including conditional generation, diffusion-based surrogate modeling, and guidance for optimization. A short discussion of them would be enough.

  4. Performance on high-dimensional MORL tasks. Could the authors provide some discussion of why the improvements on MO-Hopper and MO-Swimmer are relatively limited? In particular, RFM-Cone does not outperform the D(best) baseline on either task. I wonder whether this is related to the high dimensionality, the truncated RFM geometry, or proxy accuracy? A brief discussion based on the existing experiments would be helpful.

I would be happy to reconsider my assessment and raise my score in light of the authors' responses.

4 comments
AuthorsOpenReview ↗

We thank the reviewer for their insightful review and questions! We address them all below. [1/2]

W1: Comparisons to Relevant methods

We added both runnable methods under our protocol: RFM-Cone outperforms DOMOO and SPREAD on every MO-NAS task, by 2–11% and 12–32% of D\mathcal D(best), while both are stronger on synthetic and RE tasks, like the other surrogate-search baselines (Table R1, all tasks in App. A.6).

  • PGD-MOO: we could not run its released code under our protocol. PCD (ICLR 2026) reports the same difficulty. Its scripts cover only ZDT, DTLZ and RE, leave the seed unset and never decode discrete designs, so MO-NAS and MORL (20 of our 47 tasks) are uncovered. Its reported results are not usable either: DTLZ2–6 predate the benchmark fix (e.g., D\mathcal D(best) of 9.35 vs 20.98 on DTLZ5), and its reference values differ from ours on RE37, RE42 and RE61.
  • DOMOO learns a Pareto-set model on MLP surrogates and merges its candidates with an NSGA-II archive. It thus belongs with our DNN-surrogate baselines, and the generative baselines we follow (ParetoFlow, PCD) do not compare against it. We ran its official code unchanged (31 tasks).
  • SPREAD is primarily an online method and its published offline values cannot be compared with ours. Its baseline and D\mathcal D(best) values are reused from PGD-MOO, and its own are on a different scale (e.g., 31.19 on DTLZ4 against a D\mathcal D(best) of 10.76, 251.8 on RE61, where no method in our tables exceeds 109). We reran its offline launcher and sampling alone takes 12–21 min per seed, against seconds for RFM-Cone, yet its only gains are on some synthetic and RE tasks (e.g. RE36; see reply to v1bM W5).

Table R1 HV100 (mean ± std, 5 seeds); rest SPREAD tasks to follow in final draft

TaskD\mathcal D(best)DOMOOSPREADRFM-Cone
C-10/MOP39.5810.97±0.0810.03±0.2111.16±0.02
C-10/MOP419.7422.57±0.2818.97±0.8523.85±0.37
IN-1K/MOP814.0410.66±0.838.58±0.2212.15±0.08
IN-1K/MOP919.6614.61±0.529.81±0.2316.17±0.07
ZDT45.465.01±0.165.01±0.094.89±0.14
RE367.7610.26±0.069.95±0.089.37±0.28

W2: Learned RFM geometry

We clarify that AGOP is not an estimate of the mean gradient. It is the second moment of the gradient and identifies a subspace, of which the mean gradient is one direction. ASNS steers within this subspace in 2 ways. The simplest is one fixed displacement per weight, the form Thm. 1 analyzes and we report as RFM-AN. For it, we agree the mean gradient is the best direction: we ran Stein baseline and it matches RFM-AN. More generally, the step can depend on the sample, e.g., per-step guidance (our guided variants) or a per-sample noise step (added in the revision), both supported by the same RFM fit. For such steps, AGOP subspace is provably optimal and a single mean-gradient vector cannot be used (see new Prop. 2 in revised pdf).

  • Fixed displacement. We estimate the mean gradient with Stein’s identity from the same cached pairs, with no RFM, and compare it with RFM-AN at the same displacement norm, flow, proxies, noise and selection (5 seeds). The two are comparable: within 1% of D\mathcal D(best) on IN-1K/MOP8 and ZDT3, with RFM-AN ahead by 5.2% on C-10/MOP4 and Stein by 2.6% on DTLZ7, and both improve on unsteered sampling (App. B.3.1). This is what Thm. 1 predicts: first-order gain of any fixed displacement is set by the mean gradient (new Remark 1).
  • Sample-dependent steps. Here the geometry matters. Stein’s identity gives one vector per weight, while the RFM gives a gradient at every candidate, so each can move along its own gradient at the same average displacement, recovering the part of the gradient that cancels in the mean. Prop. 2 now quantifies it: a fixed displacement uses only a fraction cw=∥E∇Hw∥2/E∥∇Hw∥2c_w = \|\mathbb E\nabla H_w\|^2 / \mathbb E\|\nabla H_w\|^2 of the gradient energy, a sample-dependent one raises the first-order ceiling by up to 1/cw1/\sqrt{c_w}, and among rank-r subspaces the AGOP subspace maximizes it.

We test this on 4 tasks with cw≤0.3c_w \leq 0.3 at step s=∥Δz∥/D=0.25s=\|\Delta z\|/\sqrt D=0.25 (App. B.3.2), displacing same noises under both steps, comparing each candidate’s true scalarized score. The fixed comparator is the mean RFM gradient, which matches the Stein shift in mean gain (within 0.01).

Taskcwc_wCandidates improved over a fixed shiftGain ÷ Stein's gain
ZDT40.01261%2.0×
ZDT60.07859%2.2×
RE360.09247%1.2×
DTLZ50.28368%1.2×

Sample-dependent step beats the mean-gradient shift in mean gain on every seed of all 4 tasks, by 1.6× on average. The step is small to stay in Prop. 2's first-order regime, where HV100 of every arm, unsteered included, lies within ~1%, so we report pre-selection scores.

HV at full strength. The guided variants are this step at full strength: RFM-APG, RFM-DAMG apply AGOP per sample at each guided step, and both improve on the fixed displacement, lowering average HV rank from 8.93 (RFM-AN) to 8.36 and 8.49 (Table 1). More in revision.

AuthorsOpenReview ↗

Rebuttal to Reviewer isuK continued [2/2]

Q1: Related Methods

Yes. We ran DOMOO and SPREAD under our protocol, and RFM-Cone outperforms both on every MO-NAS task (Table R1). We could not run PGD-MOO's released code, whose scripts cover only ZDT, DTLZ and RE, not MO-NAS or MORL (20 of our 47 tasks). See reply to W1.

Q2: Simpler baseline

Yes, we ran this baseline: it shows no consistent difference from RFM-AN at the default step, as Remark 1 predicts, and both improve on unsteered sampling (results in App. B.3.1; see reply to W2). The 2 suggested baselines coincide: for Gaussian noise, the noise-score covariance is, by Stein's identity, the estimate of the mean gradient E∇Hw\mathbb E\nabla H_w. We compute it from the same cached samples at matched displacement norm and candidate budget (512 candidates, same selection to 256; 5 seeds).

Q3: SOO

MOO adds 2 challenges over single-objective steering, and one component addresses each: the cross-objective cache serves every trade-off without refitting, and Pareto-aware guidance (RFM-Cone) handles conflicting objectives. We added this discussion (§3.2, App. A.8) and all 5 references (§1).

With one objective, our method reduces to RFM steering of that function (Thm. 1, Prop. 1).

(1) One optimum per trade-off. Tracing a front means sweeping 256 weights, each with its own noise-space objective, so refitting the RFM per weight would repeat the precomputation 256 times. The cross-objective cache avoids this. The scalarized gradient is a weighted sum of per-objective gradients, so its metric is assembled from cached blocks in ~1 ms per weight, over 500× faster than a refit (0.3 s vs 2.4 min per sweep on ZDT2). The cache also serves Tchebycheff scalarizations, which reach non-convex parts of the front that a linear sweep cannot.

(2) Conflicting objectives. A step along the scalarized gradient can raise the weighted sum while lowering one objective. RFM-Cone avoids this by projecting the step onto the cone of directions with non-negative inner product with every proxy gradient. No objective decreases to first order, and where the projection vanishes at a Pareto-stationary point the step reduces to the unmodified flow (Thm. 2).

Related work. The 5 works differ from ASNS in where the optimization signal enters:

  • Into training: Yun et al. train a score-conditioned trajectory model, and Sun et al. fine-tune a diffusion language model, then apply RL. ASNS keeps an unconditional flow and one cache for every weight.
  • Joint model: Zhou et al. learn a joint design–score distribution and guide denoising toward target scores. ASNS models designs only.
  • Surrogate: Yang et al. use diffusion to model the score distribution given a design. Such a surrogate could serve as our proxy.
  • Sampling: Kong et al. sample from the product of the data distribution and an objective-defined density. ASNS acts on the initial noise, with guidance only late in the trajectory.

Q4: MORL

We examined each suggested factor, and the evidence points primarily to proxy accuracy.

High dimensionality. Dimensionality does not degrade the learned geometry: at D≈104D\approx10^4, the noise-space objective stays low rank (effective rank 65–100), consistent with the other tasks (App. B.1), and HV is stable across all RFM-fit settings (App. B.2). However, it makes the proxies harder to learn: on MO-Hopper they are fit on 2,389 distinct policies with ~5,000 parameters that affect the return.

Truncated RFM geometry. Tripling the eigensolver width (q=100→300q=100\to300) or steering along 300 directions (vs 100) does not change HV (all paired 95% CIs include 0).

Proxy accuracy. Our guarantees hold on the learned proxies (Thm. 1), and the recovery bound (Thm. 3) shows that proxy error directly limits the final front. On MORL this error is large for 2 reasons.

  • Noisy labels. 84% of MO-Hopper training rows are repeated policies with different returns, consistent with stochastic rollouts, so a proxy can at best learn an average return.
  • Accuracy degrades away from the data. On held-out designs, MO-Hopper has the lowest proxy accuracy of our 5 ablation tasks (Spearman 0.38, vs 0.47–0.996), falling to −0.10 among the top 10% of designs. On our generated candidates it drops further, to 0.02–0.08 (App. A.9, B.2.8). Our subsampling test (v1bM W1) did not lower this top-10% accuracy further, which is why HV stayed stable there.

Consistent with this, no generative baseline exceeds D\mathcal D(best) on either task, and the proxy-regularized MM+ICT and MM+RoMA are the only surrogate methods above it on MO-Hopper. Such proxies can be used within ASNS.

RFM-Cone and D\mathcal D(best). RFM-Cone builds its cone from each objective's full proxy gradient and the proxy scores, so it is sensitive to proxy noise. RFM-APG instead projects onto an AGOP subspace averaged over many samples, which should reduce noise. It exceeds D\mathcal D(best) on MO-Hopper and outperforms PCD on both tasks (6.40 vs 6.27, 3.59 vs 3.52).

Thank you for the additional experiments and clarifications. The new baseline comparisons and the direct comparison with mean-gradient steering largely address my main concerns.

I consider this a useful contribution to offline multi-objective optimization and have raised my rating from 2 to 3.

Thank you for taking the time to read our responses and the revision, and for raising your score! We found your questions really helpful and interesting to work through. The mean-gradient baseline and the additional comparisons you suggested pushed us to pin down what the learned geometry contributes, and gave us a clearer picture of the method as well. We have incorporated your other suggestions in the revision and will add the remaining SPREAD tasks in the final version. Thank you again for your support of the paper!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We thank the reviewers for the questions and suggestions that helped us clarify the paper and strengthen its evidence. We have responded to each review individually and uploaded a revised PDF with changes highlighted (colored text). To make it easier to navigate alongside our responses, we summarize the main changes here:

  1. What the AGOP adds over the mean gradient (isuK and v1bM). The AGOP is the second moment of the gradient and identifies a subspace, of which the mean gradient is one direction. A new remark and proposition (App. C.3) and a dedicated study (App. B.3) make its role precise. One fixed displacement per weight is the simplest instance of ASNS, the form Thm. 1 analyzes, and there the suggested Stein baseline matches RFM-AN. More generally the step depends on the sample, as in our data-adaptive guidance or a per-sample noise step, both supported by the same RFM fit. For such steps the active subspace is provably optimal and a single mean-gradient vector cannot be used, which we confirm on benchmark tasks and on synthetic objectives with exact gradients. The study also measures how consistently the fixed step improves individual candidates when gradients change sign (App. B.3.4).
  2. New baselines and positioning (isuK). App. A.6 adds DOMOO and SPREAD under our protocol, with RFM-Cone outperforming both on every MO-NAS task, and explains why PGD-MOO could not be compared. SPREAD is slow to sample, so its remaining tasks could not be completed within the discussion period and will be added in full in the final version. Section 3.2 and App. A.8 explain what multi-objective steering adds over the single-objective case, and Section 1 discusses the five suggested references.
  3. Where ASNS is and is not suitable (isuK and v1bM). App. A.7 now characterizes this by whether the data are low-dimensional and how far the front lies from the data. App. A.9 traces the limited gains on MORL to proxy accuracy, supported by a proxy-data study (App. B.2.8).
  4. Evidence that the learned geometry is intrinsic and reliably estimated (v1bM and X4RF). App. B.2.7 shows that the exact AGOP is already low rank, so the spectral decay belongs to the objective and not to the RFM. A new corollary (App. C.4) and experiment (App. B.2.6) relate the in-sample estimator of Algorithm 1 to the one our consistency result covers.
  5. Significance tests and end-to-end runtime (X4RF and v1bM). App. A.2.1 adds per-task and across-task significance tests. Section 5.2 now scopes the efficiency claim to sampling time, and App. A.4 adds end-to-end totals with the precomputation included, a break-even analysis, and the cost of a single new preference. Following v1bM's suggestion, the final version will motivate this speedup in the main text with the settings where one trained model answers many preference queries, such as per-device search in hardware-aware NAS and test-time preferences in MORL, and with the query count at which sampling overtakes training in cost (App. A.4).

We hope these revisions make the paper's central contribution clearer: in offline multi-objective optimization with generative models, we observe that the objectives are sensitive to only a few directions of the initial noise, and this geometry can be learned once from function values and reused for every trade-off, so each candidate costs one noise displacement and one ODE solve. This moves the optimization signal from every sampling step, as in ParetoFlow, PGD-MOO and PCD, to the initial noise of an unchanged sampler, and extends RFM-based steering from a single target function to a whole family of scalarized objectives. Our data-adaptive and Pareto-aware guidance operators complement this from the other direction, reaching what a single noise displacement cannot, and the new mean-gradient analysis clarifies when each is needed.

We thank the reviewers for taking the time to provide their thoughtful feedback and helpful suggestions, and would be glad to answer any remaining questions during the discussion period!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.