- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies stochastic aligned multi-objective optimization, where several objectives share a common optimum. The authors propose DUNAMOO, which constrains objective weights to the probability simplex and applies the weights one iteration after selection to avoid same-batch gradient bias. They derive a two-phase stochastic convergence guarantee and a global convex maximum-gap bound. Experiments on loss-imbalanced Multi-MNIST and analytical examples illustrate the proposed approach. However, the assumptions of this paper are restrictive and the experiments are insufficient.
Critical strengths
- The paper addresses a meaningful theoretical problem: extending aligned multi-objective optimization to noisy minibatch feedback. The delayed-weight formulation is simple and clearly motivated.
- The convergence analysis is detailed, including a noisy weight-selection bound, a one-step delay argument, and explicit limits on the resulting guarantees. The analytical experiments offer some useful verification.
Critical weaknesses
1. Limited algorithmic novelty and restrictive theoretical guarantees. The proposed algorithm mainly combines simplex normalization and delayed updates with the existing PAMOO framework. Since these techniques are already well established, the algorithmic novelty should be better justified. Moreover, the convergence analysis relies on restrictive assumptions, including a common optimum, aligned, convex, smooth, and self-concordant objectives, as well as known optimal objective values. These conditions are difficult to satisfy in practical deep learning applications and are not verified in the Multi-MNIST experiments. Furthermore, Theorem 1 only establishes finite expected entry into a neighborhood of the optimum, while its subsequent geometric bound excludes trajectories that have already exited this region. Therefore, the theoretical results do not establish global mean-square convergence in the general nonlinear setting. The authors should further clarify the theoretical advances over existing methods, relax the assumptions where possible, and strengthen the connection between theory and practice.
2. Insufficient experimental validation and unconvincing performance advantages. The experimental evaluation is relatively limited and does not adequately demonstrate the effectiveness and generality of the proposed method. The experiments mainly focus on a constructed Multi-MNIST benchmark with a small number of objectives, and the reported results primarily concern training losses rather than test accuracy or generalization performance. More importantly, the proposed delayed-update method does not consistently outperform the same-batch method, particularly in the two- and four-objective settings. In addition, the PAMOO-SGD baseline differs from its original implementation, and comparisons with other competitive stochastic multi-objective optimization methods are insufficient. The algorithm also assumes known optimal objective values, which are simply set to zero in the experiments without verifying their accuracy. More comprehensive experiments, stronger baselines, and additional ablation studies are needed to substantiate the claimed advantages.
Questions for authors
-
How can the required optimal objective values be obtained or estimated in practical applications? How sensitive is the proposed algorithm to estimation errors in these values?
-
Can the authors relax the restrictive theoretical assumptions, such as a common optimum, self-concordance and positive mixture curvature, while retaining meaningful convergence guarantees? Alternatively, can the authors better justify the necessity of these assumptions and discuss the applicability of the theoretical results to more general optimization problems?
-
Can the authors provide a more challenging stochastic benchmark that satisfies the theoretical assumptions and compare the predicted convergence behavior, including the noise floor and expected entry time, with empirical results?
5 comments
Official Comment by Authors [2/3]
Benchmarks, delay, and baseline comparisons
W2. Limited experiments on Multi-MNIST benchmark.
We also evaluate DunAMOO on the SARCOS inverse-dynamics dataset, a seven-objective regression task using real robot data. The dataset has 21 inputs (joint positions, velocities, and accelerations) and seven outputs corresponding to joint torques. The data contains 35,551 training, 4,484 validation, and 4,449 test samples. Each joint is one objective, measured by NRMSE (normalized root MSE). All methods use the same 21–128–128–128–7 ReLU MLP and plain SGD with batch size 32 for 10,000 updates (approximately 9 epoch-equivalents); the learning rate is halved at updates 4,000 and 8,000. We select each method’s learning rate using 2 development seeds and worst-joint validation NRMSE, then evaluate over 30 paired confirmation seeds using the same initialization and minibatch sequence for each seed.
| Method | Worst-joint NRMSE |
|---|---|
| EW-SGD | 0.1601 |
| PAMOO-SGD | 0.1587 |
| Simplex (same batch) | 0.1610 |
| DunAMOO (ours) | 0.1581 |
DunAMOO achieves the lowest worst-joint error among the evaluated methods. In this experiment, it assigns larger weights to objectives with higher losses, while the losses of the other objectives also decrease. This balanced progress appears to help reduce the worst-joint error without sacrificing performance on the other joints. We add these results in the revised paper (Appendix D.4.1).
W2. Additional Measurement for Multi-MNIST experiment.
We agree that the current paper reports only cross-entropy loss on the Multi-MNIST dataset. Since this is a classification task, we will add accuracy results, including the worst-objective accuracy (the minimum accuracy across objectives). At the end of training, DunAMOO and EW reach at least 99.4% worst-objective accuracy, while PAMOO reaches only 40~91%. However, DunAMOO reaches 99% worst-objective accuracy faster than the EW. This faster convergence appears to be associated with DunAMOO assigning larger weights to objectives with lower accuracy, prioritizing the underperforming objectives. We include these results in the revised paper (Appendix D.3.3).
W2. Delay and same-batch performance In our experiments, we agree that the same-batch method achieves lower final losses in some Multi-MNIST settings, particularly with (m=2) and (m=4). However, these empirical results do not theoretically establish that same-batch updates are uniformly better than delayed updates. Below is the toy experiment that delayed-update achieves faster convergence than same-batch. We add this additional toy example in our paper (Appendix D.4.2).
| Method | Worst-objective MSE |
|---|---|
| Simplex (same batch) | 0.500 |
| DunAMOO (ours) | 0.014 |
Other than the experimental results, we use a one-step delay for a theoretical analysis: it makes the weight applied to the current gradient independent of the current minibatch, allowing us to analyze noisy weight selection and its interaction with the update. Thus, the delay is not intended to guarantee lower empirical loss in every setting. Rather, it enables the convergence analysis that is a central contribution of our work. The one-step delay itself is not a contribution of our work.
W2. PAMOO-SGD baseline differs from its original implementation. The previous experimental implementation of PAMOO differs from its theoretical formulation[1]. We revised the code to better align with the theoretical PAMOO update and ran the experiments under our Multi-MNIST setup.
W2. Comparisons with other competitive stochastic multi-objective optimization. We did not include methods developed for general stochastic multi-objective optimization because they typically target Pareto-stationary solutions for potentially conflicting objectives, whereas our setting assumes a shared minimizer and studies convergence toward that common optimum. Their objectives and theoretical guarantees therefore differ from those of AMOO, making a direct comparison under our convergence criterion potentially misleading. We will clarify this distinction and the resulting scope of our baseline selection.
References
[1] Efroni et al. Aligned Multi Objective Optimization. ICML 2025
Official Comment by Authors [3/3]
Reference sensitivity and stochastic validation
Q1. Experimental assumptions and practical relevance.
We use zero as an idealized lower-bound reference for Multi-MNIST cross-entropy: zero loss would imply 100% accuracy, but we have not established that the model can attain it. Prior work also uses zero or a known lower bound as a Polyak reference [1,2].
AMOO addresses objectives that share a minimizer, a specific regime rather than general MOO. Multi-reward applications in image generation, tool use, and reinforcement learning illustrate its practical relevance [3–6]; some explicitly study rewards sharing an optimal policy [6].
For our Multi-MNIST experiments, zero is a lower-bound reference. We have not verified zero population cross-entropy or a shared population minimizer. Perfect classification alone does not establish either claim. Prior stochastic Polyak results do not certify these task-level population references.
Q1. Effect of inaccurate relative references.
Appendix A.5.2 quantifies the effect directly. If bounds the range of differential reference errors, the certified residual becomes Increasing batch size does not remove this fixed differential error. To test reference sensitivity, we use the known-optimum nonlinear family specified under Q3 below and vary the supplied references with paired stochastic paths. Each condition uses 512 paths and 2,048 updates, batch size , , , solver residual budget , and initial maximum true gap 1:
| Reference | Final mean MSE | Monte Carlo SE |
|---|---|---|
| Exact | ||
| Error range | ||
| Error range | ||
| Error range | ||
| Error range | ||
| Zero, unequal true minima |
A common shift barely changed mean final MSE, but 1/512 paths exceeded the declared coordinate tolerance, with maximum difference . This numerical discrepancy remains unresolved. Exact common-offset cancellation is an algebraic property, and we do not claim that the pathwise numerical check passed.
Q3. An admissible stochastic benchmark: entry time and finite-run error
The alignment setting is difficult to verify in real-data problems. We use a synthetic benchmark with known optima to test the theory under its assumptions. The benchmark has 8 objectives in 32 rotated coordinates, , with
\phi(t)=t\arctan t-\tfrac12\log(1+t^2).$$ Here $c_i=i/8$, and $A_{ij}=1$ iff $((j\bmod 8)-i)\bmod 8<4$, using zero-based indices. Each objective activates half the coordinates; component Hessians have rank 16, while a normalized mixture has curvature $\mu_L=0.5$. The shared minimizer is $x^\star=0$, with known minima $f_i^\star=c_i$. Independent Rademacher perturbations $sZ_i\sin y_i+uV_i$ provide known noise constants. With $s=u=0.001$, $\eta=0.5$, batch size $N=16$, and 512 paths of 2,048 updates, every path entered the outer region without censoring: | Initial distance | Bound on $\mathbb E T$ | Observed mean $T$ | |---:|---:|---:| | 2 | 82.17 | **2** | | 8 | 345.58 | **8** | | 32 | 1488.36 | **22** | Here $T=\inf\\{k:L_k\le R^2\\}$. For initial distance 8, the observed second-half mean MSE decreased with batch size: | Batch $N$ | Residual $E_\infty$ | Mean MSE | Monte Carlo SE | |---:|---:|---:|---:| | 1 | $0.0799$ | **$2.33\times10^{-7}$** | $3.18\times10^{-10}$ | | 16 | $0.0200$ | **$3.64\times10^{-8}$** | $8.82\times10^{-11}$ | | 256 | $0.0050$ | **$2.54\times10^{-9}$** | $6.17\times10^{-12}$ | The observed entry times are below the theoretical bounds, and the finite-run MSE decreases as the batch grows. These results illustrate the theory in an admissible aligned setting. The conservative residual bounds and finite-run averages do not establish a stationary noise floor or global ordinary-MSE convergence. **References** [1] Loizou et al. Stochastic Polyak Step-size for SGD. AISTATS 2021. [2] Orvieto et al. Dynamics of SGD with Stochastic Polyak Stepsizes. NeurIPS 2022. [3] Lee et al. Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation. ECCV 2024. [4] Li et al. Towards Pareto-Optimal Tool-Integrated Agents with Pareto Ranking Policy Optimization. ICML 2026. [5] Liu et al. GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization. ICML 2026. [6] Dann et al. Reinforcement Learning Can Be More Efficient with Multiple Rewards. ICML 2023.
Official Comment by Authors [1/3]
Thank you for your careful review and thoughtful questions about our contribution, assumptions, and experiments. We address each of your points in turn below. We are grateful to have the opportunity to clarify the paper’s contributions and explain their significance in greater detail. We address our contributions (W1, Q2) in Comment 1, the empirical comparisons (W2) in Comment 2, and reference sensitivity (Q1) and stochastic validation (Q3) in Comment 3.
Technical contribution and weaker assumptions
W1. Contribution beyond simplex normalization and delay.
Our contribution is the stochastic analysis of Polyak weighting selected from jointly noisy losses and gradients. The QP optimality condition gives a population-gap retention bound despite this dependence. We then transfer the bound across the intervening update, which shares the selection batch, without assuming continuity of the QP optimizer. An adapted conditional energy gives finite expected entry and localized geometric second-moment control. These steps address the dependence introduced by noisy selection and delayed application. Delay fixes the applied weights before fresh gradients are drawn, while the analysis still accounts for loss-selection error.
The distance theorem allows singular individual Hessians. Its curvature condition concerns a normalized mixture at the shared minimizer: μL=maxw∈Δmλmin(∑iwi∇2fi(x⋆))>0. It does not require global strong convexity. This curvature condition originates in deterministic AMOO. Our advance is the stochastic analysis under the stated geometric and oracle assumptions.
W1. Why the nonlinear distance guarantee is localized?
Proposition A.3 shows that a global MSE guarantee for the default fixed-step, fixed-batch algorithm would be false over the stated class: an admissible nonlinear instance has divergent ordinary MSE even with arbitrarily small noise variance. Theorem 1 instead gives almost-sure entry with finite expected time when E∞<R2, followed by a localized geometric second-moment estimate. That estimate includes the first-exit iterate and excludes paths that exited earlier. Corollary A.3 provides finite-horizon exit control. These statements identify the attainable guarantee for this algorithm and class.
Q2. Which assumptions can be relaxed?
Corollary 1 removes self-concordance and positive mixture curvature. Under smooth convex alignment and its stated oracle, reference, and stepsize conditions, EM(xˉK)=O(KβV0+σgNKV0+NνΔ), where M(x)=maxi[fi(x)−fi⋆] and V0=∣x0−x⋆∣2. This is a global averaged-gap guarantee with an explicit finite-batch residual. The stronger geometric assumptions support the distance conclusion of Theorem 1.
The shared-minimizer assumption defines the AMOO target. Removing it would require a different optimization target. We have not verified exact population alignment or the geometric assumptions for the CNN. Its experiments evaluate optimization behavior beyond verified theoretical assumptions. Comment 3 provides an admissible nonlinear stochastic construction with known constants.