acceptodds
Under review as a conference paper at ICLR 2027

Closing the Pareto-Response Loop: Preference-Aware Optimistic Selection for Multi-Objective Bilevel Learning

Abstract

Multi-objective bilevel learning (MOBL) coordinates conflicting upper-level objectives through lower-level responses. Yet prevailing methods typically use a response-first organization: they form the lower-level response first and only afterward reconcile its induced objectives or hypergradients according to user preferences, without directly feeding the Pareto mixture into the lower objective. This response-formation channel remains open-loop, although under non-convex or finite-step optimization, the reached response is highly path-dependent and fundamentally shapes the ensuing Pareto geometry. We introduce Preference-Aware Optimistic Selection (PAOS), which uses a selector-controlled lower objective for optimistic branch selection among non-unique lower solutions and preference-guided response shaping under finite optimization budgets. Specifically, a stateful response-selection variable integrates the preference-weighted Pareto mixture produced by the current subproblem and guides the subsequent lower-level response, thereby closing the Pareto-response loop. On an analytically tractable double-well problem, PAOS systematically selects the preference-compatible branch from near-boundary starts. Across data hyper-cleaning, few-shot bilevel learning, physics-informed neural networks, and medical image segmentation, PAOS achieves the strongest overall performance among the evaluated MOBL methods. These results establish preference-aware optimistic response selection as an actionable degree of freedom in multi-objective bilevel decision-making. Our code is available at https://anonymous.4open.science/r/PAOS-forreview/.

Primary area:
optimization
Keywords:
Multi-Objective Bilevel Learning, Multi-Objective Optimization, Pareto Optimization, Preference-Aware Learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_C8J1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the multi-objective bilevel optimization (MOBL) problem, focusing on a slightly different setting, Preference-Aware Optimistic Selection (PAOS), in which the user preference is explicitly applied to guide solution selection at both levels.

It reformulates the lower-level problem by adding a term that combines the upper-level function with an adaptive preference coefficient. Then, to solve this new problem, the proposed method uses several techniques, such as the conjugate gradient method and weighted-Chebyshev scalarization.

The paper provides theoretical results and numerical experiments across various benchmarks.

Critical strengths

This paper is well-motivated, as preference control in MOBL is an interesting and open question. The experiments also seem good.

Critical weaknesses

My primary concerns lie in the theoretical analysis. (i) The paper does not establish a clear relationship between the reformulated problem and the original MOBL problem (also see Question 1). (ii) The assumptions are quite restrictive. The requirements of strong convexity and three-times continuous differentiability of the objective functions limit the scope of this paper.

Questions for authors

(1) The main theorem is not very clear to me. What is the formal definition of dˉt\bar{d}_t? If I understand correctly, dˉt\bar{d}_t does not represent a combination of the exact hypergradients of the original MOBL problem. Thus, I wonder what Theorem 1 implies for the original problem.

(2) Equation 2 may not fully capture the original Pareto frontier when the functions are nonconvex. Essentially, it uses a weighted sum, or linear scalarization, of the objectives, which is known to potentially miss Pareto-optimal solutions in nonconvex settings.

(3) It seems that Equation 3 may introduce a systematic bias with respect to the user preference, and the subsequent techniques, although they adapt αt\alpha_t, do not seem to address this issue. Could the authors provide more evidence or insight?

Minor mistake: Line 210: BtB_t is not Hessian.

Please address these questions as thoroughly as possible.

5 comments
AuthorsOpenReview ↗

Response to Reviewer C8J1

Thank you for recognizing preference control in MOBL as a well-motivated open problem and for the positive assessment of our experiments. We address the theoretical and methodological concerns below.

C-W1 — Connecting the selected problem to original MOBL

1. The original-solution-set connection. We now make explicit that the original lower objective gg defines the admissible response set Y⋆(x)\mathcal Y^\star(\mathbf x) in Eq. (1), while Eq. (2) selects a preferred response within that set. The selected objective in Eq. (3), g+ρψαg+\rho\psi_{\boldsymbol\alpha}, implements this selection during response formation; it reduces to gg at ρ=0\rho=0. More importantly, Proposition 1 proves that, under its existence and compactness conditions, every accumulation point of its exact global minimizers as ρ↓0\rho\downarrow0 belongs to the optimistic set in Eq. (2). This establishes the selection mechanism’s precise connection to the original lower-level solutions (Sec. 3.2–3.4; Appendix A.2).

2. The finite-ρ\rho connection. We also quantify what changes when selection strength is positive. Near a nondegenerate original lower minimum, let G=∇yy2g≻0G=\nabla_{\mathbf y\mathbf y}^{2}g\succ0 and aα=∇yψα\mathbf a_{\boldsymbol\alpha}=\nabla_{\mathbf y}\psi_{\boldsymbol\alpha}. Proposition 2 shows that, for sufficiently small ρ\rho,

=-\rho\,\mathbf a_{\boldsymbol\alpha}^{\top}G^{-1}\mathbf a_{\boldsymbol\alpha}+O(\rho^2), \qquad \Delta g =\frac{\rho^2}{2}\mathbf a_{\boldsymbol\alpha}^{\top}G^{-1}\mathbf a_{\boldsymbol\alpha}+O(\rho^3).$$ Thus, when $\mathbf a_{\boldsymbol\alpha}\ne0$, the selected upper criterion improves at first order, while the original lower objective changes at second order (Sec. 3.4, Eqs. (9)–(10)). For a regular branch that is valid for the original lower problem, we further relate the selected direction $\bar{\mathbf d}$ to the original branch’s Pareto-stationarity residual: $$s(H_0)\leq\frac{\|\bar{\mathbf d}\|}{r_{\min}}+C_H\rho, \qquad s(H)=\min_{\mathbf w\in\Delta_S}\|H\mathbf w\|.$$ This pointwise relation is now Eq. (32) in Appendix A.3. The revised Theorem 1 (Sec. 3.4, Eq. (11)) combines it with the selected-direction bound to directly bound the mean squared Pareto-stationarity residual on the original solution branch. **3. We check original lower-level validity in the analytical study.** The double-well experiment counts a preferred response as valid only if it has the preferred sign, satisfies $|z_T|\geq0.9$, **and** meets $g(\mathbf x_T,z_T,v_T)\leq10^{-3}$ under the *original* lower objective. The feedback ablation reports: | Feedback (Appendix B, Table 8) | Preferred sign ↑ | Valid preferred branch ↑ | Median projected regret ↓ | |---|---:|---:|---:| | Static, $\beta=0$ | 67/80 | 0/80 | 0.243 | | PAOS, $\beta=0.25$ | 76/80 | 76/80 | 0.148 | | **PAOS, $\beta=1$** | **80/80** | **80/80** | **0.134** | In the separate balanced branch study, PAOS also obtains 80/80 valid preferred responses with a median terminal gap of **$2.32\times10^{-6}$ under the original $g$** (Appendix B, Table 7 and the following paragraph). These original-objective checks complement the solution-set limit and the local finite-$\rho$ bounds: **the selected response is evaluated against the original MOBL lower problem, not merely against PAOS’s surrogate.** #### C-W2 — What the regularity assumptions cover **1. Our selection result covers non-convex lower problems.** Proposition 1 requires continuity, existence of minimizers, and compactness of the relevant lower-level sublevel set (Assumption 1); it requires **neither strong convexity nor three-times continuous differentiability**. Its optimistic-selection limit therefore permits non-convex lower objectives and multiple lower-level minimizers (Sec. 3.4; Appendix A.1–A.2). **2. The stronger conditions support local quantitative guarantees.** Theorem 1 additionally requires that the unperturbed branch consists of original lower-level minimizers, that the perturbation constants are uniform along the iterates, and that the coordinator satisfies the conditional descent assumption in Appendix A.1. Under these conditions, it bounds the mean squared Pareto-stationarity residual on that original branch. Proposition 1 uses only the continuity, existence, and compactness conditions in Assumption 1. **3. We evaluate the method beyond those local guarantees.** The analytical double-well has a non-convex lower objective with two original optimal branches; PAOS obtains a valid preferred response in **80/80** paired runs (Appendix B, Eqs. (45)–(47), Table 7). We also evaluate the algorithm on four learning tasks with neural models (Sec. 4, Table 1). We present these as empirical evidence of the method’s behavior in broader settings, while keeping the formal guarantees tied to their stated assumptions.
AuthorsOpenReview ↗

Response to Reviewer C8J1

C-Q1 — The exact direction and its original-problem meaning

(Please refer to Sec. 3.4 and Appendix A.3–A.4 of the revised manuscript for the fully typeset formulas. OpenReview cannot reliably render the required notation; we apologize for using simplified notation below.)

1. What the exact direction denotes. Your interpretation of the selected direction is correct: it combines exact hypergradients of the selected response branch. At iteration t, let H*_t denote the hypergradient matrix evaluated at an exact local response of the selected lower objective, with ρ and the selector α_t held fixed during differentiation. Let λ*_t solve the weighted-Chebyshev subproblem in Eq. (5) using that response’s exact objective values and hypergradients. The reference direction is

d̄_t = H*_t (r ⊙ λ*_t).

“Exact” distinguishes this reference direction from the implemented direction, which includes finite-response, linear-solver, damping, and Pareto-subproblem approximation errors. Appendix A.1 defines the reference direction, and Appendix A.4, Eqs. (37) and (41), bounds the hypergradient and direction errors.

2. Theorem 1 directly addresses original-branch stationarity. Thank you for asking what the theorem implies for the original problem. We have revised its statement so that Theorem 1 now directly bounds the mean squared Pareto-stationarity residual on a branch of original lower-level solutions (Sec. 3.4, Eq. (11)).

Let H₀(x) collect the hypergradients of the original upper objectives evaluated along this branch. A zero residual means Pareto stationarity of the upper objectives restricted to the branch. The proof combines two results. Appendix A.5, Lemma 2, bounds the average squared exact selected direction.

For comparison with related methods, our response to Reviewer GXA8, “Theoretical guarantees and comparison with related methods,” includes a table separating lower-level assumptions, upper-level conditions, and theoretical guarantees, with separate entries for Proposition 1 and Theorem 1.

Method / resultLower-level assumptionsUpper-level and other conditionsTheoretical guarantee
MOML, Thm. 2Unique minimizer; uniformly bounded and uniformly convergent lower iteratesCompact convex upper domain; uniform Lipschitz continuity; approximate and exact reduced objectives are P-convex and strictly P-convex, respectivelySet convergence of efficient solutions and minimal objective values as the lower approximation improves
FORUM, Thm. 1Global strong convexity; Lipschitz gradientBounded objectives and gradients; Lipschitz gradients; bounded value-function constraint violation along iteratesFinite-iteration KKT-stationarity and feasibility bounds for the value-function reformulation
gMOBA, Thm. 3.9Global strong convexity; Lipschitz first and second derivativesClosed convex upper domain; smooth parts bounded below with bounded Lipschitz gradients; convex continuous nonsmooth partsEvery accumulation point of the upper iterates is Pareto stationary for the reduced problem
WC-MHGD, Thm. 5.5Global strong convexity; Lipschitz gradient and relevant second derivativesLipschitz objectives and gradients; strictly positive preferences; prescribed solver accuracy and step sizesFinite-iteration bound on the exact preference-weighted hypergradient direction, yielding approximate Pareto stationarity
MOMEHA, Thm. 1Smooth nonconvex objective; nondegenerate envelope-constraint gradientLower-bounded objectives with Lipschitz gradients; positive preferences; additional penalty boundedness and schedule for vanishing feasibility errorFinite-iteration stationarity for the relaxed envelope-constrained problem; feasibility decay under the additional penalty conditions
PAOS: Proposition 1Nonconvexity and nonunique minimizers allowed; continuity, existence, and compactnessFixed upper variable and selector; continuous selected upper criterion; exact global surrogate minimizationEvery accumulation point as ρ → 0 is an original lower minimizer that minimizes the selected upper criterion over the original lower solution set
PAOS: Theorem 1Smooth local continuation of original lower minimizers; positive lower Hessian; uniform perturbation constantsFixed small ρ; unconstrained upper domain; positive preferences; local derivative regularity; lower-bounded merit with conditional descent and Lipschitz selector dependenceFinite-iteration mean squared original-branch Pareto-stationarity residual bound, accounting for selector drift, direction bias, noise, and an O(ρ²) perturbation term
AuthorsOpenReview ↗

Response to Reviewer C8J1

C-Q2 — Linear selection is not a full-frontier parameterization

1. We clarify the role and scope of Eq. (2). We agree that linear scalarization is not a complete parameterization of a non-convex Pareto frontier. Eq. (2) has a more specific role: for a fixed upper variable x\mathbf x, the original lower objective gg first defines Y⋆(x)\mathcal Y^\star(\mathbf x), and the mixture α\boldsymbol\alpha then selects a response within that original solution set. Strictly positive weights select a Pareto-efficient response within F(x,Y⋆(x))\mathbf F(\mathbf x,\mathcal Y^\star(\mathbf x)), but varying the weights can miss unsupported efficient responses when this attainable set is non-convex (Sec. 3.2, Eq. (2); Appendix A.2).

2. Our contribution does not rely on full-frontier coverage by Eq. (2). PAOS uses this criterion for response selection, then computes a mixture from the current selected response’s objective and hypergradient geometry (Eq. (5)). That mixture updates the selector (Eq. (7)), which shapes the next lower-level response through Eq. (3), closing the loop in Eq. (8) and Fig. 2. Our claim is therefore about feedback-guided formation of preference-conditioned responses, not recovery of every unsupported point on a non-convex frontier.

C-Q3 — How the selector remains preference-conditioned

1. We distinguish the user preference from the adaptive selector. The user preference r\mathbf r remains fixed throughout optimization; αt\boldsymbol\alpha_t is a response-selection state, not a replacement for r\mathbf r. We initialize α0=r\boldsymbol\alpha_0=\mathbf r. At each iteration, r\mathbf r enters both the upper-level Pareto subproblem and update (Eqs. (5)–(6)), and the next selector is formed from the preference-conditioned target

{\mathbf 1^\top(\mathbf r\odot\boldsymbol\lambda_t)}, \qquad \boldsymbol\alpha_{t+1} =(1-\beta)\boldsymbol\alpha_t+\beta\mathbf q_t.$$ Thus, $\boldsymbol\alpha_t$ may differ from $\mathbf r$ because it incorporates conflicts revealed by the *current response*, but the feedback entering Eq. (3) remains conditioned on the user’s preference (Sec. 3.3, Eqs. (5)–(8); Fig. 2). **2. We make the preference effect explicit.** The local analysis distinguishes improvement of the selected criterion from improvement of the *fixed user criterion*. Applying Proposition 2 to the latter gives ψ_r(x, y(ρ, α)) − ψ_r(x, y⁰) = −ρ a(r)ᵀ G⁻¹ a(α) + O(ρ²). *(Please see Proposition 2 and Appendix A.3 of the revised manuscript for the fully typeset formula. OpenReview does not reliably render the required notation; we apologize for the simplified display here.)* Here a(r) and a(α) denote the lower-response gradients of the upper-objective mixtures associated with the fixed user preference and the adaptive selector, respectively, and G is the positive-definite original lower Hessian. **Positive cross-alignment yields a first-order improvement in the user’s fixed criterion** (Appendix A.3). **3. We report preference-wise evidence.** In the non-convex double-well study, the preferred branch changes with the specified user preference. PAOS reaches a valid preferred response in all 20 paired runs at **each** tested preference, whereas each open-loop control reaches 10/20 (Appendix B, Table 7): | User preference $w$ | WS-HG | TCH-HG | WC-MHGD | PAOS | |---|---:|---:|---:|---:| | 0.1 | 10/20 | 10/20 | 10/20 | **20/20** | | 0.3 | 10/20 | 10/20 | 10/20 | **20/20** | | 0.7 | 10/20 | 10/20 | 10/20 | **20/20** | | 0.9 | 10/20 | 10/20 | 10/20 | **20/20** | The learning-task evidence is consistent with this preference-conditioned behavior: on MNIST, Full PAOS improves target-aligned average accuracy from **84.66% to 84.73%** and reduces target validation CE from **0.6565 to 0.6522** relative to Static preference (Sec. 4, Table 3). Together, the mechanism, local alignment condition, and preference-wise outcomes explain why the selector adapts without simply discarding the user’s target. #### C-M1 — Correcting the derivative terminology Thank you for catching this terminology error. We have corrected the description around Eq. (4): $A_t=\nabla^2_{\mathbf y\mathbf y}\widetilde g_t+\delta I$ is the damped lower-level Hessian, whereas $B_t=\nabla^2_{\mathbf x\mathbf y}\widetilde g_t\in\mathbb R^{p\times q}$ is the **mixed derivative matrix**. We use the same terminology in the theoretical appendix.
Reviewer_GXA8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PAOS augments the lower-level objective with a stateful mixture of upper-level losses. A preference-conditioned Pareto subproblem determines both the upper update and a smoothed selector for subsequent lower-level training. Implicit hypergradients are computed through the modified lower objective. The paper provides optimistic-selection consistency, local perturbation analysis, and a conditional feedback-variation bound. Validation includes an analytical double-well problem and experiments on data hyper-cleaning, fixed-task few-shot optimization, PINNs, and medical segmentation, with ablations of static selection and feedback strength.

Critical strengths
  • The modified lower objective and corresponding implicit hypergradients provide a coherent implementation for incorporating preference-conditioned Pareto decisions into subsequent lower-level training.

  • The analytical double-well study and PINN feedback ablation provide interpretable evidence that response shaping and dynamic feedback can improve optimization under controlled settings.

Critical weaknesses
  • The open-loop framing is too broad. BDA already injects upper-level gradients into lower updates, while FORUM uses smoothed multi-objective mixtures to update lower variables. The additional contribution of PAOS’s specific selector and implicit-gradient construction is insufficiently established.
  • Comparisons with the cited WC-Penalty and MOMEHA methods are missing without clear justification. Important task controls are also absent: direct training with identical supervision, adaptive PINN weighting, and fixed or tuned segmentation losses. These omissions limit evidence of practical advantages.
  • MNIST accuracy increases from 84.66% with static selection to 84.73% with PAOS (Table 2). Independently tuned static controls, matched computational budgets, and original lower-objective residuals are needed to isolate feedback benefits, particularly because positive selection strength changes the lower problem.
  • FC-100 uses the same labeled query set for optimization and reported accuracy. This disclosed protocol measures within-task fitting, so it does not establish few-shot generalization and weakens the evidence for practical applicability.
Questions for authors

See Weaknesses.

7 comments
AuthorsOpenReview ↗

Response to Reviewer GXA8

Thank you for recognizing PAOS’s coherent implementation, the interpretable double-well and PINN evidence, and the paper’s clear presentation. We address the methodological and experimental concerns below.

G-W1 — Where PAOS differs from BDA and FORUM

1. The closed-loop distinction. Our central contribution is a feedback loop between lower-level response formation and upper-level Pareto geometry: the current response determines the objective and hypergradient geometry; its preference-conditioned Pareto decision then changes the criterion used to form the next response, which in turn produces the next geometry (Sec. 3.2–3.3, Eqs. (3)–(8); Fig. 2). This is more specific than allowing upper-level information to influence a lower variable.

BDA addresses a scalar upper objective and aggregates its gradient with that of the scalar lower objective; a prescribed scalarization does not make its selection criterion depend on the evolving multi-objective Pareto geometry. FORUM does address multiple upper objectives and uses a smoothed mixture in a joint upper–lower variable update. Its separate lower-response solve, however, still optimizes the original lower objective; the mixture is not carried into that solve as a stateful selection criterion.

We use “open-loop response formation” only for the absence of this specific Pareto-decision → next lower objective → next Pareto geometry channel, not to suggest that these methods have no upper-to-lower interaction.

2. How we close that loop. We compute the current objectives and hypergradients through the selected response, with the selector held fixed during implicit differentiation (Sec. 3.3, Eq. (4)). Their preference-conditioned Pareto mixture then updates the selector and the next lower objective:

Selector update: αt+1=(1−β)αt+βN(r⊙λt)\alpha_{t+1}=(1-\beta)\alpha_t+\beta\mathcal{N}(r\odot\lambda_t).

Next lower objective: g̃ₜ₊₁ = g + ρₜ₊₁ αₜ₊₁ᵀF.

(Please see Eq. (3) in the revised manuscript for the fully typeset lower objective and Eqs. (7)–(8) for the selector update. OpenReview does not reliably render the required notation; we apologize for the simplified display here.)

Here λt\boldsymbol{\lambda}_t is computed from the current response-conditioned objectives and hypergradients (Eqs. (5)–(7)). The implicit-gradient construction matters because it makes that Pareto signal reflect the selected response. Together, Eqs. (3)–(8) and Fig. 2 specify the dependency response → Pareto decision → next response, which is our contribution rather than upper-gradient injection or weight smoothing in isolation.

3. Evidence for the feedback channel. With the other settings matched, changing only the feedback rate improves valid preferred-branch selection from 0/80 to 80/80 in the double-well study (Appendix B, Table 7, β=0\beta=0 / 11 rows) and lowers PINN test relative L2L_2 from 0.03674 to 0.02805 (Appendix D.4, Table 9, β=0\beta=0 / 0.250.25 rows). We have narrowed the manuscript's “open-loop” wording to the reference response-formation pathway and made this specific closed-loop distinction explicit.

AuthorsOpenReview ↗

Response to Reviewer GXA8

G-W2 — Comparisons with Related Methods and Task-Specific Baselines

1. We separate solver performance from the contribution of feedback. Our central contribution is the Pareto-to-response feedback channel: the geometry of the current selected response informs the Pareto mixture, which updates the selector used in the next lower-level objective (Sec. 3.3, Eqs. (3)–(8); Fig. 2). Comparisons with WC-Penalty and MOMEHA assess complete methods. The Static–PAOS ablation instead holds the learning pipeline and upper-level supervision fixed while testing what this feedback channel adds. We report these two forms of evidence separately.

2. We have expanded the comparison with related methods. Appendix E.3 extends the heat-equation PINN comparison to both WC-Penalty and MOMEHA. WC-Penalty’s convex-lower-level penalty reformulation is not designed to correct the path-dependent responses formed by nonconvex PINN training. MOMEHA accommodates nonconvexity, but its Moreau-envelope auxiliary response targets lower-level stationarity rather than carrying the current response’s Pareto decision into subsequent training. PAOS carries that decision forward through a persistent selector, repeatedly adjusting response formation as the Pareto geometry changes. The results are reported in Table 12:

Heat-equation PINN metric ↓MOMEHAWC-PenaltyPAOS
Test relative L2L_20.045 ± 0.0080.247 ± 0.0390.028 ± 0.000
Relative-L2L_2 AUC0.119 ± 0.0020.369 ± 0.0540.082 ± 0.005
Final-20 relative L2L_20.056 ± 0.0080.265 ± 0.0450.040 ± 0.003

PINN comparison in revised Appendix Table 12; lower is better throughout.

3. We use matched ablations to identify what the closed loop adds. In the comparisons below, Static keeps preference-conditioned response selection but does not feed the newly computed Pareto mixture into the next selector; Full PAOS does (Sec. 3.3, Eqs. (3), (5)–(8); Fig. 2).

Existing manuscript ablations, with the later five-seed PINN check shown separately. Percentage changes are relative to Static unless marked “pp.”

Setting and metricStaticFull PAOSObserved changeSource
Double-well preferred sign ↑67/8080/80+13/80 runsAppendix B, Table 7
Double-well valid branch ↑0/8080/80+80/80 runsAppendix B, Table 7
Double-well median regret ↓0.2430.134−44.9%Appendix B, Table 7
PINN test relative L2 ↓, original sweep0.03674 ± 0.001420.02805 ± 0.00012−23.7%Appendix D.4, Table 9
PINN test MSE (×10⁻⁴) ↓, original sweep3.52 ± 0.911.78 ± 0.08−49.4%Appendix D.4, Table 9
PINN selected validation loss (×10⁻⁴) ↓, original sweep8.46 ± 2.746.52 ± 0.98−22.9%Appendix D.4, Table 9

As shown in the table above, double-well experiments directly confirm that the feedback mechanism changes the resulting valid response, and PINN results further support this feedback mechanism. These ablations validate the mechanism and its observable effects.

4. We add a direct, problem-matched PINN control. We evaluated Direct (alternating) while retaining the original lower-level training points, upper-level validation objectives, four non-uniform preferences, and independent test grid. At each iteration, it takes one step on the original lower objective to update the network parameters, then updates the training-point weights by differentiating the preference-weighted upper objective through that lower step. It uses neither PAOS’s Pareto subproblem nor its feedback selector. Across four preferences and five paired seeds, PAOS has lower mean error on all three reported test metrics:

Heat-equation PINN metric ↓Direct (alternating)PAOS
Test relative L2L_20.047339 ± 0.0057890.038418 ± 0.019084
Relative-L2L_2 AUC0.127132 ± 0.0084980.081691 ± 0.007004
Final-20 relative L2L_20.054805 ± 0.0082090.043168 ± 0.015587

The trajectory-AUC difference favors PAOS in all five paired seeds (PAOS minus Direct: −0.045441-0.045441; paired 95% CI [−0.057747,−0.033135][-0.057747,-0.033135]). This provides a concrete test of response quality throughout optimization against a direct alternating bilevel update.

AuthorsOpenReview ↗

Response to Reviewer GXA8

G-W3 — Feedback benefit, tuning, and computational cost

1. We ablate the stages of response formation, rather than relying on the MNIST increment alone. Our revised Table 3 distinguishes response selection, static preference conditioning, and dynamic Pareto feedback. Average accuracy rises and target validation cross-entropy falls at each stage. In particular, Static and Full PAOS both perform preference-conditioned selection; Full PAOS additionally feeds the Pareto mixture induced by the current response into the selector for the next lower-level solve (Sec. 3.2, Eq. (3); Sec. 3.3, Eqs. (4)–(8); Fig. 2).

MNIST componentAverage accuracy (%) ↑Target validation CE ↓
Open loop84.39 ± 0.020.6746 ± 0.0042
Uniform selection84.47 ± 0.050.6699 ± 0.0042
Static preference84.66 ± 0.040.6565 ± 0.0041
Full PAOS84.73 ± 0.070.6522 ± 0.0041

(Sec. 4, Table 3, Open loop / Uniform selection / Static preference / Full PAOS rows.)

2. We directly test whether feedback changes the response that is formed. In the double-well experiment, we hold the positive selection strength fixed at ρ=0.15\rho=0.15, use paired initial states, and vary only the feedback rate β\beta. As the table shows, feedback increases the number of valid, preference-consistent lower-level responses from 0/80 under Static to 80/80 under Full PAOS, while reducing median regret from 0.243 to 0.134. This is evidence about response formation itself, not merely a different score assigned after a response has been formed (Appendix B, Table 8).

Feedback ratePreferred sign ↑Valid branch ↑Median regret ↓
β=0\beta=0 (Static)67/800/800.243
β=0.25\beta=0.2576/8076/800.148
β=0.50\beta=0.5078/8078/800.147
β=0.70\beta=0.7079/8079/800.147
β=1\beta=1 (Full PAOS)80/8080/800.134

G-W4 — Performance on unused FC-100 images

1. What the original FC-100 experiment was designed to test. We use FC-100 as a multi-objective bilevel learning task: the lower level trains a task-specific classifier on support images, while the upper level coordinates five class-wise query objectives through a shared encoder (Appendix D.3, Eqs. (66)–(67); Sec. 4, Table 1). We originally reported accuracy on the optimization query set because that experiment measures how response formation affects the upper-level objectives within a fixed task (Sec. 4, Table 4).

2. Accuracy on images not used for upper-level optimization. We have now added a separate evaluation set of 100 disjoint images per sampled class, in addition to the five support and 15 optimization-query images per class. We evaluate the resulting models without using the held-out images for optimization, adaptation, or checkpoint selection. The table reports five-seed mean ± standard deviation, after averaging the five preferences within each seed. PAOS has the highest mean on all three metrics among the four methods shown.

Held-out accuracy (%)WS-HGTCH-HGWC-MHGDPAOS
Final Acc. ↑52.792 ± 5.20049.912 ± 3.55953.184 ± 4.71753.448 ± 4.494
Last-100 Acc. ↑51.463 ± 4.68549.246 ± 3.98752.359 ± 4.63753.225 ± 4.536
Acc. AUC ↑48.158 ± 4.57645.843 ± 4.05248.071 ± 4.44950.728 ± 4.769

3. How we interpret this evidence. This additional experiment shows how the learned response performs on unused images from the sampled classes. More broadly, the paper also evaluates on separate MNIST test images, a dense PINN test grid, and held-out Kvasir-SEG test images (Sec. 4, Tables 2, 5, and 6; Appendices D.2, D.4, and D.5).

Reviewer_1jnNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies multi-objective bilevel learning (MOBL) and argues that existing methods are "open-loop": the lower-level response (or optimiser) is obtained first and the Pareto mixture of upper-level objectives is applied afterwards. PAOS modifies the lower-level objective by adding a linear scalarization of the multi-objectives. The scalarizing parameters are stateful and updated by the local Pareto mixture obtained by upper-level objectives. The authors prove that solution to the modified lower-level objective is not far off from the optimizers of the original lower-level objective (remains with an error of second order scale), while reducing the linear scalarization objective in a first-order scale. The scale is charcterized by the multiplication factor of the linear scalarization term. The PAOS method is evaluated on 1 analytical mulit-objective bi-level problem and 4 learning problems.

Critical strengths

Feeding the current Pareto decision back into lower-level response formation is a neat idea. Figure 3 shows PAOS selecting the preference-compatible solution branch in 12/12 near-boundary starts versus 0/12 for open loop, a clean illustration of the effect. Four application domains, and reported memory use >10× lower than unrolled MOML.

Critical weaknesses

My critiques are mostly about writing. The paper is very poorly written and requires significant rework from the authors side, therefore I don't think this work is ready for publication yet.

The introduction does not clarify the importance of the MOBL problem. What makes it a multi-objective bi-level "learning" problem and not simple multi-objective bi-level "optimzation" problem does not become clear until one goes to the experiment section. In fact, the experiment section also doesn't cast the ML problems in the form of MOBL problems in the main draft, it remains buried in the appendix.

The motivations behind the introduced propositions are not provided. After a lot of careful reading, I figured that proposition 2 is important, cause it says the modified lower level objective "improves the multi-objective" at faster rate than it deteriorates the original lower-level optimizer.

I do not understand the importance of Theorem 1. What's the point of giving an upper bound which has an uncontrolled error term (the third term). I didn't find any result that actually attempts to control the direction error.

The contributions of this paper are not clearly distinguished from the prior art. I believe the beginning of section 3.3, response conditioned Pareto geometry, is largely the prior art with a the lower-level objective replaced with authors' modification. Therefore, it better fits in the preliminary section.

Questions for authors
  1. In the ML experiments where the higher level objectives are defined on validation data, wouldn't feeding upper-level objectives into the lower level objective cause data leakage?

  2. Can you prove or empirically bound the distance between the PAOS fixed points and stationary points of the original problem (Eq. 1) as a function of rho?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 1jnN

Thank you for your positive assessment of the paper’s soundness and for recognizing the Pareto-to-response feedback idea, and the breadth of our experiments. We have substantially improved the presentation and address each point below.

J-W0 — Concern about readability

We take your concern about readability seriously. In the submitted version, we moved substantial context and technical detail to the appendix to accommodate the experiments and visualizations within the page limit. We appreciate your pointing this out and have made the following revisions.

1. We introduce the MOBL setting earlier. We have partially reorganized the abstract and Introduction. The revised abstract directly raises the question of how lower-level responses are formed. The Introduction adds broader context and references for multi-objective bilevel optimization, then uses a concrete learning task to explain why the lower response shapes the upper-level trade-off (Abstract; Sec. 1). We have also sharpened the comparison with related methods (Sec. 2).

2. We make the closed loop traceable in the method. The revised overview and formulation start from the general MOBL problem and follow one sequence: the selector defines a lower objective, the resulting response determines the Pareto geometry, and the current Pareto mixture updates the selector for the next lower response (Sec. 3.2–3.3, Eqs. (3), (5), and (7)–(8); Fig. 2). We distinguish optimistic branch selection when lower solutions are non-unique from finite-budget response shaping when the lower response is unique.

The propositions explain which response is selected and how a small perturbation changes its objectives. Theorem 1 directly bounds the mean squared Pareto-stationarity residual on a regular branch of original lower-level solutions, accounting for selector drift and numerical errors (Sec. 3.4, Eq. (11)).

3. We connect each experiment to that mechanism. The following comparisons put the relevant evidence in one place:

What the comparison testsStatic selectionPAOSLocation
Additional MNIST accuracy from feedback84.66 ± 0.04%84.73 ± 0.07%Sec. 4, Table 3, Static preference / Full PAOS
Valid preferred response in the double-well study0/8080/80Appendix B, Table 8, β=0/1\beta=0/1
PINN test relative L2L_2 under matched settings0.03674 ± 0.001420.02805 ± 0.00012Appendix D.4, Table 10, β=0/0.25\beta=0/0.25

Together, these revisions are intended to let readers assess one claim without reconstructing it across sections: the Pareto decision can guide subsequent lower-level response formation. We address your specific questions below. We hope the revised presentation resolves your readability concern, and we would value your assessment of it.

AuthorsOpenReview ↗

Response to Reviewer 1jnN

J-W1 — Why this is a bilevel learning problem

1. Why the MOBL setting matters in learning. We have revised parts of the Abstract and Introduction to bring the practical learning settings forward. The Introduction now includes a concrete data-hyper-cleaning example: a shared weighting policy trains classifiers under different corruption conditions, and their validation losses define competing upper-level objectives (Sec. 1). This example illustrates the coupling that makes MOBL a learning problem: the upper level optimizes a shared sample-weighting policy across corruption-specific validation objectives, while the lower level trains classifiers under that policy. Changing the weights changes the trained responses; those responses, in turn, determine the upper objectives, hypergradients, and Pareto trade-off for the next update.

2. What “learning” means in our formulation. We have made the learning interpretation explicit immediately after Eq. (1) in Sec. 3.1. The upper (hyper-training) level uses x\mathbf{x} to choose data weights, an encoder, or loss weights while balancing multiple objectives. Under that choice, lower-level training produces model parameters y\mathbf{y}, idealized by y⋆(x)\mathbf{y}^{\star}(\mathbf{x}). The resulting response, together with x\mathbf{x}, determines F(x,y⋆(x))\mathbf{F}(\mathbf{x},\mathbf{y}^{\star}(\mathbf{x})) and the Pareto trade-off guiding the next upper-level update. “Learning” thus describes a coupled process of multi-objective hyper-training and model training, rather than a mathematical category separate from bilevel optimization.

3. Making every benchmark identifiable in the main paper. We have added Table 1 to Sec. 4, so readers can see each task’s variables and objectives before reaching the results. Its rows also point directly to the corresponding formulas in Appendix D.2–D.5:

TaskUpper-level variable (x\mathbf{x})Lower-level variable (y\mathbf{y})Lower training objective ggUpper objectives F\mathbf{F}
Data hyper-cleaning (MNIST)Shared sample-weighting policyFive corruption-specific classifiersWeighted training cross-entropyFive clean-validation losses
Few-shot bilevel learning (FC-100)Shared feature encoderTask-specific classifierSupport-set classification lossFive class-wise optimization-query losses
Physics-informed learning (heat equation)Training-point weightsNeural solution parametersWeighted physical training lossesResidual, initial-, boundary-, and observation-validation losses
Medical segmentation (Kvasir-SEG)Training-loss weightsSegmentation network parametersWeighted segmentation training lossesRegional, boundary, and small-lesion validation losses

The common learning structure is therefore concrete: Table 1 describes each task’s original lower training objective gg; PAOS’s selected objective g~t\widetilde{g}_t is defined separately in Sec. 3.2, Eq. (3). PAOS adds a direct feedback channel to this coupled process: the Pareto mixture induced by the current response enters the next lower-level selection objective, so multi-objective coordination also helps shape the response formed next.

AuthorsOpenReview ↗

Response to Reviewer 1jnN

J-W2 — What the two propositions explain

You have identified the motivation for Proposition 2: a small selection perturbation improves the selected upper-level criterion at first order, while changing the original lower-level objective only at second order. We now explain this benefit–cost relation in plain language before presenting the formulas in Sec. 3.4.

1. Proposition 1 explains which response is selected. For fixed x\mathbf{x} and selector α\boldsymbol{\alpha}, consider the modified lower objective g+ρ⟨α,F⟩g+\rho\langle\boldsymbol{\alpha},\mathbf{F}\rangle. As ρ\rho approaches zero, accumulation points of its global minimizers belong to the original lower-level solution set and minimize the selected upper criterion within that set. This explains the connection to optimistic selection when the original lower problem has multiple solutions (Proposition 1; Appendix A.2).

2. Proposition 2 explains why a small positive ρ\rho can help. Let ψα=⟨α,F(x,y)⟩\psi_{\boldsymbol{\alpha}}=\langle\boldsymbol{\alpha},\mathbf{F}(\mathbf{x},\mathbf{y})\rangle, a=∇yψαa=\nabla_{\mathbf{y}}\psi_{\boldsymbol{\alpha}}, and G=∇yy2g≻0G=\nabla_{\mathbf{y}\mathbf{y}}^2g\succ0 at a strict local lower-level minimizer. Proposition 2 gives

=-\rho\,a^\top G^{-1}a+O(\rho^2), \qquad \Delta g =\frac{\rho^2}{2}\,a^\top G^{-1}a+O(\rho^3).$$ When $a\neq0$, the selected criterion improves at first order, whereas the original lower-level objective changes at second order (Proposition 2; Appendix A.3). These are **orders of change in objective values**, not claims about optimization speed or simultaneous improvement of every upper-level objective. Together, the propositions explain how the selected lower objective shapes a response. The feedback update determines the selector $\boldsymbol{\alpha}$ used in that objective (Sec. 3.3, Eq. (7)). #### J-W3 — The purpose of Theorem 1 and control of direction error **1. What Theorem 1 establishes.** We have revised Theorem 1 to make its implication for the original problem explicit. It now bounds the mean squared Pareto-stationarity residual along a regular branch of original lower-level solutions (Sec. 3.4, Eq. (11)). The proof combines the selected-direction bound in Appendix A.5, Lemma 2, with the original-branch residual relation in Appendix A.3, Eq. (32). **2. How the direction error is controlled.** The direction-bias term in the earlier theorem now appears explicitly in Lemma 2. Appendix A.4, Eq. (37), bounds hypergradient error using the lower-solver residual, linear-solver residual, and damping. Equation (41) then bounds the resulting direction error using the hypergradient and objective-value input errors together with the Pareto-subproblem optimality gap. Jensen’s inequality transfers this control to the conditional direction bias. Thus, the bias term is linked to identifiable approximation and solver errors. **3. What the guarantee requires.** Appendix A.1, Eq. (20), proves fixed-selector descent for the exact weighted-Chebyshev coordinator. The analysis with biased or stochastic directions uses the conditional descent assumption stated in Assumption 4. Theorem 1 additionally requires a regular continuation of an original lower-solution branch and uniform perturbation constants. Its bound shows how initialization, selector drift, direction bias, noise, and selection strength affect original-branch stationarity. At fixed positive ρ, an O(ρ²) term remains in the mean squared residual bound even when the other contributions vanish. #### J-W4 — Separating the backbone from the contribution Eqs. (4)–(7) trace how the current selected response produces a preference-aware Pareto decision and how that decision updates the selector. Eq. (4) describes how we obtain the geometry of the current selected response. **Our contribution lies in what happens next:** the Pareto mixture derived from that geometry updates the selector $\boldsymbol{\alpha}_{t+1}$ (Eqs. (5)-(7)), which enters the next lower-level objective (Eq. (3)) and shapes the response that generates the next geometry (Eq. (8); Fig. 2). We streamline the discussion around Eq. (4) and bring this closed-loop distinction to the start of Sec. 3.3. **This response–Pareto–response loop, made explicit in Eq. (8) and Fig. 2, is our central contribution.** We have revised the opening of Sec. 3.3 to state this coupling before presenting its computational details. *For a comparison of theoretical assumptions and guarantees, please also see the table in our response to* [Reviewer GXA8](https://openreview.net/forum?id=AjJBqSxb1z&noteId=fe3eO1dYIy), “Theoretical guarantees and comparison with related MOBL methods.”
Reviewer_o7oBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that multi-objective bilevel learning (MOBL) predominantly follows an open-loop organization: a lower-level response is computed first, and user preferences are only applied post hoc to coordinate the induced upper-level objectives or hypergradients. To address this path dependency in non-convex or finite-step settings, the authors propose PAOS (Preference-Aware Optimistic Selection).

Specifically, PAOS injects the evolving preference-weighted Pareto mixture into the lower-level objective via a selector-controlled surrogate: g_tilde_t(x, y) = g(x, y) + rho_t * <alpha_t, F(x, y)>. The stateful selector alpha_{t+1} is updated via momentum using the instantaneous Pareto mixture q_t = N(r * lambda_t) obtained from a weighted-Chebyshev subproblem, while setting the feedback rate beta = 0 freezes the selector at the static user preference r.

On an analytical double-well problem, PAOS reaches the preferred branch on all 12 reported near-boundary starts, whereas the open-loop baseline reaches none. Across MNIST hyper-cleaning, one 5-way 5-shot FC-100 task, a heat-equation PINN, and Kvasir-SEG medical segmentation, the reported averages favor PAOS among the compared methods. However, the MNIST ablation shows a larger gain from static preference than from dynamic feedback (+0.27% then +0.07% average accuracy). Proposition 1-2 are small-rho local expansions, and Theorem 1 explicitly does not prove sufficient descent for the weighted-Chebyshev subproblem in Eq. (5).

Critical strengths
  1. Mechanistic Isolation and Transparent Decomposition: The paper successfully isolates the proposed feedback channel rather than presenting an entangled heuristic. Eq. (7) clearly disentangles the fixed user preference rr, the instantaneous Pareto decision λt\lambda_t, the normalized target qtq_t, and the stateful selector αt\alpha_t. Crucially, setting β=0\beta = 0 provides a genuine, well-calibrated static-preference control, enabling a rigorous ablation between static preference shaping and dynamic closed-loop feedback.

  2. Direct Alignment Between Toy Experiment and Stated Mechanism: The analytical double-well benchmark directly validates the core conceptual claim. In a setting with two equally valid lower-level minima, PAOS systematically shifts the attraction boundary zsep(α)z_{\text{sep}}(\alpha) to select the preference-compatible branch across all 12/12 near-boundary starts in Figure 1, compared to 0/12 for the open-loop control. Furthermore, the ablation in Table 2 transparently reports that on MNIST, static preference integration accounts for the primary performance gain, which gives reviewers an honest, assessable view of the method's behavior.

Critical weaknesses
  1. Disconnect Between the Decision-Relevant Claim and Actual Experimental Evidence: The defining thesis of the paper is that dynamic closed-loop Pareto feedback (β>0\beta > 0)—and not merely static preference scalarization (β=0\beta = 0)—is an actionable, high-impact degree of freedom in learning systems. However, the empirical results do not substantiate this strong claim:
  • In MNIST hyper-cleaning (Table 2), average test accuracy moves from 84.39% (open loop) to 84.66% (static preference), and reaches 84.73% for full PAOS. The incremental gain attributable to dynamic feedback is a negligible +0.07%, while the per-target standard deviation at 40% noise is 0.17%.
  • On the heat-equation PINN, while the feedback ablation shows a relative L2L_2 drop of 23.6% over static selection, Table 4 shows that PAOS is only 0.004 ahead of the standard weighted-sum and Chebyshev hypergradient baselines (0.028 vs. 0.032). The abstract's claim that these experiments "establish preference-aware optimistic response selection across domains" overstates the actual empirical delta provided by closing the loop.
  1. Protocol Discrepancies, Baseline Redundancy, and Theoretical Limitations:
  • Misleading "Few-Shot" Evaluation: Appendix D.3 explicitly states that for the FC-100 task, the query set is used both for the upper-level optimization loss and for reporting test metrics in Table 3. Evaluating on the exact 75 query images being optimized measures within-task fitting rather than out-of-sample few-shot generalization; here, the AUC margin over WC-MHGD is only 80.52 vs. 80.30.
  • Data Duplication with Baseline: In Table 1, every entry for WC-MHGD across all corruption levels (86.98±0.03, 86.05±0.13, 84.98±0.06, 83.07±0.13, 80.89±0.18, 84.39±0.02) is identical down to the last printed decimal and standard deviation to the "Open loop (ρt=0\rho_t=0)" row in Table 2. Unless clarified, WC-MHGD does not constitute an independent baseline.
  • Theoretical Gap: Theorem 1 relies on Assumption 4 (sufficient descent of the base coordinator), but lines 756–758 explicitly concede that this is an unverified assumption and not proven for the weighted-Chebyshev coordinator in Eq. (5).
Questions for authors
  1. Baseline Clarification: Is WC-MHGD in Table 1 computationally identical to the "Open loop (\rho_t = 0)" ablation row in Table 2? If not, please explain why all mean values and standard deviations match to every printed decimal.
  2. Statistical Significance of Dynamic Feedback: For the ten preference–seed pairs in the MNIST benchmark, what are the paired differences (with confidence intervals) between full PAOS (\beta > 0) and static preference (\beta = 0)? Does dynamic feedback remain statistically distinguishable from static preference at 0% and 40% noise?
  3. Generalization in Few-Shot Learning: What is the accuracy of PAOS on held-out query images that were not directly included in the upper-level optimization loss? Can you report standard out-of-sample episode generalization on FC-100?
4 comments
AuthorsOpenReview ↗

Response to Reviewer o7oB

Thank you for recognizing PAOS’s mechanistic isolation, transparent selector decomposition, and our core claim. We address the empirical, protocol, and theoretical concerns below.

O-W1 — Claim and Actual Experimental Evidence

1. What closes the loop. We clarify that PAOS does more than add a fixed preference-weighted loss to lower-level training. The current lower response determines the multi-objective geometry from which we obtain the Pareto mixture λt\lambda_t. That mixture guides the upper update and feeds back into the criterion used to form the next lower response:

=(1-\beta)\alpha_t+\beta\,\mathcal N(r\odot\lambda_t), \qquad y_{t+1} =\mathcal A_T\!\left( y_t;\,x_{t+1}, g+\rho_{t+1}\langle\alpha_{t+1},F\rangle \right).$$ With $\beta=0$ and $\alpha_0=r$, the selector remains fixed at the stated preference. With $\beta>0$, response-conditioned Pareto information changes subsequent response formation. This is the closed-loop contribution we intend to distinguish from static preference selection (Sec. 3.2, Eq. (3); Sec. 3.3, Eqs. (5)–(8); Fig. 2). **2. What the evidence shows.** The following comparisons isolate the additional effect of dynamic feedback over static preference selection: | Task | Metric | Static selection | Dynamic feedback (PAOS) | Manuscript location | |---|---|---|---|---| | Double-well | Preferred sign ↑ | 67/80 | **80/80** | Appendix B, Table 8; β = 0 vs. 1 | | Double-well | Valid preferred branch ↑ | 0/80 | **80/80** | Appendix B, Table 8 | | Double-well | Median projected regret ↓ | 0.243 | **0.134** | Appendix B, Table 8 | | PINN | Science score ↑ | 0.96480 ± 0.00105 | 0.97274 ± 0.00009 | Appendix D.4, Table 10; β = 0 vs. 0.25 | | PINN | Test relative L₂ ↓ | 0.03674 ± 0.00142 | 0.02805 ± 0.00012 | Appendix D.4, Table 10 | | PINN | Test MSE (×10⁻⁴) ↓ | 3.52 ± 0.91 | 1.78 ± 0.08 | Appendix D.4, Table 10 | | PINN | Selected validation objective (×10⁻⁴) ↓ | 8.46 ± 2.74 | 6.52 ± 0.98 | Appendix D.4, Table 10 | | MNIST | Average accuracy (%) ↑ | 84.66 ± 0.04 | 84.73 ± 0.07 | Sec. 4, Table 3 | | MNIST | Target validation CE ↓ | 0.6565 ± 0.0041 | 0.6522 ± 0.0041 | Sec. 4, Table 3 | We emphasize the different roles of these results. The double-well study shows that feedback changes which *lower-level-valid* response is formed, beyond influencing the preferred sign alone. The matched PINN comparison shows that this distinction matters in a learning task when only $\beta$ varies. The MNIST component ablation is consistent with an additional gain over static preference selection. Together, they support a measurable role for dynamic feedback beyond fixed preference conditioning.
AuthorsOpenReview ↗

Response to Reviewer o7oB

O-W1a — Is the MNIST increment reproducible?

1. The paired results show a consistent direction. We interpret the MNIST ablation across matched preference–seed runs, not solely through the 0.070.07-point difference between two averages. Full PAOS achieves the highest mean accuracy at each of the five corruption targets. Across the ten matched preference–seed pairs, its accuracy improves over Static in nine and ties in one; target validation CE decreases in all ten (Sec. 4, Table 3 and the accompanying ablation analysis).

MetricStatic preferenceFull PAOSMatched-run consistency
Average accuracy (%) ↑84.66±0.0484.66\pm0.0484.73±0.0784.73\pm0.079 improvements, 1 tie / 10
Target validation CE ↓0.6565±0.00410.6565\pm0.00410.6522±0.00410.6522\pm0.004110 improvements / 10
Accuracy at 40% corruption (%) ↑81.54±0.0081.54\pm0.0081.68±0.1781.68\pm0.17Included in the paired results above

2. The cited SD measures a different quantity. The 0.170.17-point SD describes PAOS accuracy at one corruption target; it is not the variation of the matched Full–Static difference. The paired comparisons directly track whether feedback improves outcomes under the same preference and seed. Their consistent direction, together with lower target CE in every pair, is the MNIST evidence for the added role of feedback. The double-well branch results and PINN feedback ablation provide the complementary mechanism and task-level evidence discussed in O-W1 (Appendix B, Table 8; Appendix D.4, Table 10).

O-W1b — Matched feedback versus tuned Static

1. What closing the loop contributes. We isolate feedback by comparing β=0\beta=0 and β=0.25\beta=0.25 while holding the preference, lower objective, and optimization settings fixed. Test relative L2L_2 falls from 0.036740.03674 to 0.028050.02805 (23.6%), and test MSE falls from 3.523.52 to 1.781.78 (×10−4\times10^{-4}; 49.4%). This is the component-level evidence that feeding the current Pareto mixture into subsequent response formation adds value beyond static preference selection (Appendix D.4, Table 10).

2. What the complete-method comparison shows. The 0.0040.004 terminal difference in Table 5 is an absolute error difference; 0.0280.028 versus 0.0320.032 corresponds to 12.5% lower relative L2L_2 error. The advantage also appears in trajectory-average and late-stage errors. We use these results for two distinct conclusions: Table 10 identifies the contribution of feedback, whereas Table 5 evaluates the complete method against established baselines.

Test relative L2L_2 metric ↓WS-HGTCH-HGPAOS
Final0.032±0.0010.032\pm0.0010.032±0.0000.032\pm0.0000.028±0.0000.028\pm0.000
Trajectory AUC0.087±0.0070.087\pm0.0070.087±0.0070.087\pm0.0070.082±0.0050.082\pm0.005
Final 20 iterations0.051±0.0220.051\pm0.0220.051±0.0220.051\pm0.0220.040±0.0030.040\pm0.003

O-W2a — Independent FC-100 evaluation

1. The FC-100 experiment tests bilevel response formation. In this controlled 5-way 5-shot task, the support images determine the lower-level classifier response, while the 75 query images define the five upper-level objectives. We report performance on those optimization queries because the question here is whether PAOS forms responses that yield better preference-conditioned Pareto trade-offs for the objectives being optimized—not whether it learns to generalize to unseen classes or images. This evaluation target is stated in the Table 4 caption and Appendix D.3.

2. The principal result is the trade-off geometry. The AUC difference of 80.5280.52 versus 80.3080.30 is one summary of the optimization trajectory, not the sole FC-100 evidence. Across all ten three-objective projections of the five-objective task, PAOS dominates all five reported preference means of both WC-MHGD and TCH-HG (Sec. 4, Fig. 5; Appendix Fig. 7). It also gives the highest final, last-100, and trajectory-average optimization-query accuracies (Sec. 4, Table 4).Our FC-100 conclusion is therefore specific: PAOS improves preference-conditioned trade-offs while maintaining strong optimization performance in a controlled few-shot bilevel task.

Optimization-query accuracy (%) ↑WC-MHGDPAOS
Final95.47±0.0095.47\pm0.0096.27±1.1396.27\pm1.13
Last 100 iterations94.30±0.3794.30\pm0.3794.85±0.6794.85\pm0.67
Trajectory AUC80.30±0.6080.30\pm0.6080.52±0.8080.52\pm0.80
AuthorsOpenReview ↗

Response to Reviewer o7oB

O-W2b — Data Duplication with Baseline

The identical entries are expected. Yes—Table 2’s WC-MHGD row and Table 3’s “Open loop (ρt=0\rho_t=0)” row use the same computational reference. Setting ρt=0\rho_t=0 gives g~t=g\widetilde g_t=g, so the selector no longer affects response formation or the resulting hypergradient. With the WC coordination step and numerical settings held fixed, the open-loop configuration recovers the WC-MHGD trajectory (Sec. 3.2, Eq. (3); Sec. 3.3, Eqs. (4)–(6)). We show this reference in Table 2 for the method comparison and reuse it in Table 3 as the anchor for the component ablation (Tables 2–3; Appendix C.3).

O-W2c — Descent guarantee and original-problem stationarity

1. Exact descent for the coordination subproblem. We now prove a descent property for the weighted-Chebyshev subproblem in Eq. (5). On a fixed smooth selected-response branch, with the selector and selection strength held fixed, Appendix A.1, Eq. (20), shows that the exact coordination direction decreases the merit function given by the maximum preference-weighted upper objective:

M(x − ηd̄, α) ≤ M(x, α) − (η/2)‖d̄‖².

This holds for an unconstrained upper update, L-Lipschitz weighted gradients along the update segment, and 0 < η ≤ min{1/L, 2/u}, with the second restriction omitted when u = 0.

2. How feedback and numerical errors enter the analysis. Changes in the selector are accounted for through the merit function’s Lipschitz dependence on the selector. Appendix A.4, Eq. (37), bounds hypergradient error using lower-solver residuals, linear-solver residuals, and damping; Eq. (41) then bounds direction error using the input errors and the Pareto-subproblem optimality gap. The analysis of biased or stochastic updates uses the conditional descent assumption stated in Assumption 4. Appendix A.5, Lemma 2, combines these contributions into a finite-iteration bound on the exact selected direction.

3. What the revised Theorem 1 establishes. Theorem 1 now directly bounds the mean squared Pareto-stationarity residual along a regular branch of original lower-level solutions (Sec. 3.4, Eq. (11)). Its proof combines Lemma 2 with the original-branch relation in Appendix A.3, Eq. (32). The bound includes initialization, selector drift, direction bias, stochastic noise, and an O(ρ²) perturbation term. It applies at fixed, sufficiently small ρ under the stated branch regularity, uniform perturbation bounds, positive preference weights, and conditional descent assumptions.

A comparison of lower-level assumptions, upper-level conditions, and theoretical guarantees is also provided in our response to Reviewer GXA8, “Theoretical guarantees and comparison with related methods.”

O-Q1 — Baseline Clarification

Yes. WC-MHGD in Table 2 and “Open loop (ρ = 0)” in Table 3 reuse the same run statistics. When ρ = 0, the selected lower objective reduces to the original objective g, so the selector cannot change the lower response. The identical entries are therefore expected. Table 2 uses this run for method comparison; Table 3 reuses it as the reference for the component ablation (Sec. 3.2, Eq. (3); Tables 2–3).

This common reference also isolates what PAOS adds. WC-MHGD uses the Pareto mixture for the upper update; PAOS additionally feeds that mixture into αt+1\alpha_{t+1} to shape the next lower response (Sec. 3.3, Eqs. (5)–(7)). Against the shared reference, PAOS improves MNIST accuracy at all five corruption levels: average accuracy rises from 84.39% to 84.73%, and at 40% corruption from 80.89% to 81.68%. Target validation CE also falls from 0.6746 to 0.6522 (Tables 2-3). Thus, the repeated row is one controlled reference, while the performance difference shows the value of preference-guided response formation and feedback. This shared reference makes the contribution of PAOS visible across tasks: relative to WC-MHGD, PAOS reduces PINN test relative L2L_2 error by approximately 49.1% (0.055 to 0.028; Table 5).

AuthorsOpenReview ↗

Response to Reviewer o7oB

O-Q2 — Statistical Significance of Dynamic Feedback

1. MNIST shows a consistent incremental effect of feedback. In Table 3, Full PAOS improves accuracy over Static at every corruption target, including 87.12% to 87.14% at 0% and 81.54% to 81.68% at 40%; average accuracy rises from 84.66% to 84.73%, while target validation CE falls from 0.6565 to 0.6522. The accompanying paired analysis records nine accuracy improvements and one tie, and CE reductions in all ten pairs.

2. The contribution of closing the loop is also tested beyond MNIST. In the controlled double-well feedback sweep, valid preferred-branch responses rise from 0/80 under Static to 80/80 with feedback, while median regret falls from 0.243 to 0.134 (Appendix B, Table 8). In the matched heat-equation PINN sweep, feedback reduces relative L2L_2 error from 0.03674 to 0.02805 and test MSE from 3.523.52 to 1.781.78 ×10−4\times10^{-4} (Appendix D.4, Table 10). Together, these results connect MNIST’s consistent incremental gains to the mechanism PAOS is designed to provide: the evolving Pareto decision changes subsequent lower-response formation (Sec. 3.3, Eqs. (5)–(7)).

O-Q3 — Generalization in Few-Shot Learning

1. What we originally reported. We use FC-100 as a controlled 5-way 5-shot MOBL task: the class-wise optimization-query losses define the upper-level objectives. Accordingly, Table 4 reports performance on those objectives, while Figure 5 examines the resulting preference-conditioned trade-offs (Sec. 4, Table 4 and Fig. 5; Appendix D.3). The original evaluation was designed to measure response formation for this specified bilevel problem, not accuracy on images excluded from its upper objective.

2. Held-out image evaluation. To answer the image-level question directly, we now evaluate disjoint images from the same five sampled classes that enter neither the support set nor the upper-level optimization query. We evaluate the trained states without re-adaptation. The following focused comparison reports accuracy across five seeds (mean ± standard deviation); bold marks the highest mean among these four methods. PAOS has the highest mean on all three held-out metrics within this focused comparison. This result evaluates transfer to unused images within the sampled tasks.

Held-out accuracy (%)WS-HGTCH-HGWC-MHGDPAOS
Final Acc. ↑52.792 ± 5.20049.912 ± 3.55953.184 ± 4.71753.448 ± 4.494
Last-100 Acc. ↑51.463 ± 4.68549.246 ± 3.98752.359 ± 4.63753.225 ± 4.536
Acc. AUC ↑48.158 ± 4.57645.843 ± 4.05248.071 ± 4.44950.728 ± 4.769

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and updated responses

Thank you to all reviewers for the time and care you have put into evaluating our work. We have uploaded the latest revised manuscript with changes marked in blue and updated our responses. As the discussion period approaches its close, we briefly summarize how the revisions address the reviewers’ main concerns.

1. Theoretical scope (Reviewers C8J1, GXA8, 1jnN). In response to C8J1’s and 1jnN’s questions about the original problem, Theorem 1 now directly bounds the mean squared Pareto-stationarity residual along a regular branch of original lower-level solutions (Sec. 3.4, Eq. (11)), with its assumptions and perturbation term stated explicitly. Our updated response to GXA8 also compares the lower-level assumptions, upper-level conditions, and guarantees of PAOS and related methods.

2. Experimental evidence and evaluation (Reviewers o7oB and GXA8). To address questions about the contribution of feedback, we report matched feedback comparisons and paired MNIST outcomes. We also provide additional PINN comparisons with WC-Penalty and MOMEHA, and FC-100 evaluation on held-out images from the same sampled classes, excluded from both support and optimization-query sets. In response to o7oB’s baseline question, we clarify that WC-MHGD and the open-loop ablation share the same computational reference.

3. Readability and presentation (Reviewer 1jnN). We added context on the learning setting and its significance in the Abstract and Introduction, added Table 1 to identify each benchmark’s upper- and lower-level variables and objectives, and clarified the feedback mechanism in the method overview. Section 3.4 now explains what each theoretical result establishes and why it is relevant.

We would appreciate hearing whether any specific concern remains after these updates. Thank you again for taking the time to consider our revisions and responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.