- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks when minimizing population SPO+ risk identifies the mean cost report, beyond recovering an optimal decision. It removes cost directions that cannot affect the optimization problem and studies identification in the remaining effective space. Under central symmetry, the key condition is strict crossing. Every nonzero effective displacement must have positive probability of moving the selected optimizer outside the shifted optimal face. The paper gives sufficient support conditions, distinguishes face crossing from changes in a tie-breaking oracle, and derives local risk-growth bounds.
Critical strengths
-
The distinction between a correct decision and an identified report is worth making. Example 4.1 shows that an entire ray of reports can minimize the surrogate while inducing the same optimal decision. Quotienting out the invisible directions first makes the identification target precise, including for lower-dimensional or equality-constrained feasible sets.
-
The treatment of ties is a strong part of the paper. A selected oracle can change its output while remaining within the same optimal face, so selected-output disagreement alone is not the appropriate criterion for identification.
-
The quantitative analysis is good. Theorem 8.7 derives quadratic growth from boundary mass near normal-fan facets associated with a direction-covering set of edges spanning the effective space. The one-dimensional examples also show why strict crossing alone need not imply any polynomial growth rate. Together with the reflected-distribution result in Theorem 8.8, these results help clarify which assumptions are responsible for identification and quantitative risk growth.
Critical weaknesses
-
The incremental novelty of the central identification result remains unclear. The paper acknowledges that the centered SPO+ excess-risk identity and uniqueness under stronger full-support conditions are already known. Given the nonnegative excess-risk representation, Theorem 5.2's equivalence between unique quotient-report identification and positive-probability crossing follows relatively directly after quotienting out invisible directions. Although the subsequent geometric and quantitative results are substantial, the paper does not sufficiently demonstrate important settings where the new characterization meaningfully advances existing SPO+ theory. Concrete examples where prior sufficient conditions fail but the new criterion establishes identification would strengthen the novelty claim.
-
The application experiments provide limited evidence about the effective report. For exact top-k selection with fixed cardinality, adding the same value to every cost coordinate leaves the set of optimal selections unchanged. The large raw-RMSE differences in Table 4 may therefore partly be driven by an invisible component. The same issue would arise in Table 3 if the portfolio constraints fix total investment. The authors acknowledge the limitation of raw RMSE, but decomposing prediction residuals into effective-space and invisible components would make these comparisons substantially more informative. Projected residual error would still be a predictive diagnostic, rather than evidence of conditional-mean recovery.
-
The experimental evaluation remains limited in scope, and several details needed to interpret or reproduce individual experiments are not clearly specified or mapped to the corresponding artifacts. For Table 7, the paper specifies the DGP and five seeds but not the training and held-out sample sizes, MLP architecture, optimizer, or training budget. The observational studies provide some protocol details but still lack dataset date ranges where applicable, retained sample counts, fuller preprocessing specifications, and complete feasible-set definitions. A consolidated experiment-by-experiment mapping to the relevant settings and files would improve reproducibility. The mean-recovery component of Appendix J measures Monte Carlo recovery of a sample mean and therefore does not test whether SPO+ identifies or learns that mean. Table 7 does train predictors, but only on a two-dimensional box, where the quotient coincides with the full ambient space. It therefore does not test report recovery in a genuinely nontrivial quotient setting.
Questions for authors
-
For the KuaiRec top-k and Portfolio studies, can you report the corresponding effective and invisible subspaces and decompose the prediction residuals into these two components? For Portfolio, please also provide the complete feasible-set definition. This would clarify how much of the reported RMSE differences lies in directions that are actually relevant to the optimization problem.
-
The novelty of Theorem 5.2 could be clarified. The paper notes that the centered SPO+ excess-risk identity and uniqueness under stronger full-support conditions are already known. Given that the excess risk can be written as with , the equivalence between unique quotient-report identification and positive-probability crossing appears relatively direct. Please clarify the substantive technical gain over prior SPO+ results, ideally with important settings where existing sufficient conditions fail but the new characterization yields a nontrivial conclusion.
-
There remains a substantial gap between population identification and finite-sample recovery. The identification and coercivity results are population statements, and the approximate-ERM result still assumes a local uniform-deviation bound and localization of the learned report. Can these conditions be established under standard hypothesis-class assumptions to obtain an explicit finite-sample quotient-report recovery guarantee?
-
The learned-predictor experiment does not test a nontrivial quotient space. Table 7 uses , so and ; quotient-report error therefore coincides with ordinary report error. Could the authors add a controlled learned-predictor experiment with , where ambient recovery may fail while effective-space recovery succeeds? This would more directly validate the paper’s quotient-report formulation.
4 comments
Part 2/3 — Continued from Part 1/3 (W2+Q1, then W3)
The complete Portfolio feasible set is
S_u=\left\\{w\in\mathbb R^{48}:\mathbf1^\top w=1,\quad 0\le w_i\le u,\quad 0.12\le\sum_{i\in G_g}w_i\le0.28\\ (g=1,\ldots,5)\right\\},
where the 48 industries are partitioned into five groups with indices , , , , and . Training uses , and the held-out evaluation changes the cap to . Under both constraint sets,
The KuaiRec action set at budget is
with linear-optimization relaxation , so again .
The learned-report results on these same application geometries are reported in our response to wAPG (W2). They use real feature rows and semisynthetic costs with known conditional means. In the flat-risk control, SPO+ achieves numerically zero excess risk and regret while its effective report RMSE remains large; MSE recovers the known mean much more closely.
To W3
We give the box experiment's sample sizes and training settings, the application preprocessing and feasible sets, and identify the experiment-to-artifact map for the added results. The two-dimensional box experiment of Table 7 is specified as follows. Let , , and
The three regimes (SC-on, low mass, SC-off) use , , and . Each regime uses 1024 training samples, 4096 held-out samples, and 5 training seeds. The MLP has one hidden ReLU layer of width 16; the optimizer is Adam with learning rate , batch size 64, and 60 epochs; the hybrid adds squared report error with weight . For this box, , so there are no invisible report directions.
The sample sizes, splits, and preprocessing of the application data are:
| Application | Available samples and split | Features, target, preprocessing |
|---|---|---|
| Portfolio, each return definition computed separately | 14,119 usable dates each; train/validation/test ; test period 2018-01-24 to 2026-06-30 | 355 features: five daily return lags per industry, 20-day rolling volatility, 252-day cumulative momentum, and 7 weekday plus 12 month indicators. Rolling features use lagged data; the warm-up period is dropped. Target: next-day cost . |
| KuaiRec | 1,411 users and 2,062 candidate videos; user split , disjoint user IDs | 51 user features. Numeric missing values are imputed with training-user medians and standardized with training-user means and standard deviations; categorical vocabularies are fitted on training users only. Candidates are retained only when all held-out users have watch ratios. Target: negative watch ratio. |
| Energy/Storage | 787 complete 48-slot days; chronological split ; test period 2013-09-04 to 2013-12-31 | 9 features per half-hour slot: HolidayFlag, DayOfWeek, WeekOfYear, Day, Month, Year, PeriodOfDay, ForecastWindProduction, and CO2Intensity. Rows with missing targets or allowed features are dropped before retaining complete days. Target: SMPEP2. |
Energy/Storage uses the fixed instance SchedulingInstances/load3/day01.txt with 30-minute slots, 5 machines, and 20 tasks. Writing for assigning task to machine starting at slot , the feasibility constraints are
with outside the task time windows and . The linear-programming evaluation uses the relaxation . The objective coefficients are the task power times hours times the sum of the prices over the occupied slots. The parameters , the time windows, and the powers are taken from the instance file.
We agree that the mean-recovery computation in Appendix J is a Monte Carlo estimate for a given conditional mean and trains no predictors; it is therefore not evidence of SPO+ learned-report recovery. The trained-report experiments above and under Q4 address this gap. An experiment-to-artifact map in the experimental appendix lists the settings, scripts, and machine-readable results for each added diagnostic, including the corrected Portfolio runs, known-DGP grids, and simplex learned-report experiment.
Part 3/3 — Continued from Part 2/3 (Q3 and Q4)
To Q3
Yes. The explicit high-probability bound in our response to Z9bk (Q3) applies to a finite, bounded, -valued predictor class containing , with bounded projected costs and uniform conditional quadratic growth. It gives effective report error for exact ERM. The empirical minimizer need not be assumed to lie inside the local growth radius: convexity extends the local lower bound along rays to every report reachable by the bounded class. A uniform deviation bound for a bounded class can replace the finite-class Hoeffding term.
The bounded-cost and realizability assumptions can also be relaxed. With sub-exponential centered losses and approximation error , the corresponding bound has the form
where is the sub-exponential uniform deviation term and is the growth constant extended to the bounded report range. The common conditional growth assumption remains essential.
We checked that assumption in a two-context finite-class experiment with , , and . For , the conditional excess risk equals . With , 250 exact ERM runs, and , the 95th-percentile report error divided by rises from at , to at , at , and when . A flat context therefore prevents uniform report recovery in this controlled setting.
To Q4
Yes. We trained predictors on a three-action simplex with a two-dimensional effective space and a one-dimensional invisible space. We train MLP reports on the three-action simplex
The effective space is two-dimensional and the invisible space is one-dimensional. Features are , with known conditional mean
where the columns of form an orthogonal basis of . Under SC-on, centered Gaussian noise of scale is added in ; under SC-off, costs are deterministic given . The training set has 1024 samples and the held-out set has 8192. The model is a two-layer ReLU network of width 32, with batch size 128 and 60 epochs; the Adam learning rate is for the effective network and for the common-mode intercept. The table reports means and sample standard deviations over five seeds.
| Regime | Method | Ambient RMSE | Effective RMSE | Invisible RMSE | SPO+ excess | Regret |
|---|---|---|---|---|---|---|
| SC-on | MLP-MSE | |||||
| SC-on | MLP-SPO+ | |||||
| SC-off | MLP-MSE | |||||
| SC-off | MLP-SPO+ |
Under SC-on, MLP-SPO+ has ambient RMSE , of which the invisible component is and the effective component is . The SPO+ objective has no gradient in the common-mode component, so that component retains its initialization. The five common-mode intercepts start at , while the true common-mode value is . Their invisible errors are ; across the five seeds these have mean and sample standard deviation . This is the case the reviewer asks for: ambient recovery fails, while the effective component is recovered to within , comparable to for MLP-MSE. Under SC-off, the SPO+ effective report error exceeds the MSE value, but the test excess and regret are zero.
The 10-seed scaling results, reported under wAPG (W2), show effective RMSE falling from to under crossing; without crossing, it stays near despite zero excess risk and regret.
Thank you for the detailed clarifications. I appreciate the additional analyses and experiments. However, the finite-sample guarantee still relies on additional assumptions, and the new experiment remains relatively limited in scope. More importantly, my central concern about the incremental theoretical novelty remains. Although the new example helps illustrate the characterization, the equivalence in Theorem 5.2 follows relatively directly from the previously established nonnegative excess-risk identity.
Overall, I maintain my original assessment, although I consider the paper borderline. That said, I would not oppose acceptance if the other reviewers reach a consensus in favor of the paper.
Part 1/3 — W1+Q2 and W2+Q1 (residual decomposition)
We thank Reviewer ygPw for the detailed report and the concrete suggestions.
To W1+Q2
We agree with the contribution hierarchy. Liu and Grigas (2021) give the centered SPO+ excess-risk identity; Theorem 5.2 gives its exact equality condition after removing directions in L⊥, with ties tested against the full shifted optimal face. Theorem 8.2 integrates a quantitative crossing lower bound, and the contextual statement applies the fixed-covariate result conditionally.
The equality condition covers cost laws without density or full support. For instance, on S=[−1/2,1/2], a symmetric law assigning probability 2−n−1 to each of C=1/n and C=−1/n, for every n≥1, crosses under every nonzero report displacement. Its unique Bayes report is zero. On the affine set S=(t,1−t):0≤t≤1, the cost C=Z(1,−1) with the same law for Z identifies the report modulo L⊥=span(1,1). These examples lie outside the earlier ambient sufficient conditions.
The main quantitative step is the direction-covering boundary-mass result. A positive projected-cost density near a finite family of normal-fan facets whose edge directions span L yields RSPO+(cˉ+Δ)−RSPO+(cˉ)≥C0∣Δ∣2 locally for every Δ∈L. In the four-strip example described in our response to wAPG, the density vanishes near the mean and has compact support, yet the bound holds with C0=1/16 and r=1/10. This controls effective report distance; Liu–Grigas calibration controls task regret under different conditions.
To W2+Q1
We decompose the Portfolio and KuaiRec residuals into effective and invisible components and state both feasible sets explicitly. For Portfolio with fixed total investment and for fixed-cardinality KuaiRec top-k, decision differences lie in the subspace of vectors with zero coordinate sum. Write ri=c^(xi)−ci and
L=1⊥,L⊥=span(1).
The residuals decompose as
PL⊥ri=d1⊤ri1,PLri=ri−PL⊥ri,
and the vector RMSEs satisfy
RMSEambient2=RMSEL2+RMSEL⊥2.
Portfolio uses d=48 and KuaiRec uses d=2062. The table reports means and sample standard deviations over five training seeds. OLS and the two KuaiRec fits are deterministic (Table 4), so they have no seed variation.
On KuaiRec, the ambient RMSEs of Linear SPO+ and Ridge are 71.728 and 58.921, but the effective-space RMSEs are 58.459 and 58.351: the difference lies mainly in the invisible component. The reviewer's conjecture therefore holds for KuaiRec. It does not hold for Portfolio, where the gap is in the effective component (0.0936 against 0.1879 for equal-weighted returns) and the invisible components are similar. The SPO+ invisible component shows no seed variation because the SPO+ loss has no gradient in that direction, so its value is set by the initialization. The training objective is invariant to shifts in span(1), so its gradient in that direction is zero; the fitted common-mode offset retains its initialized value. Projected residuals are held-out predictive diagnostics and do not by themselves establish conditional-mean recovery.