- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces SASS, which unifies the spectral-ranking consistency regularization used during surrogate model training with the gradient projection employed during the search phase by sharing a low-frequency Fourier projection, . The authors evaluate SASS against various offline MBO baselines across five tasks from the Design-Bench benchmark. Furthermore, the paper provides a conditional trajectory analysis to characterize the deviation of the projection-based search from standard gradient ascent.
Critical strengths
- This paper's problem positioning is novel: instead of pursuing higher ranking accuracy, it clearly distinguishes between the two distinct objectives of "ranking quality" and "search direction reliability," which is insightful for offline MBO research.
- SASS aligns learning preferences with search constraints through a single spectral projection, making its design more unified than mere surrogate regularization or simple search improvement.
Critical weaknesses
- The theoretical results are relatively weak and fail to provide guarantees for end-to-end optimization. The analysis of trajectory conditions in Section 3.3 relies on the assumption that , along with a Lipschitz condition on the gradient. However, the authors explicitly acknowledge that these assumptions are neither automatically satisfied by the consistency loss nor applicable to the ReLU surrogate networks used in practice. More importantly, this analysis merely indicates that the divergence between the trajectories of projected search and standard search is bounded by . It neither proves that the true objective function will increase nor offers any guarantees regarding the quality of the candidates. Consequently, this theoretical result serves more as a conditional explanation than as a robust validation of the method's effectiveness.
- The paper argues that accurate ranking alone is insufficient for reliable offline MBO, but the role of ranking deserves a more nuanced discussion. I encourage the authors to cite and discuss On the Learnability of Offline Model-Based Optimization: A Ranking Perspective (Lyu et al., 2026, arXiv:2603.04000), which theoretically characterizes the importance of optimization-relevant ranking errors. Clarifying the complementary roles of ranking accuracy in candidate selection and surrogate gradient geometry in guiding search would strengthen the paper's motivation and theoretical positioning.
- Evidence for frequency robustness has not yet been localized to specific components. The perturbation robustness experiments in Section 4.4 evaluate the complete SASS framework only on the Superconductor dataset. The authors explicitly note that these comparisons "do not isolate which component accounts for the difference."
- Section 4.5 evaluates the preservation of oracle quality before and after projection using the Spearman correlation coefficient and RMSE. However, such metrics only assess whether the projection retains the target correlation values or rankings; they do not directly measure the magnitude of , nor do they evaluate the degree of alignment between the projected gradient and the true local direction of the target.
Some typoes:
- In Section 2, the symbol is used in a way that conflates vectors with indices; it is recommended to change this to or to explicitly use and .
Questions for authors
- Please supplement the ablation study on the robustness of individual variants C1, C2, and C3 under high-frequency perturbations, and repeat the key robustness experiments on at least one other task (such as TF-Bind-8 or TF-Bind-10) to determine whether the robustness is consistent across tasks.
- Please report the empirical distribution of , or compare the correlation between the projected gradient and the oracle finite difference direction to directly test the core mechanism that "high-frequency gradient components lead to unreliable search".
- Clarifying the complementary roles of ranking accuracy in candidate selection and surrogate gradient geometry in guiding search.
3 comments
Response to Reviewer 8CDg [2/2]
4. Direct evaluation of gradient reliability
Spearman correlation and RMSE do not directly measure gradient reliability. Section 4.5 and Appendix E.4 acknowledge this limitation. Appendix Table 13 examines synthetic oracle-gradient preservation, rather than alignment of a learned surrogate with the oracle.
The quantity measures the removed gradient component, but its magnitude alone cannot tell us whether the remaining direction is better. We therefore evaluated the alternative diagnostic suggested by the reviewer: alignment with oracle finite-difference directions.
Additional surrogate–oracle alignment experiment. We compared raw and projected gradients of the same trained surrogate at identical evaluation points against oracle finite-difference directions. The evaluation used five matched seeds, held-out designs and optimization-trajectory points, and centered finite differences with step-size sensitivity checks. Cosine similarities were averaged within each seed before aggregation. Oracle information was used only for this post-hoc evaluation, never during training or candidate optimization.
Table R2. Surrogate–oracle gradient cosine similarity (mean ± SD).
| Task | Raw gradient | Projected gradient |
|---|---|---|
| Ant | 0.47 ± 0.07 | 0.62 ± 0.06 |
| D'Kitty | 0.42 ± 0.08 | 0.59 ± 0.07 |
| Superconductor | 0.39 ± 0.07 | 0.54 ± 0.06 |
Results are reported as mean ± standard deviation across five seed-level averages.
Projected gradients show higher mean cosine similarity on all three tasks. Because the surrogate and evaluation points are held fixed, the comparison isolates the effect of projecting the gradient. The improvement concerns local alignment; final oracle performance remains a separate question.
5. Notation
We will clarify the notation in Section 2 by explicitly defining as either or and distinguishing each vector from its scalar components.
Response to Author Rebuttal
Thank you for the detailed response and additional experiments. The TF-Bind-10 component ablation and the surrogate–oracle gradient-alignment results address important parts of my original questions.
My remaining reservation concerns the strength of the connection between the training objective and the claimed search benefits. The trajectory bound is conditional, its assumptions do not follow from the consistency loss, and it does not establish improvement in the true objective. The new alignment experiment provides relevant empirical support for gradient projection, but does not by itself establish how much the training-time consistency term contributes to this improvement or when the shared spectral bias will be beneficial.
Overall, the rebuttal strengthens the empirical case, and I continue to find the distinction between ranking quality and search-direction reliability useful. However, the theoretical support remains limited and the absolute gains over strong ranking baselines are modest. I therefore maintain my rating of 3 for now.
Response to Reviewer 8CDg [1/2]
We appreciate the reviewer’s attention to the distinction between ranking quality and search-direction reliability. Below we clarify what the theory establishes, identify the component-level analysis in the appendix, and present new robustness and gradient-alignment results.
1. Scope of the theoretical analysis
The trajectory bound does not guarantee oracle improvement or end-to-end convergence. Section 3.3 and Appendix C.4 state its assumptions separately from the consistency loss; in particular, they do not follow automatically for the ReLU implementation.
The analysis has two roles:
∥ut−vt∥≤Lϵ[(1+ηL)t−1].
The bound relates the magnitude of the removed gradient to finite-horizon trajectory discrepancy. It says nothing about which trajectory achieves a better true objective value. We will clarify this distinction and reserve claims about optimization performance for the experiments.
2. Complementary roles of ranking and gradient geometry
Ranking helps identify and select promising designs, while surrogate gradients determine the local directions used to generate new candidates. Agreement on observed rankings alone does not determine these directions outside the observed designs.
Appendix Figures 3–4 illustrate this distinction: synthetic surrogates with nearly matched ranking metrics exhibit different gradient errors, trajectories, and final oracle values. Table 5 addresses a separate question—preservation of oracle rankings under input projection—and is not a direct measure of learned ranking accuracy or gradient reliability.
Thank you for suggesting Lyu et al. (2026). Their analysis relates optimization-relevant ranking to design quality, while Section 6 explicitly identifies path-dependent optimization and local gradient behavior as a remaining issue. SASS empirically investigates this aspect by applying a shared spectral constraint to surrogate learning and search. Our focus on gradient-based candidate generation therefore complements their ranking perspective. We will clarify this connection in the revision.
3. Component-level robustness and an additional benchmark
Appendix Table 10 reports the individual-component analysis on Superconductor. Under high-frequency-directed training-input corruption at severity 0.20, the drops for Base, C1, C2, C3, and Full SASS are 0.047, 0.028, 0.027, 0.046, and 0.018. Section 4.4 discusses full-framework comparisons; we will add a clearer pointer there to this component-level analysis.
We also repeated the component-level evaluation on TF-Bind-10 to test whether the pattern holds on another task. High-frequency-directed perturbations were applied to continuous logits, rescaled to severity 0.20, using five matched seeds and identical corruption protocols across variants.
Table R1. Component-level robustness on TF-Bind-10 (mean ± SD).
Results are reported as mean ± standard deviation across five matched seeds, computed from seed-level clean-to-corrupted paired score drops.
Both tasks show the same pattern: C3 alone is close to Base, C1 and C2 reduce the drop, and Full SASS performs best. This points to a role for the training-time components in the tested corruption settings, without extending the claim to arbitrary shifts.