When High Surrogate Scores Mislead: Pairwise Ranking Reliability for Offline Model-Based Optimization

Many scientific and engineering design tasks require searching large design spaces for high-performing designs, yet costly evaluations leave only a fixed and sparsely distributed dataset. Offline model-based optimization (MBO) addresses this setting by learning surrogate models to guide candidate search. However, surrogate-guided optimization may produce misleading preferences when candidates lie beyond well-supported regions. Recent ranking-based methods improve candidate selection by learning relative candidate preferences, but a high surrogate score or rank does not indicate whether the induced pairwise ordering is reliable. Therefore, we propose TRACER, a two-stage framework for improving pairwise ranking reliability in offline MBO. Stage I improves surrogate order preservation through a Kolmogorov-Arnold Network (KAN) based surrogate, Top-$\rho$ contrastive regularization, and a two-stage candidate search strategy. We theoretically characterize the relationship between surrogate approximation error and pairwise ordering preservation, and further analyze how KAN's local spline structure supports order preservation in offline-supported regions. Stage II reassesses candidate comparisons after search by integrating reliability evidence from offline support and local prediction stability, and performs reliability-aware reranking with support-aware candidate refinement. Across five Design-Bench tasks, TRACER achieves the lowest mean rank among 18 baselines. The anonymous link of our source codes is available at https://anonymous.4open.science/status/TRACER-AF20.