- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Persistent Preference Inference (PPL) for query-efficient interactive multi-objective optimization. PPL maintains a feasible region of aspiration points consistent with historical pairwise feedback, uses LP-based certification and transitive propagation to infer pairwise relations without additional user queries, and queries the decision maker only for unresolved comparisons. The inferred preference is integrated into NSGA-II, MOEA/D, and IBEA. The paper also provides a conditional query-complexity analysis and evaluates the method on 36 multi-objective benchmark configurations, RfaH protein sequence design, and standalone preference-learning tasks based on Chatbot Arena.
Critical strengths
-
The paper addresses an important practical limitation of interactive optimization: previously collected preference feedback is often underused as the candidate population changes. Maintaining a persistent feasible preference region provides a clean and intuitive way to reuse this information across consultations.
-
The empirical evaluation is relatively comprehensive. PPL is tested with three different evolutionary optimization frameworks, multiple benchmark families, a protein-design task, and standalone preference-learning experiments. The ablations are also useful in showing that persistent memory, rather than transitive propagation alone, is a major source of the query savings.
Critical weaknesses
-
The theoretical query-complexity result is fairly conditional. The O(m log N) bound requires both persistent realizability and an η-splitting assumption on the comparisons actually queried. Importantly, the proposed median-insertion procedure does not itself guarantee this splitting property. Therefore, the result does not establish O(m log N) query complexity for PPL in general candidate populations, which makes the headline theoretical guarantee somewhat weaker than it initially appears.
-
Much of the end-to-end evaluation uses a fixed aspiration-point preference oracle that closely matches PPL’s modeling assumptions. The paper includes a linear-utility mismatch experiment, but only in a relatively limited setting. In addition, the reported 98.9% query reduction on Chatbot Arena should be interpreted cautiously: the shared query-free capability initialization already achieves approximately the final ranking accuracy, and many PPL runs therefore require no additional query. This makes it difficult to separate the contribution of persistent preference inference from the strength of the initialization and stopping mechanism.
Questions for authors
-
Can the authors empirically measure the version-space reduction or effective η for the comparisons actually queried by PPL? This would help establish whether the splitting assumption underlying the O(m log N) result typically holds in the reported experiments.
-
For the Chatbot Arena experiment, how much of the 98.9% query reduction remains when PPL and the strongest baselines are compared under exactly the same initialization and stopping criterion, or at matched numbers of target-user queries? The current results suggest that much of the ranking accuracy is already present before any target-user query.
-
Can the authors provide broader end-to-end experiments under preference-model mismatch, beyond the single linear-utility setting, for example with nonlinear or changing preferences? This would materially strengthen the practical case for persistent reuse of historical comparisons.
5 comments
Thank you for the detailed rebuttal and additional experiments. The new results clarify the method's limitations.
The theoretical query bound remains conditional, with extremely small splitting factors observed in practice. Although PPL achieves substantial query savings under controlled settings, the zero-query initialization already provides comparable accuracy. More importantly, the bimodal experiments show that fewer queries can lead to worse optimization quality, highlighting the limitations of the fixed aspiration-point assumption.
I appreciate the authors' efforts, but my main concerns remain. I therefore maintain my rating.
Response to Reviewer 6id9
Thank you for considering the additional evidence.
We agree that the extremely small observed splitting factors limit the practical informativeness of the conditional bound, and that the bimodal results demonstrate a meaningful limitation. We would like to clarify two points about the empirical contribution.
-
Historical reuse provides a measurable benefit under stable, model-compatible preferences. Compared with clearing history at each consultation, retaining history reduces mean paid queries from 344.0 to 59.9 on ZDT1 and from 432.2 to 74.7 on DTLZ2, which are approximately 83% in both cases. Final mean is unchanged at 0.1375 on ZDT1 and is 0.2319 versus 0.2303 on DTLZ2 (persistent versus reset; ten paired seeds; lower is better). These are close observed means, not a claim of statistical equivalence. This ablation directly tests the benefit of retaining earlier comparisons.
-
The zero-query finding depends on the initialization setting. Under strong initialization, we agree that query savings do not establish personalization gains. With 29 background votes, however, the shared zero-query pairwise accuracy is 63.08%; PPL reaches 65.94%, compared with 64.42% for controlled AMPLe, using 5.73 versus 6.50 queries under the same eight-query cap. These exploratory results, obtained on 44 users previously used for development, show an incremental benefit in the weakest tested initialization setting.
The bimodal failures limit applicability and should remain explicit. Our claim is therefore restricted to reducing repeated elicitation under the studied preference model. Could you please clarify whether your remaining concern is primarily the restricted preference class, or whether you also find the evidence for the benefit of historical reuse within that setting insufficient?
Ranking-Space Diagnostics, Controlled Query Savings, and Preference Robustness
Q1: Measures for Version Space Reduction
We measured ranking-space splits in 20 additional diagnostic optimization trajectories (10 each for ZDT1/DTLZ2). We counted 1044/1298 paid queries at generations 25/100/175/225; 254 queries at other generations remain uncounted. Of counted queries, 587 have exact η; 457 involve count sampling.
Entries: ZDT1/DTLZ2 (candidate N=196–200/122–128). Cumulative Q is a trajectory mean; query counts are totals at t. S_± count feasible strict rankings; η_q=min(S_+,S_-)/(S_++S_-). No-query η=NA.
† includes count sampling. Simultaneous 95% bounds for t=25 minima: [1.36612,1.36613]e-4/[2.89886,3.24180]e-9, quantifying counting error on recorded trajectories. Later minima are exact.
Measured-query η has median≈0.243/0.213 and fifth percentile 0.0109/0.00411. These query-weighted descriptive summaries include count estimates; 95% of measured queries are at t=25, so stages are unequally represented. Typical measured splits are more balanced than the minima, but some actual splits are extremely uneven. Positive, instance-dependent η establishes neither an N,m-independent lower bound nor the general O(mlogN) specialization. Auxiliary parameter-volume results cannot replace ranking counts. We will add full diagnostics/counting protocol to Appendix L and clarify query-bound scope in §4.
Q2: Controlled Arena Query Savings
With original strong-background capability initialization, aligned initial predictions and common stopping, PPL uses 82.97% fewer target-user queries than controlled AMPLe (descriptive paired-user 95% CI [73.1159, 87.5390]%). Completed replay: 44 users×5 folds×10 seeds (2,200 runs/method). We retain features, query pools and method-specific acquisition/inference, aligning analytic uniform predictions and first-MH initial means. Checks run at q=0 and after each response. Runs stop when no unresolved pair has an available vote (certification/inference or missing/exhausted pair-specific votes); ranking correctness is not guaranteed. No run is budget-censored.
Table R2. Original full-background Arena: mean actual queries/run (total over 2,200 runs); final strict user-macro PairAcc.
The shared zero-query baseline's 76.4496% matches AMPLe's final accuracy and slightly exceeds PPL's 76.4266%. Final PPL–AMPLe difference: −0.0230 percentage points (paired 95% CI [−0.0599, 0.0000]). This cost gap establishes neither personalization accuracy gains nor quality equivalence. 98.91% is for native pipelines; 82.97% is for strong-background initialization with common stopping. PairAcc is final, not AULC; intervals are descriptive on previously used development data.
Paired logs show unchanged query sequences/stops: alignment preserves ranking order, and saved posterior means agree after non-tied feedback. We will qualify the Abstract, Introduction and §6.3 accordingly.
Q3: Preference Robustness and Historical Reuse
Fixed-target historical reuse reduces paid queries by 82.6%/82.7% on ZDT1/DTLZ2, with close final quality means. Entries (means over 10 paired seeds) are persistent PPL/an ablation clearing historical comparisons at each consultation; lower quality metrics are better.
Without an equivalence test, close means do not establish statistical equivalence.
We extended end-to-end NSGA-II experiments to quartic, bimodal, abrupt and gradual preferences. For bimodal DTLZ2, retaining history reduces mean paid queries (278.8→28.4) but increases final population mean true loss (0.0253→0.0811). Thus, fewer queries can accompany worse preference quality.
Under a strong abrupt DTLZ2 change, resetting preference state and reopening consultation after external notification improves mean final D(S) (0.5298→0.1563), while mean paid queries rise (73.8→199.3). The notice supplies no target coordinates or preference answers. This evaluates externally signaled adaptation with extra feedback, not automatic detection or history clearing alone.
Appendix M will give full protocols, metric definitions and nonlinear/bimodal/changing-preference results; §6.2 will summarize quality–query trade-offs.