- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies active preference learning from pairwise comparisons under the ideal-point model: items are points in , and for a query , the user prefers the item closer to a hidden ideal point . The authors propose a confidence-aware response model in which the noise on each squared distance scales with that distance: .
Under a Gaussian posterior , Info-Synth maximizes with the query , , where is the principal eigenvector of and comes from a 1-D minimization of expected conditional entropy. For fixed datasets, Pair M-dist (a Hessian-based Mahalanobis distance) and Pair Opt-dist (a heuristic built from the optimality conditions) shortlist candidate pairs and then pick the one with the highest exact MI. Experiments cover a 20-participant color study, synthetic data (2-10 dimensions), the Reddit TL;DR dataset (128-dimensional embeddings), and simulated gain tuning of a unicycle controller. The authors report sample efficiency comparable to or better than that of pool-based selection at a lower per-query cost.
Critical strengths
The query rule is interpretable and cheap: it centers the pair at , aligns it with , and sets the spacing via a 1-D search, which, in the continuous setting, costs and requires no candidate scoring. In the discrete setting, scoring pairs jointly, rather than taking independent nearest neighbors of and , preserves the query's informativeness. In Figs 5 and 13, NN Approx and -NN Approx saturate while Pair M-dist tracks Active Discrete, so joint scoring recovers the performance that independent neighbor selection loses.
Critical weaknesses
The theory is weaker than claimed. With and , the model's argument depends only on , so ; if and is far away, the model assigns its maximal confidence , contrary to the motivation in Fig 1(c). Moderate spacing arises only from the posterior-level optimization, and since comes from the surrogate (App C.4) while comes from expected entropy, global optimality for the MI objective is not established. The experiments also fall short of the claims. In the user study (Fig 3), each condition changes both the response model and the query points, and was fit to the same participants, so Fig 3(b)-(c) cannot isolate which model better explains human choices; Fig 3(d) reports 1.61 vs 1.53 mean confidence from 20 participants without error bars or significance tests. The runtime gain (Alg 2, Fig 6) is small, 0.21-0.49 s vs 0.68-0.74 s per query, since the filter step still scores every candidate pair, and the fastest setting () needs more queries. Under model mismatch (Figs 16-20), Info-Synth plateaus at a higher error than Active Discrete and loses its edge over Gauss Search Synthesis, so the gains depend on users following the assumed model, which Fig 3 does not establish.
Questions for authors
- Can you compare the response models on a common held-out set of human comparisons, with calibrated on separate data?
- How does query-selection time scale with and , given that the filter scores all pairs, and when is the saving meaningful end-to-end?
- Can and be shown to globally maximize the original MI objective? If not, under what assumptions is the result approximate?
For the optimality concerns (Q3), please see our common response and the revised Sec. 4 and App. C in the paper. Questions about the user study calibration and computational details are addressed in the common response.
1. Distance-dependent confidence and Figure 1
Regarding the observation that ∣f∣≤1/σ0, with maximal confidence when p=w and q is far away: this is correct for a fixed user, and it is exactly why the two entropy terms cannot be separated. Such a query is answered confidently by every plausible user, so its outcome is predictable and it carries no information; the mutual information penalizes this through H(Y). The preference for moderate separations is indeed a property of the posterior-level objective, and we now say so in the paper.
Figure 1 compares different separations for equiprobable queries, for which I(W;Y)=1−H(Y∣W). Within the illustrated family, MI is lower at both separation extremes and maximized at an intermediate radius that minimizes response uncertainty averaged over the posterior. We have updated this detail in the paper.
2. Human responses and model mismatch
Figure 3 evaluates actual human judgments, providing evidence for the combined procedure in a real task, while Figures 19-23 test response mechanisms that differ from our assumed model. The results are therefore compatible. Info-Synth and Gauss Search Synthesis have similar query geometry, with separation being a principal distinction. Under the normalized mechanism, separation does not affect response probabilities at a fixed hyperplane; under the plain mechanism, larger gaps can improve reliability. The finite optimal-spacing advantage targeted by Info-Synth is therefore absent, which is consistent with the similar performance in Figures 19-20. Random gaps may occasionally favor Gauss Search; Info-Synth also updates its gap as the posterior changes. Active Discrete achieves lower errors in the reported mismatch settings at greater computational cost.
3. Reddit experiments: dimensionality and runtime
Dimensionality. All Reddit experiments now use the original 512-dimensional embeddings without PCA. The 128-dimensional setting has been removed.
Runtime. We replaced average per-query time with total query-selection time over all 80 queries. With the revised Pair Opt-dist score (Eq. 9), the smallest shortlist (γ=0.2) no longer needs more queries. Its accuracy matches Active Discrete's at every query count in 17 of 20 user–noise settings, and the largest deviation is 1.4 percentage points. Across 10 users, the median total query time is 36.8% of Active Discrete's (Figure 29). For example, for User 1 it is 124.4 s versus 339.1 s at σ0=0.1, and 86.7 s versus 263.1 s at σ0=0.3 (Figure 6). Because both methods need the same number of queries to reach a given accuracy, this saving also applies end to end, in time to a target accuracy.