acceptodds
Under review as a conference paper at ICLR 2027

Representation-Guided Preference Learning for Multi-Episode Group Decision-Making

Abstract

We study representation-guided group decision-making with a fixed pool of participants over a sequence of episodes. During each episode, a group coordinator observes scalar utility feedback from each participant in response to text descriptions of options, and seeks to find a fair solution to the posed decision problem. We formalize the setting as a bilinear low-rank bandit and apply an approach of alternating minimization with LCB-minimax exploration. We prove a cumulative pseudoregret bound that scales with the spectral dimension of the representation , rather than the ambient encoder dimension . On a controlled multi-episode simulator, the algorithm reduces the total episode-best-plan gap to vs –; per-episode error also drops sharply: from in episode 1 to in episode 3, and thereafter remains roughly in the range. The same reductions carry over to real pretrained encoders (Qwen3-8B, Llama-3.1-8B-Instruct, gemma-3-4b-it, and all-MiniLM-L6-v2), where jointly learning the shared probe and per-user preferences reduces by 1.9–2.5 versus random projections and 3.2–4.9 versus one-hot features.

Primary area:
reinforcement learning
Keywords:
Low-rank bandits, Sequential decision-making, Preference learning, Representation learning, Group decision-making

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Jx4KOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a framework for learning group decision-making over a sequence of episodes, aiming to minimize the maximum dissatisfaction across participants. It utilizes contextual bandits and bilinear modeling to represent participants' utility functions, along with a pretrained encoder, potentially an LLM-based encoder, to generate embeddings of the decision options. The proposed algorithm jointly learns a shared projection matrix and individual participant preference vectors through alternating minimization, using scalar utility feedback collected across episodes. The authors provide a theoretical cumulative pseudoregret bound that depends on the spectral dimension of the representations rather than the ambient encoder dimension. They also present experimental results evaluating the proposed framework using synthetic simulations and pretrained text encoders.

Critical strengths
  1. The main strength is the elegant problem formulation and representation-guided preference learning. The paper presents a formulation of multi-episode group decision-making, utilizing pretrained text encoders, potentially LLM-based, to represent decision options. Furthermore, the framework requires participants to provide only scalar utility feedback for each presented plan, without explicitly specifying their preference functions or ranking all possible options, adding to the simplicity and elegance of the proposed approach.

  2. The suggested algorithm of the joint learning and preference generalization is also very strong. By jointly learning a shared projection matrix and participant-specific preference vectors, the proposed method aims to identify preference-relevant similarities between options and generalize to unseen decisions across episodes.

  3. Theoretical analysis: The paper provides a theoretical cumulative pseudoregret bound that depends on the spectral dimension of the encoder representations rather than their ambient dimension. The theoretical analysis and supporting proofs are valuable contributions that help explain the potential advantages of low-dimensional representations.

  4. The results section is very informative and contains very extensive ablation studies.

Critical weaknesses
  1. Lack of real-world validation of the fundamental modeling assumptions: The proposed framework relies on strong assumptions regarding participants' utility functions, particularly that preferences can be represented through a shared low-rank bilinear model and remain stable across episodes. However, all experiments use synthetic participant preferences and utility functions, including those involving real pretrained encoders. Therefore, the experiments do not establish whether the proposed model accurately represents real human preferences or whether pretrained language representations capture the features relevant to actual decision-making. Those assumptions form the foundation of the proposed framework, thus their lack of validation on real human preference data significantly limits the practical conclusions that can be drawn.

  2. Theoretical guarantees rely on strong, unverified assumptions and an idealized algorithm: Although the paper provides a cumulative pseudoregret bound, several critical properties required for the guarantee are assumed rather than established. In particular, assumption A3(i) requires sufficient exploration of the relevant representation space, without proving that the proposed LCB-minimax strategy ensures this condition. Assumption A4(ii) directly assumes that the alternating-minimization procedure contracts toward the true parameters, rather than proving this property for the proposed algorithm. Furthermore, Theorem 1 applies to an idealized coordinator that differs from the implemented Algorithm 1 in its optimization procedure, confidence bounds, and projection estimation.

  3. Insufficient comparison with competitive baseline methods: While as mentioned in the strengths the ablations study are extensive and compares the proposed approach against variations of the same coordinator, such as random projections, PCA-based representations, there is no clear baseline of other methods or clear SOTA to compare to such as bilinear bandit, contextual bandit, or preference-learning methods. Additionally, the quality of the proposed combinatorial minimax optimization is not adequately evaluated against stronger global optimization approaches or exact solutions on smaller instances. Without these comparisons, it is difficult to assess whether the proposed method offers a substantial improvement over existing alternatives.

  4. Questionable exploration–exploitation behavior: In the synthetic experiments, the proposed joint-learning algorithm substantially outperforms the baseline provided with the true projection matrix W⋆W^\star. The authors attribute this result to the additional exploration induced by repeatedly updating WW, whereas the fixed-true-W⋆W^\star baseline reportedly becomes overconfident and exploits prematurely. However, this explanation raises concerns about the exploration–exploitation mechanism itself. An algorithm given the correct representation should not inherently be disadvantaged by this additional information. The results suggest that the observed improvement may partly arise from inadequate exploration or uncertainty calibration in the fixed-representation baseline, rather than from the superiority of the proposed representation-learning procedure. A comparison with a properly calibrated exploration strategy using the true W⋆W^\star \\ would help clarify this issue.

  5. Limited expressiveness of the additive utility model: The proposed utility representation constructs each plan's features by summing the embeddings of its selected options. Consequently, the modeled utility is additive across decisions and does not explicitly capture interactions or dependencies between different choices. For example, a participant might individually prefer remote work and frequent in-person meetings but dislike their combination. Such interactions are common in realistic group decision-making and cannot be represented directly by the proposed bilinear additive model. The paper does not investigate the effect of this limitation or evaluate more expressive alternatives.

  6. Lack of robustness to strategic preference manipulation: The proposed minimax objective relies on participants providing meaningful and comparable scalar utility ratings. However, strategic participants could exaggerate their preferences or dissatisfaction to disproportionately influence the selected plan. In particular, the minimax objective may prioritize a participant reporting exceptionally large utility differences over the remaining participants. The paper does not investigate strategic reporting, utility-scale normalization, incentive compatibility, or robustness to adversarial participants. This is particularly relevant given the paper's emphasis on fair group decision-making.

  7. Confounded experimental validation of effective-rank scaling: In Section 4.2.2, the authors investigate the relationship between effective representation rank and performance by increasing the Gaussian noise added to the synthetic embeddings. However, this intervention simultaneously increases the effective rank and deteriorates the quality of the underlying representation. Therefore, the observed increase in regret cannot be attributed specifically to effective rank, as it may simply result from a lower signal-to-noise ratio. Furthermore, the experiment measures stable rank rHr_H, whereas the theoretical result depends on the spectral dimension khk_h. A controlled experiment varying representation rank while preserving signal quality would provide more convincing evidence for the theoretical claims.

  8. Limited justification for using large LLM encoders: Although the paper motivates its approach through LLM-mediated group decision-making, the actual algorithm uses pretrained encoders only to extract representations of textual options, while participants provide scalar utility feedback. Moreover, the relatively small MiniLM sentence encoder outperforms all three larger LLMs in the reported experiments. While this demonstrates that the framework is representation-agnostic, it also raises questions about the practical benefits of using large LLM encoders instead of smaller specialized encoders or alternative learned representations.

  9. The abstract seems to contain an error "46.57–146" should be 46.57±3.4746.57\pm 3.47

Questions for authors
  1. Validation on real human preferences: Could the authors evaluate the proposed framework using real human preference data? In particular, how well do the assumptions of low-rank bilinear utilities, additive preferences, and stable participant preferences hold in realistic group decision-making scenarios?

  2. Theoretical assumptions and practical guarantees: Can the authors establish that the proposed coordinator satisfies the exploration condition A3(i) and the alternating-minimization contraction condition A4(ii), rather than assuming these properties?

  3. Comparison with established baselines: Could the authors compare their approach against existing bilinear bandit, contextual bandit, or preference-learning algorithms? Additionally, could they evaluate the quality of their minimax optimization against stronger global optimization methods or exact solutions on smaller instances?

  4. Exploration–exploitation behavior: Why does the proposed joint-learning algorithm substantially outperform the baseline using the true projection matrix W⋆W^\star? Could the authors investigate whether this result is caused by insufficient exploration or inaccurate uncertainty calibration in the fixed-W⋆W^\star \\ baseline, and compare against a properly calibrated exploration strategy?

  5. Strategic preference manipulation: How would the proposed coordinator handle participants who strategically exaggerate their utility ratings to influence the minimax objective?

  6. Effective-rank experiment: In Section 4.2.2, increasing Gaussian noise simultaneously increases effective rank and reduces representation quality. Could they provide a controlled experiment that varies effective rank while preserving signal quality?

  7. Benefits of large language model encoders and natural-language feedback: Given that MiniLM outperforms the larger LLM encoders in the reported experiments, under what circumstances do the authors expect LLM-based representations to provide an advantage over smaller specialized encoders or alternative learned representations? Furthermore, could the framework be extended to utilize LLMs to interpret and quantify natural-language feedback from participants, rather than relying exclusively on scalar utility ratings? For example, participants may express conditional preferences ("I support remote work, provided that team meetings remain in person"), conflicting priorities ("I value flexibility, but not at the expense of collaboration"), or qualitative concerns that cannot be adequately captured by a single numerical rating. Such an extension could better exploit the capabilities of LLMs and improve the applicability of the framework to realistic group decision-making scenarios.

2 comments

We thank the reviewer for describing the formulation as elegant, the joint learning and preference-generalization algorithm as strong, the theoretical analysis as a valuable contribution, and the ablation studies as extensive and informative. We also appreciate the comprehensive nine-point review. We address each item in order next. We have marked all updates in blue text within the manuscript.

W1/Q1. It is difficult to conduct a real-human experiment in the rebuttal window, but we flag this as the central empirical gap in the combined Limitations/Future Work paragraph.

W2/Q2. The reviewer is correct that A3(i) and A4(ii) are stated conditionally and not established for Algorithm 1. We have made three changes: (i) we split A4(ii) into A4(ii-W) (pooled, dominant term) and A4(ii-φ) (per-user, scaling only in mim_i), in order to preserve the dominant term; (ii) we measure sin⁡Θ(W^t,Wfit)\sin\Theta(\hat W_t,W_{\mathrm{fit}}) and the smallest top-khk_h Gram eigenvalue along Algorithm 1's trajectory on the simulator and all four real encoders, showing both stabilize as A3(i) and A4(ii) require (Figure 2, appendix); (iii) we add Proposition 2 in Appendix B.1.2 giving a trajectory-independent excitation lower bound for a forced-exploration variant of Algorithm 1 with ηt∼t−1/3\eta_t \sim t^{-1/3} uniform mixing, providing a theoretical anchor for the conditional A3(i).

W3/Q3. We add a GLM-dueling bandit (Faury et al. 2020, adapted) and a bilinear bandit (ESTR, Jun et al. 2019) on the simulator, matched in interaction budget (Appendix C.5). GLM-dueling is dominated by joint (Gtotal=97.84±6.39G_{\mathrm{total}}=97.84\pm6.39 vs joint's 22.06±1.4722.06\pm1.47), reflecting its halved per-round information budget and lack of cross-user pooling. Bilinear-bandit ESTR matches joint within one SE on both GtotalG_{\mathrm{total}} (18.24±1.1118.24\pm1.11) and RtotalR_{\mathrm{total}} (916±40916\pm40 vs 877±64877\pm64); this is because ESTR is designed exactly for the bilinear reward structure the simulator instantiates. Our contribution is the multi-episode group-coordination setting (per-user ϕi\phi_i, cross-episode option drift, plug-and-play hyperparameters).

W4/Q4. The reviewer's concern was well taken: we had not shown that the joint vs. fixed-true-W⋆W^\star gap survives proper exploration calibration. We sweep an inflation factor $\kappa\in\lbrace 0.5,1,2,4,8,16,32,64,128\rbrace $ on fixed-true-W⋆W^\star's βt\beta_t (range extended after no U-shape appeared at small κ\kappa). No value of κ\kappa closes the gap (best setting at κ∈{16,32}\kappa\in\lbrace16,32\rbrace yields Gtotal≈44G_{\mathrm{total}}\approx44 vs. joint's 22.0622.06). We now interpret the improvement as cross-episode WW-refit injecting plan-space feature diversity through the refit trajectory. We added Figure 3 and a corresponding interpretive paragraph.

W5/Q5. We agree this was missing. A paragraph has been added to Limitations.

W6/Q6. The reviewer is correct that the σh\sigma_h sweep changes both rHr_H and signal-to-noise. We have added a controlled sweep that varies the singular-value profile of W⋆W^\star (specifically, the khk_h-th eigenvalue) while fixing ∥W⋆∥F\lVert W^\star\rVert_F and the misspecification residual ε=0\varepsilon=0. Appendix C.4 shows GtotalG_{\mathrm{total}} and RtotalR_{\mathrm{total}} grow monotonically with khk_h at log-log slopes 0.140.14 and 0.280.28 — below the theoretical 0.50.5 reference, consistent with Theorem 1 providing a valid upper bound that simulator performance clears comfortably. We retain the original σh\sigma_h sweep as a joint rank-and-noise stress test and clarify its interpretation.

W7. Addressed by W6/Q6 above.

W8/Q7. The MiniLM-beats-8B finding is deliberately central to the paper's empirical claim: our method is representation-agnostic and benefits most from encoders whose top-khk_h subspace aligns well with the utility-carrying features, regardless of model size. This is consistent with the theorem's khk_h-dependence. We also acknowledge an extraction-method confound in the comparison: MiniLM uses mean-pooled sentence embeddings (its canonical extraction) while the three 8B LLMs use last-token last-layer activations (the standard extraction for causal LLMs). A controlled ablation applying mean-pooling to the 8B hidden states would separate these contributions; we have added this caveat to Limitations and defer the ablation to future work. The reviewer's natural-language-feedback suggestion is excellent and we have added it to Future Work as a natural methodological extension.

W9. The "46.57–146" is the range 46.57–145.56 written in shortened form; we have expanded it to 46.57–145.56 in the revision.

Dear Reviewer,

With the discussion period closing soon, we wanted to kindly follow up and ask whether our rebuttal and revisions have adequately addressed your concerns. We would be happy to provide any further clarification if needed.

Thank you for your time and consideration.

Reviewer_27zAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies fair coordination for a fixed participant pool across decision episodes, each with multiple interaction rounds and fresh text-described options. Utilities are modeled using pretrained representations, a shared low-rank probe and persistent per-user preferences, allowing bounded misspecification; the objective minimizes worst-participant divergence from each user's own optimum. The practical method alternates probe refits and posterior rebuilding, and selects plans using LCB-minimax, progressive widening and coordinate descent. Under five assumptions, Theorem 1 gives a spectral-dimension-dependent cumulative pseudoregret bound for an exact-LCB theoretical variant. Experiments use a synthetic multi-episode simulator and features from four real pretrained encoders on author-generated option texts; preferences and target utilities remain synthetic.

Critical strengths

1.The paper connects group coordination, low-rank bilinear bandits and representation spectral geometry, offering a testable account of how spectral dimension affects the representation-dependent part of interaction complexity. The worst-participant divergence objective is better aligned with the stated fairness motivation than average utility. Within the stated assumptions, I find this theoretical perspective novel and a key argument in the paper's favor. 2.The multi-seed experiments help distinguish cross-episode probe transfer from posterior persistence: in these experiments, the latter mainly reduces round-wise exploration cost rather than improving the best visited plan. Comparisons include random projections, PCA, one-hot features and a supervised reference. With joint learning, the small MiniLM encoder outperforms the tested 8B LLMs on this corpus under native feature scaling, a result the paper reports explicitly. Discussion of four differences between the theoretical variant and practical algorithm, together with deployment risks, improves interpretability.

Critical weaknesses

1.Calibration of the practical utility intervals is not reported. As already disclosed in Section 3.2(c), Algorithm 1 omits the probe-bias, misspecification and cross-term corrections in the theoretical radius. The existing known-utility simulator could test empirical coverage and clarify when this simplification is adequate. A theoretical reference should include the additive utility-error slack in Appendix B.2, not just the parameter radius. The empirical reliability of these practical intervals therefore remains unclear. 2.The actual-encoder experiments provide a useful controlled comparison, but preferences and utility targets remain synthetic and low-rank. Table 2's held-out probe residuals and Table 4's feature-noise sweep already provide related diagnostics; neither isolates the effect of adding nonlinearity to the utility generator while holding the rest fixed. A controlled nonlinear-utility test would be a useful optional robustness extension to assess sensitivity to this modeling assumption.

Questions for authors

1.On the existing known-utility simulator, could you assess empirical coverage of the practical utility intervals and, where the relevant assumptions hold, compare it with that of a theory-based reference using the projected estimator analyzed in Appendix B.2? Please state the reference confidence level, required constant bounds and coverage definition, include the full theoretical radius and additive utility-error slack, and specify the rounds satisfying Lemma 3's conditions. The practical schedule has no explicit nominal-confidence parameter; please clarify any nominal level assigned to it. 2.As an optional extension, could you add a bounded nonlinear utility component of varying strength to the existing simulator, keeping features, participants and interaction budgets fixed, and compare the joint method with the existing baselines on worst-participant divergence? A curve versus misspecification strength would clarify the tested robustness range.

3 comments

We thank the reviewer for calling the connection between group coordination, low-rank bilinear bandits, and representation spectral geometry novel, and for highlighting that our multi-seed experiments cleanly separate cross-episode probe transfer from posterior persistence. We also appreciate the pointed calibration critique and the constructive suggestion of a theory-based coverage comparison. We address each weakness/question next. We have marked all updates in blue text within the manuscript.

W1/Q1. The reviewer is correct: Algorithm 1 uses a simplified radius that drops the probe-bias, misspecification, and cross-term corrections in the full Appendix B.2 radius, and the empirical reliability of these practical intervals was not previously reported. We have run exactly the experiment the reviewer proposes. On the known-utility simulator, for every round of Algorithm 1 we check whether Ui(xt)∈[LCBt(xt),UCBt(xt)]U_i(x_t)\in[\mathrm{LCB}_t(x_t),\mathrm{UCB}_t(x_t)] under (a) Algorithm 1's simplified radius and (b) the full Appendix B.2 radius with the projected estimator, reporting mean coverage against a 90%90\% coverage reference across four settings of the feature generator h(o)=Aψ(o)+b+ηh(o)=A\psi(o)+b+\eta:

  • On the Section 4.1 simulator (with feature offset bb and nuisance noise η\eta, neither in Lemma 3's model): simplified-radius coverage is 17%17\%, full-radius coverage is 100%100\%; Lemma 3's basin condition holds in 0%0\% of rounds, and top-1 plan-ordering agreement between the two radii is 4%4\% (near the 3.1%3.1\% chance level).
  • In the well-specified regime (no bb, no η\eta): both radii achieve 100%100\% coverage, Lemma 3's basin condition holds in 96.9%96.9\% of rounds, and top-1 ordering agreement is 91.5%91.5\%.

The under-coverage on the Section 4.1 simulator therefore traces to feature-noise misspecification, not to simplifying the radius: in the regime where Lemma 3 was derived (well-specified), both radii are well-calibrated. Algorithm 1's good empirical performance despite miscalibrated intervals under feature noise is evidence of robustness beyond theory. We now disclose this in Section 3.2, add the full coverage table as a Remark in Appendix B.2, and label Algorithm 1's schedule with a 90%90\% coverage reference in Appendix A.1 to make the comparison explicit.

W2/Q2. We agree that Table 2's held-out probe residuals and Table 4's feature-noise sweep do not isolate the effect of adding nonlinearity to the utility generator while holding the rest fixed. We have added a bounded nonlinear perturbation Uinl(x)=ϕi⋆⊤ψ(x)+Atanh⁡(ϕi⋆⊤ψ(x)/su)U_i^{\mathrm{nl}}(x)=\phi_i^{\star\top}\psi(x)+A\tanh(\phi_i^{\star\top}\psi(x)/s_u) to the simulator, keeping features, participants, and interaction budgets fixed. We sweep α\alpha under two parameterizations (A=αA=\alpha raw amplitude and A=αsuA=\alpha s_u with su=Jr≈10.95s_u=\sqrt{Jr}\approx10.95 scaled to the utility magnitude), with $\alpha \in \lbrace 0,0.1,0.25,0.5 \rbrace $; only the utility-scale version shows degradation (the raw α\alpha barely saturates at the relevant utility scale). Figure 4 (appendix) plots GtotalG_{\mathrm{total}} for joint vs. baselines against α\alpha. Joint's advantage over random-WW remains ≥3.6×\ge3.6\times through α=0.25\alpha=0.25 and shrinks to 2.8×2.8\times at α=0.5\alpha=0.5 as the bilinear approximation degrades. This clarifies the range over which the bilinear modeling choice is empirically robust.

Thank you for adding both experiments. The coverage comparison addresses my question about the reliability of the simplified intervals and clarifies the effect of feature noise. The nonlinear perturbation experiment also shows how the advantage of joint learning changes as the perturbation increases over the tested range.

Thank you for your valuable feedback and suggestions for additional experiments and improvements. We are glad that our revisions have addressed your concerns. We sincerely appreciate your time and consideration, and would be happy to address any further questions.

Reviewer_SgwHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies repeated group decision-making with a fixed set of participants and changing text-described options. The paper models utilities through a shared low-rank projection of pretrained encoder features and persistent user-specific preference vectors, and proposes a coordinator that learns these quantities from scalar feedback while selecting plans to minimize the largest participant dissatisfaction. The main theoretical contribution is a conditional cumulative-regret bound for an idealized coordinator whose representation-dependent term involves the encoder's spectral dimension rather than its ambient dimension. Experiments with a synthetic simulator and four pretrained encoders report improvements in best-visited-plan quality and cumulative regret over fixed-projection and one-hot baselines.

Critical strengths
  1. The combination of persistent participant preferences, changing option sets, and a minimax dissatisfaction objective provides a setting for studying how representation learning can reduce repeated preference elicitation.

  2. The experiments include several representation and persistence ablations, four pretrained encoders evaluated against the same underlying utility target, and both best-plan and cumulative-regret metrics.

Critical weaknesses
  1. The theoretical guarantee relies on estimation and exploration assumptions that are insufficiently justified for the proposed algorithm. In Appendix B.1, A4(ii) defines a joint error eke_k that includes both the shared representation error and the maximum individual preference error, but assumes contraction toward a pooled statistical floor proportional to 1/Ntk1 / \sqrt {Nt_k} when other parameters are fixed. This improvement is plausible for the shared probe, but not generally for each separately estimated preference vector. Even with a known scalar probe and independent Gaussian noise, each participant contributes only tt observations of their own parameter, giving a ridge-estimation error scale of σobs/t\sigma_{\mathrm{obs}}/\sqrt{t}, independently of NN. The subsequent oracle calculation establishes a pooled rate for the probe with known preferences.

    Although the theorem is explicitly conditional, A4 therefore assumes a central statistical benefit that has not been established. The shared-probe and individual-preference rates need to be separated and their interaction justified. A3(i) additionally requires persistent excitation along the selected plans, which a finite random cold start does not ensure.

    The practical optimization guarantee also needs correction. Section 3 states that both ξt\xi_t and ζt\zeta_t vanish once the active set covers the relevant decisions. However, full active-set coverage does not make coordinate descent globally optimal, so ξt\xi_t can remain positive.

  2. The experimental controls and metrics do not cleanly establish the claimed persistence mechanism or final-plan performance. In joint-coldstart, participant posteriors are reset between episodes while the historical probe-refitting buffer is retained. However, Algorithm 1, line 24, rebuilds participant posteriors from the full buffer after each refit, potentially restoring the historical information that the ablation is intended to remove. Unless its replay rule is also modified, this comparison does not cleanly isolate preference persistence. Similarly, joint-newusers resamples preferences while retaining old feedback under the same user indices, confounding the absence of persistent users with inconsistent historical labels.

    The metric Ge=min⁡t∈[T]Φe(xt)−Φe⋆G_e=\min_{t\in[T]}\Phi_e(x_t)-\Phi_e^\star also selects the best visited plan retrospectively using ground-truth utilities. The algorithm does not specify how that plan would be identified from noisy observations, so this metric does not directly establish the quality of an implementable final recommendation.

Questions for authors
  1. How can the individual-preference component of eke_k attain the pooled 1/Ntk1/\sqrt{Nt_k} floor in A4(ii), given that each participant contributes only tkt_k observations of their own preference parameter?

  2. What conditions or exploration mechanism ensure A3(i) along the coordinator’s selected trajectory? Also, how is the practical optimization error ξt\xi_t controlled? The claim that it vanishes with full active-set coverage appears incorrect because coordinate descent can remain at a suboptimal local solution.

  3. In joint-coldstart, does posterior rebuilding after a probe refit use observations from previous episodes or only the current episode? If it uses the full history, how should the reported comparison be interpreted as an ablation of preference persistence? For joint-newusers, why are observations from different participants retained under the same preference indices?

2 comments

We thank the reviewer for recognizing the formulation's value for studying how representation learning reduces repeated preference elicitation, and for calling out the breadth of our encoder and ablation experiments. We also appreciate the precise, actionable critiques on the pooled-vs-per-user rate conflation in A4(ii), the full-buffer confound in joint-coldstart, and the optimization-error overclaim. We address each weakness/question next. We have marked all updates in blue text within the manuscript.

W1/Q1. The reviewer is correct: the per-user ϕi\phi_i term cannot attain the pooled 1/NETeff1/\sqrt{NET_{\mathrm{eff}}} floor because each user contributes only observations of their own parameter. Our theorem's dominant term is the WW-term, which does pool across users, so the overall rate is unaffected, but the original A4(ii) mistakenly suggested both rates pool. We have rewritten A4(ii) as two separate conditions:

  • A4(ii-W): ∥W^t−W⋆Pkh∥op\lVert \hat W_t - W^\star P_{k_h} \rVert_{\mathrm{op}} contracts to the pooled statistical floor ρ∞W(mpool)\rho_\infty^W(m_{\mathrm{pool}}) with mpool≍NETeffm_{\mathrm{pool}} \asymp NET_{\mathrm{eff}}.
  • A4(ii-φ): each ∥ϕ^i(t)−Qtϕi⋆∥\lVert \hat\phi_i^{(t)} - Q_t\phi_i^\star \rVert contracts to the per-user floor ρ∞ϕ(mi)\rho_\infty^\phi(m_i) with mim_i the observations specific to user ii.

Numerical regret is unchanged because the WW-term dominates in all our regimes. We thank you for catching this.

Q2. The reviewer is correct: a finite random cold start gives A3(i) only with high probability under the generator, not along the subsequent LCB trajectory. We address this both theoretically and empirically. On the theoretical side, Appendix B.1.2 introduces a forced-exploration variant of Algorithm 1 that mixes an ηt∼t−1/3\eta_t\sim t^{-1/3} uniform-random action into each round, and proves (Proposition 2) a lower bound on the played-design excitation on the top-khk_h subspace: the time-averaged Gram 1t∑s≤tPkh⊤H(xs)H(xs)⊤Pkh\tfrac{1}{t}\sum_{s\le t} P_{k_h}^\top H(x_s) H(x_s)^\top P_{k_h} has smallest eigenvalue at least 12γexp t−1/3\tfrac12\gamma_{\mathrm{exp}}\ t^{-1/3} (equivalently, the unnormalized Gram grows as Ω(γexpt2/3)\Omega(\gamma_{\mathrm{exp}}t^{2/3})). On the empirical side, we measure the excitation along Algorithm 1's actual trajectory (no forced exploration) and find the excitation ratio (normalized by the i.i.d.-random-plans baseline) exceeds 11 on the top-kh50k_h^{50} subspace across all four encoders by episode-boundary 44–55 (Qwen3 ∼5×\sim 5\times, Llama ∼3×\sim 3\times, Gemma ∼2×\sim 2\times, MiniLM ∼30×\sim 30\times; Figure 2, appendix). Together these support Theorem 1's conditional dependence on A3(i): the deployed Algorithm 1 is empirically well-excited on the subspace that carries the dominant regret term, and the forced-exploration variant provides a trajectory-independent excitation guarantee.

W1. The reviewer is right that full active-set coverage does not imply global optimality of coordinate descent. We have removed the "vanishes" claim from Section 3.2 and replaced it with: "the optimization error ξt\xi_t is bounded by the coordinate-descent gap at convergence, which need not be zero even under full active-set coverage."

W2/Q3. The reviewer is correct. Algorithm 1 rebuilds the per-user posterior from the full buffer, so the original coldstart only resets the prior and the historical observations are immediately re-ingested. We now report a clean coldstart (Appendix C.1) that flushes both the posterior and the per-user buffer at each episode boundary, retaining only the WW-refit's encoder-side data. Result: clean-coldstart yields Gtotal=18.14±1.26G_{\mathrm{total}}=18.14 \pm 1.26 (vs joint's 22.06±1.4722.06\pm1.47) and Rtotal=1806±61R_{\mathrm{total}}=1806\pm61 (vs the original coldstart's 1799±581799\pm58); both are ≈2×\approx 2\times joint's 877877, and the paired difference between the two coldstart variants is only +7+7, providing no clear evidence of an additional regret penalty from removing historical posterior replay. Our claim "persistence of per-user posteriors matters for exploration efficiency, not final plan quality" survives this ablation, with the clean coldstart confirming that full-buffer rebuild is nearly inert in this regime.

W2. We agree. This comparison does not isolate "lack of persistent users" because the retained observations are now mislabeled relative to the new ϕi⋆\phi_i^\star. We have reframed its caption to "a label-noise stress test under identity drift".

W2. We now report a GdeployG_{\mathrm{deploy}} metric (Appendix C.3): the quality of the plan selected by Algorithm 1's own LCB-minimax rule at episode end, without oracle access. GdeployG_{\mathrm{deploy}} preserves the mode ordering of GtotalG_{\mathrm{total}} (joint beats non-coldstart baselines by 1.81.8–4.2×4.2\times) and rises by ∼1.1\sim 1.1–2.6×2.6\times in absolute terms, as expected since the deploy plan is not retrospectively best.

Dear Reviewer,

With the discussion period closing soon, we wanted to kindly follow up and ask whether our rebuttal and revisions have adequately addressed your concerns. We would be happy to provide any further clarification if needed.

Thank you for your time and consideration.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General comment to all reviewers and the AC

We thank all three reviewers for their detailed and constructive reviews. They converged on four strengths. All three highlighted the problem formulation: Reviewer SgwH noted that combining persistent participant preferences, changing option sets, and a minimax dissatisfaction objective yields a clean setting for studying how representation learning reduces repeated preference elicitation; Reviewer 27zA called the connection between group coordination, low-rank bilinear bandits, and representation spectral geometry novel and the "key argument in the paper's favor"; Reviewer Jx4K called the formulation "elegant," particularly the choice to require only scalar utility feedback without asking participants to specify preference functions or rank options. Second, two reviewers praised the algorithmic design, with Jx4K highlighting that jointly learning the shared projection and per-user preferences identifies preference-relevant similarities between options and generalizes to unseen decisions, and Reviewer 27zA noting the paper's "discussion of four differences between the theoretical variant and practical algorithm, together with deployment risks," which improved interpretability. Third, the theoretical contribution was recognized: 27zA described the spectral-dimension-dependent regret bound as giving "a testable account of how spectral dimension affects the representation-dependent part of interaction complexity," and Jx4K called the analysis and supporting proofs "valuable contributions that help explain the potential advantages of low-dimensional representations." Finally, all three reviewers positively noted the empirical breadth.

We have run a number of new experiments and revised some of the theoretical claims in the manuscript.

Theorem-to-algorithm gap (SgwH, Jx4K).

SgwH correctly identified that A4(ii) conflates pooled-WW and per-user-ϕ\phi rates, and both reviewers flagged that A3(i) is assumed rather than established for Algorithm 1. We (i) have split A4(ii) into pooled WW-rate and per-user ϕ\phi-rate conditions; (ii) measure sin⁡Θ(W^t,Wfit)\sin\Theta(\hat W_t,W_{\mathrm{fit}}) and the top-khk_h Gram eigenvalue along Algorithm 1's trajectory, showing both stabilize as A3/A4 require (new Figure 2 in Appendix B.1.1); (iii) add Proposition 2 in Appendix B.1.2 giving a trajectory-independent excitation lower bound for a forced-exploration variant with ηt∼t−1/3\eta_t\sim t^{-1/3}, providing a theoretical anchor for the conditional A3(i). We also correct the Section 3.2 claim that the optimization error vanishes with active-set coverage.

Ablation (SgwH).

SgwH correctly noted that joint-coldstart was confounded by the full-buffer posterior rebuild, and joint-newusers by mislabeled retained observations. We add a clean coldstart (flushes both posterior and per-user buffer) and reframe joint-newusers as a label-noise stress test. We also add a GdeployG_{\mathrm{deploy}} metric using Algorithm 1's own LCB-minimax recommendation (no oracle) to complement the retrospective best-visited metric.

Calibration and fixed-W⋆W^\star anomaly (27zA, Jx4K).

We add (i) empirical LCB coverage on the known-utility simulator under both the simplified Algorithm 1 radius and the full theoretical radius (new Remark in Appendix B.2); and (ii) a matched-exploration sweep κ∈{0.5,1,2,4,8,16,32,64,128}\kappa\in\lbrace 0.5,1,2,4,8,16,32,64,128 \rbrace on fixed-true-W⋆W^\star's βt\beta_t (new Figure 3 in Appendix C.2), which distinguishes representation-learning benefit from baseline miscalibration; no κ\kappa closes the gap.

Other items (Jx4K, 27zA).

We add SOTA baselines (GLM-dueling and bilinear-bandit ESTR) matched to our interaction budget on the simulator (Appendix C.5), a controlled khk_h sweep varying W⋆W^\star's singular-value profile at fixed norm (decouples rank from SNR, Appendix C.4), and a nonlinear-utility robustness sweep (tanh⁡\tanh-perturbed utilities under two parameterizations, Appendix C.6) showing the joint-advantage ratio degrades gracefully from 4.5×4.5\times to 2.8×2.8\times as nonlinearity grows.

Per-reviewer replies address each question in order. The major revisions are marked in blue in the revised manuscript.

AuthorsOpenReview ↗

Summary of Discussion and Changes for AC

Dear Area Chair,

We thank the AC and reviewers for the thorough engagement during the discussion period. A brief summary of the current state of the discussion:

  • Reviewer 27zA (confidence 2) acknowledged that our two added experiments, the empirical LCB-coverage analysis (Remark 3, Appendix B.2) and the nonlinear-utility robustness sweep (Appendix C.6), address their questions, and raised their rating to 3.
  • Reviewer SgwH (confidence 3) acknowledged the split of A4(ii) into pooled-WW and per-user-ϕ\phi rates, the correction to the optimization-error claim, and the empirical and theoretical treatment of A3(i), and raised their rating to 3.
  • Reviewer Jx4K (confidence 2) raised nine items. Each is addressed in the rebuttal and the revised manuscript (marked in blue): pooled-vs-per-user rate split (W2), SOTA baselines GLM-dueling and ESTR (W3, Appendix C.5), matched-exploration κ\kappa sweep for fixed-true-W⋆W^\star (W4, Appendix C.2), limitations paragraph on strategic manipulation (W5), controlled khk_h sweep decoupling rank from signal-to-noise (W6, Appendix C.4), and abstract wording (W9). The one item that we could not address in the rebuttal window, real-human-preference validation (W1), is explicitly flagged in the Limitations and Future Work paragraph. The reviewer did not engage further after our rebuttal or after a subsequent reminder.

Thank you for your help in handling the manuscript’s review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.