acceptodds
Under review as a conference paper at ICLR 2027

Learning Marginal Viewpoint Utility for Budgeted Selection in RAG

Abstract

Retrieval-augmented generation over multi-viewpoint queries must preserve distinct answers and positions from the retrieved evidence while operating under a fixed context budget. We introduce LMVU, a lightweight ranker that learns to prioritize candidate passages by the additional viewpoint coverage they contribute to the selected context. Training uses viewpoint annotations on training examples from each benchmark to compare candidate passages added to the same context. At inference, LMVU repeatedly scores passages that fit the remaining context budget as the context grows, without accessing these annotations. We evaluate recovered viewpoint coverage in terms of breadth () and repeated support from distinct sources (). Exact oracles measure the maximum coverage achievable from the same evidence pool within the same budget. Across four viewpoint-annotated benchmarks at a 1,024-token context budget, LMVU achieves the highest average coverage on both metrics. It exceeds the strongest non-oracle baseline average by points and points, and reaches the exact viewpoint-breadth ceiling on two benchmarks.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Retrieval-augmented generation, evidence selection, viewpoint coverage, learning to rank, budget-constrained retrieval, diverse evidence

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MnGYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to select evidence under a token budget when a query has several relevant viewpoints. LMVU uses a small linear model to score passages by the viewpoints they add to the evidence already selected. Its features describe passage relevance, geometric relations to evidence and viewpoint atoms, and the selection state. The model uses frozen embeddings and pairwise supervision derived from viewpoint coverage. Experiments on four datasets measure coverage breadth and depth, with further analyses of budget, state conditioning, transfer, and oracle behavior. The evaluation focuses on evidence selection; it does not test downstream answer generation.

Critical strengths
  1. Selecting passages by relevance alone can miss alternative answers or perspectives. LMVU addresses this with a linear model over frozen embeddings, which is inexpensive to use, and supervision based on marginal viewpoint coverage. The paper also acknowledges that its budget-feasible greedy procedure is a heuristic and does not claim an unproven approximation guarantee.

  2. The evaluation provides useful detail beyond average performance. Table 1 shows better average breadth than MMR at the primary budget, while also showing where LMVU is not the best method. The appendix reports uncertainty and transfer results, and explains why state-constant features cancel in pairwise comparisons. The Perspective coverage-depth interval is above zero even though the breadth evidence is less conclusive.

Critical weaknesses
  1. The objective gives equal weight to all annotated viewpoints, regardless of their prevalence or credibility. This is reasonable for queries asking for a range of perspectives, but may be less appropriate when users want the dominant view or the best-supported answer. The evaluation measures coverage of annotated viewpoints, without testing whether the resulting answers are more complete while still accurately reflecting the evidence. The paper should make the intended query scope clearer and avoid treating higher coverage as a general improvement in answer quality.

  2. The experiments leave uncertainty about both practical efficiency and the size of the gains. LMVU updates features and rescores candidates after each selection, but the paper reports training time rather than selection latency or scaling with candidate-pool size. Some benchmarks also have few independent queries. In addition, JPR and RichRAG use adapted scoring procedures and consider only the top 64 candidates, while LMVU can access larger pools. These differences make it difficult to determine how much improvement comes from the learned ranking rule and what additional inference cost it requires.

Questions for authors
  1. Which types of queries are intended to benefit from equal-weight viewpoint coverage? How should the method be used when viewpoint prevalence or evidence credibility matters? Please clarify whether the claim is limited to evidence coverage or also extends to final-answer quality. Evaluating downstream answers would be useful future work.

  2. Are selection-time measurements available, including feature updates and repeated scoring? How does latency compare with static ranking and MMR as the candidate pool and number of selected passages grow? Please distinguish these measurements from the reported training times; a new large-scale benchmark is not required for the rebuttal.

  3. Are results available with identical candidate access for LMVU, JPR, and RichRAG? Given the small evaluation sets, can the existing results also be summarized with paired uncertainty at the independent query or topic level?

Reviewer_DeyKOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies budget-constrained retrieval-augmented generation for questions whose answers require multiple viewpoints. Under a limited context budget, relevance-based rerankers may repeatedly select passages supporting the same viewpoint, and semantic diversification methods such as MMR do not guarantee viewpoint coverage, so the selected context can miss viewpoints needed to answer the question. The authors propose LMVU, a lightweight linear ranker trained on viewpoint annotations to predict each candidate’s marginal viewpoint gain given the currently selected context, and use it for greedy selection within the budget. Across four viewpoint-annotated benchmarks, the authors show that LMVU achieves the highest average viewpoint coverage compared with other neural reranking baselines.

Critical strengths
  1. The authors identify and formalize a problem in retrieval augmented generation that purely based on semantic similarity, under a budget limited condition, the context window may be occupied by the contents share the viewpoint. I found the problem itself can be meaningful in some specific circumstances that require context diversity.
  2. The author propose LMVU which provides a computationally efficient approach to evidence selection by training a lightweight linear ranker on viewpoint-labeled data. Using predefined relevance, length, and similarity features, it learns to select document that can provide higher marginal viewpoint benefits to the current context window.
Critical weaknesses
  1. The usefulness of proposed method in practical RAG setting not clear. The main experiments rely on all-MiniLM-L6-v2, a lightweight embedding model, which provides limited evidence that the identified viewpoint-coverage problem exists with stronger dense retrievers. Although the authors include additional experiments with bge-base and bge-m3, these use a different candidate-pool and the improvements are not consistent. Also, these is no evaluation and experiments of how the proposed problem will affect the downstream generation.

  2. The main evaluation's scale is limited, with only six evaluation queries each for WVS and ArgKP. The six queries in WVS are only correspond to three topics with two paraphrases each. The scale of the experiment is far blow the standard RAG[1] and reranking[2] evaluation. At the same time, the datasets are adapted from survey and argumentations resources, but the paper provides limited validation that resulting queries and synthetic task. It remains questionable that the provided evaluation can represent realistic multi-viewpoint RAG tasks.

  3. The paper motivates viewpoint coverage in the context of RAG, but does not evaluate downstream generation quality. So, it is unclear whether improved coverage of annotated viewpoints leads to more comprehensive or useful answers.

[1] Active Retrieval Augmented Generation Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, Graham Neubig

[2] FIRST: Faster Improved Listwise Reranking with Single Token Decoding Revanth Gangi Reddy, JaeHyeok Doo, Yifei Xu, Md Arafat Sultan, Deevya Swain, Avirup Sil, Heng Ji

Questions for authors
  1. Could the authors evaluate stronger and more diverse and modern embedding backbones under the same candidate-pool for the main experiments?

  2. Could the authors include established benchmarks such as ASQA [1] to evaluate both evidence coverage and downstream answer completeness? Although ASQA focuses on multiple interpretations and answers rather than opinion diversity, it would provide a complementary test of whether LMVU’s coverage gains can result in better generation.

  3. Could the authors provide evidence that improved viewpoint coverage may affect downstream generation? It is surprising that a paper motivated by RAG and even with RAG in the paper title provides neither quantitative nor qualitative evaluation of its performance on downstream generation.

[1] ASQA: Factoid Questions Meet Long-Form Answers Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, Ming-Wei Chang

Reviewer_mAufOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LMVU is a small linear reranker that picks passages one at a time under a token budget, favoring passages that add viewpoints not yet covered by the selected set. It is trained with a pairwise loss on dev-set viewpoint labels (which of two candidates adds more new viewpoints) and run as greedy selection at inference, without labels. On four viewpoint-annotated benchmarks at 1,024 tokens it has the highest average viewpoint coverage and hits the oracle on two of them.

Critical strengths
  1. The problem is real: relevance ranking and MMR fill the budget with passages for the same viewpoint, and the exact oracles show how much room is left.
  2. The method is cheap (1.3 s CPU fit vs. 962 s GPU for RichRAG) and needs no labels at inference.
  3. Paired CIs and sign tests are reported, which is more than most papers in this space do.
Critical weaknesses
  1. Novelty is unclear. Diversity-aware reranking that conditions on what is already selected is an old idea in retrieval and recommendation. MMR is a special case of LMVU (Remark 8), and learned versions (R-LTR [1], structural SVM diversification [2]) do the same thing with a linear scorer and greedy. RVC is subtopic recall [3]. None are cited. The twist here is the viewpoint label; the rest of the pipeline is standard.
  2. A straightforward baseline is missing. An ordinary reranker (e.g. the paper's own M0) trained with an added viewpoint-diversity term on the same labels and budget would be the direct comparison. Without it I cannot tell whether the gain comes from the state-dependent design or just from labels the baselines never see.
  3. The practical benefit is not shown. Covering more labeled viewpoints does not mean a better answer, and RVC is exactly what the loss supervises. The author should show that LMVU's selection improves answer completeness without hurting correctness, with a generator in the loop. Whether coverage matters also depends on how each benchmark defines a viewpoint, which is not discussed.
  4. The margin over MMR is within noise. No CI in Table 12 excludes zero, WVS has 6 queries shared between dev and eval, and half of the average gain comes from one WVS topic (Sec. 5.7). JPR and RichRAG only score the top-64 while LMVU scores the full pool, so that comparison is not fair either.
Questions for authors
  1. What does LMVU add over R-LTR / PAMM style learned diversification, beyond the label?
  2. Train M0 or a standard reranker with a viewpoint-diversity term on the same labels and budget. Does LMVU still win?
  3. Any downstream result, i.e. an LLM answering from LMVU vs. MMR context, scored on viewpoint completeness and correctness, even on one benchmark?
  4. Leave-one-benchmark-out model evaluated on the protected WVS fold, and JPR / RichRAG rerun on the full pool.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.