acceptodds
Under review as a conference paper at ICLR 2027

Reader-Oriented Evidence Reshaping: Improving RAG from the Reader’s Perspective

Abstract

Retrieval-Augmented Generation (RAG) enhances large language models by incorporating external evidence. However, existing post-retrieval processing methods remain largely evidence-centric approaches that tend to ignore how the target Reader interprets and utilizes the resulting representation. Insights from prompt engineering suggest that different representations of the same information lead to significant variations in model outputs. Therefore, we propose Reader-Oriented Evidence Reshaping (ROER), a framework designed to explicitly align the representation of evidence with the intrinsic reasoning preferences of the target Reader. Instead of relying on a fixed processing strategy, our framework represents evidence organization as executable Evidence Organization Programs (EOPs) and evolves candidate programs guided by feedback from the target Reader. These evolved programs dynamically determine how retrieved evidence is selected, structured, and presented. To support this, we introduce a multi-view retrieval module that integrates evidence from multiple perspectives, supplying diverse evolutionary materials for subsequent evidence reshaping. Experiments on CosmosQA, QuALITY, and LongBench-v2 demonstrate that ROER delivers consistent improvements over competitive baselines across short-context, document-level, and ultra-long-context reasoning settings. In particular, ROER achieves a 21.21% relative accuracy improvement over the strongest baseline in the ultra-long-context setting, highlighting its particular strength in complex long-context reasoning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
RAG, Evidence Reshaping, Reader-Oriented Optimization, Long-Context Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Y9PpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Reader-Oriented Evidence Reshaping (ROER), a framework for optimizing retrieved evidence for a frozen language-model reader. It combines multi-view retrieval with executable Evidence Organization Programs (EOPs), which compose operators for evidence selection and organization. Candidate programs are evolved offline using reader-derived utility, including proposals informed by execution failures. A learned selector then chooses one program per query, requiring one EOP execution and one reader call at inference time.

Experiments evaluate 200 examples each from CosmosQA, QuALITY, and LongBench-v2. With Qwen3-8B, ROER achieves the highest average accuracy and the best results on two benchmarks. Its largest improvement is on LongBench-v2: 40% versus 33% for the strongest reported baseline. Additional experiments evaluate two other readers, component ablations, and inference latency.

Critical strengths
  1. A practical design for adapting evidence processing while keeping the reader frozen. The constrained program space makes the transformations explicit, while offline search and a lightweight selector avoid searching over programs at inference time. This is a useful engineering contribution: it offers an interpretable way to customize evidence processing without updating the reader’s parameters.

  2. Encouraging long-context results across several readers. The reported LongBench-v2 improvement from 33% to 40% with Qwen3-8B is meaningful enough to warrant investigation. ROER also outperforms the evaluated baselines with the other two readers, and removing the reshaping module reduces accuracy from 40% to 30.5%. These results support the usefulness of the overall system, subject to the protocol and attribution concerns below.

Critical weaknesses
  1. The conceptual novelty is overstated, and the specific increment over existing work is not sufficiently established.

    Optimizing retrieved context for a downstream reader is an established research direction. RECOMP trains compressors using downstream language-model performance, and PRCA explicitly uses a frozen generator’s feedback to optimize an adapter between retrieval and generation. FaviComp, which this paper evaluates, already adapts compressed evidence to the target model’s familiarity. Evolution guided by execution traces also has a clear precedent in GEPA.

    Against this background, ROER’s potentially distinctive contribution is its compositional EOP search space and query-dependent program selection. However, the experiments do not separate the benefits of evolution from those of the eight manually designed initial programs, or adaptive selection from simply using the best fixed program. Furthermore, EOPs change evidence content and coverage as well as presentation, so the reported gains cannot yet be attributed specifically to alignment with a reader’s representation preferences.

    Theorem 1 does not substantially strengthen this argument. Its condition Sψ1→ψ2(E~;E1∗)>Gψ1(E~;E1∗)S_{\psi_1\rightarrow\psi_2}(\widetilde E;E_1^*)>G_{\psi_1}(\widetilde E;E_1^*) is algebraically equivalent to L(E~;Mψ2)<L(E1∗;Mψ2)L(\widetilde E;M_{\psi_2})<L(E_1^*;M_{\psi_2}). The result is correct, but it restates the existence of a better representation under the second reader rather than explaining when such a preference reversal should occur or why ROER will discover it.

  2. The evaluation leaves important uncertainty about independence, comparability, and robustness.

    Appendix C.1 states that each dataset’s original public training split is used for optimization and its public validation split for evaluation. However, the official LongBench-v2 release contains one split named train, comprising 503 benchmark examples, rather than separate public training and validation splits. A custom split could be legitimate, but the actual partition and its independence from EOP evolution, failure-trace collection, and selector tuning need to be documented before the main result can be assessed confidently.

    The comparison protocol also needs clarification. ROER limits both intermediate evidence packing and final reader evidence to 8,000 characters, while corresponding evidence access, output budgets, model settings, and timing boundaries for the baselines are not sufficiently documented. Consequently, the accuracy and latency tables do not isolate the effect of evidence organization.

    Finally, each benchmark uses only 200 evaluation examples, without uncertainty estimates or repeated sampling. The headline LongBench-v2 improvement corresponds to 14 additional correct answers; the CosmosQA improvement over the strongest baseline corresponds to three. These gains may be real, but paired uncertainty estimates are needed to assess their robustness. On QuALITY, ROER trails the strongest baseline, so the claims should also distinguish improved average performance from improvement on every benchmark.

Questions for authors
  1. Evaluation protocol clarification: What exact LongBench-v2 examples were used for EOP evolution, selector training and validation, and final evaluation? Please provide the split construction, seeds, and example IDs, and clarify whether any evaluation examples or their labels influenced program proposals, failure traces, or tuning.

  2. Evidence needed to assess the claimed increment: With the retrieved evidence pool and output budget held fixed, how does ROER compare with the best fixed EOP and a selector trained only over the eight initial EOPs? Does transferring an EOP bank and selector between readers measurably reduce performance relative to optimizing them for the destination reader? These comparisons would directly affect my assessment of whether evolution and reader-specific adaptation provide distinct benefits.

  3. Comparison and robustness clarification: What evidence inputs, length budgets, exact model checkpoints, and decoding settings were used for each baseline? What stages are included in the latency measurements? Please also provide paired uncertainty estimates for the main accuracy differences and report the offline reader-call and proposal-model costs.

Reviewer_bke8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studies how retrieved evidence can be better organized to match the reasoning preferences of different language model Readers in RAG systems. The authors argue that existing post-retrieval methods primarily focus on evidence relevance and information preservation, while overlooking how effectively the target Reader utilizes the resulting representations. To address this, they propose Reader-Oriented Evidence Reshaping (ROER), which combines multi-view retrieval with executable Evidence Organization Programs (EOPs). These programs are evolved using feedback from the target Reader to determine how evidence should be selected, structured, and presented. A learned selector then chooses an appropriate program for each query. Experiments on CosmosQA, QuALITY, and LongBench-v2 show improvements over existing baselines, particularly on LongBench-v2, where ROER achieves a 21.21% relative accuracy improvement over the strongest baseline. Additional experiments across different Reader models suggest the potential of Reader-oriented evidence organization.

Critical strengths
  1. The paper addresses an important problem in RAG, highlighting that relevant evidence does not necessarily guarantee effective utilization by the target Reader. The Reader-oriented perspective provides an interesting direction for improving post-retrieval processing.

  2. Representing evidence organization strategies as executable programs is an interesting design. The combination of evolutionary optimization and adaptive program selection allows the framework to flexibly construct different evidence representations without modifying the target Reader.

  3. The experiments cover multiple benchmarks with different context lengths and several Reader models. The improvements on LongBench-v2 are promising, and the inference latency results suggest that the proposed framework can be relatively efficient.

Critical weaknesses
  1. My main concern is about the evaluation protocol. Appendix C.1 states that the original training and validation splits are used for EOP optimization and evaluation, respectively. However, LongBench-v2 does not provide a separate official training/validation split in its standard release. It is unclear how the authors construct these splits and ensure that training and evaluation examples do not overlap. This should be clarified.

  2. The central claim of Reader-specific evidence alignment is not sufficiently validated. Although the method achieves improvements across different Readers, this does not necessarily demonstrate that the learned programs capture Reader-specific preferences. Evaluating programs optimized for one Reader on other Readers would provide more convincing evidence.

  3. The evaluation is conducted on only 200 examples per dataset, which raises concerns about the stability and reliability of the reported results, especially given the relatively small improvements over some baselines.

  4. The theoretical analysis is relatively straightforward. The main theorem essentially reformulates the observation that an evidence representation optimal for one Reader may not remain optimal for another, with the proposed condition being algebraically equivalent to the conclusion. While this provides some intuition for Reader-oriented optimization, it offers limited insight into when such differences arise or why the proposed EOP evolution is effective.

Questions for authors

see weakness

Reviewer_wa5ZOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Reader-Oriented Evidence Reshaping (ROER), a RAG framework that adapts retrieved evidence to a fixed, frozen answering model. It combines multi-view retrieval with executable Evidence Organization Programs (EOPs), which specify how evidence is selected, organized, and presented. Candidate programs are evolved offline using the target Reader’s task performance and a representation-cost penalty. A trained selector then chooses one program for each new query before execution, avoiding exhaustive program evaluation during inference.

Validation covers CosmosQA, QuALITY, and LongBench-v2, primarily using Qwen3-8B, with additional Reader comparisons using DeepSeek-8B and GLM-4-9B on LongBench-v2. ROER achieves the highest average accuracy across the three benchmarks and improves LongBench-v2 accuracy from the strongest baseline’s 33% to 40%, although it does not lead on QuALITY. Component ablations, online latency measurements, and cross-Reader comparisons support the usefulness of adaptive evidence organization. The central contribution is treating evidence presentation as a Reader-dependent optimization problem and learning efficient query-specific selection among reusable transformation programs.

Critical strengths

The long-context results show a meaningful empirical benefit from reshaping. On LongBench-v2, ROER improves accuracy from the strongest baseline’s 33% to 40%, with improvements also reported for DeepSeek and GLM Readers. Removing the reshaping module reduces accuracy from 40% to 30.5%, compared with 36.5% when removing multi-view retrieval. These results support evidence organization as an important contributor to performance, even though they do not fully isolate Reader-specific adaptation.

Critical weaknesses

W1. The central claim of Reader-specific adaptation is not convincingly isolated. Improvements across several Readers, differing baseline rankings, and illustrative examples do not establish that programs optimized for one Reader are specifically better suited to that Reader. The reported ablations remove entire modules rather than isolate Reader-specific optimization, program evolution, or adaptive selection. Cross-Reader program-bank comparisons and a comparably optimized fixed-program baseline would help distinguish Reader adaptation from generally better evidence processing or additional task supervision. Without this distinction, the experiments under-support the paper’s central explanation for its gains.

W2. The practical advantage is not established under comparable computational resources. ROER requires offline program generation, repeated evaluation using the target Reader, and selector training, whereas the reported efficiency comparison measures online inference. The comparison does not establish the total preparation cost, the query volume needed to amortize it, or whether this optimization must be repeated for each Reader or task. This does not invalidate the online latency measurements, but it leaves the benefit of the complete pipeline unresolved. Given that efficient Reader adaptation is a principal motivation, the added optimization complexity requires a clearer resource–performance justification.

Questions for authors

Q1. Which existing comparisons establish that the learned programs are specifically suited to their target Reader? Were program banks evolved separately for each Reader, and were cross-Reader bank transfers or comparisons against the best fixed EOP evaluated under matched optimization budgets?

Q2. Which EOP operators invoke auxiliary LLMs, and which models perform these operations? Can transformations introduce inferred facts or candidate answers, or are they restricted to evidence-supported content? How is factual preservation assessed, so that gains can be attributed to presentation rather than additional reasoning by another model?

Q3. Which data splits are used for program evolution, utility evaluation, and selector training, and how are they separated from final testing? What are the sample counts for each benchmark, and do the reported results use complete evaluation sets or selected subsets?

Q4. Were program evolution and selector training repeated across random seeds? What uncertainty estimates or paired significance tests support the accuracy gains, particularly the smaller improvements for DeepSeek and GLM? Do the selected programs and their relative performance remain stable across runs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.