acceptodds
Under review as a conference paper at ICLR 2027

Realize What Matters: Principled Context Representation for Large-Scale Reasoning

Abstract

Solving complex tasks in domains such as science, medicine, law, and finance often requires assembling interdependent information scattered across vast, heterogeneous sources far beyond model context limits. Existing approaches tackle this challenge by organizing information into more manageable representations over which models can reason, such as graphs, textual memories, and retrieval collections. These representations dictate what downstream reasoning is possible and, ultimately, whether it succeeds; yet their design and construction remain largely ad hoc. In this work, drawing on the cognitive theory of relevance realization, we propose concrete principles for designing AI systems that construct effective representations of very large contexts. We analyze existing approaches and show how their successes and failures map onto their alignment with these principles, and introduce R3Con, a harness designed to operationalize the principles more systematically. We evaluate R3Con against nine state-of-the-art baselines on two recent benchmarks of reasoning over large document corpora. On these benchmarks, R3Con substantially outperforms the strongest baseline, by and percentage points. It also enables smaller models to outperform much larger ones: R3Con with 4B and 9B models outperforms all evaluated 35B baselines, while R3Con with a 35B-A3B model outperforms Claude Code with Claude-Sonnet-5 at lower cost. Our results show that context representations following our principled approach can reduce reliance on model scale, pointing toward a future of AI systems with frontier-level performance powered by smaller models. Our code is available at https://anonymous.4open.science/r/r3con-evaluation-anon.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long-context Reasoning, Context Representation, R3Con, LLM agents, Relevance Realization, External Memory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_r3j5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes principles for constructing task-specific representations of large document collections, inspired by relevance realization. R3CON implements these through iterative document summarization, task-conditioned schema generation and extraction, and downstream code-based reasoning. On CorpusQA and Loong, it reports performance gains over the strongest evaluated same-backbone baselines, with additional experiments on smaller models, cost, and component ablations.

Critical strengths
  1. The same backbone improvements are substantial across two benchmarks and several domains. Smaller-model experiments further support the practical value of explicitly constructing task-relevant representations.

  2. Ablations show that both relevance extraction and structuring contribute to accuracy. The intermediate representations and detailed examples also make the pipeline and its behavior relatively easy to inspect.

Critical weaknesses
  1. The evidence for the proposed principles is less conclusive than the overall performance results. Figure 13 shows nearly identical accuracy for N=1N=1 and N=2N=2, so cross-round reinterpretation has limited demonstrated benefit on these benchmarks. Removing entire stages also changes computation and available information, rather than isolating the principles themselves.

  2. The efficiency and frontier-model claims need qualification. R3CON rebuilds representations for every query, whereas several baselines can reuse their indexes; multi-query amortized costs are not evaluated. Moreover, the roughly one-point advantage over Claude Code with Sonnet-5 is reported without a paired uncertainty analysis, making the superiority in accuracy less convincing than the cost advantage.

Questions for authors
  1. How does the accuracy-cost tradeoff change when answering multiple queries over the same corpus and amortizing baseline indexing costs?

  2. Does the accuracy difference from Claude Code with Sonnet-5 remain reliable under paired evaluation and repeated runs?

1 comment

Given that the discussion period ends in two days, we would greatly appreciate any additional comments or feedback while there is still time for us to clarify any remaining points. We are more than happy to engage in further discussion.

Reviewer_mnuiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper contributes R3Con, a new context representation method for large-scale reasoning.

Critical strengths
  • The paper is overall well written. The arguments and figures/tables are presented well.
  • The evaluation seems to be comprehensive -- many baselines compared (with multiple base models, too).
  • The performance improvement is significant. (e.g., Table 1)
Critical weaknesses

I don't see critical weaknesses. However, for Table 1, it is helpful to expand baselines into conceptual diagrams and show how conceptually the proposed method, R3Con, can be ablated into these baselines. With this, more targeted ablation studies (on specific components of R3Con) can be performed.

Questions for authors
  • The figures in the paper are beautiful. Did you use AI to create them (and which)? Many icons/symbols are used in these figures, did you create them by yourself?
  • Does it make sense to expand the style and domain of the evaluation dataset (e.g., long-horizon coding tasks)?
1 comment

Given that the discussion period ends in two days, we would greatly appreciate any additional comments or feedback while there is still time for us to clarify any remaining points. We are more than happy to engage in further discussion.

Reviewer_5TJfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes R3CON, a method for answering questions over document corpora which is too large to fit in a model's context. The authors argue that existing approaches build representations of the corpus in ad hoc ways, without regard to the task, so the resulting representation can discard the information needed to derive answer for a specific task. R3CON solves this by summarizing each document with respect to the task. After that, it summarizes each document again with the other documents' summaries as context. Second, the model proposes a schema for the task and fills it from each document. Lastly, a coding agent reads the summaries and the filled schema and produces the answer. The authors conduct experiments on CorpusQA and Loong, with Qwen3.5-35B for R3CON and nine baselines. Results show that R3CON outperforms the strongest baseline, With 4B and 8B models as backbone, it still beats all 35B baselines. Ablations show that removing task-focused summarization or structuring costs the performance.

Critical strengths

Experiments show R3CON beats the strongest baseline by +19.9 on CorpusQA and +8.4 on Loong, with standard errors reported for every cell. The comparison is fair that all baselines and R3CON use the same model and the smaller model experiments in Table 8 shows that the gain is not just from the backbone. R3CON with a 4B model beats every baseline running on 35B. The ablation studies show both stages are important.

The method is simple, it's three prompts plus a coding loop, without training.

Critical weaknesses

The paper’s key claim, iterative reinterpretation. does not have major effect. According to Figure 13, accuracy goes from 52.6->69.9->70.6 on CorpusQA and 24.0->36.9->37.4 on Loong. In round 1 the other summaries are empty, round 2 is the first time a document is reread with others in the context, that step gives +0.7 and +0.5, inside the standard errors. Nearly all the improvement comes from round 1, which is task-focused summarization from Dang[1].

The problem setting is different from what the baselines are built for. RAG index the corpus once and for every query it retrieves a small part of it, whereas R3CON reads every document in full three times per query and produces nothing reusable. Comparing them on accuracy shows that reading everything with the question in hand beats retrieving a fraction, not that R3CON builds a better representation. The missing baseline is summarize each document with the question, concatenate, answer. Otherwise the performance gain cannot be attributed to the method.

[1] Hoa Trang Dang. Overview of DUC 2005.

Questions for authors
  1. Figure 13 shows that going from N=1 to N=2 adds -.7 and 0.5 points on different datasets, both inside standard errors. In this case, what evidence supports the claim that reinterpretation is what makes R3CON work, as opposed to task-focused summarization alone?

  2. R3CON reads every document in full three times per query. Can the authors add a simple baseline with the same reading budget: summarize each document with the question, put all the summaries together, and have the model answer from them directly?

  3. Several baselines were run with hyperparameters far from their defaults (RAPTOR leaf 100→2,000 tokens, ReadAgent page 600→6,000 words, HippoRAG2 8,000-token chunks) for compute reasons, and five baselines score under 12% on CorpusQA. Can the authors confirm that under this setting the baselines can even reproduce its published numbers on a benchmark it was designed for? How do the strongest baselines perform with their best-performing hyperparameters?

2 comments

Given that the discussion period ends in two days, we would greatly appreciate any additional comments or feedback while there is still time for us to clarify any remaining points. We are more than happy to engage in further discussion.

With the discussion closing today, we wanted to ask whether you have any remaining questions. We’d be happy to clarify any points while we still can.

If our responses have addressed your concerns, we would appreciate hearing whether you would consider raising your score.

Thank you for your time in reviewing our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Rebuttal

We thank all reviewers for their feedback. We are glad that the reviewers recognized the substantial gains of R3Con (5TJf, r3j5, mnui), i.e., approximately 20pp and 8.5pp in accuracy on two benchmarks and various domains compared to nine SOTA methods. We also appreciate that the reviewers found our evaluation to be comprehensive and convincing (mnui, 5TJf), recognized the strong results of R3Con with smaller models (5TJf, r3j5), and highlighted the inspectability of R3Con (r3j5) and the presentation of the paper (mnui).

To recap, we have addressed the comments raised in the reviews with new experiments, analyses, and clarifications. The table below summarizes our responses; full details are in our replies to each reviewer.

Reviewer commentOur response
Iterative reinterpretation appears to have limited effect (5TJf, r3j5)We conducted a new full-benchmark ablation on Loong isolating iterative reinterpretation by removing the structuring phase. We show that increasing the number of rounds from 1 to 3 improves accuracy from 21.2% → 26.7% → 30.2% with Qwen3.5-4B, and 30.3% → 33.0% → 35.3% with Qwen3.5-35B-A3B. These results show consistent benefits from additional reinterpretation rounds.
Suggested additional task-focused summarization baseline (5TJf)We implemented the suggested baseline and evaluated it on the full Loong benchmark. R3Con achieves 37.43% vs. 30.31% (+7.12 pp), providing further evidence that its improvements go beyond task-focused summarization alone.
Suitability of RAG baselines for the problem setting and budget (5TJf)We clarified that evaluating RAG-based methods on Loong follows prior work, including StructRAG (ICLR 2025). Our evaluation also includes memory agents, coding agents, and other sophisticated graph- and RAG-based approaches. Also, we highlighted how R3Con with smaller models outperforms baselines at comparable or lower inference cost, with 6.58× and 2.52× cost reductions on Loong and CorpusQA, respectively (relative to the 35B config; Figure 6, Table 8).
Robustness of scaled-up baseline hyperparameters (5TJf)We explained that we scaled hyperparameters to accommodate larger corpora, following prior work (e.g., Lee et al., ICML 2024). To demonstrate these choices, we also ran new experiments with the original default settings on 83 financial CorpusQA tasks. Our settings achieve higher accuracy for all three baselines: RAPTOR 11.0% vs. 4.8%, HippoRAG2 6.0% vs. 4.8%, and ReadAgent 49.4% vs. 39.8%. These results support that our scaled-hyperparam choices improve, rather than weaken, baseline performance.
Amortized costs of baselines (r3j5)We analyzed the five baselines that support representation reuse: on Loong, reusing representations would reduce their total costs by 1.1–1.5×. We will include this analysis and the corresponding CorpusQA analysis in the appendix. Notably, the four baselines with reusable offline indices are also the four lowest-performing baselines on Loong. This is consistent with our discussion of the importance of being task-specific in realizing relevance (empirically and theoretically).
Paired uncertainty analysis against Claude Code (r3j5)We ran a paired McNemar's test and the accuracy difference between R3Con using the 35B-A3B model (71%) and Claude Code with Sonnet-5 (70%) or Sonnet-4.6 (67%) is not statistically significant. We will clarify this and emphasize that R3Con achieves comparable accuracy at 3.7–3.9× lower cost, using smaller open-source models (Qwen3.5-35B-A3B vs. Claude Sonnet 5).
Conceptual diagram comparing R3Con with the baselines (mnui)We will add a diagram showing the shared components and conceptual differences between R3Con and the baselines in the appendix to make this Table 1 even more interpretable.
Additional application domains (mnui)We discussed potential applications to large codebases, server and unit-test logs, and safety monitoring of multi-agent swarms through their logs and shared filesystems. We plan to include such discussions of future applications and impact of R3Con in the revised paper.

We believe these additional experiments and clarifications have thoroughly addressed the points raised in the reviews. We thank the reviewers and Area Chair again for their time and consideration. We remain happy to address any comments with the time left in our discussion phase.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.