- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes principles for constructing task-specific representations of large document collections, inspired by relevance realization. R3CON implements these through iterative document summarization, task-conditioned schema generation and extraction, and downstream code-based reasoning. On CorpusQA and Loong, it reports performance gains over the strongest evaluated same-backbone baselines, with additional experiments on smaller models, cost, and component ablations.
Critical strengths
-
The same backbone improvements are substantial across two benchmarks and several domains. Smaller-model experiments further support the practical value of explicitly constructing task-relevant representations.
-
Ablations show that both relevance extraction and structuring contribute to accuracy. The intermediate representations and detailed examples also make the pipeline and its behavior relatively easy to inspect.
Critical weaknesses
-
The evidence for the proposed principles is less conclusive than the overall performance results. Figure 13 shows nearly identical accuracy for and , so cross-round reinterpretation has limited demonstrated benefit on these benchmarks. Removing entire stages also changes computation and available information, rather than isolating the principles themselves.
-
The efficiency and frontier-model claims need qualification. R3CON rebuilds representations for every query, whereas several baselines can reuse their indexes; multi-query amortized costs are not evaluated. Moreover, the roughly one-point advantage over Claude Code with Sonnet-5 is reported without a paired uncertainty analysis, making the superiority in accuracy less convincing than the cost advantage.
Questions for authors
-
How does the accuracy-cost tradeoff change when answering multiple queries over the same corpus and amortizing baseline indexing costs?
-
Does the accuracy difference from Claude Code with Sonnet-5 remain reliable under paired evaluation and repeated runs?
Given that the discussion period ends in two days, we would greatly appreciate any additional comments or feedback while there is still time for us to clarify any remaining points. We are more than happy to engage in further discussion.