acceptodds
Under review as a conference paper at ICLR 2027

ParSer: Read in Parallel, Reason in Depth for Long-Context LLM Agents

21 views

Abstract

Reasoning over documents far beyond an LLM's context window remains challenging, as evidence may be sparsely distributed across hundreds of thousands of tokens. Sequential memory agents stream documents chunk by chunk into a compact recurrent memory, enabling bounded-context processing of arbitrarily long inputs. However, because every reading step immediately updates the memory on which subsequent processing depends, these agents couple document traversal with sequential reasoning. This coupling makes the reasoning sensitive to evidence position and forces the sequential inference path to grow with document length. We introduce **ParSer** (**Pa**rallel **R**eading, **Se**quential **R**easoning), a framework that separates document reading from question reasoning. To realize this separation, ParSer assigns local reading to chunk-bound subagents and global reasoning to a lead agent. The subagents read in parallel under the lead agent's queries; the lead agent integrates their findings and iteratively refines its queries as evidence accumulates. This decoupled design concentrates all learnable behavior in the lead agent and optimizes it with reinforcement learning, while the lightweight chunk readers remain frozen off-the-shelf models. On multi-hop QA with contexts ranging from 7K to 896K tokens, ParSer with a 4B backbone beats the strongest sequential memory baseline by 5.7 points on average and by 12.0 points at 896K tokens. With a 9B backbone, ParSer surpasses DeepSeek-V4-Pro by 6.3 points. Experiments show that ParSer remains robust to changes in evidence position, order, and distance that cause large accuracy swings in sequential methods. By parallelizing document coverage, ParSer reduces inference latency by up to .

Primary area:
foundation or frontier models, including LLMs
Keywords:
long-context, LLM agent, question answering, RLVR

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NHobOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ParSer answers questions over long documents by dividing reading and reasoning between different agents. Each frozen reader is assigned one document chunk. An RL-trained lead agent sends focused questions to all readers, combines their returned findings, and either asks another question or produces the final answer. This replaces sequential document traversal with parallel reading rounds while keeping multi-step reasoning in the lead agent. Experiments on HotpotQA and 2WikiMultiHopQA span 7K–896K tokens, with additional RULER tests. The paper reports improved accuracy over sequential memory agents, greater robustness to evidence arrangement, and runtime gains that depend on the deployment setting.

Critical strengths
  1. The architecture addresses a clear limitation of sequential memory updates. In sequential agents, early evidence can be discarded before later information reveals its relevance. ParSer retains access to every chunk and lets the lead agent revisit them with questions informed by earlier findings. This provides a concrete mechanism for handling evidence presented in an inconvenient order.

  2. The experiments provide substantial evidence of practical benefit over the evaluated baselines. Important sequential baselines are reimplemented using common backbones and training data. The paper evaluates several context lengths, an out-of-distribution QA benchmark, and ablations of RL training and reader size. The evidence-order experiment is particularly informative because it directly tests the failure mode that repeated parallel access is intended to address.

Critical weaknesses
  1. The closest parallel-reading methods are discussed but not evaluated. ParSer combines a lead agent, chunk-specific readers, and repeated question-driven interactions. LongAgent [1] and XpandA [2] already use closely related multi-round architectures, while learned orchestration has precedent [3, 4]. The current comparisons focus mainly on sequential memory systems. The no-RL ablation shows that training helps ParSer, but does not establish its advantage over existing parallel approaches under comparable resources. Table 7 makes this gap more important: ParSer without RL, which is essentially a hand-specified parallel reader, already reaches 82.85 on 2WikiMultiHopQA against 60.61 for RL-trained MemAgent, and RL adds only 4.2 points. So most of the robustness may come from the parallel structure that LongAgent and XpandA already have.

  2. The latency claims are stronger than the evidence. The 11× speedup is measured at concurrency 1, where MemAgent processes one chunk at a time on a mostly idle H100 while ParSer batches all chunks, so it mainly reflects how well each method uses an idle GPU. At concurrency 16 and 32, the gap shrinks to 1.7× and 1.3× (Table 2), and these are also the settings where ParSer's extra RTX3090 can work in parallel with the H100. In addition, the paper says latency scales with reasoning depth and not with document length, but ParSer's time at concurrency 1 roughly doubles with each doubling of length beyond 112K (13.3s, 22.4s, 42.2s, 78.2s), even though the number of steps stays near 4 (Figure 3). A comparison with matched hardware is needed.

  3. The compact-context argument depends on readers returning little information. Each reader sees a short chunk, but the lead agent gathers all non-abstaining findings. If many chunks are relevant, or many readers mistakenly return irrelevant findings, the lead agent’s input can still grow with document length. The current sparse-evidence QA setting does not sufficiently test this aggregation bottleneck. Document coverage alone therefore does not establish that the full system maintains a compact reasoning context. Table 6 already shows this trend in sparse-evidence QA: the subagents' total output grows from 312 tokens at 7K to 804 tokens at 896K. The failure case in Appendix G.2 also shows that one wrong finding from an unrelated chunk can mislead the lead agent, and more chunks give more chances for such findings. The RULER aggregation tasks contain only a few key items, so they do not test the case where many chunks are relevant.

  4. The evidence-distance experiment changes more than distance. To separate the two supporting paragraphs, the experiment inserts additional distractors between them while keeping the outer padding fixed. This increases total document length and distractor volume at the same time. The observed stability is useful, but the comparison cannot attribute the sequential baselines’ degradation specifically to evidence separation.

[1] LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration. 2024.

[2] Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration. 2025.

[3] Multi-Agent Collaboration via Evolving Orchestration. 2025.

[4] WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning. 2026.

Questions for authors
  1. Is there an explicit bound or filtering rule for the lead agent’s accumulated observations?
8 comments
AuthorsOpenReview ↗

Response to Reviewer NHob (Part 1)

We sincerely thank you for your time, constructive feedback and valuable suggestions. We are deeply grateful for your recognition of ParSer's novelty. We also appreciate your kind words regarding the experiment soundness of our work. Below, we respond point-by-point to address your concerns, clarify details, and provide additional analyses.


Response to W1

Clarification on our research question

The closest parallel-reading methods are discussed but not evaluated. ParSer combines a lead agent, chunk-specific readers, and repeated question-driven interactions. LongAgent [1] and XpandA [2] already use closely related multi-round architectures, while learned orchestration has precedent [3, 4]. The current comparisons focus mainly on sequential memory systems. The no-RL ablation shows that training helps ParSer, but does not establish its advantage over existing parallel approaches under comparable resources.

Thank you for raising this concern and for the chance to further clarify our research question. Our research asks how far LLMs can go in long-context QA when simple reading and reasoning workflows rely primarily on the models’ native capabilities. This motivates two design principles.

  1. Native capabilities with minimal workflow design. To study this question, for the baselines, we start with full-context answering, where the model directly reads and reasons, and MemAgent, where it reads and reasons sequentially in a chunk-wise manner. We then explore decoupling reading from reasoning: each reader performs direct QA over a chunk, while the lead agent performs the LLM’s native agentic reasoning. These approaches rely on the model’s basic capabilities of reading and reasoning. We acknowledge that LongAgent [1] and XpandA [2] adopt related multi-agent architectures. They additionally introduce highly customized components/workflow mechanisms: LongAgent uses inter-member communication to share chunks and resolve conflicting answers, while XpandA combines dynamic partitioning and selective replay. In contrast, our design keeps the reading and reasoning interfaces simple without additional specialized designs.
  2. Simple outcome feedback to reinforce reasoning patterns. Building on this investigation, and following MemAgent, we also study whether Reinforcement Learning can further stimulate the model’s potential within ParSer framework. To this end, we optimize the lead agent to reinforce its own beneficial reasoning patterns with a standard and simple RL algorithm: we use final-answer correctness as the sole task reward; subagents remain frozen. Unlike the auxiliary reward designs in [3, 4], we do not add rewards for cost, tool use, or intermediate collaboration, nor do we optimize multiple agents, since proposing a new optimization algorithm for orchestration within multi-agent systems is outside the scope of this work. Our focus is the effectiveness of this reading and reasoning organization and its potential for improvement through simple feedback.

Our primary comparisons focus on basic ways of organizing an LLM’s reading and reasoning with minimal workflow design. Although LongAgent and XpandA incorporate specialized mechanisms beyond this focus, they remain closely related approaches to long-context QA. To address the reviewer’s concern, we therefore extend our evaluation to these methods: LongAgent is adapted using its released codes, and XpandA is implemented from its published algorithm, preserving their respective conflict-resolution and exploration/replay mechanisms. For fair comparison, we enable thinking for LongAgent’s leader and XpandA’s decider, with a maximum of 2,048 output tokens per call, a same level of budget as ParSer. None of the methods have been trained.

BackboneDatasetLongAgentXpandAParSer w/o RL
Qwen3.5-4BHotpotQA41.9942.7774.42
Qwen3.5-9BHotpotQA45.9055.1876.24
Qwen3.5-4B2WikiMultiHopQA40.3337.2182.85
Qwen3.5-9B2WikiMultiHopQA38.4858.2084.86

ParSer (w/o RL) achieves higher accuracy in every evaluated backbone–dataset–length setting. These results support our central design choice: a simple reading-reasoning interface that follows the model’s native working mode can be more effective than the specialized workflows evaluated here. Our results provide empirical support for leveraging the model’s native agentic capabilities more, rather than relying on highly customized workflows, especially in the era of agentic LLMs.

References:

[1] LongAgent: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration.

[2] Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration.

[3] Multi-Agent Collaboration via Evolving Orchestration.

[4] WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning.

AuthorsOpenReview ↗

Response to Reviewer NHob (Part 2)

Response to W1 (Cont'd)

Gains of RL

ParSer without RL, which is essentially a hand-specified parallel reader, already reaches 82.85 on 2WikiMultiHopQA, and RL adds only 4.2 points. So most of the robustness may come from the parallel structure that LongAgent and XpandA already have.

2Wiki is an OOD benchmark that is not used for training. Despite the strong initial performance, RL improves its average accuracy from 82.85% to 87.04% for the 4B model and from 84.86% to 88.48% for the 9B model. For comprehensive interpretation of the gain of RL used in ParSer, we additionally provide another OOD comparison on RULER Variable Tracking (VT), which requires following chains of variable assignments. For Qwen3.5-4B, the existing trained-model evaluation averages approximately 98.40% over 64K–512K, compared with 87.46% for the evaluated w/o-RL model.

Method64K128K256K512KAvg.
ParSer w/o RL82.1988.1387.8191.7287.46
ParSer w/ RL97.9797.6699.0698.7598.36

Response to W2

Benefit of parallel processing for GPU utilization

The 11× speedup is measured at concurrency 1, where MemAgent processes one chunk at a time on a mostly idle H100 while ParSer batches all chunks, so it mainly reflects how well each method uses an idle GPU.

We acknowledge that exploiting parallelism within a document contributes to the large single-request gain. This is an intended property of the architecture: sequential memory updates depend on the preceding update, whereas ParSer exposes independent reader calls within each round, allowing one question to use the available batching capacity.

Resource equivalence

these are also the settings where ParSer's extra RTX3090 can work in parallel with the H100.

While RTX 3090’s BF16 compute performance is about 7% of the H100 (71 vs. 989 TFLOPS), we agree that the comparison does not establish strict resource equivalence. We have therefore re-evaluated ParSer, MemAgent, and Full-context with each complete method restricted to one H100; ParSer’s lead and readers share that GPU , with the lead agent allocated 30% of the GPU memory and the subagent utilizing the remaining 70%. Each cell of the updated efficiency table reports the average per-question latencies / amortized inference time.

ConcurrencyMethod7K14K28K56K112K224K448K896K
1Full-context1.58 / 1.581.68 / 1.682.74 / 2.743.74 / 3.748.05 / 8.0517.06 / 17.0652.28 / 52.28186.42 / 186.42
1MemAgent10.56 / 10.5615.99 / 15.9930.40 / 30.4058.49 / 58.49112.59 / 112.59222.86 / 222.86438.61 / 438.61876.20 / 876.20
1ParSer4.37 / 4.375.04 / 5.046.10 / 6.107.62 / 7.6211.14 / 11.1417.73 / 17.7329.38 / 29.3854.27 / 54.27
16Full-context6.71 / 0.6410.01 / 0.8715.68 / 1.0033.39 / 2.36139.58 / 9.19416.15 / 26.581394.46 / 90.82—
16MemAgent18.72 / 1.2131.72 / 2.0857.49 / 3.64111.22 / 7.05215.93 / 13.64435.27 / 27.38853.56 / 53.611652.67 / 103.65
16ParSer17.21 / 1.1822.30 / 1.5036.27 / 2.3760.56 / 3.94125.14 / 8.01242.25 / 15.51473.48 / 30.51932.85 / 60.78
32Full-context11.18 / 0.6417.91 / 0.6330.85 / 1.26104.22 / 3.65269.62 / 8.84785.18 / 26.582739.39 / 91.12—
32MemAgent26.66 / 0.8843.97 / 1.4681.52 / 2.59157.36 / 5.01309.91 / 9.78611.45 / 19.261221.31 / 38.352396.76 / 75.21
32ParSer27.14 / 0.9437.26 / 1.2867.67 / 2.27131.78 / 4.41237.97 / 8.00467.93 / 15.63917.11 / 30.581824.88 / 61.09

Clarifying the latency-scaling claim

the paper says latency scales with reasoning depth and not with document length, but ParSer's time at concurrency 1 roughly doubles with each doubling of length beyond 112K, even though the number of steps stays near 4.

Thank you for the opportunity to clarify this claim. In Lines 74-76, we mainly discuss the constructural advantage of ParSer and our claim refers to idealized execution with sufficient compute and memory. Under these conditions, ParSer’s per-round latency is approximately fixed and independent of document length, so its idealized latency scales with the number of query-driven reasoning rounds. In contrast, MemAgent requires a dependent memory update for each chunk. So, even at the condition of sufficient resource, its sequential latency scales with document length.

For ParSer, with fixed GPU resources (the practical condition in our experiments), longer documents or higher concurrency can require more reader execution, increase resource contention, and limit cache retention. Consequently, each round takes longer, and then leads to longer total latency even when the number of rounds remains stable. This is the reason for the latency trend reported in the above table and Table 2, which has been discussed in Appendix B. Thanks again for raising this point. We will describe these assumptions more clearly in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer NHob (Part 3)

Response to W3

Context compactness. We agree that our design cannot guarantee bounded context provided to the lead agent with longer document, as the lead-context size depends on the findings returned across queries, including false-positive findings. As Table 6 shows, a 128-fold increase in document length, from 7K to 896K tokens, corresponds to an increase in reported subagent output from 312 to 804 tokens, approximately 2.6-fold. While the observed growth is modest, we agree that specialized filtering methods like employing stronger readers could further improve context compactness.

Robustness to misleading findings. Stronger readers may reduce the occurrence of misleading findings in the lead agent's context, but more specific follow-up queries and better conflict resolution by the lead agent, especially the one after RL training, can also help. To explain this, we add an ablation study for 4B lead agent where the finding that does not hit a supporting evidence is discarded. This provides an oracle filtered condition for findings. The below table shows the findings precison (the fraction of returned findings that hit a supporting evidence) and accuracy.

Sub_EM/Findings precision (%)Before RLAfter RLRL gain (pp)
Original setting74.41/63.0184.57/60.97+10.16
Oracle filtered setting80.47/95.8487.99/95.02+7.52

For the trained model, moving from the oracle filtered condition to the original setting reduces finding precision from 95.02% to 60.97%, while Sub_EM decreases by only 3.42 percentage points, from 87.99% to 84.57%. This shows that the trained lead agent is more robust to noisy findings.

RULER evaluation on dense-information QA. Appendix C.2 evaluates Frequent Words Extraction (FWE) on RULER, which asks for the three most frequent coded words across a long text. Although its answer contains only three words, the underlying frequency information is densely distributed: in the 512K test set, each example contains approximately 91K–104K non-filler word occurrences. FWE therefore provides complementary evidence on processing densely distributed repeated information. ParSer achieves a Sub_EM of 94.53% with Qwen3.5-4B backbone and 98.96% with 9B backbone, outperforming MemAgent (Sub_EM of 22.31% and 97.38% respectively). The evaluation details are in Appendix C.2 in our revised submission.


Response to W4

Appreciate the opportunity to further clarify our design considerations for the experiment of evidence position and distance control in Sec. 4.1. The two experiments vary complementary aspects of evidence layout. Evidence Position Control keeps the relative separation between the supporting evidence paragraphs fixed and varies their proximity to the document boundaries. Evidence Distance Control, conversely, keeps the prefix before the first evidence paragraph and the suffix after the second fixed, and varies the separation between the evidence paragraphs by inserting additional distractors between them. The latter therefore increases the intervening interference while leaving the outer padding unchanged. We acknowledge that this construction increases total document length. This increase is part of the intended test: one reason longer documents degrade sequential memory agents is that more distractors intervene between dependent evidence, requiring earlier evidence to survive more recurrent memory updates before it can be combined with later evidence.


Response to Q1

Our filtering relies on subagent abstention: if a subagent judges its chunk irrelevant to the query, it returns Unknown, which is discarded. All non-abstaining responses are passed to the lead agent. We do not impose an explicit bound on the accumulated observations.

Reviewer_RjAvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Long-document question answering can lose early evidence when a model reads chunks sequentially through a compressed memory. ParSer instead broadcasts a trained lead agent's queries to frozen, chunk-local readers in parallel; the lead gathers evidence and iterates before answering. The paper specifies lead-only GRPO training, prompts, and inference settings. On HotpotQA, mean Sub_EM improves from 78.91% for the strongest tested 4B sequential baseline to 84.57%, a gain of 5.66 percentage points. Controlled evidence-order results also favor ParSer over the tested sequential-memory baselines. At 896K tokens, ParSer reports an 11.20x speedup over MemAgent with one concurrent request, falling to 1.73x at concurrency 16 and 1.28x at concurrency 32. The higher-concurrency figures measure amortized time per question rather than individual response latency, and the comparison uses an H100 plus an RTX 3090 for ParSer versus one H100 for MemAgent, not equal hardware resources.

Critical strengths
  • A different response to a known ordering problem. Chow-Liu Ordering [A] improves the processing order within a sequential chain. ParSer instead lets a document-blind lead repeatedly query all chunk readers in parallel. In the controlled reversed-evidence test, ParSer scores 97.1 versus 23.3 for MemAgent and 61.7 for ReMemR1. This is targeted evidence for the architecture against the tested baselines, although it does not establish superiority over order-aware sequential reading.
  • Meaningful scoped contribution: HotpotQA mean Sub_EM improves from 78.91 to 84.57 for the strongest tested 4B sequential baseline, and the lead-RL ablation increases 4B performance from 74.42 to 84.57. Combined with frozen chunk readers and repeated broadcast, this is a useful long-document application distinct from prior leader/member reading [B], question-driven collaboration [C], and general RL orchestration [D].

References

  • [A] Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents, ICLR 2026
  • [B] LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration, EMNLP 2024
  • [C] Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration, ICML 2026
  • [D] Multi-Agent Collaboration via Evolving Orchestration, NeurIPS 2025
Critical weaknesses
  • Missing comparison with MemAgent plus improved chunk ordering. A central argument for ParSer is that sequential memory loses evidence when chunks arrive in an unfavorable order. However, Chow-Liu Ordering [A] shows that changing the processing order can improve sequential Chain-of-Agents performance. ParSer cites this work but does not test MemAgent with Chow-Liu reordering. It is therefore unclear how much of the accuracy gap requires ParSer's parallel, iterative architecture and how much could be recovered by improving the sequential reader's chunk order. A matched comparison on the main benchmarks and reversed-evidence test would address this gap. Inference time should include the reordering cost; ParSer may retain a speed advantage even if the accuracy gap narrows.
  • Fixable evaluation limits: Table 8 reuses just 128 questions per QA benchmark across eight lengths; Table 1 reports three-run means without dispersion, and Appendix F.6 uses bidirectional substring Sub_EM rather than strict exact match. The 4B 2Wiki mean also trails the DeepSeek comparator, and the RULER results have no matched baseline. These gaps weaken confidence in broad accuracy claims; report paired question/seed uncertainty, strict-EM sensitivity, and broader matched tests.
  • Fixable efficiency comparison: The 896K, concurrency-one 11.20x wall-time advantage uses an H100 plus RTX3090 for ParSer versus one H100 for MemAgent; it falls to 1.73x at concurrency 16. Appendix B also assumes effective parallelism and cache retention, while Appendix F.4 reports substantial lead-training resources. Thus a favorable deployment cost is not yet established; compare matched GPU budgets, per-request latency, and GPU-hours per correct answer under cache pressure.
  • Missing Comparison with Tool Use: Agents can use search tools such as grep to locate relevant passages without reading every chunk in each round. Work such as Dr-DCI [B] explores such direct corpus interaction ideas. Comparing ParSer with a tool-based agent using the same backbone and documents would help establish when repeated all-chunk reading is worth its cost. The comparison should measure answer accuracy, latency, and token usage, including on questions that require combining evidence across distant passages.

References

  • [A] Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents, ICLR 2026
  • [B] Dr-DCI: Scaling Direct Corpus Interaction via Dynamic Workspace Expansion, ArXiv 2026
Questions for authors
  1. Comparison with improved sequential ordering: How does ParSer compare with MemAgent augmented with Chow–Liu ordering, using the same backbone and chunks? Please report accuracy on the main benchmarks and reversed-evidence test, together with inference time including reordering cost. How much of the accuracy gap remains after improving the sequential processing order?
  2. Query quality and credit assignment: The final-answer reward does not distinguish a query that returns focused evidence from one that returns a useful finding alongside substantial noise. Does lead training improve the relevance of returned findings or mainly teach the lead to answer despite noise? Could you measure these changes before and after training and test the effect of removing irrelevant or misleading findings?
  3. Speedup under matched resources: The reported 11.20x speedup falls to 1.73x at concurrency 16 and 1.28x at concurrency 32, with different hardware allocations for ParSer and MemAgent. Under matched GPU budgets, what are the per-request latency and throughput gains? How much do these depend on retaining chunk KV caches?
  4. Reliability and limits of the accuracy gains: Do the gains persist under strict exact match, larger question sets, and paired uncertainty estimates? What explains the weaker 4B results on 2WikiMultiHopQA, and which failure types remain despite repeated access to every chunk?
  5. Comparison with tool-based document navigation: How does ParSer compare with an agent that uses grep-like search and targeted passage reading, such as Dr-DCI, with the same backbone and documents? On questions requiring evidence from distant passages, does repeated all-chunk reading offer better accuracy, and at what cost in latency and token usage?
9 comments
AuthorsOpenReview ↗

Response to Reviewer RjAv (Part 1)

We sincerely thank you for your time, constructive feedback.


Response to W3&Q3

Matched GPU resources

Appendix B also assumes effective parallelism and cache retention, while Appendix F.4 reports substantial lead-training resources. Thus a favorable deployment cost is not yet established.

We would like to clarify that Appendix F.4 demonstrates the GPU resources used for training the lead agent, instead of inference. As RL training undergoes hundreds of training step involving rollout and policy update, we use multiple GPUs to speed up the training. For the re-implementation of MemAgent and ReMemR1, 8 H100 GPUs is used for training. Furthermore, in Appendix B, we assume sufficient GPU memory solely for the purpose of complexity analysis. In our experiments, all methods are evaluated under comparable GPU memory budgets (the RTX 3090’s BF16 compute performance is about 3.6% of the H100). We also re-evaluate the latencies of ParSer running on a single GPU under the strictly matched budget with 1 H100 GPU in the following response.

Re-evaluation with identical resource

The 896K, concurrency-one 11.20x wall-time advantage uses an H100 plus RTX3090 for ParSer versus one H100 for MemAgent; it falls to 1.73x at concurrency 16; compare matched GPU budgets, per-request latency, and GPU-hours per correct answer under cache pressure. Under matched GPU budgets, what are the per-request latency and throughput gains? How much do these depend on retaining chunk KV caches?

While RTX 3090’s BF16 compute performance is about 7% of the H100 (71 vs. 989 TFLOPS), we agree that the experiment in Sec. 4.2 does not establish strict resource equivalence: the original two-GPU deployment has additional memory capacity. We have therefore re-evaluated ParSer, MemAgent, and Full-context with each complete method restricted to one H100; ParSer’s lead and readers share that GPU , with the lead agent allocated 30% of the GPU memory and the subagent utilizing the remaining 70%. Each cell of the updated efficiency table reports the average per-question latencies / amortized inference time. Note that the experiment is based on practical and identical GPU resources for all methods.

ConcurrencyMethod7K14K28K56K112K224K448K896K
1Full-context1.58 / 1.581.68 / 1.682.74 / 2.743.74 / 3.748.05 / 8.0517.06 / 17.0652.28 / 52.28186.42 / 186.42
1MemAgent10.56 / 10.5615.99 / 15.9930.40 / 30.4058.49 / 58.49112.59 / 112.59222.86 / 222.86438.61 / 438.61876.20 / 876.20
1ParSer4.37 / 4.375.04 / 5.046.10 / 6.107.62 / 7.6211.14 / 11.1417.73 / 17.7329.38 / 29.3854.27 / 54.27
1ParSer (no reader prefix reuse)4.78 / 4.784.84 / 4.845.64 / 5.647.94 / 7.9411.59 / 11.5919.69 / 19.6935.91 / 35.9165.25 / 65.25
16Full-context6.71 / 0.6410.01 / 0.8715.68 / 1.0033.39 / 2.36139.58 / 9.19416.15 / 26.581394.46 / 90.82—
16MemAgent18.72 / 1.2131.72 / 2.0857.49 / 3.64111.22 / 7.05215.93 / 13.64435.27 / 27.38853.56 / 53.611652.67 / 103.65
16ParSer17.21 / 1.1822.30 / 1.5036.27 / 2.3760.56 / 3.94125.14 / 8.01242.25 / 15.51473.48 / 30.51932.85 / 60.78
16ParSer (no reader prefix reuse)18.47 / 1.2627.20 / 1.7842.75 / 2.7974.33 / 4.80133.51 / 8.68244.24 / 15.88510.97 / 32.88985.01 / 64.84
32Full-context11.18 / 0.6417.91 / 0.6330.85 / 1.26104.22 / 3.65269.62 / 8.84785.18 / 26.582739.39 / 91.12—
32MemAgent26.66 / 0.8843.97 / 1.4681.52 / 2.59157.36 / 5.01309.91 / 9.78611.45 / 19.261221.31 / 38.352396.76 / 75.21
32ParSer27.14 / 0.9437.26 / 1.2867.67 / 2.27131.78 / 4.41237.97 / 8.00467.93 / 15.63917.11 / 30.581824.88 / 61.09
32ParSer (no reader prefix reuse)30.34 / 1.1045.63 / 1.5977.50 / 2.61134.30 / 4.56258.59 / 8.71490.05 / 16.47959.45 / 32.371849.45 / 62.96

With the lead agent now running on the H100 instead of the RTX3090, the single-concurrency results show a larger advantage over MemAgent: at 896K, ParSer takes 54.27 s versus 876.20 s, giving a 16.15× ratio compared with 11.20× in the submitted paper. At concurrency 16 and 32, ParSer retains advantages of a similar order of magnitude to the paper’s reported gains on documents of at least 56K tokens. At higher concurrency, MemAgent can better utilize the GPU by batching work across independent questions, narrowing the benefit of ParSer’s within-question parallelism. Meanwhile, ParSer’s parallel readers face greater contention for the shared GPU’s compute and memory resources (discussed in Appendix B).

Effect of chunk KV-cache retention. The table also shows the latencies without enabling KV cache for subagents. Disabling reader chunk-prefix KV-cache reuse generally increases ParSer’s latency compared with the default configuration. Nevertheless, ParSer remains faster than MemAgent on long documents (≥56K tokens).

AuthorsOpenReview ↗

Response to Reviewer RjAv (Part 2)

(following previous discussion)

Response to W1 & Q1

We thank the reviewer for suggesting this comparison. We agree that improved chunk ordering is a meaningful augmentation of sequential memory agents, and have completed an inference-time CL–ORDER [1] adaptation for the frozen Qwen3.5-4B MemAgent checkpoint on both main benchmarks and the logical/reversed-evidence test.

Experiment settings. We preserve the settings and checkpoints used in the paper. Following CL–ORDER’s construction, we encode the full chunks using frozen Qwen3-Embedding-8B, compute the complete pairwise cosine-similarity graph, construct an exact maximum-weight spanning tree, select the chunk most similar to the question as the root, and obtain the reading sequence through breadth-first traversal. The encoder revision and tie-breaking rules are fixed. Neither gold answers nor supporting-fact or logical-order annotations are used to construct the traversal. Each test sample retains its original fixed chunks and receives an independently constructed CL traversal. We perform three inference repeats with different seeds.

HotpotQA

Method50 (7K)100 (14K)200 (28K)400 (56K)800 (112K)1600 (224K)3200 (448K)6400 (896K)Avg
MemAgent81.25 ± 2.0681.51 ± 0.9083.59 ± 1.5677.86 ± 1.6377.86 ± 0.9079.69 ± 0.7873.96 ± 1.1972.92 ± 1.9778.58 ± 0.41
MemAgent + CL85.68 ± 0.9082.55 ± 1.6385.68 ± 2.2682.81 ± 0.7878.39 ± 1.6381.51 ± 2.5180.73 ± 2.2679.17 ± 2.7482.06 ± 0.74
ParSer85.68 ± 1.6284.64 ± 2.3983.60 ± 2.0785.68 ± 1.1985.42 ± 1.6383.07 ± 1.2083.07 ± 1.9785.42 ± 1.1984.57 ± 0.93

2WikiMultiHopQA

Method50 (7K)100 (14K)200 (28K)400 (56K)800 (112K)1600 (224K)3200 (448K)6400 (896K)Avg
MemAgent67.45 ± 2.7473.18 ± 3.1669.01 ± 0.4561.46 ± 1.1954.17 ± 5.5454.43 ± 1.1960.16 ± 5.4145.05 ± 1.6360.61 ± 0.79
MemAgent + CL77.34 ± 1.3579.69 ± 4.1375.00 ± 0.7871.88 ± 3.5865.89 ± 3.6966.41 ± 1.3566.67 ± 1.1963.02 ± 1.6370.74 ± 0.71
ParSer86.98 ± 1.1984.64 ± 1.2088.28 ± 0.0087.76 ± 0.4587.50 ± 1.5685.94 ± 2.0688.28 ± 1.3586.98 ± 1.9687.04 ± 0.39

Evidence order on 2WikiMultiHopQA (6,400 paragraphs)

MethodLogicalReversedReversed − logical (pp)
MemAgent66.47 ± 1.8123.32 ± 1.77-43.15 ± 3.58
MemAgent + CL74.54 ± 1.6372.59 ± 1.58-1.95 ± 0.39
ParSer96.42 ± 0.4197.14 ± 0.410.72 ± 0.41

CL–ORDER is shown able to improve MemAgent's performance and alleviate the directional effect of evidence reversal. Nevertheless, it still performs worse than ParSer. CL–ORDER optimizes semantic similarity between chunks rather than the directed, question-specific dependencies required by multi-hop QA. Most distractor chunks contain content that appears relevant to the questions, and their embedding may favor shared topics instead of identifying the necessary reasoning links. For example, a chunk identifying a film’s director may be more similar to unrelated movie descriptions than to a chunk containing that director’s death date. As a result, semantic similarity may miss the entity–attribute link required for multi-hop reasoning. Therefore, the resulting order does not provide obvisous gains in our multi-hop QA settings. In contrast, ParSer does better in identifying evidence through iterative parallel reading.

Inference overhead. As shown in the above discussion about latency , at concurrency of 16, ParSer has lower average latencies than MemAgent. Now with CL, MemAgent needs to spend extra time for ordering preprocessing.

Number of paragraphs50100200400800160032006400
Mean CL ordering preprocessing time (s/document)1.822.785.3710.5220.5540.4481.16162.00

References:

[1] Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents, ICLR 2026.

AuthorsOpenReview ↗

Response to Reviewer RjAv (Part 3)

Response to W2&Q4

Comprehensive evaluation

Table 8 reuses just 128 questions per QA benchmark across eight lengths; Table 1 reports three-run means without dispersion, and Appendix F.6 uses bidirectional substring Sub_EM rather than strict exact match. Report paired question/seed uncertainty, strict-EM sensitivity, and broader matched tests.

For comprehensive interpretation of the effectiveness of ParSer, we also report the metric of dispersion and exact match here.

Dispersion We follow the established HotpotQA evaluation protocol and directly use the long-context test sets released by MemAgent [1] and ReMemR1 [2] to maintain comparability with prior work. Reusing the same 128 questions across eight context lengths is intentional: it keeps question identity, allowing us to assess robustness to context expansion while controlling for variation in question difficulty. Our controlled evaluations in Section 4.1 additionally use 512 questions per evaluation . These larger question sets provide complementary evidence about robustness to controlled changes in evidence distribution.

To quantify uncertainty in performance differences, we report 95% confidence intervals (CIs) from 10,000 paired question-block bootstrap resamples on HotpotQA used in the main experiment. We first average the observed inference repeats within each method. Each sampled question retains all its context-length variants and observed repeats, preserving their dependence; lengths and inference repeats are not treated as independent questions. The additional analysis uses three complete inference evaluations per method. Avg denotes the average over the eight context lengths. The table reports ParSer’s Sub_EM gains over the baselines and paired 95% CIs in percentage points.

BackboneBaselineParSer's avg gain [95% CI]896K gain [95% CI]
Qwen3.5-4BMemAgent+5.99 [1.99, 10.09]+12.50 [6.24, 18.75]
ReMemR1+5.66 [2.18, 9.41]+11.98 [7.55, 16.93]
Agentic RAG+25.72 [20.44, 30.99]+38.80 [29.69, 47.66]
Qwen3.5-9BMemAgent+6.82 [3.01, 10.81]+10.81 [5.60, 16.02]
ReMemR1+8.48 [4.51, 12.68]+9.77 [4.82, 14.97]
Agentic RAG+27.33 [22.35, 32.45]+40.76 [31.90, 49.74]

We can see all reported Avg and 896K intervals exclude zero, supporting positive differences over these matched baselines on HotpotQA under the evaluated checkpoints and original Sub_EM metric. These are pointwise intervals conditional on the saved inference outputs, based on 128 question blocks.

We also report standard deviation for sub_em across inference runs of the same checkpoint. We donot report Full-context's SD here as they use a temperature of 0 for inference.

BackboneMethodAvg: mean ± SD896K: mean ± SD
Qwen3.5-4BParSer84.57 ± 0.9385.42 ± 1.19
MemAgent78.58 ± 0.4172.92 ± 1.97
ReMemR178.91 ± 0.6173.44 ± 3.58
Agentic RAG58.69 ± 0.2046.09 ± 1.63
Qwen3.5-9BParSer86.79 ± 0.4085.94 ± 1.59
MemAgent80.05 ± 0.6975.00 ± 2.07
ReMemR178.39 ± 0.4176.04 ± 2.26
Agentic RAG59.59 ± 0.6446.88 ± 1.19

Exact match We follow the evaluation protocol of previous work including MemAgent and ReMemR1, which use Sub_EM to meaure model performance. For comprehensive evaluation, we shows the Exact Match scores on HotpotQA. Note that the full-context baselines undergo no task-specific fine-tuning and tend to produce more verbose answers, which can lower strict EM even when the correct answer is included

BackboneMethodAvg: mean ± SD896K: mean ± SD
DeepSeek-V4-ProFull-context (non-think)52.8341.41
Full-context (think-max)56.1557.81
Qwen-4BFull-context (non-think)48.2427.34
ParSer84.57 ± 0.9385.42 ± 1.19
MemAgent78.58 ± 0.4172.92 ± 1.97
ReMemR178.91 ± 0.6173.44 ± 3.58
Agentic RAG58.69 ± 0.2046.09 ± 1.63
Qwen3.5-9BFull-context (non-think)48.4438.28
Full-context (think)50.9835.94
ParSer86.79 ± 0.4085.94 ± 1.59
MemAgent80.05 ± 0.6975.00 ± 2.07
ReMemR178.39 ± 0.4176.04 ± 2.26
Agentic RAG59.59 ± 0.6446.88 ± 1.19

References:

[1] MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent, ICLR 2026.

[2] Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents, ICLR 2026.

Reviewer_ZdLnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ParSer separates long-context processing into frozen, parallel chunk readers and an RL-trained lead agent that iteratively queries and integrates evidence. Experiments on HotpotQA, 2WikiMultiHopQA, and RULER demonstrate strong accuracy and robustness to evidence placement, with reduced inference time relative to sequential memory agents.

Critical strengths
  1. The research questions are interesting and well motivated.
  2. The experiments are extensive, covering multiple models and quantization settings.
Critical weaknesses
  1. The incremental contribution over prior parallel systems remains insufficiently established. LongAgent and XpandA already explore closely related leader–worker and question-driven reading workflows. Although discussed, neither is directly evaluated. Matched-backbone comparisons are needed to establish ParSer’s advantage over these alternatives.

  2. The efficiency comparison needs stronger resource controls. ParSer uses an H100 plus an RTX3090, whereas the baselines use one H100. Alternating execution within a request does not establish equivalent resources under concurrent workloads. Moreover, Table 2’s concurrent measurements quantify amortized throughput rather than individual request latency, limiting the practical interpretation of the speedup.

Questions for authors
  1. How does ParSer compare with LongAgent or XpandA using the same backbone and comparable inference budgets?

  2. Do the efficiency gains persist with matched hardware? Please distinguish request latency, throughput, and peak memory.

6 comments
AuthorsOpenReview ↗

Response to Reviewer ZdLn (Part 1)

We sincerely thank you for your time, constructive feedback and valuable suggestions. We are deeply grateful for your recognition of ParSer's novelty. We also appreciate your kind words regarding the experiments of our paper. Below, we respond point-by-point to address your concerns, clarify details, and provide additional analyses.


Response to W1 & Q1

The incremental contribution over prior parallel systems remains insufficiently established. LongAgent and XpandA already explore closely related leader–worker and question-driven reading workflows. Although discussed, neither is directly evaluated. Matched-backbone comparisons are needed to establish ParSer’s advantage over these alternatives. How does ParSer compare with LongAgent or XpandA using the same backbone and comparable inference budgets?

Thank you for raising this concern and for the chance to further clarify our research question. Our research asks how far LLMs can go in long-context QA when simple reading and reasoning workflows rely primarily on the models’ native capabilities. To study this question, for the baselines, we start with full-context answering, where the model directly reads and reasons, and MemAgent, where it reads and reasons sequentially in a chunk-wise manner. We then explore decoupling reading from reasoning: each reader performs direct QA over a chunk, while the lead agent performs the LLM’s native agentic reasoning. These approaches rely on the model’s basic capabilities of reading and reasoning. We acknowledge that LongAgent [1] and XpandA [2] adopt related multi-agent architectures. They additionally introduce highly customized components/workflow mechanisms: LongAgent uses inter-member communication to share chunks and resolve conflicting answers, while XpandA combines dynamic partitioning and selective replay. In contrast, our design keeps the reading and reasoning interfaces simple without additional specialized designs.

Our primary comparisons focus on basic ways of organizing an LLM’s reading and reasoning with minimal additional workflow design. Although LongAgent and XpandA incorporate specialized mechanisms beyond this focus, they remain closely related approaches to long-context QA. To address the reviewer’s concern, we therefore extend our evaluation to these methods, complementing our main analysis with direct evidence of how ParSer’s simpler design compares with these alternatives. LongAgent is adapted using its released codes, and XpandA is implemented from its published algorithm, preserving their respective conflict-resolution and exploration/replay mechanisms. For fair comparison, we enable thinking for LongAgent’s leader and XpandA’s decider, with a maximum of 2,048 output tokens per call. None of the methods have been trained.

HotpotQA

BackboneMethod7K14K28K56K112K224K448K896KAvg.
Qwen3.5-4BLongAgent47.6643.7543.7539.0640.6238.2842.9739.8441.99
Qwen3.5-4BXpandA57.8153.1253.9147.6640.6239.8431.2517.9742.77
Qwen3.5-4BParSer w/o RL72.9276.0473.9674.4876.0473.9675.0072.9274.42
Qwen3.5-9BLongAgent46.8853.1248.4444.5343.7543.7544.5342.1945.90
Qwen3.5-9BXpandA71.8864.8458.5957.0353.1257.8143.7534.3855.18
Qwen3.5-9BParSer w/o RL77.6074.2274.4874.7479.4377.3476.5675.5276.24

2WikiMultiHopQA

BackboneMethod7K14K28K56K112K224K448K896KAvg.
Qwen3.5-4BLongAgent50.0049.2232.8140.6239.8438.2839.0632.8140.33
Qwen3.5-4BXpandA55.4760.9447.6639.0627.3432.8117.1917.1937.21
Qwen3.5-4BParSer w/o RL82.2984.3885.6884.9080.9981.5183.3379.6982.85
Qwen3.5-9BLongAgent42.1945.3142.9738.2834.3833.5937.5033.5938.48
Qwen3.5-9BXpandA61.7271.8862.5061.7255.4757.8152.3442.1958.20
Qwen3.5-9BParSer w/o RL84.3786.4686.9883.0783.5983.8685.9484.6484.86

ParSer (w/o RL) achieves higher accuracy in every evaluated backbone–dataset–length setting. These results support our central design choice: a simple reading-reasoning interface that follows the model’s native working mode can be more effective than the specialized workflows evaluated here. Our results provide empirical support for leveraging the model’s native agentic capabilities more, rather than relying on highly customized workflows, especially in the era of agentic LLMs.

AuthorsOpenReview ↗

Response to Reviewer ZdLn (Part 2)

(following previous discussion)

Response to W2 & Q2

The efficiency comparison needs stronger resource controls. ParSer uses an H100 plus an RTX3090, whereas the baselines use one H100. Alternating execution within a request does not establish equivalent resources under concurrent workloads. Moreover, Table 2’s concurrent measurements quantify amortized throughput rather than individual request latency, limiting the practical interpretation of the speedup. Do the efficiency gains persist with matched hardware? Please distinguish request latency, throughput, and peak memory.

Resource equivalence. We agree that alternating lead/reader execution within one request does not establish resource equivalence: the original two-GPU deployment has additional memory capacity, though RTX 3090’s BF16 compute performance is about 7% of the H100 (71 vs. 989 TFLOPS). We have therefore re-evaluated ParSer, MemAgent, and Full-context with each complete method restricted to one H100; ParSer’s lead and readers share that GPU , with the lead agent allocated 30% of the GPU memory and the subagent utilizing the remaining 70%. Each cell of the updated efficiency table reports the average per-question latencies / amortized inference time. The amortized inference time can be used for comparison against the original results in the paper. Full-context uses non-thinking mode.

ConcurrencyMethod7K14K28K56K112K224K448K896K
1Full-context1.58 / 1.581.68 / 1.682.74 / 2.743.74 / 3.748.05 / 8.0517.06 / 17.0652.28 / 52.28186.42 / 186.42
1MemAgent10.56 / 10.5615.99 / 15.9930.40 / 30.4058.49 / 58.49112.59 / 112.59222.86 / 222.86438.61 / 438.61876.20 / 876.20
1ParSer4.37 / 4.375.04 / 5.046.10 / 6.107.62 / 7.6211.14 / 11.1417.73 / 17.7329.38 / 29.3854.27 / 54.27
16Full-context6.71 / 0.6410.01 / 0.8715.68 / 1.0033.39 / 2.36139.58 / 9.19416.15 / 26.581394.46 / 90.82—
16MemAgent18.72 / 1.2131.72 / 2.0857.49 / 3.64111.22 / 7.05215.93 / 13.64435.27 / 27.38853.56 / 53.611652.67 / 103.65
16ParSer17.21 / 1.1822.30 / 1.5036.27 / 2.3760.56 / 3.94125.14 / 8.01242.25 / 15.51473.48 / 30.51932.85 / 60.78
32Full-context11.18 / 0.6417.91 / 0.6330.85 / 1.26104.22 / 3.65269.62 / 8.84785.18 / 26.582739.39 / 91.12—
32MemAgent26.66 / 0.8843.97 / 1.4681.52 / 2.59157.36 / 5.01309.91 / 9.78611.45 / 19.261221.31 / 38.352396.76 / 75.21
32ParSer27.14 / 0.9437.26 / 1.2867.67 / 2.27131.78 / 4.41237.97 / 8.00467.93 / 15.63917.11 / 30.581824.88 / 61.09

With the lead agent now running on the H100 instead of the RTX3090, the single-concurrency results show a larger advantage over MemAgent: at 896K, ParSer takes 54.27 s versus 876.20 s, giving a 16.15× ratio compared with 11.20× in the submitted paper. At concurrency 16 and 32, ParSer retains advantages of a similar order of magnitude to the paper’s reported gains on documents of at least 56K tokens.

Throughput and memory. The original amortized quantity, total split time divided by 128, describes inverse throughput. For memory, SGLang preallocates its KV-cache pool, so peak GPU-resident memory includes reserved capacity as well as model and runtime allocations. Its interpretation is the footprint of the configured deployment, rather than the minimum memory required by the workload. The new experiments control physical memory capacity by placing the entire deployment on one H100. Both MemAgent and ParSer can run with one H100 GPU, while Full-context fails on 896K-token samples.

We will update the paper with the single-H100 mean-latency table, and the revised discussion of hardware fairness, efficiency metrics. Thanks again for the suggestion.

References:

[1] LongAgent: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration.

[2] Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration.

Reviewer ZdLn

Thank you again for your feedback. In addition to our above response, we have revised Section 4.2 and Table 2 to address your concerns about resource equivalence and the interpretation of concurrent timing measurements. All methods now run entirely on a single H100, including ParSer’s lead agent and readers, and the table reports mean per-question inference latency rather than amortized inference time. We have also updated the discussion to distinguish sequential reasoning steps from wall-clock latency under limited resources.

As the rebuttal period is nearing its end, we would greatly appreciate any further feedback on our responses and manuscript revisions, especially whether these changes address your efficiency concerns.

Comments on the paper

1 comment

We thank all three reviewers for their careful and constructive reviews and AC for coordinating the review process. As the rebuttal period concludes, we summarize our discussion process.


1. What the Reviewers Found Valuable

  1. Principled Remedy for Sequential Memory Bottlenecks (NHob, RjAv, ZdLn): Reviewers highlighted that sequential memory agents risk discarding early evidence before downstream hops reveal its relevance (NHob, RjAv). Decoupling frozen, parallel chunk-bound readers from an RL-trained, document-blind lead agent offers a clean, well-motivated mechanism to revisit all chunks iteratively under refined queries (ZdLn, NHob).
  2. Scoped Contribution & Targeted Controlled Diagnostics (RjAv, NHob): Reviewer RjAv specifically recognized that combining frozen chunk readers, repeated broadcast, and lead-only GRPO forms a "meaningful scoped contribution... distinct from prior leader/member reading, question-driven collaboration, and general RL orchestration." Both RjAv and NHob praised our controlled evidence-order, position, and distance perturbations for directly validating the exact failure mode ParSer targets.
  3. Extensive, Matched Empirical Evidence & Clear Presentation (ZdLn, NHob, RjAv): Reviewers commended our evaluation across 7K–896K tokens, matched-backbone RL baseline re-implementations (MemAgent, ReMemR1), OOD benchmarks (2WikiMultiHopQA, RULER), RL/reader-size/quantization ablations (ZdLn), and clear writing (Presentation: 3 across all reviewers).

2. How the New Results Address the Main Concerns

Concern / ExperimentReviewer(s)Main Conclusion
1. Matched hardware, per-question latency & KV cacheZdLn, RjAv, NHobRestricting each complete method to one H100 GPU, ParSer cuts mean per-question latency by up to 16.15× vs. MemAgent at concurrency 1 and remains faster under high concurrency (where Full-context OOMs), even without reader prefix KV-cache reuse on long documents (≥56K).
2. Sequential reordering (MemAgent + CL–ORDER)RjAvInference-time Chow–Liu reordering narrows MemAgent's gap at extra preprocessing cost, yet ParSer retains clear accuracy leads (especially on OOD 2WikiMultiHopQA and reversed evidence), as static semantic similarity misses directed multi-hop links.
3. Tool-based search (DR-DCI)RjAvAcross 4B/9B backbones (7K–896K) and controlled order/distance splits, ParSer outperforms DR-DCI in every setting while generating 6×–9× fewer output tokens, showing superior accuracy and efficiency over grep-style search loops.
4. Statistical rigor (Uncertainty & strict EM)RjAvPaired question-block bootstrap 95% CIs exclude zero across all HotpotQA baseline comparisons, with low cross-run variance (<1% SD) and consistent gains under strict Exact Match.
5. Parallel Systems (LongAgent, XpandA)ZdLn, NHobUnder matched 4B/9B backbones without RL (7K–896K), ParSer w/o RL beats both across every dataset and length, confirming that clean reading–reasoning decoupling scales far better than complex, hand-crafted coordination heuristics.
6. Comparison with baselines on RULERRjAv, NHobAcross 10 OOD RULER tasks (8K–512K), ParSer achieves the highest overall mean over matched Full-context (with YaRN) and MemAgent baselines on both 4B and 9B, including on densely distributed aggregation (Frequent Words Extraction).
7. Lead-Agent RL: Evidence acquisition & Noise robustnessRjAv, NHobTrajectory diagnostics show RL substantially boosts evidence recall, complete coverage, and effective-query rate, while an oracle noise-removal test confirms high robustness to false-positive findings.
8. Lead-Agent RL: OOD reasoning gainsNHobTrained solely on HotpotQA with final-answer rewards, lead-agent RL generalizes effectively out-of-distribution—consistently improving 2WikiMultiHopQA and substantially boosting RULER Variable Tracking (+10.9 pp on 4B).

3. Main Revision in the Paper

  1. Sec. 4.2 & Table 2 & Intro (Inference Latency): Replaced two-GPU amortized timing with matched 1×H100 mean per-question latency across concurrency 1, 16, and 32 (30%/70% memory split) and revised scaling/concurrency analysis.
  2. App. C.2 & Fig. 5 (RULER OOD): Added task-level Full-context and MemAgent comparisons across 10 RULER tasks (8K–512K, 4B/9B), inference/YaRN settings, and discussion of Full-context's strong scores.
  3. New App. C.3 (Reader Prefix Reuse): Added 1×H100 latency ablations with and without subagent chunk-prefix KV-cache reuse.
  4. New App. C.4 (Evidence Acquisition Before/After RL): Added recall, complete coverage, effective-query rate, and finding precision before/after RL for 4B/9B lead agents.
  5. New App. C.5 (MemAgent + CL–ORDER): Added frozen 4B MemAgent + CL–ORDER comparisons on HotpotQA and 2WikiMultiHopQA (7K–896K) and analyzed why semantic ordering leaves a multi-hop gap.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.