- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ParSer answers questions over long documents by dividing reading and reasoning between different agents. Each frozen reader is assigned one document chunk. An RL-trained lead agent sends focused questions to all readers, combines their returned findings, and either asks another question or produces the final answer. This replaces sequential document traversal with parallel reading rounds while keeping multi-step reasoning in the lead agent. Experiments on HotpotQA and 2WikiMultiHopQA span 7K–896K tokens, with additional RULER tests. The paper reports improved accuracy over sequential memory agents, greater robustness to evidence arrangement, and runtime gains that depend on the deployment setting.
Critical strengths
-
The architecture addresses a clear limitation of sequential memory updates. In sequential agents, early evidence can be discarded before later information reveals its relevance. ParSer retains access to every chunk and lets the lead agent revisit them with questions informed by earlier findings. This provides a concrete mechanism for handling evidence presented in an inconvenient order.
-
The experiments provide substantial evidence of practical benefit over the evaluated baselines. Important sequential baselines are reimplemented using common backbones and training data. The paper evaluates several context lengths, an out-of-distribution QA benchmark, and ablations of RL training and reader size. The evidence-order experiment is particularly informative because it directly tests the failure mode that repeated parallel access is intended to address.
Critical weaknesses
-
The closest parallel-reading methods are discussed but not evaluated. ParSer combines a lead agent, chunk-specific readers, and repeated question-driven interactions. LongAgent [1] and XpandA [2] already use closely related multi-round architectures, while learned orchestration has precedent [3, 4]. The current comparisons focus mainly on sequential memory systems. The no-RL ablation shows that training helps ParSer, but does not establish its advantage over existing parallel approaches under comparable resources. Table 7 makes this gap more important: ParSer without RL, which is essentially a hand-specified parallel reader, already reaches 82.85 on 2WikiMultiHopQA against 60.61 for RL-trained MemAgent, and RL adds only 4.2 points. So most of the robustness may come from the parallel structure that LongAgent and XpandA already have.
-
The latency claims are stronger than the evidence. The 11× speedup is measured at concurrency 1, where MemAgent processes one chunk at a time on a mostly idle H100 while ParSer batches all chunks, so it mainly reflects how well each method uses an idle GPU. At concurrency 16 and 32, the gap shrinks to 1.7× and 1.3× (Table 2), and these are also the settings where ParSer's extra RTX3090 can work in parallel with the H100. In addition, the paper says latency scales with reasoning depth and not with document length, but ParSer's time at concurrency 1 roughly doubles with each doubling of length beyond 112K (13.3s, 22.4s, 42.2s, 78.2s), even though the number of steps stays near 4 (Figure 3). A comparison with matched hardware is needed.
-
The compact-context argument depends on readers returning little information. Each reader sees a short chunk, but the lead agent gathers all non-abstaining findings. If many chunks are relevant, or many readers mistakenly return irrelevant findings, the lead agent’s input can still grow with document length. The current sparse-evidence QA setting does not sufficiently test this aggregation bottleneck. Document coverage alone therefore does not establish that the full system maintains a compact reasoning context. Table 6 already shows this trend in sparse-evidence QA: the subagents' total output grows from 312 tokens at 7K to 804 tokens at 896K. The failure case in Appendix G.2 also shows that one wrong finding from an unrelated chunk can mislead the lead agent, and more chunks give more chances for such findings. The RULER aggregation tasks contain only a few key items, so they do not test the case where many chunks are relevant.
-
The evidence-distance experiment changes more than distance. To separate the two supporting paragraphs, the experiment inserts additional distractors between them while keeping the outer padding fixed. This increases total document length and distractor volume at the same time. The observed stability is useful, but the comparison cannot attribute the sequential baselines’ degradation specifically to evidence separation.
[1] LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration. 2024.
[2] Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration. 2025.
[3] Multi-Agent Collaboration via Evolving Orchestration. 2025.
[4] WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning. 2026.
Questions for authors
- Is there an explicit bound or filtering rule for the lead agent’s accumulated observations?
8 comments
Response to Reviewer NHob (Part 2)
Response to W1 (Cont'd)
Gains of RL
ParSer without RL, which is essentially a hand-specified parallel reader, already reaches 82.85 on 2WikiMultiHopQA, and RL adds only 4.2 points. So most of the robustness may come from the parallel structure that LongAgent and XpandA already have.
2Wiki is an OOD benchmark that is not used for training. Despite the strong initial performance, RL improves its average accuracy from 82.85% to 87.04% for the 4B model and from 84.86% to 88.48% for the 9B model. For comprehensive interpretation of the gain of RL used in ParSer, we additionally provide another OOD comparison on RULER Variable Tracking (VT), which requires following chains of variable assignments. For Qwen3.5-4B, the existing trained-model evaluation averages approximately 98.40% over 64K–512K, compared with 87.46% for the evaluated w/o-RL model.
| Method | 64K | 128K | 256K | 512K | Avg. |
|---|---|---|---|---|---|
| ParSer w/o RL | 82.19 | 88.13 | 87.81 | 91.72 | 87.46 |
| ParSer w/ RL | 97.97 | 97.66 | 99.06 | 98.75 | 98.36 |
Response to W2
Benefit of parallel processing for GPU utilization
The 11× speedup is measured at concurrency 1, where MemAgent processes one chunk at a time on a mostly idle H100 while ParSer batches all chunks, so it mainly reflects how well each method uses an idle GPU.
We acknowledge that exploiting parallelism within a document contributes to the large single-request gain. This is an intended property of the architecture: sequential memory updates depend on the preceding update, whereas ParSer exposes independent reader calls within each round, allowing one question to use the available batching capacity.
Resource equivalence
these are also the settings where ParSer's extra RTX3090 can work in parallel with the H100.
While RTX 3090’s BF16 compute performance is about 7% of the H100 (71 vs. 989 TFLOPS), we agree that the comparison does not establish strict resource equivalence. We have therefore re-evaluated ParSer, MemAgent, and Full-context with each complete method restricted to one H100; ParSer’s lead and readers share that GPU , with the lead agent allocated 30% of the GPU memory and the subagent utilizing the remaining 70%. Each cell of the updated efficiency table reports the average per-question latencies / amortized inference time.
| Concurrency | Method | 7K | 14K | 28K | 56K | 112K | 224K | 448K | 896K |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Full-context | 1.58 / 1.58 | 1.68 / 1.68 | 2.74 / 2.74 | 3.74 / 3.74 | 8.05 / 8.05 | 17.06 / 17.06 | 52.28 / 52.28 | 186.42 / 186.42 |
| 1 | MemAgent | 10.56 / 10.56 | 15.99 / 15.99 | 30.40 / 30.40 | 58.49 / 58.49 | 112.59 / 112.59 | 222.86 / 222.86 | 438.61 / 438.61 | 876.20 / 876.20 |
| 1 | ParSer | 4.37 / 4.37 | 5.04 / 5.04 | 6.10 / 6.10 | 7.62 / 7.62 | 11.14 / 11.14 | 17.73 / 17.73 | 29.38 / 29.38 | 54.27 / 54.27 |
| 16 | Full-context | 6.71 / 0.64 | 10.01 / 0.87 | 15.68 / 1.00 | 33.39 / 2.36 | 139.58 / 9.19 | 416.15 / 26.58 | 1394.46 / 90.82 | — |
| 16 | MemAgent | 18.72 / 1.21 | 31.72 / 2.08 | 57.49 / 3.64 | 111.22 / 7.05 | 215.93 / 13.64 | 435.27 / 27.38 | 853.56 / 53.61 | 1652.67 / 103.65 |
| 16 | ParSer | 17.21 / 1.18 | 22.30 / 1.50 | 36.27 / 2.37 | 60.56 / 3.94 | 125.14 / 8.01 | 242.25 / 15.51 | 473.48 / 30.51 | 932.85 / 60.78 |
| 32 | Full-context | 11.18 / 0.64 | 17.91 / 0.63 | 30.85 / 1.26 | 104.22 / 3.65 | 269.62 / 8.84 | 785.18 / 26.58 | 2739.39 / 91.12 | — |
| 32 | MemAgent | 26.66 / 0.88 | 43.97 / 1.46 | 81.52 / 2.59 | 157.36 / 5.01 | 309.91 / 9.78 | 611.45 / 19.26 | 1221.31 / 38.35 | 2396.76 / 75.21 |
| 32 | ParSer | 27.14 / 0.94 | 37.26 / 1.28 | 67.67 / 2.27 | 131.78 / 4.41 | 237.97 / 8.00 | 467.93 / 15.63 | 917.11 / 30.58 | 1824.88 / 61.09 |
Clarifying the latency-scaling claim
the paper says latency scales with reasoning depth and not with document length, but ParSer's time at concurrency 1 roughly doubles with each doubling of length beyond 112K, even though the number of steps stays near 4.
Thank you for the opportunity to clarify this claim. In Lines 74-76, we mainly discuss the constructural advantage of ParSer and our claim refers to idealized execution with sufficient compute and memory. Under these conditions, ParSer’s per-round latency is approximately fixed and independent of document length, so its idealized latency scales with the number of query-driven reasoning rounds. In contrast, MemAgent requires a dependent memory update for each chunk. So, even at the condition of sufficient resource, its sequential latency scales with document length.
For ParSer, with fixed GPU resources (the practical condition in our experiments), longer documents or higher concurrency can require more reader execution, increase resource contention, and limit cache retention. Consequently, each round takes longer, and then leads to longer total latency even when the number of rounds remains stable. This is the reason for the latency trend reported in the above table and Table 2, which has been discussed in Appendix B. Thanks again for raising this point. We will describe these assumptions more clearly in the revised paper.
Response to Reviewer NHob (Part 3)
Response to W3
Context compactness. We agree that our design cannot guarantee bounded context provided to the lead agent with longer document, as the lead-context size depends on the findings returned across queries, including false-positive findings. As Table 6 shows, a 128-fold increase in document length, from 7K to 896K tokens, corresponds to an increase in reported subagent output from 312 to 804 tokens, approximately 2.6-fold. While the observed growth is modest, we agree that specialized filtering methods like employing stronger readers could further improve context compactness.
Robustness to misleading findings. Stronger readers may reduce the occurrence of misleading findings in the lead agent's context, but more specific follow-up queries and better conflict resolution by the lead agent, especially the one after RL training, can also help. To explain this, we add an ablation study for 4B lead agent where the finding that does not hit a supporting evidence is discarded. This provides an oracle filtered condition for findings. The below table shows the findings precison (the fraction of returned findings that hit a supporting evidence) and accuracy.
| Sub_EM/Findings precision (%) | Before RL | After RL | RL gain (pp) |
|---|---|---|---|
| Original setting | 74.41/63.01 | 84.57/60.97 | +10.16 |
| Oracle filtered setting | 80.47/95.84 | 87.99/95.02 | +7.52 |
For the trained model, moving from the oracle filtered condition to the original setting reduces finding precision from 95.02% to 60.97%, while Sub_EM decreases by only 3.42 percentage points, from 87.99% to 84.57%. This shows that the trained lead agent is more robust to noisy findings.
RULER evaluation on dense-information QA. Appendix C.2 evaluates Frequent Words Extraction (FWE) on RULER, which asks for the three most frequent coded words across a long text. Although its answer contains only three words, the underlying frequency information is densely distributed: in the 512K test set, each example contains approximately 91K–104K non-filler word occurrences. FWE therefore provides complementary evidence on processing densely distributed repeated information. ParSer achieves a Sub_EM of 94.53% with Qwen3.5-4B backbone and 98.96% with 9B backbone, outperforming MemAgent (Sub_EM of 22.31% and 97.38% respectively). The evaluation details are in Appendix C.2 in our revised submission.
Response to W4
Appreciate the opportunity to further clarify our design considerations for the experiment of evidence position and distance control in Sec. 4.1. The two experiments vary complementary aspects of evidence layout. Evidence Position Control keeps the relative separation between the supporting evidence paragraphs fixed and varies their proximity to the document boundaries. Evidence Distance Control, conversely, keeps the prefix before the first evidence paragraph and the suffix after the second fixed, and varies the separation between the evidence paragraphs by inserting additional distractors between them. The latter therefore increases the intervening interference while leaving the outer padding unchanged. We acknowledge that this construction increases total document length. This increase is part of the intended test: one reason longer documents degrade sequential memory agents is that more distractors intervene between dependent evidence, requiring earlier evidence to survive more recurrent memory updates before it can be combined with later evidence.
Response to Q1
Our filtering relies on subagent abstention: if a subagent judges its chunk irrelevant to the query, it returns Unknown, which is discarded. All non-abstaining responses are passed to the lead agent. We do not impose an explicit bound on the accumulated observations.
Response to Reviewer NHob (Part 1)
We sincerely thank you for your time, constructive feedback and valuable suggestions. We are deeply grateful for your recognition of ParSer's novelty. We also appreciate your kind words regarding the experiment soundness of our work. Below, we respond point-by-point to address your concerns, clarify details, and provide additional analyses.
Response to W1
Clarification on our research question
Thank you for raising this concern and for the chance to further clarify our research question. Our research asks how far LLMs can go in long-context QA when simple reading and reasoning workflows rely primarily on the models’ native capabilities. This motivates two design principles.
Our primary comparisons focus on basic ways of organizing an LLM’s reading and reasoning with minimal workflow design. Although LongAgent and XpandA incorporate specialized mechanisms beyond this focus, they remain closely related approaches to long-context QA. To address the reviewer’s concern, we therefore extend our evaluation to these methods: LongAgent is adapted using its released codes, and XpandA is implemented from its published algorithm, preserving their respective conflict-resolution and exploration/replay mechanisms. For fair comparison, we enable thinking for LongAgent’s leader and XpandA’s decider, with a maximum of 2,048 output tokens per call, a same level of budget as ParSer. None of the methods have been trained.
ParSer (w/o RL) achieves higher accuracy in every evaluated backbone–dataset–length setting. These results support our central design choice: a simple reading-reasoning interface that follows the model’s native working mode can be more effective than the specialized workflows evaluated here. Our results provide empirical support for leveraging the model’s native agentic capabilities more, rather than relying on highly customized workflows, especially in the era of agentic LLMs.
References:
[1] LongAgent: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration.
[2] Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration.
[3] Multi-Agent Collaboration via Evolving Orchestration.
[4] WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning.