- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AttriSwarm, a multi-agent framework for cross-page visual attribution. It uses a leader agent for query decomposition and global reasoning, while parallel subagents independently identify relevant pages and localize supporting evidence regions. The authors also introduce AttriDoc, a dataset with page- and region-level annotations, and develop role-specific SFT and RL training. Experiments across multiple models and datasets show improvements in answer accuracy, attribution quality, and robustness to distractor pages.
Critical strengths
- The paper addresses a meaningful problem of cross-page visual attribution, where both the supporting pages and the relevant evidence regions need to be identified.
- The experiments cover multiple backbones and datasets, with encouraging improvements in answer accuracy and attribution quality. The new AttriDoc dataset is also a useful contribution.
Critical weaknesses
1. Insufficient insights into the proposed architectural design.
The main idea seems to be a combination of existing techniques. LongAgent [1] has explored leader–worker decomposition for document reasoning, while ORCA [2] and MACT [3] have investigated specialized multi-agent collaboration for visual document understanding. Fine-grained visual source attribution has also been studied in VISA [4].
Combining existing ideas is not a problem in itself. However, I would expect a deeper discussion of why this particular combination works well for cross-page visual attribution. The paper does not sufficiently explain what challenges are unique to this task, or why the proposed coordination between global reasoning and local attribution is more effective than simpler alternatives. More analysis of the components and their interactions would help clarify the contribution.
2. Insufficiently controlled comparisons to establish the benefits of the proposed architecture.
Although AttriSwarm achieves substantial improvements, the current comparisons make it difficult to determine how much of the gain comes from the architecture itself. In Table 2, the trained AttriSwarm is compared mainly against training-free baselines or existing checkpoints without task-specific adaptation. Interestingly, the training-free AttriSwarm does not consistently outperform the single-pass baseline.
Table 3 shows that role-specific training contributes substantially to the improvements, but it remains unclear whether a simpler architecture could achieve similar results with the same training data. Without a training-matched comparison, the advantages of the proposed architecture are not fully established.
[1] Zhao et al. LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration. EMNLP 2024.
[2] Lassoued et al. ORCA: Orchestrated Reasoning with Collaborative Agents for Document Visual Question Answering. CVPR 2026.
[3] Yu et al. Visual Document Understanding and Reasoning: A Multi-Agent Collaboration Framework with Agent-Wise Adaptive Test-Time Scaling. CVPR 2026.
[4] Ma et al. VISA: Retrieval Augmented Generation with Visual Source Attribution. ACL 2025.
Questions for authors
I suggest the author to provide more detailed architectural ablations and analysis to explain the contributions of context isolation, query decomposition, and leader-based evidence aggregation, particularly their benefits for cross-page reasoning. Also, it's also suggested to include a single-pass or shared-context baseline trained with the same AttriDoc supervision and comparable training budgets to better distinguish architectural improvements from the effects of task-specific training. I would like to reconsider the rate if the rebuttal is sufficient.
5 comments
Rebuttal for Weakness 2 and Question 2
W2&Q2: Insufficiently controlled comparisons to establish the benefits of the proposed architecture. Table 3 shows that role-specific training contributes substantially to the improvements, but it remains unclear whether a simpler architecture could achieve similar results with the same training data. Also, it's also suggested to include a single-pass or shared-context baseline trained with the same AttriDoc supervision and comparable training budgets to better distinguish architectural improvements from the effects of task-specific training.
Thank you for highlighting the need for a trained simpler baseline. We agree that comparisons against training-free baselines alone cannot establish whether a simpler architecture could achieve similar results with task-specific supervision. We therefore added a single-pass baseline using the same Qwen3-VL-4B-Instruct backbone, trained with SFT and GRPO on AttriDoc supervision to generate the answer and visual attributions together.
AttriSwarm relies on page-wise agents to select relevant evidence before global reasoning. Without task-specific training, these agents may incorrectly filter out necessary pages, leaving the leader unable to cite their evidence and potentially lacking information needed to answer. This is one reason why training-free AttriSwarm does not consistently outperform single-pass, which retains all retrieved candidate pages in its input. Role-specific training helps the agents make more reliable relevance and localization decisions.
| Dataset | Method | Acc. | Page F1 | Reg. IoU | JAS |
|---|---|---|---|---|---|
| AttriDoc-Paper | Single-pass (trained) | 4.78 | 36.11 | 23.31 | 14.04 |
| AttriDoc-Paper | AttriSwarm (trained) | 25.86 | 62.53 | 47.91 | 37.40 |
| AttriDoc-Slide | Single-pass (trained) | 54.32 | 28.77 | 17.95 | 14.62 |
| AttriDoc-Slide | AttriSwarm (trained) | 64.73 | 71.91 | 46.57 | 40.71 |
| CiteVQA | Single-pass (trained) | 34.55 | 31.57 | 16.75 | 11.03 |
| CiteVQA | AttriSwarm (trained) | 45.61 | 53.80 | 31.30 | 24.07 |
Trained AttriSwarm outperforms the trained single-pass baseline in answer accuracy and all three attribution metrics across the three datasets. Single-pass uses the original query with Top10 retrieval, whereas AttriSwarm uses its full retrieval and coordination pipeline. This comparison supports the complete trained system; the architectural ablations separately assess the contributions of individual design choices.
Rebuttal for Question 1
Q1: I suggest the author to provide more detailed architectural ablations and analysis to explain the contributions of context isolation, query decomposition, and leader-based evidence aggregation, particularly their benefits for cross-page reasoning.
Thank you for this suggestion. We agree that more detailed ablations help to explain the contributions of different components. Here, we add three ablation studies.
Context isolation. Context isolation allows each attribution subagent to assess and localize partial evidence on its assigned page without competing page content, while the Leader connects these findings for cross-page reasoning. To examine shared-context attribution as an alternative to page-wise processing, we trained a 4B attribution subagent to jointly assess relevance and localize evidence across candidate pages in one response, rather that isolating different page contexts to attribution agent swarms. We evaluated it on all 669 AttriDoc-Paper questions, retaining the trained leader and the retrieval pipeline with the same retrieval and interaction limits.
| Attribution Context | Acc. | Page F1 | Reg. IoU | JAS |
|---|---|---|---|---|
| Trained shared-context attribution | 8.97 | 34.55 | 11.16 | 7.64 |
| AttriSwarm (trained page-wise attribution) | 25.86 | 62.53 | 47.91 | 37.40 |
Page-wise attribution achieves higher scores across all four metrics, supporting its effectiveness.
Query decomposition. Query decomposition targets complementary facts required by cross-page questions, helping retrieval collect evidence for different parts of the answer. On the AttriDoc-Paper test set, we restrict retrieval to the original question's Top10 results while retaining the trained Leader and attribution subagents.
| Retrieval setting | Acc. | Page F1 | Reg. IoU | JAS |
|---|---|---|---|---|
| Original query, Top10 | 22.42 | 58.98 | 48.03 | 36.37 |
| Full AttriSwarm (with query decomposition) | 25.86 | 62.53 | 47.91 | 37.40 |
Executing decomposed queries improves answer accuracy, page selection, and JAS. Region IoU is similar in the two settings. Because the retrieved candidate pages also change, this experiment measures the contribution of decomposed retrieval within the pipeline, rather than question decomposition in isolation.
Leader-based evidence aggregation. Individual pages may provide only partial facts needed for a cross-page answer. To examine whether independent local answers sufficent, we replay the archived retrieval and attribution schedules, preserving the original boxed overviews and detail-crop configurations. The same trained Leader answers each retained page in a separate branch with its own history. We then concatenate these answers without a final cross-page synthesis step.
| Dataset | Per-page answers + concatenation: Acc. | Full AttriSwarm: Acc. |
|---|---|---|
| AttriDoc-Paper | 2.09 | 25.86 |
| AttriDoc-Slide | 35.98 | 64.73 |
| CiteVQA | 32.34 | 45.61 |
Concatenation collects local answers but does not explicitly compare, reconcile, or combine their facts into a joint conclusion. Independent page-level answers with concatenation perform substantially worse across all three datasets, suggesting that collecting local answers alone without further selection and global understanding is insufficient for these questions. These results support the value of jointly interpreting complementary evidence to produce a coherent answer. The reference scores are from the reported full pipeline, while the concatenation variant replays its archived evidence acquisition schedule.
Rebuttal for Weakness 1
Thank you for asking us to clarify the architectural rationale and why these components are combined for cross-page visual attribution. We agree that leader-worker coordination, agent specialization, and visual source attribution have each been studied before. Our design is motivated by the need to satisfy two requirements together: each evidence region must be grounded on the correct page, while the answer may depend on complementary facts distributed across several pages. A page can therefore be relevant even when it cannot answer the question on its own. Missing a necessary page prevents complete attribution and may leave insufficient evidence for answering.
AttriSwarm assigns these requirements to complementary roles. The leader identifies the evidence needed for the question; page-wise attribution agents assess whether each page supports any part of it and localize the corresponding regions in separate contexts. Each attribution agent returns the relevant regions and findings from its assigned page. The leader combines these relevant regions to answer questions that require information from multiple pages. This allows each agent to focus on local evidence while the leader handles cross-page reasoning.
This also explains the choice relative to simpler alternatives: (1) Jointly processing all pages allows cross-page reasoning, but similar content across pages may interfere with local relevance assessment and region localization. Figure 4 in the submission provides system-level evidence of robustness to such distractors. (2) Separating the leader and attribution roles alone does not remove this shared context if the attribution agent still reads all pages together. To examine this distinction, we additionally evaluate a shared-context attribution variant trained to assess and localize evidence across multiple pages in one response; this comparison is reported below in response for Q1. (3) A single attribution agent could process pages one at a time in separate contexts, but sequential processing increases latency as the number of pages grows. AttriSwarm executes these independent page-wise calls in parallel. Figure 5 in the submission directly supports this choice, showing approximately a 4.1× speedup over the serial variant with 32 distractors on successful post-retrieval runs.
To sum up, the contribution of AttriSwarm lies in coordinating isolated, parallel page-wise attribution with global answer synthesis, allowing local evidence to remain linked to its source page while being combined for cross-page reasoning.