- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes LAM-RAG, a multi-agent framework for multi-document, multi-answer QA. Document-specific agents communicate through LoRA-based messages, with reliability-based filtering and scaling, before final answer aggregation. The method aims to reduce conformity during inter-agent communication while preserving complementary correct answers.
Critical strengths
The paper identifies an important failure mode of multi-LLM debate in multi-answer QA: agents may discard correct, locally supported answers after adopting their peers’ responses.
The experimental evaluation is extensive, covering multiple datasets and model backbones, with a broad range of baselines, ablations, and analyses of agent behavior.
Critical weaknesses
The paper does not adequately explain why low-rank parameter updates should specifically address conformity. The intuition connecting this representation to the preservation of independent judgments, while still allowing useful information exchange, remains underdeveloped.
The reduced loss of correct answers may reflect weaker peer influence under a restricted communication channel. Although improvements over no exchange demonstrate the value of sharing complementary information, they do not establish why LoRA is preferable to other forms of partial interaction. The current experiments do not sufficiently distinguish the proposed mechanism from this alternative explanation. Comparisons that control the amount or strength of communication would help justify the design choice.
Table 2 shows modest, setting-dependent gains over the no-exchange setting, which already aggregates independent local answers. This raises the question of whether similar performance could be achieved through simpler procedures, such as retaining the union of independently supported answer candidates or having a central agent verify independently generated document summaries and proposed answers. More direct comparisons are needed to establish the value of mutual exchange over these alternatives.
Questions for authors
What makes LoRA particularly suitable for reducing conformity beyond limiting peer influence? Can the authors provide a clearer explanation and controlled comparisons with alternative forms of partial interaction?
How does LAM-RAG compare with preserving the union of initial answer candidates or centrally verifying document-level summaries and answers, under comparable evidence and computational budgets?