- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
BiGate proposes selective communication for heterogeneous multi-agent reasoning. Each agent pair receives directional ternary judgments. The sender's judgment determines whether feedback is needed, while the receiver's judgment determines whether the sender's full reasoning can accompany that feedback. Separate verification and revision contexts enforce this distinction. Experiments use three models and five benchmarks, with 200 sampled examples per benchmark. The paper reports 73.90% macro-average accuracy versus 71.60% for Peer Review, alongside fewer reported model calls and response rounds. Gate ablations and a persistent-error analysis examine how the mechanism changes communication and restricts admission of erroneous full reasoning into subsequent answer revision.
Critical strengths
-
Clear separation of two communication decisions. The sender controls whether feedback is needed, while the receiver controls whether full reasoning is admitted. Separate revision contexts make this distinction operational and preserve potentially useful local criticism from untrusted solutions.
-
Relevant comparisons and targeted ablations. The evaluation uses shared models and examples across five benchmarks. Table 4 compares both gates individually and jointly, with BiGate reporting higher accuracy and lower token usage than either single-gate configuration.
Critical weaknesses
- Call accounting appears inconsistent with the stated protocol. Each revision stage requires three verification calls and at least one revision call. Thus, 2.48 response rounds imply at least
calls, exceeding the reported 8.80. Caching, skipped verification, or a different round definition might explain this, but none is specified. This affects the central efficiency claim.
- The incremental accuracy benefit needs stronger support. Peer Review and FullConnect have identical aggregate accuracy and cost metrics, yet their implementation relationship is unclear. Moreover, the gains over Peer Review and Sender-Only---2.30 and 1.50 percentage points---lack paired uncertainty estimates; the two reported seeds concern sample selection rather than repeated generation.
Questions for authors
1.Can you reconcile the reported calls and response rounds by providing separate initial-generation, verification, and revision counts? Please clarify whether termination checks count as response rounds and whether verification is cached or skipped.
2.Are Peer Review and FullConnect the same implementation? Please clarify their relationship and provide paired uncertainty estimates for BiGate versus Peer Review and Sender-Only.