acceptodds
Under review as a conference paper at ICLR 2027

SEPAL: Separated Expert Pairs with Answer-Level Fusion for Reliable LLM Collaboration

Abstract

Multi-agent collaboration lets large language models (LLMs) improve question answering through deliberation and feedback. Yet shared discussion couples correction with exposure to the same mistakes, which can erode the diversity needed for voting. Self-consistency offers sampling diversity without feedback, while single-pair Actor–Critic collaboration refines only one candidate. We introduce SEPAL, which assigns three private Actor–Critic teams to direct reasoning, evidence grounding, and verification. Role-specific training gives the teams different reasoning objectives beyond sampling variation. Each Critic guides revisions within its own team, preventing feedback from carrying errors across candidates. Once revision ends, majority voting combines only the final answers, keeping the reasoning histories separate until the decision. Across five open-weight backbones and five question-answering benchmarks, SEPAL improves mean accuracy by 1.81 percentage points over a matched single Actor–Critic pair, with improvements across all five backbones.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, multi-agent collaboration, actor-critic, iterative refinement, answer-level fusion, preference optimization, majority voting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YRcrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SEPAL, a multi-agent LLM collaboration framework designed to improve reasoning reliability while reducing error propagation among agents. SEPAL employs three independent Actor–Critic teams specializing in direct reasoning, evidence grounding, and verification. Each team undergoes role-specific supervised fine-tuning and preference optimization, followed by private iterative revisions. The final answers are combined through majority voting without sharing intermediate reasoning histories. Experiments across five LLM backbones and five question-answering benchmarks demonstrate consistent improvements over a single Actor–Critic baseline, with an average accuracy gain of 1.81 percentage points. Ablation studies further highlight the effectiveness of iterative revision and answer-level fusion.

Critical strengths

1. Clear Motivation and Practical Design: The paper addresses error propagation in multi-agent collaboration through independent Actor–Critic teams and a simple answer-level voting mechanism.

2. Consistent Experimental Results: Experiments across five LLM backbones and five benchmarks show consistent improvements over the single Actor–Critic baseline, supported by detailed ablation and revision analyses.

Critical weaknesses
  1. Insufficient Validation of the Core Isolation Mechanism: The paper argues that private Actor–Critic interactions prevent error propagation and preserve reasoning diversity. However, no direct ablation compares private and shared communication under identical settings (same roles, training, revision rounds, and computational budget). Therefore, it remains unclear whether the improvements actually come from the proposed isolation mechanism or simply from using multiple independently generated answers.

  2. Unfair Computational Budget and Limited Efficiency Analysis: SEPAL requires 30 model generation calls per question (15 Actor and 15 Critic calls), compared with only 10 calls for the single-pair ACC-Collab baseline, while achieving an average accuracy improvement of just 1.81 percentage points. The paper lacks comparisons under matched inference budgets, as well as detailed latency and token consumption analyses. Moreover, the revision analysis shows that the first revision already contributes approximately 89.2% of the total improvement, raising questions about the necessity of four revision rounds.

  3. Limited Benefits from the Complex Training Pipeline: Table 3 shows that adding Critic DPO improves average accuracy from 75.74% to 76.30%, while further Actor DPO provides only a 0.01-point improvement. Furthermore, simpler configurations outperform the full method on some backbones, such as Qwen2.5-3B and Phi-4-mini. These results raise concerns about whether the additional preference optimization stages are necessary.

  4. Insufficient Evidence for Role Specialization: Although SEPAL introduces Direct, Evidence, and Verification roles, all three teams share the same backbone and training questions. The paper does not compare the proposed role-specialized ensemble with three independently trained teams using identical roles. Consequently, it is difficult to determine whether the performance gains arise from meaningful role specialization or simply from independent sampling and majority voting.

  5. Limited Statistical Reliability and Generalization: All results are based on a single evaluation run without standard deviations, confidence intervals, or statistical significance tests, making the relatively modest improvements difficult to assess. Additionally, experiments are restricted to short-answer question-answering benchmarks, leaving the effectiveness of SEPAL on open-ended generation and more complex reasoning tasks insufficiently explored.

Questions for authors
  1. Effectiveness of the Isolation Mechanism: The central claim of SEPAL is that private Actor–Critic interactions prevent error propagation and preserve reasoning diversity. However, the paper lacks a direct comparison between private and shared communication. Could the authors provide an ablation that changes only the communication strategy while keeping the roles, training configurations, revision rounds, and computational budget identical? This would help determine whether isolation itself is responsible for the observed improvements.

  2. Computational Fairness and Necessity of the Full Pipeline: SEPAL requires 30 model generation calls per question, compared with 10 for ACC-Collab, but achieves only a 1.81-point average accuracy improvement. Moreover, Table 3 shows that Actor DPO contributes only 0.01 points, while the first revision accounts for approximately 89.2% of the total revision gain. Could the authors clarify whether SEPAL still outperforms simpler alternatives under matched inference budgets, particularly configurations with fewer revision rounds or without Actor DPO? These comparisons would substantially affect the assessment of the method's efficiency and practical value.

Reviewer_cRa8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SEPAL combines three independently trained Actor–Critic teams with majority voting over final answers. The teams specialize in direct reasoning, evidential support, and verification, and exchange feedback only within their respective pairs. Actors first undergo role-specific supervised initialization, after which Critics and Actors receive preference training based on the correctness of subsequent Actor continuations. At inference time, each team independently performs four rounds of revision before a deterministic vote combines final answers. Evaluation spans five open-weight backbone models and five question-answering benchmarks, while training is restricted to MMLU. Relative to a matched single Actor–Critic pair, average macro accuracy improves by 1.81 percentage points. Component and trajectory analyses indicate that most gains arise in the first feedback exchange, with diminishing returns from later revision rounds.

Critical strengths
  1. The independent-team architecture and deterministic final vote are simple and interpretable, enabling local correction without exposing every candidate answer to the same feedback.
  2. Evaluation spans five backbone models and substantial test sets, with four benchmarks excluded from training. Improvements over the single-pair ACC baseline appear in 24 of 25 model–dataset combinations.
  3. The component and round-wise analyses are candid and informative: they distinguish contributions from feedback, initialization, and preference learning, and quantify both correction and degradation instead of portraying extra revision as uniformly beneficial.
Critical weaknesses
  1. The main comparison does not control computational budget: SEPAL uses three Actor–Critic trajectories totaling 30 generations, versus 10 for ACC. Comparisons with three-sample or three-pair ACC ensembles and self-consistency under matched training and inference budgets are needed to establish value beyond ordinary ensemble scaling.
  2. The central isolation and diversity hypotheses are not directly tested. All component ablations retain private histories and the same three roles; thus, neither a matched shared-history variant nor a multiple-pair same-role control is provided to show that communication isolation and role specialization explain the gains.
  3. The results offer limited support for the necessity of the full training-and-revision pipeline. After feedback from a trained Critic, Actor DPO adds only 0.01 percentage points of macro accuracy, while rounds R2–R4 contribute only another 0.40 percentage points relative to R1. Cost–accuracy comparisons with these simpler configurations would better justify the full method.
Questions for authors

See Weaknesses.

Reviewer_aiq8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SEPAL, a multi-agent question-answering framework with three separately trained Actor–Critic pairs assigned to direct reasoning, evidence grounding, and verification. Each pair performs four private revisions, after which only the final answers are combined by majority voting, with the Direct answer used as a fixed fallback. Training combines role-specific supervised fine-tuning with continuation-based preference learning. Experiments cover five open-weight backbones and five question-answering benchmarks, using MMLU for training.

Critical strengths
  1. Clear aggregation procedure. The method separates private revision from final-answer voting and specifies a fixed fallback without an additional model judge. This makes the decision procedure easy to follow.
  2. Evaluation across several models and datasets. The experiments cover five backbones and five QA benchmarks, including four datasets not used for training. This provides a broader empirical view than evaluation on a single model or dataset.
Critical weaknesses
  1. Although the method uses three separate teams, their substantive differences are unclear. Regarding the error-propagation problem emphasized in the paper, keeping the teams separate prevents errors from spreading between them during revision, but incorrect answers may still affect the final decision through fusion. It is therefore unclear to what extent the method addresses this problem.
  2. The evaluation uses general-purpose benchmarks. Could the authors also consider benchmarks specifically designed to evaluate multi-agent methods?
  3. The experimental evaluation could be strengthened. Beyond accuracy, additional analysis of how errors accumulate across rounds would help assess whether the method reduces error accumulation. Case studies showing how the actual outputs change during revision would also be useful.
  4. The method involves both multiple agents and separate model training. How do its training and inference costs compare with those of other methods, and do these costs remain manageable as the system scales?
Questions for authors

Just see weeknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.