- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper introduces TrustFlow, a collaborative framework for LLM multi-agent systems based on counterfactual information flow control. The framework employs a joint reliability belief estimator to assess the joint correctness states of multiple agents and identify those agents requiring intervention. It also evaluates the downstream benefits of various communication topologies and intervention actions. The authors utilize GRPO-style policy optimization to train a controller that dynamically adjusts information flow based on the current collaborative state. They conduct experiments on MATH, MMLU, HumanEval, and TruthfulQA, showing that this method improves average reasoning performance, enhances error recovery capabilities, and reduces the consumption of reasoning tokens.
Critical strengths
- The paper introduces a joint reliability belief over agent correctness configurations, potentially capturing correlated errors beyond independent trust scores.
- The paper evaluates multiple reasoning benchmarks and includes ablation, robustness, recovery, and efficiency analyses, demonstrating promising performance and reduced inference token consumption.
Critical weaknesses
- The method's novelty is limited. The proposed framework mainly combines existing reliability estimation, counterfactual assessment and strategy optimization techniques. Specifically: (i) Although the joint reliability confidence is intended to capture the correlation between agent errors, the selection of intervention anchors is based on marginal accuracy, and no comparison with independent reliability estimates is provided to demonstrate the necessity of joint modeling; (ii) The GRPO-based counterfactual controller is built on mature reward-driven topology optimization and policy learning techniques, but the paper lacks comparative experiments with simpler outcome-based policy optimization methods, failing to establish the unique advantages of counterfactual feedback and GRPO.
- The core contributions have not been experimentally verified. On MMLU (+0.50%), HumanEval (+1.22%), and TruthfulQA (+0.72%), the performance improvements relative to the strongest baseline were negligible and lacked sufficient statistical significance analysis. More importantly, ablation experiments failed to distinguish the advantages of counterfactual feedback and GRPO from simpler outcome-based optimization methods. Furthermore, the efficiency improvements in the report overlooked the additional training costs associated with counterfactual rollouts.
Questions for authors
- Compared with existing counterfactual and reward-based topology optimization methods, what are the key methodological advancements of TrustFlow? Could the authors provide comparative experiments with independent reliability assessments and simpler strategy optimization alternatives to demonstrate the necessity of the proposed component?
- Could the authors provide statistical significance analysis and additional ablation experiments to compare counterfactual GRPO training with standard outcome-based policy learning? How much additional computational overhead does performing counterfactual rollouts during training incur?
3 comments
Thank you for your reponse. I have raised my score.
Thank you for the update and for reconsidering our response. We hope our response reflects the rigor and care we have brought to this work, and we will address the remaining concerns carefully in the revision.
Thank you for your feedback.
W1 & Q1. Methodological Novelty and Component Necessity
(1) Methodological contributions.
TrustFlow goes beyond conventional reward-driven topology optimization through two complementary mechanisms:
The key distinction is the integration of joint reliability beliefs and counterfactual intervention feedback into adaptive information-flow control.
(2) Joint vs. Independent Reliability Estimation.
To directly examine the necessity of joint reliability modeling, we conducted additional comparisons against independent agent reliability estimation.
The joint reliability approach consistently outperforms independent estimation on both evaluated datasets, with a particularly notable improvement on HumanEval. These results provide empirical support for modeling dependencies among agent correctness states rather than relying exclusively on independent reliability estimates.
(3) Counterfactual vs. Outcome-Based GRPO.
We further evaluated a standard outcome-based GRPO baseline to isolate the contribution of counterfactual feedback from that of GRPO optimization itself.
Counterfactual GRPO outperforms outcome-based GRPO by 2.22 percentage points, demonstrating the benefit of counterfactual feedback beyond final task rewards.
(4) Reliability-only Topology Selection.
To address the reviewer's concern, we introduce a training-free reliability-only baseline that selects communication topologies directly from estimated agent reliability scores, without counterfactual feedback or a learned Controller. The experimental results on TruthfulQA are summarized below:
Over three runs on 64 questions, TrustFlow outperforms the reliability-only baseline by 1.08 percentage points, suggesting benefits beyond reliability-based topology selection.
Q2. Statistical Significance, Ablation Studies, and Computational Overhead
(1) Statistical reliability and additional ablations. To address the reviewer's concerns, we conducted supplementary evaluations with three repeated runs per method. On TruthfulQA, TrustFlow achieves 59.37 ± 1.56%, outperforming EIB Learner. We further evaluated 50 challenging MATH problems (difficulty Levels 4–5) to examine performance on complex reasoning tasks, where TrustFlow achieves 46.67 ± 1.15%, compared with 44.67 ± 2.06% for EIB Learner.
(2) End-to-end inference efficiency. Figure 5 reports inference token consumption across three rounds of multi-agent discussion and Controller execution. We additionally measured end-to-end inference latency under identical hardware and model settings. TrustFlow reduces average latency by 25.25% and 18.76% compared with LLM Debate and DoV-Follower, respectively. Counterfactual rollouts are performed only during offline training, while inference directly uses a lightweight Belief Estimator and a compact Qwen3-1.7B Controller, avoiding rollout overhead during deployment.
(3) Training Overhead. On TruthfulQA with an A100 GPU and 600 matched optimizer updates, counterfactual GRPO required 3.957 hours, versus 2.586 hours for outcome-based GRPO (+53%). Most additional time (79.2 of 82.3 minutes) came from counterfactual rollout collection, which can be parallelized or precomputed offline. Thus, the overhead is manageable and does not affect inference efficiency.
We thank the reviewer for their constructive feedback. All additional results and clarifications will be included in the revised PDF.