- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes FCA for training the operator-selection controller in multi-agent architecture search. During training, FCA uses both architecture-level and operation-level advantages: it computes operation rewards from the correctness of intermediate answers and token costs, then estimates operation-level advantages by comparing rewards under similar reasoning states. The authors also use operator distribution projection to maintain sampling diversity. The experiments use Qwen3-8B and GPT-4o-mini and cover MATH, GSM8K, HumanEval, MBPP, and BBH. Compared with MaAS and RAAS, FCA achieves better task performance and consumes fewer inference tokens in most settings.
Critical strengths
-
The experiments show positive results for both performance and inference costs. With both models, task performance on all five benchmarks exceeds that of the authors' reproductions of MaAS and RAAS, while fewer inference tokens are consumed in most settings.
-
The training analysis on BBH gives a fairly direct illustration of the credit assignment problem. By tracking the answers at each step, the authors find that the training signal for the whole architecture may not match the effect of a specific operation. They also show cases where computation continues after a correct answer is obtained and where a later operation turns a correct answer into an incorrect one. These observations show how redundant computation and answer degradation occur and explain why operation-level feedback is needed.
Critical weaknesses
-
Necessity and novelty of the research, and the independent contribution of the core mechanism
An important role of early multi-agent workflows of this kind, which split a long reasoning process into multiple stages, was to alleviate limitations in models' context capacity and reasoning ability. However, as model capabilities improve, context windows expand, and latent chain-of-thought techniques develop, the need for splitting tasks into external stages requires renewed justification. In this paper, the authors evaluate only two models and use short, linear workflows. I think this is insufficient to show that the framework still offers clear additional value with stronger models. In addition, the paper retains the existing operator-selection controller and search space, with its main improvements focusing on operation-level rewards, state-conditioned advantage estimation, and exploration control. I view these designs as closer to incremental optimizations of an existing framework.
The “+SRA” setting in Table 3 simultaneously introduces operation-level rewards, the reward rules for the first correct answer and early stopping, discounting and operation-level cost penalties, and a reference reward based on state similarity. Therefore, the current ablation cannot distinguish whether the improvements in performance and reductions in inference cost come from the operation-level rewards themselves or from SRA's state-conditioned comparison. Although Table 5 in the appendix compares soft weighting with hard grouping, both use state similarity and cannot replace a control that does not use state information. Without this control, it is currently not possible to determine whether state-conditioned comparison provides clear additional value over a simpler operation-level reward mechanism.
-
Full training costs, the cost-benefit trade-off, and the statistical reliability of performance improvements
Regarding training costs, although the paper reports the local computational overhead of components such as SRA and ODP in the appendix, it does not adequately report the total token consumption and wall-clock time for complete training on each benchmark, and it lacks training-cost comparisons with MaAS and RAAS under comparable conditions. Another point to note is that, when recording wall-clock time, the number of concurrent calls to the model API needs to be confirmed. This is very important for a fair comparison of the measurements across different methods. The authors should include training-stage costs, test performance improvements, and reductions in inference costs in an overall evaluation to explain whether the method offers a reasonable cost-benefit trade-off at different scales of use. This information is currently missing, so it is not yet possible to judge whether the benefits of the fine-grained reward and advantage estimation mechanisms justify the corresponding training costs.
The main experiments report only a single aggregate performance value, without variance or other statistical data, and the overall performance improvements are not large in many settings. I am concerned that these improvements could come from random fluctuations caused by uncertainty in model outputs, rather than stable gains from the method itself.
Questions for authors
-
Please explain the additional value of this framework with stronger models and what the substantive novelty of this paper is relative to existing stepwise credit assignment and exploration methods.
-
Although the main text and appendix specify the number of training epochs and that sampling is performed eight times per question to construct training signals, the description of how the training data are organized into batches and how parameters are updated remains insufficient. I think the authors need to clarify the following information: when training on a given benchmark, how many optimization steps does each epoch contain? How many training questions are used in each step/batch? After sampling eight times per question, how are the samples organized into batches for optimization? How many parameter updates are performed per batch, and are the sampled results reused?
-
Please provide the total token consumption and wall-clock time for complete training on each benchmark, along with training-cost comparisons with MaAS and RAAS under comparable conditions. Also specify the number of concurrent calls to the model API when recording wall-clock time to ensure a fair comparison across methods.
-
Please provide the variance or other uncertainty measures for the performance results to further assess whether these improvements are stable gains from the method itself.
-
While keeping operation-level rewards, costs, sampling, and optimization settings unchanged, please compare SRA with a reference reward computed without state similarity, such as uniformly weighting the operation rewards from other architectures, to assess whether state-conditioned comparison provides additional value.