- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes STAR, a trajectory-aware sentence-level routing framework for efficient multi-LLM ensembles. The method addresses the computational redundancy of frequent multi-model collaboration and the limitations of local candidate selection. STAR introduces two key components: Sentence-level Disagreement Prediction (SDP), which identifies when multi-model collaboration is necessary, and Tree-Advantage Routing Optimization (TARO), which uses trajectory-level rewards and counterfactual advantages to optimize candidate selection. Experiments across mathematical reasoning, code generation, and general reasoning benchmarks demonstrate that STAR achieves superior average accuracy while significantly reducing inference costs compared with existing ensemble methods. Ablation studies and additional analyses further support the effectiveness and efficiency of the proposed framework.
Critical strengths
1. Clear Motivation and Novel Design: The paper addresses computational inefficiency in multi-LLM ensembles through a well-motivated combination of disagreement prediction and trajectory-aware routing.
2. Strong Experimental Results: Extensive experiments demonstrate improved accuracy and reduced inference costs, supported by comprehensive ablation studies and efficiency analyses.
Critical weaknesses
-
Unclear Data Splits and Inconsistent Accuracy Reporting: The paper does not clearly explain whether the training data for SDP and TARO are strictly separated from the evaluation benchmarks, raising potential data leakage concerns. Moreover, some reported accuracy values appear inconsistent with the dataset sizes under single-run evaluation. For example, MATH500 contains 500 samples, meaning accuracy should change in increments of 0.2%, yet Table 1 reports 56.69%. Similarly, the reported 69.80% on HumanEval (164 samples) does not correspond to an integer number of correct predictions. The authors should clarify whether these results are averaged over multiple runs and provide details of the evaluation protocol.
-
Questionable Assumption and Insufficient Evaluation of SDP: The method assumes that semantic disagreement indicates when multi-LLM collaboration is necessary. However, models may agree on incorrect reasoning or disagree while producing equally valid solutions. Therefore, semantic disagreement does not necessarily reflect the actual benefit of collaboration. Furthermore, the paper only reports the agreement between LLM-generated and human annotations and the overall ensemble activation rate, without directly evaluating SDP's precision, recall, F1-score, or false negative rate. These metrics are important for validating its ability to identify critical states.
-
Limitations of TARO and Insufficient Justification of Its Design: The proposed max-value backup (Eq. 7) may produce overly optimistic value estimates, as a candidate is considered valuable if only one of its downstream trajectories achieves a high reward. Additionally, the counterfactual advantage (Eq. 8) may provide weak or zero learning signals when different candidate branches have similar returns. Although some ablations are provided, comparisons with simpler alternatives, such as ranking-based objectives or advantage-weighted classification, are missing.
-
Potential Training–Inference Distribution Mismatch: SDP is trained using sentence-level states collected along trajectories generated by the base model (Algorithm 1). However, during inference, the context may contain sentences selected from different LLMs (Algorithm 3). This creates a potential distribution shift between training and inference states, which could affect disagreement prediction and routing reliability. The paper does not investigate this issue or provide robustness analysis under mixed-model trajectories.
-
Incomplete Cost and Statistical Analysis: Although the paper provides extensive inference efficiency comparisons, it does not quantify the substantial offline costs associated with LLM-based semantic annotation, multi-model candidate generation, and tree rollouts. These costs are important for assessing the overall economic benefits of STAR. In addition, the paper lacks results across multiple random seeds, standard deviations, or confidence intervals, making it difficult to assess the statistical reliability of the reported improvements.
Questions for authors
-
Clarification on Data Splits and Accuracy Reporting: Could the authors clarify whether the original questions used for SDP and TARO training are strictly separated from the evaluation benchmarks? Additionally, how are the accuracy values in Table 1 calculated? For example, MATH500 contains 500 samples, so accuracy should change in increments of 0.2% under single-run evaluation, yet 56.69% is reported. Similarly, 69.80% on HumanEval (164 samples) does not correspond to an integer number of correct predictions. Are these results averaged over multiple runs? If so, how many runs were conducted?
-
Clarification on SDP Reliability and Collaboration Necessity: How do the authors validate that semantic disagreement actually identifies states where multi-LLM collaboration improves final task accuracy? Since models may agree on incorrect reasoning, could the authors report the precision, recall, and false negative rate of SDP, particularly on mixed-model trajectories during inference? These results would help determine whether the proposed disagreement criterion reliably identifies critical decision points.