- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes DWELL, a cost-sensitive framework for selecting among predefined, fixed-depth multi-LLM deliberation policies by maximizing expected accuracy minus a weighted inference cost. Under diminishing marginal returns along the accuracy–cost frontier, it derives a threshold rule for selecting the cheapest utility-maximizing endpoint. It also establishes an upper bound on the expected benefit of further deliberation, assuming state-conditionally calibrated confidence.
Validation consists of a five-seed controlled simulation and a preliminary ARC-Challenge evaluation with 60 held-out questions and three API-served models. The simulation reports a selected endpoint with 2.7× lower normalized cost than mixture-of-agents at a 1.4-percentage-point accuracy loss.
Critical strengths
The proposed problem is important in practice.
Critical weaknesses
The paper is very poorly written and difficult to follow. It is full of undefined terms and uninformative, defensive statements.
Questions for authors
None.