- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
PROTEUS formulates self-evolution of skill-composing agents as stochastic mutation, evaluation, and selection updates. It combines bounded reflection-guided mutation, Beta competence posteriors, lower-confidence scoring, and an elite/full-candidate selection mixture. Under explicit assumptions, the paper analyzes stationarity of a reward surrogate and a pre-resampling selection-KL bound. Evaluation covers program repair, literature synthesis, and long-horizon tool use; five-seed results improve over the strongest reported baselines by 5.4, 3.9, and 6.3 percentage points, with additional ablations and diagnostics
Critical strengths
Decomposing self-evolution into mutation, evaluation, and selection with explicit noise, drift, and local-alignment conditions is a useful conceptual contribution. It provides a common framework for discussing evolutionary systems beyond their final task scores. The paper also distinguishes analysis of the reward surrogate from the full diversity-regularized objective. This formulation has merit independently of the proof concerns below.
Critical weaknesses
- For the stated Q=Unif(32), H=Unif(4), and Pbar=0.8Q+0.2H, elite/nonelite masses are 0.075/0.025. Hence KL(Pbar$\Vert$Q)=4×0.075×ln(2.4)+28×0.025×ln(0.8)=0.10644 nats, independent of candidate rewards at these fixed settings. Figure 2's fluctuating curve and approximately 0.78 peak, and Table A.8's 0.62 median, cannot measure this quantity. The conflict directly undermines the empirical support for controlled selection drift; being below the theoretical bound does not resolve it.
- The stationarity proof accumulates improvements for the pre-resampling mixture Pbar, whereas the next iteration uses a finite resampled population P. These terms do not automatically telescope without an expectation relation or a resampling-error argument, leaving applicability to Algorithm 1 insufficiently established. Also, removing nonelite support can make reverse KL(Q$\Vert$H) infinite, not forward KL(H$\Vert$Q), making the selection-drift discussion internally inconsistent
Questions for authors
- What quantity is actually computed in Figure 2 and Table A.8? Please provide Q, H, and Pbar probability vectors and the log base for one iteration. If implementation uses nonuniform weights, dynamic parameters, or duplicate aggregation, how does it relate to the stated algorithm and theorem?
- Which proof step transfers the improvement bound for Pbar to the next finite population P? Is the theorem in expectation, does it include a resampling-error term, or does it analyze only a virtual-distribution process