1. Substantial concerns regarding the consistency of the probabilistic formulation and the validity of the theoretical guarantees.
My primary concern is that the proposed Bayesian formulation does not appear to be internally consistent, which directly affects one of the paper's central contributions.
First, Eq. (4) defines the conditional distribution of the agent variable as p(θ∣ϕ=χi)=N(μ+δi,σ2). However, Algorithm 1 (Line 2) samples ϕ and θ separately, with θ∼N(μ,σ2). It is therefore unclear whether the actual implementation follows the conditional generative model assumed in the theoretical derivation. In particular, Lemma 1 relies on the conditional dependence between these variables to obtain a trajectory-return distribution with mean μ+2δi.
Second, the derivation of Lemma 2 appears problematic. Marginalizing the context variable from Eq. (6) generally yields a Gaussian mixture rather than the single Gaussian N(θ,στ2) claimed in the paper. The mixture weights should account for the dependence between ϕ and θ specified in Eq. (4). Even under an independent-sampling approximation, the additional variance induced by the context-dependent offsets does not generally disappear. Consequently, Eq. (8), while algebraically correct for a Gaussian prior and Gaussian likelihood, is not established as the exact Bayesian posterior under the originally specified joint model.
More importantly, Theorem 2 claims that E[vτk−θk]>0 and that the expected agent variable increases monotonically across iterations. However, under the joint model defined by Eqs. (4) and (6), the stated constraint ∑iωiδi=0 implies E[vτ−θ]=0, which is inconsistent with the claimed strict inequality. Furthermore, the proof in Appendix A.4 appears to rely on selecting the coefficient κto ensure sufficient action rewards without establishing that such rewards are achievable in the underlying environment. A Bayesian posterior update also does not, by itself, guarantee monotonic improvement in the posterior mean or actual task return.
These issues substantially weaken the paper's claims of a unified Bayesian inference framework with rigorous monotonic improvement guarantees. Clarifying the underlying generative model, distinguishing exact inference from approximation, and establishing the necessary assumptions for the theoretical results would be essential.
2. Insufficient experimental evidence to isolate the contribution of Bayesian co-evolution from additional test-time computation and reward-guided action selection.
The proposed method performs multiple evolution iterations, generates multiple trajectories per iteration, and considers multiple candidate actions whose probabilities are subsequently adjusted. This introduces additional inference and interaction costs. However, the paper does not sufficiently establish whether the reported improvements arise specifically from the proposed Bayesian belief updates or primarily from increased test-time computation.
Although the ablation study compares the full method against context-only and agent-variable-only optimization, it does not isolate the contribution of Bayesian inference itself. Comparisons against simpler alternatives, such as budget-matched repeated sampling, reward-guided candidate reranking, or heuristic context-selection updates, would be particularly informative.
Moreover, the continuous variable θ does not represent an updated LLM policy parameter; instead, it controls action selection through a return-dependent probability transformation. This raises an important question about the distinction between the proposed agent-model co-evolution mechanism and existing test-time sampling or reranking strategies.
The paper would benefit from clearer reporting of the number of sampled trajectories, candidate actions, total model calls, token consumption, and tool interactions, together with matched-budget comparisons. Without these controls, it remains difficult to attribute the performance gains specifically to the proposed coupled Bayesian formulation rather than to additional exploration and computation.
Overall, these concerns affect both the theoretical soundness and the empirical justification of the paper's principal contribution.
Dear Reviewer,
Thank you again for your great work on reviewing this paper submission to ICLR 2027.
The authors have now submitted their rebuttal. The open reviewer-author discussion phase will end soon, on November 18 at 11:59 PM AoE.
Please read the authors' responses, determine whether any further clarification is needed, and respond to the rebuttal at your earliest convenience. Please be sure to acknowledge the authors' responses and update your review accordingly.
Best regards, Your AC