- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies endogenous curriculum generation for verifiable self-play through KL-regularized control on a syntax-constrained program tree. The proposed projected soft-Q generator is designed to maintain non-vanishing support over legal programs while favoring tasks with high estimated information gain. The paper derives several theoretical guarantees, including completed-program probability floors, Bellman-residual-based tracking bounds, comparator-relative information certificates, and finite-hypothesis posterior identification results. It also introduces sharper direct residual certificates based on performance identities, quadratic residual penalties, and contrast-aware estimator errors. Experiments are conducted on a finite Boolean DSL with 192 programs and 16 truth-table hypotheses. Under a matched pure-information objective, the proposed curriculum improves target identification and semantic coverage relative to the reference policy. However, the original residual-composed information bound is nonpositive throughout the tested configurations, while the newly introduced direct certificates have not yet been evaluated on the reported experimental trajectories.
Critical strengths
The paper is unusually careful about the scope of its theoretical claims. It explicitly distinguishes support guarantees, information-value certificates, posterior identification results, and empirical learning gains, and repeatedly states where one result does not imply another. This makes the theoretical assumptions and limitations relatively transparent.
The paper does not hide negative or vacuous empirical results. The authors explicitly report that the original residual-composed information lower bound is nonpositive in all tested configurations, rather than presenting only favorable diagnostics. This level of transparency is valuable for assessing what the proposed certificate currently does and does not establish.
Critical weaknesses
-
The main certificate is empirically vacuous in the reported experiments. The residual-composed lower bound is nonpositive in every tested configuration, and even the exact-TV version is positive only in a subset of rounds. This substantially weakens the practical significance of the paper’s central theoretical construction.
-
The paper introduces stronger direct certificates after observing this failure, but does not evaluate them on the main experimental trajectories. The manuscript explicitly states that these direct refinements have not been tested on the v2/v3 records. As a result, the strongest theoretical contribution is disconnected from the empirical evidence, and it remains unclear whether it actually resolves the weakness exposed by the original certificate.
Questions for authors
What do the authors consider to be the single primary scientific claim of this paper?
The manuscript develops a sequence of increasingly elaborate results—support guarantees, residual-based information certificates, direct certificates, quadratic refinements, and posterior-identification guarantees—while several of these results address limitations of earlier ones, and the strongest new certificates are not evaluated on the main v2/v3 experiments. At present, I find it difficult to identify which result is intended to constitute the central contribution and what empirical evidence directly validates that claim. Please clarify the main claim of the paper and explain which theoretical and empirical results are necessary to support it.
3 comments
Thank you for the clarification. However, I remain seriously concerned about the overall presentation and coherence of this manuscript.
First, the paper is extremely difficult to follow. The manuscript is 41 pages long, yet it does not contain a single figure. There is no overview of the proposed framework, no visualization of the theoretical results, and not even a learning curve in Section 5.2, which is explicitly titled "Learning Curves and the Objective Trade-off." For a paper combining theoretical analysis, algorithm design, and empirical evaluation, I find this presentation highly unusual and unnecessarily difficult to understand.
Second, the manuscript contains numerous unexplained references to internal experimental versions. The authors repeatedly refer to "v2" and "v3" without ever explaining why these versions are named this way or what happened to "v1." Similarly, what exactly are "Schema 17" and "Schema 18," and why should readers be expected to understand these internal identifiers? The manuscript even discusses inconsistencies across document versions and analyses that have been superseded. This is supposed to be a research paper, not a draft experimental report or an internal development log. I would expect the authors to consolidate their experiments and present a self-contained, coherent account of the research.
Third, I remain concerned about the coherence of the scientific contribution. Throughout the manuscript, the authors repeatedly explain which claims have not been established, which experiments cannot be combined, and which theoretical results have not yet been empirically validated. Acknowledging limitations is important, but the sheer number of qualifications, successive refinements, and disconnected experimental versions makes the paper read more like an accumulation of incremental revisions than a unified scientific investigation. This raises serious concerns about whether the different components have been sufficiently synthesized into a coherent scientific contribution.
Finally, I have concerns about the overall writing style and the degree of independent intellectual synthesis reflected in the manuscript. The writing relies heavily on repetitive, defensive qualifications and carefully hedged statements, which contribute to the impression that the manuscript has been assembled through successive rounds of revision rather than developed around a clear scientific argument. Even in the rebuttal, the authors continue to rely on such defensive qualifications, including:
- "It does not assert that every round has such a margin..."
- "This is evidence about finite-model certification, not broad scaling or computational dominance."
- "A nonpositive sufficient bound is not a counterexample to its mathematical validity..."
While such qualifications may be appropriate individually, their repeated use throughout the paper and rebuttal reinforces my concern that the authors are more focused on defending the limitations of individual claims than on presenting a clear and convincing central scientific contribution. I recognize that such stylistic observations cannot establish whether any content was AI-generated, and I do not make that allegation. I leave any assessment related to the conference's AI-use policies to the Area Chair.
My concerns are ultimately about the clarity, coherence, and scientific maturity of the submission. I therefore maintain my strong rejection recommendation, as I do not believe this manuscript is ready for publication in its current form.
Thank you for identifying these presentation obstacles. We will consolidate the manuscript around one question: when does an implemented executable-task generator preserve program support and certify an information advantage over its same-state, information-ablated comparator? Theorem 4.14 is the principal information result. Theorem 4.12 supplies support and a general distributional route to the same information comparison. This relationship organizes the theory, algorithm, and experiments.
1. The connection from Theorem 4.12 to Theorem 4.14
Both theorems use the same implemented policy, frozen learner state, reference, temperature, clipped scores, and comparator. In Theorem 4.12, projection gives a completed-program probability floor. Its information branch follows Bellman residual → terminal-law TV → information transfer, controlling information through the entire terminal distribution.
Theorem 4.14 retains the additional structure of the objective. Suppressing the frozen state index, write , where includes non-information utility and the KL cost, and let maximize . The implemented-policy gap is . The exact decomposition is
=\Delta_t^{0,\mathrm{imp}}+J(\pi_t)-J(\pi^0).$$ Bounding full-objective suboptimality by $\Xi_t$ and accounting for the estimation error shared by the two policies gives the objective branch of Theorem 4.14: $$\mathcal D(\pi_t)-\mathcal D(\pi^0) \ge\frac{\Delta_t^{0,\mathrm{imp}}-\Xi_t}{\lambda_I}-E^\Delta_{t0}.$$ The performance identity yields a quadratic residual penalty near the optimum; the contrast error charges estimation uncertainty through differences between the policies' terminal weights. Local and symmetric refinements retain more of this structure. The same iterate retains its support floor. **The progression is from general distributional control to direct analysis of the same information comparison.** Subtracting the comparator's value in Theorem 4.12(iii) makes this common target explicit. The earlier tables reported positivity of the absolute information lower bound; the direct audit reports the comparator-relative margin. Retaining the distributional bound makes its conservatism measurable. In the historical baseline, exact TV gives positive information lower bounds in 288/1,600 rounds, whereas residual-to-TV composition gives 0/1,600. Their difference on the same states isolates the loss introduced by that relaxation. The direct analysis quantifies optimization loss and shared estimation error at the objective level. ## 2. The roles of v1, v2, v3, and the schema identifiers The labels record distinct analytical and experimental roles: - **v1:** the preliminary analysis using cross-trajectory local-objective comparisons. C.4 - **v2:** the main study with 17 configurations, 40 seeds per configuration, and 40 rounds per seed. - **v3:** the separate completed-program support audit, with seven reported configurations and additional realized-row floor measurements. Keeping the batches separate preserves measurement provenance: additional support fields belong to the runs that recorded them. The superseded v1 analysis also carries a methodological lesson. Local information values must be compared at the same learner state, because a faster learner can have less uncertainty remaining later in its trajectory. **Shared frozen beliefs evaluate the local information comparison; separate adaptive rollouts evaluate learning progress.** We organize the experiments by purpose: same-state objective evaluation, completed-program support, direct-certificate evaluation, and adaptive-selector comparison. One table will specify objectives, horizons, temperatures, update budgets, information access, seeds, and statistical units. ## 3. Make the appendix follow the mathematical argument We consolidate Appendices D–G as the **derivation and proof of Theorem 4.14**: the performance identity, residual penalties, and contrast-aware information comparison. The opening on page 33 will refer to the model already defined in the main text and introduce only the additional proof notation. ## 4 .Qualifications in the manuscript and our earlier rebuttal During presubmission checking, feedback through ICLR's Paper Assistant Tool program highlighted concerns about rigor and potentially overstated wording. To preserve the exposition, we used LLM-assisted polishing for minimal, local corrections. We often appended scope qualifications rather than integrating them into the argument this is the simplest way to distinguish computational guarantees from claims about learning, generalization, and convergence rates.
Thank you for asking us to identify one scientific claim and the evidence needed for it. The principal claim is a conditional information-robustness theorem for the actual generator: its advantage over the same-state IG-ablated optimum can be bounded directly by optimization and shared estimation errors, while the same projected iterate preserves program support. This is Theorem 4.14 in the reviewed submission
Its simplest branch says
\ge \frac{\Delta_t^{0,\mathrm{imp}}-\Xi_t}{\lambda_I}-E^\Delta_{t0}.$$ The claim is therefore precise: information advantage is guaranteed when the implemented policy's non-IG objective sacrifice exceeds its certified optimization and estimation penalties. It does not assert that every round has such a margin, or that our selector learns faster than every adaptive alternative. The results have distinct roles rather than forming a succession of replacement claims. Appendix E supplies the exact performance/information identity; Appendices F–G turn it into policy-relevant residual and estimation-error bounds. Appendix H establishes that the positive-margin region is nonempty. The support result ensures task availability under the same decoder. Appendix J separately explains when accumulated support on distinguishing tasks yields finite-model posterior identification. The two-time-scale results concern tracking under additional hypotheses and are not required by the finite-round information theorem. **The empirical gap identified in the review is now addressed by fresh trajectory audits, rather than by the two-query construction alone.** In balanced 192-program experiments, refined direct margins are numerically positive in **1,950/5,120 composite one-update states (38.09%)**, **2,920/5,120 twenty-update states (57.03%)**, and **1,549/5,120 pure-information states (30.25%)**. Direct interval evaluation is stronger on those same records: its corresponding positive counts are **3,631, 3,701, and 3,434**. The original residual-TV-composed absolute-information bound remains nonpositive. Thus the added evidence establishes positive content for the direct sufficient conditions on a subset of audited frozen states, not a claim that the old bound became useful or that every trajectory is certified. The branch breakdown is informative: the local quadratic bound improves the objective-only certificate, while the symmetric branch supplies most positive states in the principal configurations. A separate 2,944-program extension gives lower refined positive fractions, 23.75% and 10.86%; the hypothesis family still has size 16. We retain these weaker results and the negative mean margins in the one-update configurations. This is evidence about finite-model certification, not broad scaling or computational dominance. The adaptive-baseline experiment answers another necessary question. In the Monte Carlo cohort, projected one-step selection succeeds in 195/256 runs versus 144/256 for fixed reference sampling, but greedy IG with 5% reference mixing succeeds in 256/256 and learns faster. Hence our justification is **not** that this decoder is the best active-query rule. It is the quantitative analysis of when a specified regularized curriculum preserves information value despite implementation errors. Eq. (102)'s interval dominance remains part of that analysis, not a result we attempt to overturn. A nonpositive sufficient bound is not a counterexample to its mathematical validity, but it is a real limitation of its usefulness. We address that distinction with proofs, branch-level positive/negative margin distributions, direct-interval comparisons, and separate learning results.