- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper rigorously examines design choices for an inference-time steering algorithm that uses gradient guidance and particle selection. It suggests the following: 1) adaptive temperature selection, 2) gradient from amortizing over gumbel noise or the leave-out-out reinforce estimator, 3) leaving out proposal correction. The paper contributes a theoretical analysis on 3). It achieves strong empirical results on 3 AI-for-science tasks.
Critical strengths
-
The strongest aspect of the paper is the results section, where it shows clear empirical improvement on 3 standard tasks. This is useful as it is the ultimate metric by which different methods are judged.
-
The adaptive temperature selection appears to be novel and generally useful.
Critical weaknesses
I am not sure about the paper's algorithmic novelty. Each of the three things, on its own, seems somewhat straightforward. Combined, it seems like the adaptive temperature selection is the main improvement. But again, that also feels quite straightforward.
Questions for authors
I like the empirical results; this is the strongest part of the paper.
However, I would like a clearer conceptual/algorithmic story. It sounds like you've examined many algorithmic details (e.g. gradient guidance pre or post softmax, adaptive temperature selection, variance over gumbel noise, proposal correction). This is nice. In terms of a major, novel takeaway/result, it seems like that would be adaptive temperature selection? In and of itself, it doesn't seem particularly novel/groundbreaking. But you are getting better empirical results due to this, whereas adding it directly to other methods helps but is also insufficient?
If you could help me understand: 1) is adaptive temperature selection the main contribution, 2) is it sufficient to integrate this with other methods to achieve good performance, or does it uniquely achieve good performance with your method, and 3) an explanation for the answer in 2 - then I'd be able to understand the paper's contributions better.
For now, I am going to select 2 b/c I don't clearly undersand these points. However, I'm quite open to raising my score if you can help me contextualize the paper's contributions and significance.
4 comments
A Kind Reminder for Comments
We sincerely thank Reviewer VA8j for the constructive questions on the main contribution and on the role of adaptive temperature selection. Our responses and the revised paper address them as follows:
- Main contribution as one chain: Sec. 1 and Sec. 3 now present variance-reduced guidance, the omission of the proposal correction, and Adaptive Monte Carlo (AMC) as one chain, in which each step is the reason for the next, with AMC as the main component (new Figure 2).
- AMC on other samplers: With only the selector replaced by AMC, the reward rises on SMC (4.09 to 5.13), TDS (4.75 to 5.17) and SMC-DDM (5.71 to 5.85), so AMC is general and extends to existing samplers (Table 5b in Sec. 5).
- Why the best result needs our method: Ours is the only sampler in which both the guidance and the selection enter the target, and it reaches 7.33 under AMC, against 5.13 to 5.85 for the others (Sec. 3.2, Sec. 5).
We appreciate the reviewer's time and would welcome any remaining questions. Changes in the revised paper are marked in blue.
Reviewer Response
Thanks. I have updated my score to 3.
Upon thinking about it, although the algorithmic novelty isn't huge, I do think the algorithm's simplicity will make it something that stands the test of time & is actually used, compared to very complex techniques that are not compatible with each other and get ignored.
Dear Reviewer VA8j,
Thank you very much for taking the time to read our responses and for raising your score. We are truly grateful for your kind words, and we are especially glad that you see value in the simplicity of our method and in its potential to be used in practice.
We sincerely appreciate your thoughtful evaluation.
Reply to Reviewer VA8j (2/2)
[
VA8j] asked whether integrating adaptive temperature selection into other methods is sufficient for good performance, or whether good performance is achieved only with our method.Thank you for this question. We would kindly refer the reviewer to Table 4b of the original manuscript (Table 17 in the revised one), where AMC is already integrated into SMC-DDM. To test this further on other methods, we added two samplers, SMC and TDS. As shown in Table R1, with all other settings left at their defaults, replacing only the selector by AMC raises the reward of every sampler.
Table R1. Selector replaced by AMC (DNA, 3 seeds). Each sampler keeps its proposal, and the row below it replaces only the selector by AMC. †: re-run together with its AMC variant.
AMC therefore generalizes beyond our sampler: with no other setting changed, it raises the reward of a sampler without guidance (SMC), of samplers that keep the proposal correction (TDS, SMC-DDM), and of our own proposal (Table 5a), which we find encouraging. At the same time, AMC alone is not sufficient: with AMC, the other samplers reach 5.1 to 5.9, whereas the best result, 7.33, is obtained with our full design, which validates the design.
Revision: Table R1 replaces the earlier Table 4b and is now Table 5b in Sec. 5.
[
VA8j] asked for an explanation of this answer, namely why AMC helps on every sampler and yet the best performance is reached only with our method.AMC helps on every sampler because all of them prescribe the selection temperature before sampling, and AMC replaces it with a measured one. The best performance requires our method for two reasons.
First, a selector can only choose among the candidates that its proposal produces. Our guided proposal alone, without any selection, already reaches 6.35 (Table 14), above the other samplers with AMC at their default settings (5.13 to 5.85).
Second, the targets differ, where the target is the distribution that the samples of a method follow (Sec. 3.2). TDS and SMC-DDM keep the proposal correction, which cancels the guidance in the target, so only the selection tilts their target toward the reward (Lemma 1). Our method omits the correction, so the guidance and the selection both tilt the target (Proposition 1). Table R2 summarizes this difference.
Table R2. What tilts the target toward the reward in each sampler, with the Pred-med under AMC. ✓: enters the target. △: used in the proposal, but cancelled in the target by the correction. ✗: not used.
As Table R2 shows, ours is the only sampler in which both the guidance and the selection enter the target, and it reaches the highest reward under AMC. This is the chain of our first answer: AMC is general, and the best result is reached when it is paired with our guided proposal.
Revision: Sec. 3.2 states this consequence where the effective temperature is introduced.
We hope that this clarifies the contribution and its significance, and we would be glad to run any further ablation during the discussion period.