(1) The presentation is very bad. First of all, the style file used is different. This alone can be a valid reason for a rejection. Even if the style file was correct, there are still major concerns:
(a) The experiment setup is missing. There is mention of DeepSea and FrozenLake, which I didnt not find the citations to, so am not sure what they look like, what the problem is, what the prompts/responses/trajectories look like. Still, DeepSea and FrozenLake experiments are mentioned in section 5.4. What about all the previous experiments? What domain are they in?
(b) What does "Representative gates." mean in line 118? Seems very weird. Should it be a paragraph header or sub section? Otherwise it a two word sentance without any verb.
(c) The algorithm is missing. With a confusing method, atleast an algorithm box could help.
(d) The paper title in the pdf is "Idol learning" while in the submission is, "Idol Learning: Probe Before Commit for Rare-Success Reinforcement Learning".
(2) I am not sure I am convinced by the premise though. RL does rely on lucky successes to some extent and that is why the best practice is to run experiments over large sets of random seeds. In a sparse reward setting, when you are exploring, you do rely on discovering states with non-zero reward. And as I said, the best way to study how much the algorithm relies on this is by taking an expectation across seeds.
(3) If I understand correctly, there is a batch sampled (its called mini-batch not main batch), the policy makes an update, then somehow you believe that some successful trajectories should further update the weights. Then you compute a Delta^h. But the new weights do not depend on Delta^h, rather is simply an EMA averaging strategy. I do not know what is special about this algorithm. Delta^h is unused. And once policy updates are made on a mini-batch, you do not need to further update the policy using some trajectories from the mini-batch. While you can develop an algorithm that does this, I believe this is the common strategy.
(4) In section 4, Rglobal and Rmask are defined which I am not sure about how they are used. And there are a lot of number mentioned here without any explanation. For instance, "uniform-policy skill is 4−H", "Bernoulli(10.24⋅4−H)", etc. Where are these coming from?
(5) What are FR-0x being used throughout the experiments?
(6) It is difficult to analyse the experiments without understanding the setup and the algorithm.
We thank the reviewer for highlighting the importance of scalability to complex environments. We agree that extending the framework to large-scale and real-world domains is an important direction for future research.
We would like to emphasize, however, that the primary contribution of this paper is to introduce and investigate a new learning principle, rather than to establish a general-purpose solution for complex RL environments.
The central insight of Idol Learning is that rare successful experiences should be treated as hypotheses to be tested before being reinforced. This motivates a new probe-before-commit framework that separates temporary behavioral amplification, used to evaluate a candidate update, from permanent policy commitment. We believe this conceptual distinction is valuable in its own right and opens up a different perspective on learning from rare successes.
Our experimental design is deliberately controlled to isolate and systematically investigate this principle. By separating genuine behavioral skill from favorable environmental randomness, we can directly examine when reinforcing rare successes is beneficial or harmful, why temporary amplification and permanent commitment should be treated differently, and what additional value active validation provides. These findings establish the motivation and empirical foundation for the proposed framework.
We fully acknowledge that constructing informative probes becomes more challenging as environments grow in complexity. Developing scalable candidate construction and efficient validation mechanisms is a substantial research problem in its own right, and we see it as a natural next step building on the framework introduced here.