- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies how learners preserve or discard past information in changing environments, using five approaches:
- HGF (hierarchical Gaussian Filter) simulations testing whether forgetting helps learners adapt after changes while retaining useful information;
- Model-recovery experiments testing whether persistence control can be distinguished from volatility adaptation;
- A non-stationary bandit comparing control over forgetting with control over learning rates;
- A reversal-learning experiment with 74 participants, comparing models and relating fitted persistence to behaviour;
- EEG analyses testing whether frontal-midline theta is associated with fitted persistence.
It proposes “persistence control”: before each learning update, the learner relaxes its previous beliefs toward an initial prior. The amount of forgetting is either constant or determined by the previous outcome’s surprise.
The conclusion is that persistence is a distinct target of adaptive control, with evidence from simulations, human behaviour and EEG.
My main concern is that the results are either given in advance or already known. While the human and EEG analyses do not convincingly demonstrate the claimed distinction.
Critical strengths
The authors validate their counterfactual replay, separate bandit-controller selection from evaluation, and report negative results. These include a competitive leaky-RL baseline, a null choice-stream protection effect and null results in the stricter EEG analyses.
Critical weaknesses
I found the paper to be hard to follow, poorly structured and the tasks, simulation, and fitting procedures lacked explanations. My assessment is that there was very little surprise in the findings presented in the paper, everything follows naturally from the setting.
-
The novelty is unclear. Interpolating Gaussian natural parameters toward a reference prior implements established stabilised forgetting. Moreover, related work also infers the forgetting weight from data, rather than specifying it through a fixed surprise-dependent function, which is more robust and less prone to leaking the design knowledge in the modelling assumptions. Relevant comparisons include Kulhavý & Kraus (1996), Masegosa et al. (2017), Moens (2018), Moens & Zénon (2019), and Yu & Cohen (2009). The paper needs to clearly explain what is new beyond applying this operation within the HGF.
-
The experiments do not establish one coherent claim. Forgetting improving responsiveness and helping more in rapidly changing environments are expected consequences of the tracking–noise trade-off. Additionally, the long-history failure that motivates the simulations isn't clearly tested by the human task which is very short. The learning-rate comparison uses a different learner. The changepoint baseline is a restrictive threshold-reset model, rather than a full Bayesian changepoint or inferred-forgetting alternative.
-
The model may produce these effects by design. With the fitted settings, it keeps more of its past beliefs after a surprising outcome. This may explain the reported “protection” effect. The model only uses surprise from the last trial. Consequently, forgetting after a second unexpected outcome does not show that it has combined evidence across trials. Better players also experience different outcomes which could explain the link between protection and performance. The authors should check whether this link remains when they use the same model settings for every participant.
-
The model comparison uses participants’ choices, but the protection and forgetting effects come from a separate fit to their rewards. This matters because the fit that wins the comparison does not show the protection effect. Its success therefore does not directly support the main finding. The authors also leave unclear how the model predicts choices. This makes the comparison hard to assess. They do not test whether the task provides enough information to estimate the model’s settings reliably. Along the same line, the EEG comparison contrasts an unexpected non-reward with an expected reward. This is a problem because reward versus no reward already affects frontal-midline theta. The observed difference could therefore have a familiar explanation.
-
The presentation completely obscures the contribution: a) the abstract should clearly identify the simulations, bandit study and human EEG experiment. There should be a list of tasks that are done in the paper (eg, "We test our hypothesis in simulation, behavioural experiments and in an EEG recording setting). Methods and results need organizing around explicit experiments and questions, and it should be clear when skimming over the paper what experiment (simulation/behavioural/EEG) we are looking at, and what questions is being answered.
-
State variables and update operators require clearer definitions, terminology should be consistent, and several settings and figure elements are insufficiently explained.
Questions for authors
How do the fitted surprise slopes vary across participants? Does the protection effect show anything beyond the direction of those slopes?
What is the full observation and response model for the human task? How do the choice-stream and reward-stream fits relate? Can the gate parameters be recovered in the actual task?
Do the behavioral correlations survive when identical model parameters are applied to every participant’s outcome stream? Does theta predict persistence beyond surprise and reward versus non-reward?
How does the method compare with inferred-forgetting models and stronger Bayesian changepoint baselines? What happens when forgetting is applied separately at each HGF level?
Please report the missing simulation settings and joint-controller results. Please also define “reversal surge” and “reconfiguration.” Which result most clearly distinguishes this account from existing explanations?
2 comments
Response to LWfg (2/2)
(2/2)
Q1. "How do the fitted surprise slopes vary across participants? Does the protection effect show anything beyond the direction of those slopes?"
We estimated each participant's slope over a grid from −6 to +6 under the same N(0, 1) prior used elsewhere. The direction is consistent across people (posterior-mean slope median −0.54, negative in 68 of 74), but 200 trials determine each person's slope only moderately, so we make no individual-slope claims. Both signs were available, and a positive slope fits worse than no slope at all. Beyond the direction, the slope captures three concrete features of behaviour:
- People lapse even during a run of correct choices; the negative slope relaxes slightly when nothing is surprising, which keeps the model from becoming overconfident.
- A first error predicts another: right after a surprising error, participants chose the best option only 31% of the time, and the negative slope holds the updated belief rather than resetting it.
- People then recover: once the surprise has passed, the slope releases the belief, and choices of the best option return to 38%.
A positive slope gets all three wrong. What goes beyond the direction is the strength of the evidence for surprise dependence, calibrated against gate-free learners (point 3). The slope distribution and this decomposition are now in C.15. This probe was really helpful for us, and we thank the reviewer for it!
Q2. "What is the full observation and response model? How do the choice-stream and reward-stream fits relate? Can the gate parameters be recovered?"
We should have specified the model fully in the submission; it is now in Supplement B.4–B.5. The input on each trial is whether the participant chose the objectively best option, with all 200 trials fitted as one sequence. A three-level binary HGF predicts this outcome from its own history, with the gate βₜ relaxing the inherited beliefs before each update; the likelihood is the model's one-step prediction. Free parameters are ω2, ω3 and b0, with bᵥ shared across participants; estimation is MAP, with Laplace model evidence and BIC. The reward-stream fit applied the same model to whether each trial was rewarded; its parameters are only weakly related to the choice-stream fit (ρ = 0.21 and 0.09), so the two estimate different quantities. Recovery: see point 4.
Q3. "Do the behavioral correlations survive identical model parameters? Does theta predict persistence beyond surprise and reward?"
Identical parameters: see point 3. Within a participant, fitted persistence is a fixed function of the previous trial's surprise, so no signal can predict it beyond surprise at the trial level, and we now state this explicitly. Of the tests that can be identified: theta is larger at a contradicting loss with reward held constant (dz = 0.40–0.50, q < .01); between participants it does not track how strongly each person gates; and it does not predict the next choice once surprise and reward are accounted for (F.5).
Q4. "How does the method compare with inferred-forgetting models and stronger Bayesian changepoint baselines? What happens when forgetting is applied separately at each HGF level?"
The dynamic belief model: see point 1. A leaky delta rule fits about as well by model evidence (ΔLME −2.9) and better by BIC (ΔBIC −385), but its fitted leak did not distinguish better from worse performers, as persistence did (C.12); we present our model as one implementation of regulated carryover. The known-reversal reset bounds any changepoint model that resets the learner, including a full Bayesian one, since no detector can place its resets better than the true reversal times; graded release still fits better (ΔBIC +390; C.11). Forgetting at each level: level 2 alone retains part of the gain (ΔLME −51 relative to both levels), the mean alone also (−38), while level 3 alone (−500) and precision alone (−476) fail. The effect therefore depends on relaxing the inherited level-2 belief: persistence control acts on what the learner believes, not on how volatile it takes the world to be (C.9).
Q5. "Missing simulation settings and joint-controller results; definitions; which result most clearly distinguishes this account?"
Settings and definitions: see point 5. The joint controller beat the better single controller in only 20% of environments; its search grid was not matched in complexity, so we draw no conclusion from it, and in aggregate held-out performance learning-rate control or no control did best. This is why the paper frames the result as an environmental boundary rather than dominance. The distinguishing result is the comparison with learners that lack the mechanism: ΔLME 31 to 57 (52 to 65 when fitted to each participant's choices) against 81 in participants. The mechanism is also recovered without confusion from volatility-based accounts (30/30 each way), and reversing the fitted gate worsens every participant's fit (74/74).
Response to LWfg (1/2)
(1/2)
We really appreciate and thank the reviewer for taking out the time on our paper to produce such a thorough and constructive review. It pushed us to test directly whether our results follow from the model's design, and those tests now anchor the revised paper!
We accept the criticism of the presentation and have restructured accordingly (point 5). Some patterns do follow from the setting, and we now say so. The central result does not: participants' evidence for surprise-dependent persistence control (ΔLME = 81) exceeds that of all 20 simulated cohorts without the mechanism (31 to 57), and of all 20 cohorts of learners fitted to each participant's own choices (52 to 65).
1. Novelty relative to stabilised and inferred forgetting. We agree that the operator is stabilised forgetting in its Gaussian form, and we now state this explicitly, with a comparison to the works the reviewer lists in the Supplement (B.6). Our weight is estimated, not fixed by design: both its baseline and the sign of its surprise dependence are fitted, and both signs were available. We also fitted the dynamic belief model of Yu & Cohen (2009), with exact Bayesian inference over every possible changepoint history: it fits comparably by BIC (ΔBIC = 7 in favour of DBM) but less well by model evidence (ΔLME = 233 in favour of our model), and we report both. What is new is not the operator but what we establish with it:
2. "The experiments do not establish one coherent claim."
3. "The model may produce these effects by design." We thank the reviewer for this point; it led to the analyses we now consider most important. With one shared parameter set for every participant, the protect-then-release pattern remains (dz 2.05 → 3.40) and the reversal-surge link to performance survives (ρ = 0.36 with the learning index, 0.46 with reward); the protection–performance link does not (ρ = 0.18), and we have removed it (C.16). Gate-free learners also show the protect-then-release trajectory and the surge link, so we present these as descriptions of the fitted gate. What exceeds these learners is the evidence for surprise-dependent persistence control (ΔLME 81 vs 31 to 57; C.13).
4. Streams, response model, recoverability and the EEG contrast.
5. Definitions and settings. We added a definitions box and harmonised notation (2.1). Reversal surge: the rise in persistence immediately after a true reversal, relative to the preceding trials. Reconfiguration: the absolute trial-to-trial change in persistence. All simulation settings are in the Supplement (C.7, D.4).