- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduce an interesting question: whether resetting an agent also removes the effects of its earlier capability changes. The answer can be no: an earlier agent may have changed the world, which then changes what a fresh agent learns. The authors trace this pathway using world swaps, replay swaps, and interventions before and after learning. A small exact model describes when history affects construction. The full pathway is shown in a controlled Fork environment; tests in other environments show where the evidence stops.
Critical strengths
1. The intervention design is the main strength. Swapping the actual worlds and replay buffers, then removing the world after learning, helps pin down where the effect is stored. This gives a concrete way to check what an agent reset does and does not remove.
2. Paper is careful about what its results show. It treats topology skeletons as independent units, reports unsuccessful extensions, and distinguishes a small residual effect from a statistical null. I also found the future-task result helpful: the same learned difference can help on one task and hurt on another.
Critical weaknesses
1. I couldn't reconstruct the central environment from the paper. Sections 4–5 and Appendix D.2 give the network and optimizer, but the four control actions, their transition and reward rules, the collection of the 24 training transitions, and the Fork layouts need more detail. Since the contribution rests on this constructed example, one fully worked instance would make a big difference.
2. The full lifecycle is shown in 12 designed Fork skeletons, but the separate locked audit finds none, even among its 48 Fork variants. These are different experiments, so this isn't a contradiction. Still, I couldn't find a breakdown of which checks fail for that Fork subset. This makes it hard to judge which settings actually support the complete mechanism.
3. Scrubbing the world shows that the learned difference survives removal of its physical cause. It doesn't show how long it survives further learning. The MLP readout uses saved parameters after 400 updates on 24 transitions, without subsequent common experience. Whether the effect persists through further practice or quickly washes out remains unclear.
4. The demand experiment uses two directional tasks and one approximately symmetric task. It shows that the sign can change, but doesn't establish how sensitive the small balanced effect is to less neatly balanced demands. A small asymmetry might make the inherited difference matter again. This limits the practical meaning of cancellation, while leaving the existence result intact.
5. Some figures are hard to read on their own. Figure 2A has no vertical-axis metric label, and its caption doesn't identify the learner. Section 6 points to Figure 3 for the empirical phase scans, but the figure shows the analytic window schematically, with only aggregate coverage counts in the caption. Labeled slices of the actual scan, along with clear metric and learner labels, would make the evidence easier to check.
6. I found it harder than necessary to keep the notation and study versions straight. The symbol d denotes future demand in Eqs. (2)–(3) but healthy cross-method disadvantage in Proposition 1, and 'Fork' covers both the positive lifecycle study and the separate locked audit. Distinct symbols and consistent study labels would make the argument much easier to follow.
Questions for authors
1. Could you provide one complete Fork example, including actions, rewards, transition collection, training targets, and future-task evaluation, with a link to the corresponding artifact?
2. Which checks fail for the 48 Fork variants in the locked audit, and which differences in construction or sampling explain the contrast with the 12 successful Fork skeletons?
3. In the current Fork setup, what happens if both reset agents get the same additional practice before evaluation? Does the effect fade, and if so, how quickly?
4. With the learned snapshots fixed, what happens under small asymmetries in the balanced future task? How much change is needed to move the contrast outside the stated ±0.02 practical region?