- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper asks whether visual history actually helps an agent choose better actions. The authors keep the current image fixed, change the past images, and separate the first action from what the agent does afterward. They test the measurements on simple tasks and then apply them to Dreamer in Craftax. Recent history helps the agent collect wood from a specific tree, but improvements in overall reward and normal play remain unclear. The measurement idea is useful and the experiments are careful, but the confirmed finding is narrow, and its value for broader task performance remains unclear.
Critical strengths
The paper asks a meaningful question. Remembering past information, or changing an action because of it, does not necessarily help an agent succeed. Measuring what happens after the action gives a more practical view of whether that information is useful.
Critical weaknesses
-
The practical benefit is still limited. History helps the agent collect wood from the selected tree, but it does not clearly help it collect wood overall or earn more reward. It may mainly change which tree the agent chooses. The study also focuses on one local decision in one game, with the target usually visible just one frame earlier, so it says little about longer-term memory.
-
It is unclear how much the findings carry over to normal play. The main experiments hide relevant information in both the past and later observations. With full history and normal observations, the benefit is uncertain. Editing the images may also confuse the model, so the results could partly reflect damage from the edits rather than useful information in the original history.
Questions for authors
-
Why does the improvement for the selected tree not lead to clearer gains in collecting any wood or earning reward? Does history mainly shift the agent from another tree to this one, or is there evidence that it helps with the broader task?
-
What evidence separates the benefit of useful history from the harm caused by edited images? Are there controls that keep the inputs closer to normal observations while making comparable changes? How strongly do the current results support a benefit during normal play?
Dear Reviewers,
We’re about halfway through the discussion period. Please read the authors’ rebuttal and assess whether it adequately addresses the key concerns raised in your review. We encourage you to leave a follow-up comment for the authors, indicating which concerns have been resolved and whether any important issues remain or require further clarification. Please update your review and scores as appropriate in light of the rebuttal and discussion.
Thank you for your time and participation.
Best regards,
AC