- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes a method for test time adaption of latent world models that works even before "the next real state becomes available". Three mechanisms are presented: 1. The encoder is updated by sending the transition history through a filter bank, encoding the modified observations and choosing the the filter that the world model is best at predicting next latents. Then the encoder is trained toward predicting this latent from the original observation. 2. They limit the drift of the encoder through this mechanism by interpolating between the original parameters and the updated ones. 3. The target-local dynamics adaption aims at only updating the model such that it changes the prediction in the dimensions that the history actual informs (only the subspace defined through the transitions in the history). The method is evaluated on the PushT task and PointMaze and compared against an AdaJEPA variant.
Critical strengths
- Limiting the dynamics adaption to what is supported by the history makes sense to me. I think using the subspace defined by the latent transitions is the right move.
Critical weaknesses
I want to preface this by saying I am not familiar with test time adaption, but with world models in general. For me the paper was hard to read and I think a clearer writing could greatly improve it.
Here are my main concerns:
-
The method seems very hand-crafted/engineered: Especially two mechanisms rely on assumptions that may not generally hold: 1. The encoder adaption mechanism uses a set of visual hypotheses. I do not see any justification, why specifically these hypotheses. 2. The latent space coordination is mainly a hyper parameter on adaption strength and the method seems to be rather sensitive to its value. Because of this I have doubts how generally applicable the method is and under which circumstances it can actually work.
-
The experiments are not significant enough: Given that the mechanisms seems rather engineered to me, I would expect more convincing results. There is only a baselines for comparison on only two tasks. For PushT the improvement is less than 1% and for PointMaze the AdaJEPA is a tiny bit better even (also considering it is only three seeds). Given that the work overall does not give much new insight into test time adaption, but presents a method, that is not very well supported by the empirical results, this become a major blockage to recommend acceptance.
Questions for authors
- Can you give some additional details on the experiments? How long are the episodes? How often is the world model updated (also in comparison to the baseline V0-X1)?
- At several places the paper emphasizes that HATTA adapts before the next outcome arrives. Isn't this always true? Don't the existing approaches for TTA also do this, just in a different way? I really struggle to understand how this is a contribution or a differentiating property of HATTA.
- Is there any way to decide what the latent space coordination parameter should be before running the agent?
- Did you also try larger K? Isn't the history much larger in reality with many tasks or set of tasks spanning hundreds to thousands of transitions and thus, providing a much larger history to learn from?