- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper present a video game benchmark. It assumes some sort of state machine or an internal representation. Then using that information a vlm uses a genAI model to generate the interaction and evaluate the outputs. It makes sure that the agent
Critical strengths
There is benchmark introduced.
Critical weaknesses
The paper in its current format is too hard to read, the complexity of the sentences are too high. I am sure the content is interesting, but if I cannot read this paper as a native speaker, I doubt others can actually learn much from it.
Questions for authors
Here are my raw notes while reading the abstract / intro.
bstract. No need to speak about camera control. “support stateful and rule-governed interaction” this is a little vague, what does these words mean. “It maintains an evolving world state under specified rules, renders each resulting event with a frozen video model, and uses the generated clip to update the world state for the next turn” this sentence is overwhelming. event? what is the world state. “at each turn”? which turn? “instruction”? like prompt? “3d scene”? How is this hidden state constructed? like the benchmark is for a single object.
I dont understand why this paper is motivated by the navigation world models. I think controlling rendering artefacts is strong enough. I think the intro is clumsy, it does not clearly state the problem early. Sounds like control based on hidden state is the problem. Abstract failed at stating that. “Given a world description and an initial image” are the authors describing a game world? This is confusing. “select an actions” from where? Is the 3d scene an actual game state? What is described in the text of the intro is far more complex than the figure 1. “Together, rule execution, video generation, and visual observation form a closed loop that yields state world models.” So it is a game loop that is slow? Again “hidden state” what is this state? How is the state connected to rendering? Change-to-Comply protocol, what is that? this intro is confusing. looking at table 1, why is there a benchmark if the authors already solved it?
2 comments
Thank you for the great answer. But to me this paper is not ready for publication and I will not change my score.
Thank you for the questions. We will improve the abstract and introduction.
We acknowledge the presentation concerns. We would also welcome the opportunity to address any methodological, experimental, or soundness concerns you may have.