- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
A benchmark is introduced built from 31 open-ended browser games. The paper is difficult to follow. Many sentences consist of a long list of somewhat related topics that are not explained in the paper. The paper uses much jargon, is difficult to understand by outsiders. The paper contributed a benchamrk. No research questions are posted, no answers are given, no conclusion is present. This is not written in the style of a scientific paper. Please rewrite or submit to a suitable venue.
There is no supplementary material. No AI use statement.
reject
Critical strengths
- a benchmark is introduced of substantive size for real time web interaction
Critical weaknesses
- paper is difficult to follow
- paper reports on the benchmark, but has no research questions, conclusions, or research insights.
- no supplementary material, no AI use statement.
Questions for authors
ICLR 2027 RTWORLD: BENCHMARKING REAL-TIME GAME
What is a fail-closed runtime? Abstract states that 13 VLM are evaluated with common seeds. How can it be relevant that seed values of a random number generator are shared between VLMs? What is meant by "distinct factors", does that mean relevant, or determining? What does it mean that faster responses do not by themselves guarantee use-ful control? What does it mean that the opportunity has passed, which opportunity: RTWorld provides a reproducible foundation for studying agents that must reason and act before the opportunity has passed.
Abstract must be rewritten.
Real world: environment may move on while agent is thinking. Writing style is difficult: many sentences with long item lists, that are not explained further on: "Without replica identity, immutable traces, synchronized video, and explicit exclusions, apparent gains can arise from hidden pauses, stale actions, mid-episode failover, or incomplete evidence."
What is a source-audited browser game? Who is the auditor? What is DOM state?
"The world clock advances through capture, inference, ad- mission, and dispatch. Schema, interruption-epoch, and source-age checks prevent invalid or stale plans from controlling the game. Each episode is bound to one model replica and yields hash-linked inputs, outputs, traces, results, and video." Please rewrite, and move out of introduction, to experimental setup.
What is freshness? What does this sentence mean: "Latency and freshness thus become observable properties of the evaluated system rather than experimental noise." How can freshness be experimental noise? What is experimental noise?
The list of contributions should be rewritten using full sentences. There are no scientific insights listed as contribution, only a benchmark set.
What does this sentence mean: "Dustline is implemented but not evaluated here."
Related work is unreadable. Please rewrite using english sentences. What are the conclusions, how is it related to your work? Please have a look at other relevant scientific papers, and produce a paper in a proper scientific format, where research questions are posed and answered, or follow a course in academic writing. Note that ICLR is one of the premier AI conferences, and standards are high, for readability and scientific contributions.
Section 7, results, does not have conclusions. Please provide a clear problem statement, and list contributions to the scientific community.
The final sentence of the conclusion is a riddle: The evidence is deployment-conditioned, with no supported rank reversal or burst-ordering effect; broader cross-family replication remains essential.
I do not think that this paper belongs at ICLR in its current form.
reject