- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the trade-off between task accuracy and context cost when feeding ego-vehicle kinematic histories into a vision-language model, primarily Qwen3-VL-2B-Instruct with LoRA. Then it measures four interfaces that receive the same ego-state history: decimal text, LLMTime-style text, xVal-style scalar embeddings and learned sixteen-position adapters.
The main finding is that accuracy drops monotonically as the state representation is compressed, and the paper appends over 30 pages of post-hoc tables and subgroup breakdowns to support the idea.
Critical strengths
- Useful core question and baseline comparison: It is a practical and useful baseline for embodied VLMs to measure the accuracy-vs-budget trade-off of text serialization against continuous embeddings on a shared multivariate state history.
Critical weaknesses
- Auto-research presentation with post-hoc excuses instead of a research paper: As noted in the AI Use Statement, generative AI was used across ideation, code revision, analysis suggestion, diagram drafting, and writing. In practice, the 43-page manuscript reads like an uncurated automated-research log: 31 pages of appendices and 41 tables mechanically dump numbers accompanied by repetitive, formulaic disclaimers while providing very little genuine scientific insight.
Questions for authors
N/A
3 comments
Response to Reviewer wTHq (2/2)
4. Clarification of the quoted examples
The review describes every appendix section as ending with repetitive automated disclaimers. The examples it cites require the following clarifications:
| Example cited in the review | Manuscript wording, context, and purpose |
|---|---|
| “Ranks are descriptive” | This phrase recurs in table captions to distinguish observed rankings from statistical significance claims. We repeated it so that readers consulting individual tables would have the qualification available alongside the results. |
| “These comparisons condition on the checkpoints and do not estimate retraining variability” | This exact sentence occurs once, in the introductory paragraph of §E.2 (p. 22), not at the subsection’s end. It states the uncertainty scope defined in §3.4 (p. 6): intervals conditional on evaluated checkpoints do not estimate variability under retraining. Variations of this qualification are used to keep the scope of individual comparisons explicit. |
| “This does not establish a common mechanism” | The manuscript states, “This does not identify a common mechanism,” within §E.2 (p. 22), before the magnitude-shift paragraph. It cautions against inferring a shared causal mechanism from a shared performance pattern. |
We accept the criticism that the appendix needs further language revision. We reject the implication that this shortcoming reflects research conducted without substantive author direction, scientific judgment, or verification. We apreciate the positivity about our general idea and would be open to discuss the scientific content.
I appreciate the experimental effort and the inclusion of controls. However, the reason why I flagged the paper a fully auto research paper is that I find the paper reads more like an experimental coverage report than a research paper with clear motivation, interpretation, and actionable conclusions.
-
The reason behind the comparison needs clearer framing.
The interfaces have origins, but the paper does not sufficiently explain why these particular methods lead to the appropriate comparison or what hypothesis each tests.
-
The practical value of the accuracy–cost trade-off remains unclear. The adapter reduces state-segment length by 48.8×, but the full input reduction is 2.16× and the measured prefill speedup is 1.54×, with noticeable accuracy decreases from 96.00% to 88.57%.
-
The experiments reveal differences without sufficiently explaining their causes. Interfaces differ in architecture and optimization, and the three staged runs differ in preparation length as well as seed. These comparisons therefore do not isolate whether the observed failures arise from representation capacity, alignment, training, or other factors.
-
The recommendations largely restate the measured trade-off. Section 4.5 recommends text for accuracy, adapters for context savings, and xVal as an intermediate option. This offers limited guidance beyond the reported rankings.
-
The organization emphasizes experimental execution over the central argument. Numerous recipes, stages, and diagnostics make the paper difficult to follow.
Overall, I do believe the paper contains useful measurements, but it needs a stronger connection between the application need, experimental design, and resulting guidance. At present, I will keep my clear rejection recommendation.
Response to Reviewer wTHq
Dear Reviewer,
Thank you for recognizing the practical relevance of our research question. We were shocked that our transparent AI-use statement and the appendix’s length and presentation could lead to the impression of autonomous research without substantive author direction and curation. We strongly disagree with that characterization and would like to explain our research process.
This was author-led research. The authors designed the study, made the methodological decisions, selected and interpreted the analyses, and determined the conclusions. We reviewed AI-assisted writing and code, checked the reported results and citations, and take responsibility for the submission.
1. What our AI-use statement describes
We reported assistance wherever it occurred, including activities beyond language polishing, to describe our workflow transparently.
Writing and figure preparation. The authors developed the storyline. For some sections, an LLM drafted paragraphs based on that structure, which we rewrote and refined. Figure 1 was produced through Python code written with LLM assistance. The authors directed its concept, layout, and revisions, and checked the numerical values and reported results.
Literature discovery. We used AI-assisted search, including Gemini functionality in Google, and related-work suggestions from LLM feedback. The authors checked the citations included in the submission.
Methodological discussion and analyses. We used LLM feedback, including simulated reviews, to discuss methodological choices and identify questions for investigation, similar in purpose to the optional automated author-feedback programmes offered at NeurIPS 2026 and ICLR 2027. The authors assessed that feedback, decided which analyses to pursue, and interpreted the evidence. We disclosed this as research ideation and execution because the feedback influenced scientific decisions.
Code assistance. Coding agents assisted with parts of the project, particularly analysis and plotting code. The authors reviewed the generated code and resulting plots.
Synthetic caption generation. Our study investigates VLMs, where model-generated training supervision is established practice, as illustrated by LLaVA (Liu et al., 2023) and BLIP (Li et al., 2022). Qwen3-8B generated the wording of Stage-2 alignment captions from deterministic motion facts (§3.3). We disclosed this training-data construction step.
2. Relationship to ICLR’s policies
The ICLR AI Policy for Authors requires disclosure of assistance with hypotheses, experimental methodology, method implementation, interpretation, and synthetic-data generation. It recommends disclosure of drafting, editing, literature discovery, figures, and other code assistance. We reported the relevant activities across both groups. The policy also invites detailed accounts: “we encourage you to provide as much detail as you want!”
3. Purpose and presentation of the appendix
Our study investigates how trained VLMs use numerical state. Given the complexity of these learned interfaces, we used additional empirical analyses to examine the scope of the main findings and aimed for a sufficient documentation of limitations and failure modes. The main paper presents the central argument and principal evidence. The appendix provides further experiments and methodological detail.
The opening guide, “Where each claim is supported,” within the appendix maps claims to the relevant sections. Introductions such as “This section supports Section …” recur because readers were expected to consult individual sections through that guide. We therefore repeated those connections without prioritizing variation in their wording and provided an overview table at the beginning of the appendix (similar to a glossar).
We acknowledge that we concentrated on conducting experiments and interpreting results and did not revise the appendix’s language sufficiently for a full read of the appendix rather than a glossary lookup. Some passages are consequently repetitive and insufficiently polished. We accept responsibility and will streamline the repeated framing and improve the explanations of each analysis’s purpose and findings.