- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors introduce RVPA, a framework that uses VLMs to encode scenes and environments into a different, cheaper representation that can subsequently be queried for in-domain queries without needing to re-use the VLM.
Critical strengths
Tradeoff: I appreciate the framing of Figure 1: trading off per-domain construction with per-inference cost. With more clarity in the text, I can appreciate the potential of exploring the pareto frontier in this regard. This allows downstream users to "tune" how important each criteria is: if you have many future domains you may want cheap domain construction, but if you are constrained to one domain with many inferences you will prefer the opposite.
Critical weaknesses
Method Clarity: The current text in S4 is very hard to follow. Even after multiple reads I am still unsure what each component looks like. Almost all relevant entities are under-described. For example: what is the state s, how is it represented? How is the renderer instantiated? What is the reader G? What do "plans" look like? How is "solvable" determined by the Candidate Planner? All of these entities are described at a high level and I still do not understand what they encode and how they encode it. Similarly, Figure 2 does little to help: the authors should replace the box that says "State" with an actual visualization of what the "state" is, likewise for the reader G and Dynamics. I understand the flow of the Figure, but there is not sufficient information to understand each component.
As the method section is severely underspecified, I find it difficult to contextualize the experiments. What is "more reusable code" (RQ1)? Does this refer to the Readers? Likewise there are no qualitative figures to help explain where the differences stem from in comparing the methods in experiments. In the papers' current state, I cannot evaluate the results in the experiments and do not understand why the proposed method outperforms the baselines.
Questions for authors
Please see the weakness. The paper is highly underspecified and describes things only at a high-level, this severely impedes the ability to assess it's soundness and contributions.