- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This method proposes putting raw tactile images into the demonstration context of a frozen, tool-using agent, plus a controlled comparison that holds current sensing fixed. The main claim is that tactile content in demonstrations helps, beyond current tactile sensing.
Critical strengths
The research question is well scoped and clear. The main comparison between Visual–motion and Tactile-ETA that use the same expert episodes, segmentation, sensing, controller, and budgets seems reasonable. Only the tactile content in the demos differs, which is the right comparison to make. I also like that the negative results are reported honestly. The authors have also done a good job with reproducibility by giving all the details in the appendix.
Critical weaknesses
While sensible, this work doesn’t seem to be a contribution on its own that is well established in the paper. Here are some of my main concerns with the paper: There seems to be limited novelty. Each component described in the pipeline like, frozen-LLM in-context imitation, VLM agents using demos plus closed-loop feedback, tactile reasoning in VLMs, etc., already exists.
The experiments also seem sparse and low in number. There's one demonstration pair, one run per condition, and no confidence intervals which make the results anecdotal and not high in confidence.
The results also seem a bit fragile. In one of the applications, success goes from 36% to 2% with four demos instead of two; from 36% to 2% with all frames instead of Selected; from 36% to 0% with a different model. This sensitivity makes it hard to tell real tactile in-context learning apart from a prompt configuration tuned to these conditions.
The real-robot experiment doesn't test the central claim. It compares no demo against one tactile demo, with no visual–motion arm, and runs only five trials. It shows that demonstrations help, not that tactile content helps.
The results also rely on a model at a specific reasoning setting, and model versions keep changing. Naming is also inconsistent: "GPT-6" in the main text, "GPT-6 Astra" in the appendices.
Questions for authors
These are my main questions:
- How does performance change if the demonstration tactile strips are shuffled across segments or swapped with strips from a different episode?
- Can you report results over multiple demonstration pairs and multiple seeds, with confidence intervals?
- Why were the representation and model ablations run only on Lift Bottle? Can you report the others too?
- On the real robot, what is the success rate with a visual–motion demonstration and no tactile content?
- What explains Lift Bottle dropping to 2% with four demos and with all frames? Is it context length, image count, or contradictory demonstrations? Do episode traces show a characteristic failure mode?