- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents FeelCap, a multisensory data-collection device; FEELING, a dataset of manipulation recordings; and FEEL, a representation learning method. The method combines semantic alignment and sensory reconstruction, with evaluation on retrieval, reconstruction, and real-robot manipulation.
Critical strengths
I like the hardware design. Integrating vision, depth, touch, force, and audio into a device that supports both handheld collection and robot use is a useful contribution. The modular design looks well thought out.
The dataset is another strong point. It contains 9,636 episodes and 37,391 action segments, covering a range of materials and interactions. The synchronized recordings and segment-level annotations could be valuable beyond the proposed model.
The paper is clear and easy to follow. I also appreciate the real-robot experiments. Both policies improve on all three tasks in the reported trials, which provides some evidence of practical value.
Critical weaknesses
My main concern is environmental diversity. Many examples show similar green tabletop setups. This does not establish that collection was limited to one environment, but I couldn’t find a breakdown of locations, backgrounds, lighting, or acoustic conditions. The paper reports material and action diversity, while the benchmark holds out objects. Neither tells us how well the representations transfer to different recording environments.
The robot evaluation needs more detail. Section 5.4 gives the tasks, demonstration counts, and success criteria, but leaves the policy integration and test conditions unclear. Which sensors does each baseline receive? How are FEEL features added? What changes between training and testing? There are also only 10 trials per setting. For example, π0’s pouring improvement from 40% to 50% is one additional success. The results are encouraging, but still preliminary.
The learning method would benefit from cleaner comparisons. I’d like to see a simple fusion baseline with the same sensors and encoders, plus ablations of the three-space design and reconstruction losses. The Stage-1/Stage-2 results use different evaluation protocols, so they do not isolate the effect of the second training stage.
Questions for authors
- How many collection locations and setups are included? How much variation is there in backgrounds, lighting, and acoustic conditions? Are any environments held out for evaluation?
- What microphone is used, and how do you handle background noise and robot motor noise during deployment?
- Which sensors do the baseline and FEEL-augmented policies receive? How are FEEL features integrated and sensor streams synchronized?
- What varies across robot trials—for example, object poses, initial contents for pouring, or board placement? Are more trials or same-input fusion comparisons available?