- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces a benchmark over dynamic household scenes to examine how agents receive spoken and written information of unknown reliability and verify it through visual observation at a navigation cost. Evaluations of six open omni-modal models reveal uneven performance across capabilities and persistent difficulty in forming reliable trust judgments from verification history.
Critical strengths
- The paper addresses an important and timely problem about trust calibration in embodied agents.
- The tasks are clearly defined. the figures and tables are well presented.
Critical weaknesses
- The main text reports experimental results under only a limited set of settings, weakening the contribution of the proposed benchmark.
- Some values in Tables 3 and 4 appear unusual, such as the highly repetitive standard deviations for CRA in Table 3 and the identical 95% confidence intervals in Table 4. More extensive experimental evidence would help resolve concerns about the reported results.
Questions for authors
See weakness