acceptodds
Withdrawn from ICLR 2027

Seeing is Believing: An Embodied Benchmark for Trust Calibration in Multimodal Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Embodied agents receive instructions and reports on channels that are not equally reliable, and only their own perception can settle what is true. We ask whether current omni-modal models know whom to trust, and whether they can earn that knowledge by looking. We introduce , an embodied benchmark over dynamic household scenes in which a spoken and a written informant carry hidden, persistent reliability profiles, and the agent's own camera is the only means of verification, at a cost in navigation steps. Six evaluation levels, from perceiving the scene to acting on unverified claims, each isolate one capability with one primary metric, so that a failure is attributed to a capability rather than folded into one score. The capstone metric counts only test informants and only decisions the agent chose not to verify, so it cannot be farmed by trusting, declining, or verifying everything, and a ground-truth perception variant separates perception failures from trust failures. Models are evaluated without parameter updates, with the agent's own verification history as the only state carried across episodes. Across six open omni-modal models, the levels rank the models differently, so a single score would hide where each model fails. Several models favor whichever informant is labeled as speaking, the model with the highest memory score is not the one that acts best without checking, and only one model beats trusting every claim by a margin that does not survive our robustness checks. thus provides a farm-resistant testbed for embodied trust and a map of where current agents fail to earn it.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
embodied AI, trust calibration, multimodal agents, benchmark, audio-visual reasoning, verification, reputation, vision-language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_uPo5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces a benchmark over dynamic household scenes to examine how agents receive spoken and written information of unknown reliability and verify it through visual observation at a navigation cost. Evaluations of six open omni-modal models reveal uneven performance across capabilities and persistent difficulty in forming reliable trust judgments from verification history.

Critical strengths
  1. The paper addresses an important and timely problem about trust calibration in embodied agents.
  2. The tasks are clearly defined. the figures and tables are well presented.
Critical weaknesses
  1. The main text reports experimental results under only a limited set of settings, weakening the contribution of the proposed benchmark.
  2. Some values in Tables 3 and 4 appear unusual, such as the highly repetitive standard deviations for CRA in Table 3 and the identical 95% confidence intervals in Table 4. More extensive experimental evidence would help resolve concerns about the reported results.
Questions for authors

See weakness

Reviewer_VYLQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces a multimodal embodied intelligence benchmark called Seeing Is Believing, which aims to evaluate how embodied agents respond to statements from unreliable information sources, decide whether to trust them, whether to actively verify them, and how to leverage past verification experiences to improve subsequent decisions. The paper designs multiple evaluation levels and employs different metrics at each level to measure agent performance. It evaluates six omnimodal large models and identifies the limitations of existing models.

Critical strengths
  1. The problem studied in this paper is meaningful, as embodied agents in real-world scenarios may need to handle inaccurate statements from external information sources. By integrating source reliability, active verification, and the use of historical information, this paper proposes an evaluation direction with certain research significance.

  2. The paper includes several control experiments, such as label swapping, informant identity shuffling, empty verification history, and presenting spoken claims as text, providing a relatively comprehensive experimental evaluation.

Critical weaknesses
  1. (Critical) The data scale is insufficient. the benchmark proposed in the paper comprises a total of only 104 episodes, with the test set containing just 27 episodes and 12 informants. The data scale and diversity are insufficient to fully support the evaluation requirements of a benchmark.

  2. (Critical) The real-world relevance of the benchmark is not clear. The paper simulates information conflicts by manually specifying the reliability of informants, room-specific and object-category-specific error rates, and rules for generating true and false statements. These manually designed settings lack grounding in real-world scenarios and exhibit a gap from real environments. Moreover, the paper does not sufficiently point out the rationale behind these manually designed settings, making the benchmark overly subjective.

  3. The audio in the benchmark is generated entirely via TTS. Compared to the text, it introduces no distinct informational content but may instead introduce additional perceptual errors. Although the authors conducted a speech-to-text control experiment, the study lacks a sufficient justification for the necessity of the speech modality, as well as a systematic analysis of the relationship between speech comprehension errors and the ability to detect information conflicts.

  4. Innovation is limited compared to SpaMEM. It directly reuses SpaMEM’s embodied environment, action sequences, and visual data, primarily replacing the originally reliable text history with unreliable synthetic statements while adding mechanisms to assess speech and source reliability. Since the core embodied environment is a contribution of existing work, this paper represents an incremental improvement. Furthermore, as all data is synthetic, there is a gap with the real world, making it difficult to demonstrate that this benchmark accurately reflects issues regarding information conflict and trust assessment in real-world scenarios.

  5. The analysis of the experimental results is insufficient. Most models in the study performed poorly, and the rankings across different levels lacked a consistent pattern, in some instances, larger models or "thinking mode" models from the same series actually underperformed. Although the authors conducted some analysis, they failed to provide a deeper explanation for these unusual phenomena and did not include mainstream closed-source models in the evaluation.

  6. The data distribution and training set lack convincing design. The test set manually includes 6 highly reliable and 6 highly unreliable informants, resulting in a clear distributional discrepancy between the training and test sets. Although a training set is provided, the paper does not conduct model training experiments to validate it.

  7. The presentation of the paper is relatively poor. The paper focuses primarily on describing the benchmark itself but lacks sufficient justification for its design choices, such as why 6 evaluation levels are defined and why specific metrics are selected for each level. Since the proposed benchmark is specialized and does not have widely adopted conventions in the community, these design choices require clear explanations and supporting evidence. However, the paper does not provide enough explanations or evidence.

Minor weaknesses:

  1. Figs 1, 2, 3, and 5 are not vector graphics, and their quality becomes poor when zoomed in, which affects reading.

  2. In Figure 5, text like “history (infxx)” appears after “A: act without looking”, indicating a formatting or rendering issue.

  3. The experimental setup lacks citations, and citations for the comparison models are missing from the text or tables. The paper needs to improve its academic writing standards.

  4. Although the paper presents the evaluation results to show the limitations of existing models on the proposed benchmark, it does not propose any methods to improve the capabilities of these models.

Questions for authors

see weaknesses, mainly focus on the following issues:

  1. There are many manually designed rules in the benchmark's data construction. For example, why is informant reliability related to room and object categories? I am curious about the rationale behind these human-designed rules. If relying only on subjective manual designs, it may fail to accurately reflect the information conflicts encountered in real-world scenarios.

  2. The innovation of building upon SpaMEM. It seems that the core contribution of the newly introduced task compared to SpaMEM primarily involves replacing the originally reliable text history with unreliable synthetic statements, while adding speech and mechanisms to assess source reliability. But I am curious about why does this benchmark need to be conducted in an embodied setting? The paper could better clarify why an embodied setting is essential for studying source reliability and information conflicts.

  3. Multimodal design. The speech in this paper is entirely generated via TTS. I am curious about the necessity of introducing the speech modality for the tasks of source trust and information conflict that this paper focuses on.

  4. Experimental results. Most models performed poorly on key tasks, and model rankings across different levels lacked a consistent pattern. In some cases, larger models or "thinking mode" models even underperformed. How should these counter-intuitive results be interpreted? Do they truly reflect differences in the models' ability to handle conflicting information, or are they influenced by the design of the benchmark tasks and the choice of metrics? What is the performance for human or advanced closed-source models, such as GPT-6, on the proposed benchmark.

  5. Data scale and distribution. The test set comprises only 27 episodes and 12 informants, with each informant appearing in just 3–7 episodes. I don't think this scale is sufficient to support a reliable evaluation. Furthermore, the manually designed categorization of informants into "highly reliable" and "highly unreliable" groups might make the evaluation results overly dependent on a specific data distribution.

  6. Design of the evaluation. The paper categorizes the benchmark into 6 levels and assigns distinct metrics to each, yet it lacks a sufficient rationale for this design. The author should clarify whether such a classification is necessary or sufficient. Do these metrics genuinely measure the ability to assess the paper's primary focus of information conflict without being confounded by other capabilities such as visual perception or navigation?

Reviewer_FXZwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SeeingIsBelieving, an embodied benchmark built on SpaMEM/ProcTHOR houses. In each episode, a spoken (TTS) informant and a written informant make claims about a changing household scene. Each informant has a hidden, persistent reliability that depends on room and object type, and the agent can verify any claim with its own camera at a cost in navigation steps. Six levels each isolate one capability with one primary metric: perception (GS), single-claim veracity (VF1), conflict adjudication (CRA, with a modality-bias diagnostic), verification efficiency (VE), cross-episode memory (MS), and trust correctness on unverified test-informant decisions (TCS-NV, with ECE). At L4-L5 the 27 evaluation episodes form one fixed sequence, and the agent's own verification history is the only state carried across episodes; there are no parameter updates. Six open omni-modal models are compared with analytic baselines, oracles, and Bayesian policies simulated from generator parameters, under five controls: text-only, label swap, identifier shuffle, empty history, and a warning prompt.

The main findings are as follows. Three models prefer the informant labelled SPOKEN, and this preference follows the label rather than the audio. VE and MS effects are small and none survives multiple-comparison correction. Only Qwen3-Omni-Instruct exceeds the trust-all baseline on TCS-NV (+0.12), and this margin vanishes on a second corpus, under moderate stratification, and when receptacle swaps are excluded.

Critical strengths
  1. Well-motivated problem formulation with a metric design that resists degenerate policies. Unlike knowledge-conflict and modality-bias benchmarks, which use anonymous, one-shot, uncheckable sources, here informants recur, verification is costly, and the agent's own perception is the arbiter. Because TCS-NV is restricted to unverified decisions on held-out test informants, it cannot be inflated by trusting, declining, or verifying everything, or by memorising the training pool. The ground-truth-perception variant separates perception failures from trust failures. This design is the main source of the paper's value, and the generator could become a useful RL environment.

  2. Careful statistics and controls, yielding one robust, non-obvious finding. The paper uses a wild cluster bootstrap-t over informants, reports which results survive Bonferroni correction, includes threshold sensitivity and second-corpus replication, and reports negative results honestly. The text-only and label-swap controls (Table 7) show that the SPOKEN preference attaches to the presentation label rather than to the audio modality. This runs opposite to the text dominance reported by Deng et al. (2025) and Wang et al. (2025), and it survives correction. It is the clearest empirical contribution.

Critical weaknesses
  1. The benchmark appears underpowered for the capabilities it targets, and the headline results are mostly null or fragile. The evaluation split has only 27 episodes and 12 test informants. The minimum detectable MS is 0.10 (App. F), while the largest observed MS is +0.11, which does not survive correction. No positive VE gain survives correction either. Kendall's W across levels is 0.19 (p ≈ 0.25), so the claim that the levels "rank the models differently" and therefore measure different capabilities cannot be distinguished from ranking noise; the Figure 6 caption concedes that most first places lie within noise. The only above-baseline result at L5 disappears on the second corpus, under moderate stratification, and without receptacle swaps (App. D, Table 11). For a benchmark paper, the central question is whether it reliably separates agents along its proposed axes, and the current evidence largely says it does not yet. Since the generator exists, a substantially larger evaluation split seems feasible.

  2. The trust signal exists only under an artificial reliability distribution, and the key reference comparison is not like-for-like. Headroom depends on stratifying test informants to extremes: six at R_p ∈ [0.12, 0.31] and six at [0.82, 0.96]. With training-process informants the headroom is 0.04, and with moderately unreliable ones no model clears the baseline (Table 11). Informants who are wrong 70-88% of the time and can simply be inverted are a narrow test of "whom to trust".

    The Bayesian reference policies behind the "32%" headline are also not run in the simulator. They are simulated from generator parameters, with a scalar i.i.d. reader accuracy and an assumed look cost, and the group-aware variant is told the true reliability groups.

    Finally, the gap decomposition is order-dependent, and the paper reads it two ways. §5 attributes 0.32 of the gap to perception, whereas App. C concludes that "the shortfall lies mainly in how they use the recorded outcomes" (+0.33 vs. +0.84 under ground-truth perception). The central diagnostic claim is therefore not settled.

Questions for authors
  1. Can you report results on a substantially larger evaluation split from the same generator (e.g., ≥50 test informants with more exposures each), together with the minimum detectable MS and TCS-NV margins? If the qualitative conclusions hold at that scale, my soundness concern would be largely addressed.

  2. Please report both orderings of the gap decomposition, or a symmetric (Shapley-style) attribution, and reconcile §5 ("perception costs 0.32") with App. C ("the shortfall lies mainly in how models use outcomes").

  3. Can the Bayesian reference policies be executed inside the simulator with a real VLM reader, rather than simulated with a scalar reader accuracy? How much does the 32% figure change? Should the group-agnostic policy (24%), which is not told the reliability groups, be the headline reference?

  4. The Qwen3-Omni models carry nothing across episodes, and the history-use prompt changes them by at most 0.01. How long, in tokens, and in what format is the verification history in late episodes, and is it ever truncated? Would a compact per-informant summary (e.g., "3 held / 5 did not hold") change the memory results?

  5. Is the order of the SPOKEN and WRITTEN claims in the user turn counterbalanced? Could position or prompt wording, rather than the label itself, drive the sign reversal under the label swap?

  6. Clarifications: (a) Why is ACT on an off-topic claim scored +1? (b) At L2, 84 of 148 gold answers are NEITHER, so a constant answer is unbeaten. Would a class-balanced metric be more informative? (c) Please report |N| per model alongside TCS-NV. (d) Why evaluate at temperature 0.7 rather than with greedy decoding? (e) The Reproducibility Statement promises release of the corpus, evaluation code, and per-episode result files. Since the contribution is a benchmark, can these be made available, in anonymized form, for verification? (f) The paper is very dense, with key numbers scattered between §5 and App. F, and many overlapping reference points (pooled, informant-level, and group-level oracles; three Bayesian variants with two readers each). Consolidating the main comparisons into one table in the main text would substantially improve readability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.