acceptodds
Under review as a conference paper at ICLR 2027

From Peer Reports to Revisable Intent Evidence: Grounded Peer Evidence Revision for Embodied Collaboration

Abstract

Grounded Peer Evidence Revision (GPER) raises the rate of completed goal units in a Cooperative Embodied Language Agent (CoELA) pipeline by 3.3 and 2.9 points. The gains hold on two suites derived from the PARTNR and TDW- MAT simulators, with 300 and 200 episodes. Each suite carries ambiguity that a peer can resolve, a regime in which a supported report can still leave competing intentions open. The control receives the same qualified facts with no intent update (QF), and both combined 95% intervals exclude zero. On a diagnostic suite of 600 snapshots with matched inputs, GPER makes the fewest harmful revisions among the tested conditions. GPER is an add-on to CoELA. GPER separates whether observations support a claim from how far that claim, assumed true, shifts competing explanations of teammate behavior. Authorization never conditions on active intent preferences, and compatibility never observes support strength or source reputation. An adapter multiplies record support, freshness and a conflict flag into a support strength in [0, 1]. The support strength scales a compatibility score from an evaluator that never sees strength or source. A ledger indexed by events stores each contribution as an active event group and keeps that contribution revisable. We will make the code, prompt templates, diagnostic snapshots and logs for every call available upon acceptance

Primary area:
applications to robotics, autonomy, planning
Keywords:
Multi-agent Embodied AI, Embodied Multi-Agent Collaboration, Intent Reasoning, Evidence-Grounded Reasoning, Evidence Revision

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8htHOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces Grounded Peer Evidence Revision (GPER), an add-on to cooperative embodied-agent pipelines that separates authorization of a peer report from its conditional compatibility with candidate teammate intentions. GPER uses a deterministic support/freshness/conflict strength, an LLM-based compatibility score that is intended to be isolated from support and source information, and an event-indexed ledger to revise or withdraw contributions. Experiments on screened PARTNR-derived and TDW-MAT-derived suites report improved scores.

Critical strengths
  • I appreciate the modular design, which separates evidence grounding, intent compatibility, and revision. I find this separation conceptually appealing because it could make the framework easier to extend.
Critical weaknesses
  • The motivation, proposed mechanism, and claimed novelty are not connected tightly enough to establish what is fundamentally new. The key ideas are distributed across multiple concepts and procedural details, leaving the reader to infer the central methodological contribution. The authors should state the specific research gap, isolate the novel elements.
  • The writing and presentation need to be significantly improved for ICLR.
Questions for authors
  • Which specific components of GPER are novel compared with existing evidence-grounding and belief-revision methods?
  • Can individual modules be replaced or extended without degrading overall performance?
1 comment
AuthorsOpenReview ↗

Response to Reviewer 8htH

Thank you for the review and for the positive view of the modular design. In short: (1) we state the research gap and the two novel elements below. (2) Individual modules can be replaced, and Tables 2 and 3 measure the cost of each replacement. (3) The revised PDF rewrites the presentation.

Weaknesses

1. The specific research gap and the novel elements.

The gap is how a cooperative embodied agent should use a peer report that is supported by observation but compatible with several intentions of the teammate, or that later becomes invalid. Existing pipelines either pass the report to the planner as context (CoELA, LLaMAR), weight it by source reliability (credibility scoring), or update a joint world and mental-state representation (CoBel-World). None separates whether the report is supported from what it implies about intent, and none can withdraw the effect of one report after it has been used.

GPER contributes two elements.

  • Factorized evidence weight. The effect of a report is the product of a support strength, computed deterministically from record support, freshness and conflict, and a compatibility score from an evaluator that sees neither support strength nor source. Removing physical support adds 38 errors [17, 59], and binarizing support strength adds 43 [20, 66] (Table 3). On 120 semantic-direction cases, the separated compatibility evaluator assigns the correct neutral or directional relation in 109 cases versus 94 for a joint evaluator, a paired advantage of 15 [6, 24] (Tables 43 and 44).
  • Event-indexed revisable ledger. Each contribution is stored as an event that can expire, be withdrawn or be refreshed. Removing withdrawal adds 25 errors [7, 43], and removing compatibility refresh adds 18 [2, 34] (Table 3).

Together, the update and the ledger add +2.3 and +2.1 points over a call-matched control that spends the same calls without an update (general response, item 1).

2. Writing and presentation. The revised introduction states the gap and the two elements before any procedural detail.

Questions

Q1. Which components are novel. The two elements above. Grounded support checks and belief revision exist separately in prior work. The novelty is their factorization with an evaluator blind to support and source, and the event-level revision of each contribution.

Q2. Can individual modules be replaced. Yes. Table 2 replaces the support adapter with an LLM verifier (Four-check, thresholded and soft verifiers) and the compatibility evaluator with ported evaluators (CoELA, PCE). Every replacement increases total errors relative to GPER (102, versus 133 to 165). Four-check improves the balanced criterion at the reference setting at the cost of more harmful revisions (54 vs. 29). Each module can therefore be replaced, and Tables 2 and 3 report the cost.

Reviewer_B3FTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces Grounded Peer Evidence Revision (GPER), an extension to CoELA that separates whether a teammate’s report is supported from how that report, assumed true, should affect hypotheses about the teammate’s intention. A deterministic adapter and conflict checker produce a support weight, a separate LLM evaluates compatibility with candidate intentions, and an event-indexed ledger supports expiration, withdrawal, and deduplication. On screened PARTNR-derived and TDW-MAT-derived tasks, GPER improves completed goal units over Qualified Facts Only (QF) by 3.3 and 2.9 percentage points. The paper also presents extensive diagnostic experiments, component ablations, and robustness analyses.

Critical strengths
  1. A clear and useful problem formulation. A report can be well supported yet provide little information about a teammate’s intention. Treating support and compatibility separately is sensible, particularly when observations become stale or must be withdrawn. The treatment of absence claims—requiring evidence of adequate coverage—is also well motivated.
  2. Positive controlled results with meaningful internal controls. The gains over QF have reported confidence intervals excluding zero on both screened suites. QF receives the same qualified facts, making it substantially more informative than a comparison against an unmodified planner alone. The call-matched QF control and Revised State Only condition further help distinguish additional inference, factual information, and explicit preference revision.
Critical weaknesses
  1. The central mechanism is not sufficiently isolated in the strongest experiments. The evidence does not yet cleanly separate the benefits of explicit intent revision and ledger maintenance from those of additional inference, nor establish the downstream contribution of input isolation. Against call-matched QF, GPER’s closed-loop advantage is 2.3 percentage points on PARTNR-derived tasks and 2.1 points on TDW-MAT-derived tasks, but paired confidence intervals for these direct contrasts are not reported. Significance against ordinary QF does not establish significance against call-matched QF. More directly, the isolation intervention in Tables 52–53 reduces errors from 122 to 110, but the paired confidence interval for the reduction is [−4,28][-4,28], with p=0.182p=0.182. Thus, the assertion in Section 5.1 that the gains “stem from” separating grounding from intent compatibility is stronger than the evidence supports. The semantic-direction result is promising, but it does not establish a downstream benefit from isolation in deployed episodes. Direct paired comparisons against call-matched QF and a closed-loop isolation ablation would strengthen the causal attribution.

  2. The primary diagnostic endpoint does not favor GPER. Appendix F identifies the balanced mean of correction-error and harmful-revision rates as the primary diagnostic endpoint. On this measure, Four-check achieves 0.279 versus GPER’s 0.330, with a paired difference favoring Four-check and a confidence interval excluding zero. GPER is not statistically separated from QF or CoBel-World on this endpoint. The trade-off is substantial: GPER makes 29 harmful revisions versus Four-check’s 54, but incurs 245 correction errors versus 175. Moreover, Errors/600 uses the harmful-revision versions of the 200 overlapping bases rather than their correction-error versions. Consequently, this aggregate does not capture GPER’s poor correction performance on those harder versions, where it incurs 172/200 errors. Although the paper transparently acknowledges these distinctions, the results support a trade-off between avoiding harmful changes and correcting mistaken beliefs, rather than overall superiority in evidence revision. Comparing correction accuracy at matched harmful-revision rates across revision-strength settings would clarify whether GPER improves this trade-off.

Questions for authors

1.What are the direct paired confidence intervals for GPER versus call-matched QF on both closed-loop suites? Do the remaining gains of 2.3 and 2.1 percentage points remain statistically significant? 2. Does GPER outperform Four-check on correction accuracy when their harmful-revision rates are matched by tuning revision strength? A comparison across revision-strength settings would help distinguish an improved correction–harm trade-off from a more conservative operating point. 3. Since the coverage and capability fields are supplied by simulator geometry (Appendix D.7), how does performance change when these fields are estimated from observations? Can you also provide a complete trace from observation through authorization, compatibility scoring, ledger revision, and action selection? 4. Tables 8 and 20 imply approximately 4.13k generated tokens per episode (842×1,472/300842 \times 1{,}472 / 300), whereas Table 1 reports 29.4k. What differences in accounting or aggregation explain this discrepancy? 5. Does call-matched QF receive compatibility annotations at the same decision as GPER? Appendix I.2 mentions passing annotations on the next step. Please clarify the timing, since a delay would confound the comparison between explicit preference revision and supplying compatibility scores as annotations.

1 comment
AuthorsOpenReview ↗

Response to Reviewer B3FT

Thank you for the detailed and precise review. In short: (1) the residual gains over call-matched QF are significant, and the annotation timing is the same step. (2) The operating curves answer your matched-rate question. (3) The token figures in Tables 1 and 8 count different quantities.

Weaknesses

1. The central mechanism is not sufficiently isolated. See item 1 of the general response. The paired intervals against call-matched QF exclude zero, so the update and the ledger contribute beyond the extra call. The revised text attributes the gains to these components, and reports the effect of input isolation alone as unresolved.

2. The primary diagnostic endpoint does not favor GPER. At the reference setting this is correct, and Errors/600 does use the harmful-revision versions of the 200 overlapping bases. On their correction versions GPER makes 172 errors of 200, versus 96 for Four-check. Table 22 of the revised PDF reports CE-only, CE-other and HR separately for every condition. Most of the 172 errors (111) come from updates too small to overturn the prior, which is consistent with a conservative revision scale (general response, item 2).

Questions

Q1. Paired intervals against call-matched QF. +2.3 [1.0, 3.6] on PARTNR-derived and +2.1 [0.6, 3.6] on TDW-MAT-derived tasks. Both remain significant.

Q2. Correction accuracy at matched harmful-revision rates. Yes. On the GPER operating curve, β = 0.75 gives 171 correction errors and 47 harmful revisions versus 175 and 54 for Four-check, and interpolation to 54 harmful revisions gives about 166 correction errors. GPER also makes fewer correction errors than the thresholded verifier at matched harmful-revision rates. These swept points are descriptive (general response, item 2, and Appendix O.1).

Q3. Estimated fields and a complete trace. All runs use simulator-supplied fields, and Table 24 reports the effect of field noise. A complete update for one record: EVT_001 from PEER_A states that a cabinet interior is empty, with full interior coverage, acquired at step 8 and used at step 10. Authorization gives α = exp(−2/5) = 0.670. The compatibility call scores the hypotheses retrieve from cabinet, search other room and deliver carried item as 1, 5 and 3, mapped to r = (−1, 1, 0). The preference update with β = 0.25 moves the distribution from (0.468, 0.468, 0.064) to (0.391, 0.546, 0.063), and the continuation call selects an action for the leading hypothesis. When an insertion at step 11 invalidates the absence record, α becomes 0, the event is withdrawn and the distribution returns to (0.468, 0.468, 0.064). The revised PDF gives this trace in Appendix O.3 (Table 57).

Q4. Token accounting. Table 8 reports generated output tokens per decision (Appendix D.2). Table 1 reports input and output tokens per episode. On the primary seed, 842 generated tokens per decision over 1,472 decisions and 300 episodes give about 4.13k generated tokens per episode, as you computed, and the remaining 25.3k are input tokens. GPER uses 1.1k more tokens per episode than call-matched QF (3.9%). The revised captions state both definitions.

Q5. Timing of compatibility annotations. Call-matched QF receives the score in the same decision step, before action selection. The revised Appendix I.2 states this timing (general response, item 1).

Reviewer_FsGCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GPER augments CoELA by separating observational support for peer reports from their implications for intent hypotheses, then updating intent preferences through a revisable event ledger. Compared with a qualified-facts control, it improves the proportion of completed goal units by 3.3 and 2.9 percentage points on screened PARTNR- and TDW-MAT-derived tasks. Unscreened gains are smaller, with intervals including zero. Diagnostics show fewer harmful revisions but a trade-off in correcting existing errors.

Critical strengths
  1. The separation of factual support, intent relevance, and evidence invalidation yields a concrete, inspectable design. The revisable ledger allows outdated or invalidated evidence to be withdrawn from subsequent judgments, addressing an important limitation of permanently accumulating peer messages.

  2. Qualified-facts and call-matched controls, fixed-input diagnostics, noise analyses, and negative controls provide useful checks. Reporting unfavorable unscreened and balanced-metric results also makes the scope of the controlled benefits easier to assess.

Critical weaknesses
  1. Validation of observational support depends on idealized metadata. Coverage and capability fields come from simulator geometry, while diagnostic support labels use the same underlying rules. These results therefore do not independently validate grounding under realistic perception. At 20% field noise, the advantage over Four-check is no longer established, indicating that this assumption materially affects the findings.

  2. The core mechanism and overall revision advantage remain unresolved. On the designated primary balanced diagnostic endpoint, GPER underperforms Four-check, 0.330 versus 0.279, with a paired interval excluding zero. The improvement attributable specifically to information isolation has an interval including zero. The evidence supports a particular trade-off that reduces harmful revisions rather than a generally superior intent-revision mechanism.

Questions for authors

Continue from Weakness,

  1. Is there support annotation independent of the adapter rules, or measured perception error, that connects the gains under idealized metadata to realistic observation conditions?

  2. What are the paired intervals for GPER’s remaining 2.3- and 2.1-point gains over call-matched QF? Which existing results separate information isolation from the update rule and ledger?

  3. Given that the balanced diagnostic measure is designated as the primary endpoint, what task costs or application requirements justify fewer harmful revisions at the expense of more correction errors? Can the corresponding instance-level results be provided to examine this trade-off?

1 comment
AuthorsOpenReview ↗

Response to Reviewer FsGC

Thank you for the careful review. In short: (1) the residual gains over call-matched QF are +2.3 and +2.1 points with intervals excluding zero. (2) A sweep of the revision scale shows that the diagnostic trade-off reflects a conservative operating point. (3) The gains depend on accurate metadata, and Table 24 bounds this dependence.

Weaknesses

1. Validation of observational support depends on idealized metadata.

Item 3 of the general response gives the noise results and the revised scope.

2. The core mechanism and the overall revision advantage.

Items 1 and 2 of the general response give the call-matched contrasts, the isolation result and the operating curves.

Questions

Q1. Support annotation independent of the adapter, or measured perception error.

All runs use simulator-supplied fields. The noise analysis in Table 24 measures how the results change as these fields degrade, and the revised limitations state this scope explicitly.

Q2. Paired intervals against call-matched QF, and results that separate isolation from the update rule and the ledger.

The intervals are +2.3 [1.0, 3.6] and +2.1 [0.6, 3.6] (general response, item 1). Existing results separate the factors as follows.

  • Extra inference without an update. Call-matched QF over QF: +1.0 [0.1, 1.9] and +0.8 [−0.3, 1.9].
  • Belief update and ledger. GPER over call-matched QF: +2.3 and +2.1, both intervals excluding zero.
  • Evaluator organization. One shared call to two isolated calls: 20 fewer errors [3, 37]. Input isolation alone: 12 fewer errors [−4, 28], unresolved (Tables 52 and 53).
  • Ledger operations. Removing withdrawal adds 25 errors [7, 43], removing temporal applicability 26 [8, 44] and removing compatibility refresh 18 [2, 34]. Removing deduplication is unresolved (+12, [−2, 26], Table 3).
  • Revised state without fact text. Revised State Only makes 119 errors versus 125 for QF (point estimates).

Q3. Why fewer harmful revisions can justify more correction errors, and instance-level results.

In embodied collaboration a harmful revision overturns a correct belief about a teammate during execution, whereas a correction error leaves an existing wrong belief open to later evidence. With a loss of CE + λ·HR on the diagnostic suite, the reference GPER has lower loss than Four-check whenever a harmful revision costs more than 2.8 correction errors. The operating curves in the general response also show that a larger revision scale reaches Four-check's operating point, so an application that weights correction more can choose a larger β. Table 22 gives the per-subset decomposition for every condition, and the instance-level outcomes will be released with the diagnostic snapshots and call logs.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers (1/2)

We thank the reviewers. Three points recur: the contrast against call-matched QF, the balanced diagnostic endpoint, and the reliance on simulator metadata. We address them here and answer the remaining points individually. We have uploaded a revised PDF.

1. Paired contrasts against call-matched QF.

Against call-matched QF, the paired residual gain of GPER is +2.3 points on PARTNR-derived tasks (70.4 vs. 68.1, combined 95% interval [1.0, 3.6]) and +2.1 points on TDW-MAT-derived tasks (69.7 vs. 67.6, [0.6, 3.6]). Both intervals exclude zero. The intervals use the protocol of Table 1, and the revised PDF reports them in Section 5.1 and Appendix H.7. The residual retains 70% and 72% of the gain over standard QF, and risk counts also favor GPER against the matched control (30 vs. 34 of 300 and 21 vs. 23 of 200).

Call-matched QF receives the compatibility score from the same dedicated call as GPER, as an annotation to the continuation call in the same decision step and before action selection. The revised Appendix I.2 states this timing explicitly, replacing a description in the submitted version that placed the annotation on the next step. The contrast therefore isolates the belief update and the ledger at a fixed call budget and fixed prompt fields.

The revised Section 5.1 attributes the gains to the belief update and the ledger at a matched call budget. The full evaluator organization, from one shared call to two isolated calls, reduces errors by 20 with interval [3, 37], while the effect of input isolation alone (+12 errors, [−4, 28], p = 0.182, Tables 52 and 53) is reported as unresolved.

2. The balanced diagnostic endpoint.

At the reference setting GPER trades correction errors for fewer harmful revisions. It makes 29 harmful revisions of 380, versus 43 for QF and 54 for Four-check (HR-only paired intervals [2, 26] and [10, 40], Table 31), but 245 correction errors of 420, versus 175 for Four-check. On the balanced criterion Four-check is better (0.279 vs. 0.330, paired difference −0.050 [−0.090, −0.012]), and GPER ranks third of fourteen. The revised abstract and Section 5.2 state this trade-off.

AuthorsOpenReview ↗

General Response to All Reviewers (2/2)

2 (continued). Operating curves (new analysis).

To test whether the trade-off reflects the operating point or the method, we swept the revision scale β of GPER and the threshold of the thresholded verifier on the 600 diagnostic snapshots. The reference β = 0.25 was selected on 90 development bases before any diagnostic evaluation (Appendix N.3), so the sweep below is descriptive and does not select β on the test snapshots. The new Appendix O.1 reports the full sweep (Table 55) and plots it against all conditions of Tables 2 and 3 (Figure 4).

ConditionCE/420HR/380Balanced
GPER, β = 0.10268180.343
GPER, β = 0.25 (reference)245290.330
GPER, β = 0.50195400.285
GPER, β = 0.75171470.265
GPER, β = 1.00160630.273
Thresholded verifier, best (0.5)215520.324
Four-check (reference)175540.279

As β grows, GPER repairs more beliefs and makes more harmful revisions. At β = 0.75 it has fewer correction errors (171 vs. 175) and fewer harmful revisions (47 vs. 54) than Four-check, and interpolating to Four-check's harmful-revision count of 54 gives about 166 correction errors. At matched harmful-revision rates GPER also makes fewer correction errors than the thresholded verifier at every swept threshold. The reference setting is therefore a conservative point on a curve that reaches Four-check's operating point. These swept points have no paired intervals, and we report them as descriptive. The submitted reference setting remains the basis of all confirmatory results.

The causes of the reference-setting correction errors agree with this reading. Of the 172 errors on the correction versions of the 200 overlapping bases, 111 come from an update too small to overturn the prior, 27 from the continuation call, 20 from zero authorization and 14 from the compatibility score (Appendix O.2, Table 56).

3. Simulator-supplied metadata.

Coverage and capability fields come from simulator geometry (Appendix D.7), and the diagnostic labels use the same rules, so the diagnostic suite does not independently validate grounding under realistic perception. Table 24 bounds this dependence. With 10% noise in these fields GPER keeps its advantage over Four-check (118 vs. 138 errors, [2, 38]). At 20% the advantage is no longer established (137 vs. 145, [−10, 26]), and at 35% the point estimate favors Four-check by 4 errors ([−22, 14]). The revised limitations state that the results hold under accurate metadata and weaken with field noise.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.