- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces Grounded Peer Evidence Revision (GPER), an add-on to cooperative embodied-agent pipelines that separates authorization of a peer report from its conditional compatibility with candidate teammate intentions. GPER uses a deterministic support/freshness/conflict strength, an LLM-based compatibility score that is intended to be isolated from support and source information, and an event-indexed ledger to revise or withdraw contributions. Experiments on screened PARTNR-derived and TDW-MAT-derived suites report improved scores.
Critical strengths
- I appreciate the modular design, which separates evidence grounding, intent compatibility, and revision. I find this separation conceptually appealing because it could make the framework easier to extend.
Critical weaknesses
- The motivation, proposed mechanism, and claimed novelty are not connected tightly enough to establish what is fundamentally new. The key ideas are distributed across multiple concepts and procedural details, leaving the reader to infer the central methodological contribution. The authors should state the specific research gap, isolate the novel elements.
- The writing and presentation need to be significantly improved for ICLR.
Questions for authors
- Which specific components of GPER are novel compared with existing evidence-grounding and belief-revision methods?
- Can individual modules be replaced or extended without degrading overall performance?
Response to Reviewer 8htH
Thank you for the review and for the positive view of the modular design. In short: (1) we state the research gap and the two novel elements below. (2) Individual modules can be replaced, and Tables 2 and 3 measure the cost of each replacement. (3) The revised PDF rewrites the presentation.
Weaknesses
1. The specific research gap and the novel elements.
The gap is how a cooperative embodied agent should use a peer report that is supported by observation but compatible with several intentions of the teammate, or that later becomes invalid. Existing pipelines either pass the report to the planner as context (CoELA, LLaMAR), weight it by source reliability (credibility scoring), or update a joint world and mental-state representation (CoBel-World). None separates whether the report is supported from what it implies about intent, and none can withdraw the effect of one report after it has been used.
GPER contributes two elements.
Together, the update and the ledger add +2.3 and +2.1 points over a call-matched control that spends the same calls without an update (general response, item 1).
2. Writing and presentation. The revised introduction states the gap and the two elements before any procedural detail.
Questions
Q1. Which components are novel. The two elements above. Grounded support checks and belief revision exist separately in prior work. The novelty is their factorization with an evaluator blind to support and source, and the event-level revision of each contribution.
Q2. Can individual modules be replaced. Yes. Table 2 replaces the support adapter with an LLM verifier (Four-check, thresholded and soft verifiers) and the compatibility evaluator with ported evaluators (CoELA, PCE). Every replacement increases total errors relative to GPER (102, versus 133 to 165). Four-check improves the balanced criterion at the reference setting at the cost of more harmful revisions (54 vs. 29). Each module can therefore be replaced, and Tables 2 and 3 report the cost.