acceptodds
Under review as a conference paper at ICLR 2027

CausGround: Causal Interaction Grounding for Adaptive Embodied Planning

Abstract

Multimodal large language models (MLLMs) have demonstrated promising reasoning capabilities, but deploying them for physical interaction requires reliable planning that accounts for when actions are executable and how they change the world. These preconditions and effects are often environment-dependent and are not fully captured by MLLMs, making adaptation to new environments challenging. Yet existing adaptation approaches either update model parameters through costly environment-specific training or rely on unstructured interaction experience that is difficult to ground and reuse. To address this challenge, we introduce CausGround, a non-parametric causal interaction grounding framework that distills interaction experience into reusable action-centered causal knowledge. CausGround learns from experience by identifying action preconditions and effects, and organizes them in a factorized world-state representation that jointly captures object attributes and inter-object relations. At planning time, task-relevant causal dependencies are grounded in the current observation, supporting backward reasoning over action preconditions and forward reasoning over action effects for action selection and multi-step causal planning. We demonstrate the effectiveness of CausGround across diverse long-horizon embodied settings spanning simulated high-level planning, interaction reasoning, and real-world robot manipulation. Evaluated on the adapted VirtualHome and RoboCasa benchmarks, CausGround consistently improves vision-based high-level planning across MLLM backbones, improving average task success from 24.9% to 53.3% on VirtualHome and closed-loop planning success from 43.0% to 57.0% on RoboCasa. Consistent gains are also observed on SWITCH benchmark, with average interaction reasoning accuracy improving from 25.8% to 31.9%, indicating stronger reasoning over action selection, recovery, and verification. Moreover, real-world experiments further show that CausGround can reason backward from MLLM-interpreted goal states to derive executable high-level action sequences for sequential manipulation. These results demonstrate that explicit action preconditions and effects provide a reusable causal grounding mechanism for long-horizon embodied planning across diverse environments without parameter updates.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Planning, Multimodal Large Language Models, Causal Interaction Grounding, Factorized World States, Non-Parametric Adaptation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_zmNxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

CausGround extracts action preconditions and effects from interaction data and uses the resulting rules to guide a frozen multimodal planner. The paper reports improvements across several embodied benchmarks.

Critical strengths

The problem is relevant, the evaluation spans multiple settings, and the retrieval and rule-component ablations are useful.

Critical weaknesses
  1. The incremental contribution is unclear. Learning action preconditions and effects for planning has a long history, while InterPreT and PSALM-V address closely related embodied settings. The paper does not adequately distinguish its mechanism from these works or compare against a learned action-model planning baseline.
  2. The “causal” claim needs qualification. Appendix C.3.1 uses controlled factor variation in VirtualHome, but extracts rules from observational consistency in RoboCasa and SWITCH. The latter does not by itself establish that a retained factor is a causal prerequisite.
  3. The method requires an environment-specific rule library built from substantial interaction data. The paper should report this construction cost and test whether its gains persist under a matched data and computation budget.
Questions for authors

What improvement remains over a standard learned action-model planner given the same observations and interaction data? How are correlated but non-causal conditions excluded in RoboCasa and SWITCH?

5 comments

We sincerely appreciate your thorough review and insightful feedback. In response to your concerns, we have clarified the distinctions between CausGround and related symbolic planning methods, introduced additional learned action-model and trajectory-retrieval baselines, and provided further data-efficiency and construction-cost analyses. We have also clarified the different levels of causal evidence across environments, supported by additional rule validation. We hope these additional comparisons and clarifications address your concerns and better establish the distinct contributions and practical value of CausGround.

W1: Incremental Contribution Beyond Symbolic Action-Model Planning

We appreciate you for highlighting these closely related works. InterPreT (Han et al., 2024) [1] learns symbolic predicates from human language feedback and action operators from interaction data, while PSALM-V (Zhu et al., 2026) [2] infers symbolic action semantics through visual observations and execution feedback. Both primarily integrate the acquired knowledge into PDDL-based symbolic planning. As detailed in Section 3, CausGround differs from these approaches in two key aspects. First, causal interaction knowledge acquisition: we use controlled factor variations in VirtualHome to test action dependencies and action-conditioned state-transition regularities in RoboCasa and SWITCH. Second, knowledge-grounded MLLM planning: Instead of relying on a symbolic solver, we strengthen a frozen MLLM using retrieved action dependencies grounded in the current visual state. Given the increasingly strong embodied reasoning capabilities of modern MLLMs (e,g., Fable 5.2, GPT-6 Astra), enhancing their planning capabilities with explicit, environment-grounded interaction knowledge represents a valuable complementary direction to traditional symbolic planning.

To further evaluate this distinction, we introduced SAM [3] + Fast Downward [4], a learned symbolic action-model planning baseline, using the same VirtualHome interaction data and 75 evaluation tasks. SAM + Fast Downward achieves 12.0% (9/75) task success, compared with 44.0% (33/75) for CausGround using visual observations. These results provide additional evidence for the effectiveness of our knowledge-grounded MLLM planning framework. We have added the relevant citations and discussion in Section 2, clarified the knowledge acquisition protocols in Appendix C.3.1, and included the learned symbolic planning baseline in Appendix D.5 of the revised manuscript.

[1] InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planning. RSS, 2024.

[2] PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Models. ICRA, 2026.

[3] Safe Learning of Lifted Action Models. KR 2021

[4] The Fast Downward Planning System. JAIR 2006

W2 & Q2: Causal Grounding and Filtering of Spurious Dependencies

We appreciate the reviewer for highlighting this important distinction. CausGround focuses on grounding action preconditions and effects in environmental interactions, with the strength of supporting evidence depending on the available feedback. In VirtualHome, controlled factor variations directly test action dependencies, achieving 100% rule consistency under these tests. In RoboCasa and SWITCH, we apply counterexample-based filtering: a candidate precondition is removed if the action succeeds without it, while a candidate effect is removed if it does not occur following a successful action under applicable conditions. We further examine the extracted knowledge against the official simulator execution code, confirming 98.1% of preconditions and 98.8% of effects, providing stronger mechanistic evidence for the extracted dependencies beyond observational consistency. For SWITCH, we also evaluate the practical utility of the extracted knowledge through action selection, recovery, and verification tasks (Table 2), while developing a controllable testbed for future interventional validation. We have clarified these distinctions in Appendix C.3.1 of the revised manuscript.

W3 & Q1: Knowledge Construction Cost and Learned Action-Model Comparison

We appreciate the reviewer's emphasis on construction cost and fair comparisons. To quantify the data requirements of CausGround, we evaluated different interaction-data budgets in VirtualHome and report the corresponding knowledge coverage and generation costs below. Notably, with only 10% of the interaction data, CausGround already achieves the full-data 44.00% success rate. Additional data further broadens knowledge coverage across different action families and object instances. Importantly, as described in Section 3.3, CausGround requires no planner parameter updates, avoiding gradient-based optimization.

Data RatioProgramsRecordsAction CoverageGeneric SetsSpecific CandidatesSuccess Rate (%)Generation Time
1%1570026/4010634.6715 min
5%753,20637/40171241.3370 min
10%1495,85038/40191544.00130 min
30%48018,96540/40201744.007 h
60%89535,53240/40201544.0013 h
100%1,49158,45840/40202144.0022 h

We additionally evaluated two complementary baselines using the same interaction-data pool: Raw Trajectory Retrieval, which directly retrieves historical interaction trajectories under a matched context-token budget, and SAM + Fast Downward, which learns symbolic action models for classical planning. As shown below, CausGround achieves 44.00%, outperforming both baselines and demonstrating the effectiveness of structured interaction knowledge compared with raw experience retrieval and learned symbolic planning. We have added the data-efficiency and estimated generation-cost analysis in Appendix D.4, and the Raw Trajectory Retrieval and SAM + Fast Downward comparisons in Appendix D.5 of the revised manuscript.

MethodSuccess Rate (%)
Raw Trajectory Retrieval30.67
SAM + Fast Downward12.00
CausGround44.00

Dear Reviewer zmNx,

As the ICLR 2027 discussion period approaches its conclusion, we would greatly appreciate it if you could share your thoughts on our rebuttal and revised manuscript. We would be happy to discuss and address any remaining concerns.

Thank you again for your time and consideration!

Best regards,
The Authors

Reviewer_grCsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CausGround distills interaction experience into explicit action preconditions and effects. A frozen MLLM planner receives them as planning-time context, adapting across environments without parameter updates. World states are represented through object attributes and typed relations. Preconditions are built by controlled intervention in VirtualHome. In RoboCasa and SWITCH they come from a consistency filter over official interaction trajectories. Retrieved rules are grounded in the current observation. Across VirtualHome, RoboCasa, and SWITCH the framework improves over baseline planners. The gain holds across six MLLM backbones on VirtualHome. VirtualHome ablations attribute most of the gain to the causal knowledge and its retrieval path. Two qualitative real-world demonstrations complete the validation.

Critical strengths

Making action preconditions and effects explicit gives a frozen planner reusable structure. The factorized world state makes the knowledge groundable against the current observation. The VirtualHome library builds its preconditions by controlled intervention. A candidate is retained only when violating it prevents execution. This necessity test is designed to exclude spurious preconditions that rules mined from observed successes can retain.

Average success on VirtualHome climbs from 24.9% to 53.3%. The gain holds across six MLLM backbones from three model families. All methods share backbone, observations, and protocol. Ablations locate the source: causal knowledge alone recovers most of the gain. Causal retrieval far exceeds semantic retrieval, and preconditions and effects each contribute.

Critical weaknesses

The knowledge is validated only at construction time. There is no held-out validation of the consolidated VirtualHome rules and no error analysis for RoboCasa and SWITCH. In those two environments the main text describes controlled variations. Appendix C.3.1 shows a consistency filter over successful executions, with no comparison against failures. A factor holding in every recorded success can survive alongside the true preconditions.

The libraries are built from official interaction trajectories and evaluated against the same recorded transitions. Beyond calling the 100 RoboCasa test cases held-out, the paper does not state what they are held out from. On SWITCH, rules extracted from overlapping interaction data could encode the answers being tested.

On RoboCasa, scores are computed only on cases where no method meets an UNKNOWN transition. The one-shot and replanning columns may therefore rest on different subsets, with neither size reported.

Questions for authors

Were the construction data disjoint from the evaluation data? The evaluation data are the 100 held-out RoboCasa test cases and the 768 SWITCH examples; how were the splits defined?

All current evidence for the consolidated knowledge's quality is either construction-time or downstream. Could you report a held-out measure, for example re-testing retained VirtualHome rules on fresh simulator states, or an error analysis for RoboCasa and SWITCH? Relatedly, what exact rule decides consistency, for example the fraction of supporting trajectories required?

A line of work induces the same kind of action knowledge for LLM-based planners and is not discussed. COWP augments preconditions and effects at execution time (Ding et al., 2023). LASP builds precondition knowledge from execution errors (Chen et al., 2024). LODGE induces predicates, operators, preconditions and effects with simulator grounding (Kienle et al., 2025). What do the factorized schema, the intervention- or consistency-based extraction, and the planning-time grounding add over this line? Could the paper cite it?

5 comments

We sincerely thank you for your careful review and valuable suggestions. In response to your concerns, we have conducted additional rule-quality validation and data-efficiency experiments, clarified data separation and evaluation protocols, and introduced raw-trajectory retrieval and symbolic-planning baselines. We have also extended the evaluation with SWITCH final-state prediction and quantitative real-world experiments. We hope the new experimental evidence and methodological clarifications help resolve your concerns about the fairness and reliability of our evaluation.

W1 & Q2: Rule Quality and Library Statistics

Thank you for your insightful comments. As described in Section 3.2 and Appendix C.3.1, CausGround constructs action preconditions and effects using different interaction-based protocols across environments. To further assess the reliability of the extracted knowledge beyond downstream planning performance, we conducted additional action-level rule validation. In VirtualHome, the evaluated preconditions and effects achieved 100% consistency under controlled factor variations, with evaluation trajectories excluded from knowledge construction.

In RoboCasa, the evaluated preconditions and effects achieved 98.1% and 98.8% semantic validity, based on the official simulator's execution logic, with minor errors primarily involving object binding and missing execution conditions. The RoboCasa library statistics are already reported in Appendix C.3.1, including 98 grounded action mechanisms, 254 condition records, and 416 effect records.

For SWITCH, we evaluate the practical utility of interaction-derived knowledge through real-world action selection, recovery, and verification tasks. Its library contains 3,119 occurrence-level semantic action hyperedges from 1,090 source video records, alongside 835 aggregated branch rules.

We have included the additional validation results and SWITCH library statistics in Appendix C.3.1 of the revised manuscript.

Q2: Rule Consistency and Knowledge Consolidation

Thank you for your insightful question. As described in Section 3.2, CausGround extracts action preconditions and effects from interaction experience and organizes them into generic and instance-specific knowledge. To clarify the consistency criterion, we adopt a strict counterexample-based rule: a candidate precondition is removed if the action succeeds without it, while a candidate effect is removed if it does not occur after a successful execution. A single valid counterexample is sufficient for rejection, and no additional minimum support threshold is imposed. Following this criterion, we transform 58,458 semantic action-hyperedge records into 5,614 action rules in VirtualHome through deduplication, generic and instance-specific knowledge abstraction. The reliability of the retained knowledge is further supported by our RoboCasa validation against the official simulator's execution logic, achieving 98.1% and 98.8% semantic validity for preconditions and effects, respectively. We have clarified these details in Appendix C.3.1 of the revised manuscript.

W2 & Q1: Data Separation

Thank you for raising this important concern. We clarify that knowledge construction and evaluation data are strictly separated across all three environments. In VirtualHome, causal knowledge is constructed through independent action-level controlled interventions. In RoboCasa, we use independent episodes with different random seeds and task initializations for knowledge construction and evaluation, ensuring no overlap between the two sets. For SWITCH, we follow the official train and test split, using the training set for knowledge construction and the test set for evaluation. Therefore, no evaluation trajectories are used during knowledge construction. We have clarified these data separation protocols in Appendices A.2.1 and A.3, with the VirtualHome knowledge construction protocol detailed in Appendix C.3.1 of the revised manuscript.

W3: UNKNOWN Transitions in RoboCasa

Thank you for your valuable suggestion. For RoboCasa, as described in Appendix A.2.3, our evaluation focuses on high-level planning using officially annotated skill-level state transitions. We mark unrecorded transitions as UNKNOWN rather than assuming unverified successor states. For fair comparison, all methods are evaluated on the same resolvable subset under each backbone and planning protocol. The retained sample sizes are reported below.

BackboneOne-shot RetainedReplanning Retained
Qwen3.5-9B71/10049/100
Qwen3.5-27B86/10064/100
Qwen3.5-35B-A3B73/10061/100

We further analyzed all 100 original test cases. CausGround achieves the best success rates on the shared subsets and the highest number of confirmed successes across all backbones and protocols. Notably, under 35B-A3B one-shot planning, CausGround achieves 39 confirmed successes, exceeding the strongest baseline's maximum of 37, even if all its UNKNOWN cases are considered successful. We have included these statistics and the corresponding analysis in Appendix A.2.3 of the revised manuscript.

Q3: Distinction from Related Action-Knowledge Planning Methods

Thank you for pointing us to these closely related works. COWP (Ding et al., 2023)[1] augments an existing PDDL action model with LLM commonsense to handle unexpected situations; LASP(Chen et al., 2024)[2] incrementally refines incomplete symbolic knowledge using execution errors; and LODGE (Kienle et al., 2025)[3] learns hierarchical predicates and operators through LLM-based domain generation and simulator-grounded refinement. As detailed in Section 3, CausGround differs from these approaches in two key aspects. First, causal interaction knowledge acquisition: we extract action-centered condition–effect knowledge from environmental interactions, using controlled factor variations in VirtualHome and action-conditioned state-transition regularities in RoboCasa and SWITCH.

Second, knowledge-grounded MLLM planning: CausGround strengthens a frozen MLLM through task-relevant dependency retrieval and visual grounding in a shared factorized world-state space instead of primarily using acquired action knowledge for symbolic planning. Given the growing embodied reasoning capabilities of modern MLLMs (e,g., Fable 5.2, GPT-6 Astra), enhancing their planning with explicit interaction knowledge represents a valuable complementary direction. On VirtualHome, dependency-based retrieval achieves 42.7% success, compared with 18.7% for semantic retrieval alone (Table 8). Furthermore, using the same interaction-data pool, SAM [4] + Fast Downward [5], a learned symbolic planning baseline evaluated with ground-truth symbolic states, achieves 12.0% (9/75) success, compared with 44.0% (33/75) for CausGround. These results demonstrate the effectiveness of integrating structured interaction knowledge into MLLM planning. We have added the relevant citations and discussion in Section 2, clarified the knowledge construction protocols in Appendix C.3.1, and included the learned symbolic-planning comparison in Appendix D.5 of the revised manuscript.

[1] Integrating Action Knowledge and LLMs for Task Planning and Situation Handling in Open Worlds Autonomous Robots 2023

[2] Language-Augmented Symbolic Planner for Open-World Task Planning RSS 2024

[3] Learning Hierarchical Domain Models Through Environment-Grounded Interaction ArXiv 2025

[4] Safe Learning of Lifted Action Models. KR 2021

[5] The Fast Downward Planning System. JAIR 2006

Additional Experiments and Analyses Motivated by Earlier Review

Beyond the concerns addressed above, we also conducted several additional experiments in response to the reviewer's earlier comments. We summarize these results below to provide further evidence of CausGround's effectiveness and practical applicability.

SWITCH Subtask Selection and Final-State Prediction

Thank you for your insightful suggestion. For SWITCH, we originally selected four subtasks (Action, Recovery, Verification Action, and Verification State) to evaluate action-level interaction reasoning, using all 768 test examples without additional sampling. To further evaluate the forward reasoning over action effects described in Section 3.3, we extended our evaluation to include State-Transition Prediction (Final-State, 72 examples), yielding 840 test examples across five subtasks. As shown below, CausGround consistently outperforms Vanilla across all three backbones, demonstrating its effectiveness in predicting action-conditioned state transitions in real-world interaction data.

BackboneVanilla (%)CausGround (%)
Qwen3.5-9B30.5634.72
Qwen3.5-27B19.4426.39
Qwen3.5-35B-A3B23.6129.17

These results have been incorporated into Section 4.3 (Table 2), with the corresponding evaluation details and subtask statistics provided in Appendix A.3 (Table 6) of the revised manuscript.

Raw Trajectory Retrieval Baseline

Thank you for your helpful suggestion. As motivated in the Introduction, CausGround distills interaction experience into structured precondition–effect knowledge to support reusable and grounded planning. Following your advice, we further introduced a Raw Trajectory Retrieval baseline that directly retrieves historical interaction trajectories as planning context, without extracting structured preconditions and effects. Both methods use Qwen3.5-9B on VirtualHome, with the same interaction experience and a matched retrieved-context token budget.

MethodSuccess Rate (%)Progress (%)
Raw Trajectory Retrieval30.6743.89
CausGround44.0061.50

CausGround improves success rate by 13.33% and progress by 17.61%, demonstrating the benefit of structured precondition–effect knowledge over raw trajectory retrieval. For example, in a web-browsing task, the raw retrieval baseline repeatedly attempted Sit without satisfying the precondition, whereas CausGround identified the missing precondition, approached the chair, and continued the task successfully. These results have been incorporated into Appendix D.5 of the revised manuscript.

Quantitative Real-World Evaluation

Thank you for your valuable suggestion. To complement the two real-world demonstrations in Section 4.6 with quantitative evidence, we further conducted 10 trials for each task. CausGround achieved 70% (7/10) success on the multi-object placement task and 60% (6/10) on the microwave-and-bread task. Notably, CausGround consistently generated correct high-level action sequences across the evaluated trials, while most execution failures arose from low-level manipulation limitations, particularly unstable grasping under variations in object positions. These results highlight the stability of CausGround's high-level planning and its applicability to real-world manipulation. We have included these quantitative results and failure analyses in Appendix E of the revised manuscript.

Knowledge Construction Cost and Data Efficiency

We appreciate the reviewer's thoughtful feedback. Building on the non-parametric planning framework described in Section 3.3, we further analyzed the interaction-data requirements and estimated data-generation costs under different budgets in VirtualHome. Notably, with only 10% of the interaction data, CausGround achieves the same 44.00% success rate as the full dataset. Additional data further broadens knowledge coverage across different action families and object instances. We also report the corresponding data generation costs, as summarized below. Importantly, CausGround requires no planner parameter updates, avoiding the gradient-based optimization required by parametric adaptation methods. These results and the corresponding cost analysis have been incorporated into Appendix D.4 of the revised manuscript.

Data RatioProgramsRecordsAction CoverageGeneric SetsSpecific CandidatesSuccess Rate (%)Generation Time
1%1570026/4010634.6715 min
5%753,20637/40171241.3370 min
10%1495,85038/40191544.00130 min
30%48018,96540/40201744.007 h
60%89535,53240/40201544.0013 h
100%1,49158,45840/40202144.0022 h
Reviewer_nRtaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CausGround, a framework that learns reusable action preconditions and effects from interaction experience to improve embodied planning without updating model parameters. It represents world states using object attributes and relations, and retrieves task-relevant causal knowledge to guide planning. Experiments on VirtualHome, RoboCasa, and SWITCH show improvements over existing baselines across different MLLM backbones. The paper also includes qualitative real-world robot demonstrations.

Critical strengths

Explicitly representing action preconditions and effects provides useful structure for long-horizon planning. The factorized world-state representation also makes it easier to connect learned knowledge with current observations.

On VirtualHome, average success improves from 24.9% to 53.3%. The results are consistent across multiple MLLM backbones, and the ablations show that both causal knowledge and the factorized state representation contribute to performance.

Critical weaknesses

In VirtualHome, the authors use controlled interactions to identify preconditions and effects. However, in RoboCasa and SWITCH, causal knowledge is extracted from existing trajectories. It is unclear whether this can reliably distinguish necessary preconditions from correlations in the demonstrations. The paper also does not evaluate the accuracy of the extracted causal rules or how much interaction experience is needed to construct the knowledge library. This makes the claim of efficient adaptation less convincing.

In RoboCasa, the authors evaluate only cases where none of the methods encounters an UNKNOWN transition, potentially introducing selection bias. The real-world evaluation is also limited to two qualitative demonstrations without success rates or comparisons with baselines. These limitations make it difficult to assess how well the method generalizes beyond the evaluated high-level planning settings.

Questions for authors

How much interaction experience is required to build the causal knowledge library? Can the authors evaluate the accuracy of learned preconditions and effects, particularly when extracted from passive trajectories rather than controlled interventions?

For RoboCasa, how many test cases remain after excluding UNKNOWN transitions? How sensitive are the reported results to this filtering? It would also be helpful to report quantitative success rates for the real-world experiments.

3 comments

We sincerely thank the reviewer for the encouraging assessment and constructive feedback. In response to your concerns, we have conducted additional rule-quality validation and data-efficiency experiments, provided a more comprehensive RoboCasa UNKNOWN-transition analysis, and reported quantitative real-world evaluation results. We hope these additional evaluations and analyses adequately address your concerns regarding knowledge reliability, evaluation robustness, and real-world applicability.

W1 & Q1: Rule Quality & Data Efficiency

Thank you for your insightful suggestion. Rule Quality: As described in Appendix C.3.1, CausGround adopts different knowledge construction protocols according to the interaction feedback available in each environment. Following your suggestion, we further conducted additional action-level rule validation to directly assess the reliability of the extracted knowledge. In VirtualHome, the evaluated preconditions and effects achieved 100% consistency under controlled factor variations. In RoboCasa, the evaluated preconditions and effects achieved 98.1% and 98.8% semantic validity, respectively, based on the official simulator's execution logic, with minor errors primarily involving object binding. Moreover, for SWITCH, we evaluate the practical utility of the extracted knowledge through complementary real-world interaction reasoning tasks(e.g., action selection, recovery, and verification). We are also developing a controllable testbed to enable further intervention-based validation of the learned rules.

Data Efficiency. We further investigated the data efficiency of CausGround by constructing knowledge libraries with varying proportions of interaction experience in VirtualHome, using Qwen3.5-9B for evaluation. As shown in the table below, with only 10% of the data, CausGround achieved the same 44.00% success rate as the full dataset. Even smaller data proportions yielded competitive performance, demonstrating the efficient utilization of limited interaction experience. Importantly, additional data is also meaningful since it further broadens knowledge coverage across different action families and object instances, as reflected by the Generic Sets (shared action knowledge) and Specific Candidates (instance-specific action knowledge).

Data RatioProgramsSemantic RecordsAction CoverageGeneric SetsSpecific CandidatesSuccess Rate (%)
1%1570026/4010634.67
5%753,20637/40171241.33
10%1495,85038/40191544.00
30%48018,96540/40201744.00
60%89535,53240/40201544.00
100%1,49158,45840/40202144.00

We have included the rule validation results and knowledge construction details in Appendix C.3.1, and the data-efficiency analysis in Appendix D.4 of the revised manuscript.

W2 & Q2: Unknown Transitions in RoboCasa

Thank you for your valuable feedback. As described in Appendix A.2.3, our RoboCasa evaluation focuses on high-level planning by leveraging officially annotated skill-level state transitions, thereby isolating planning performance from low-level control. When a predicted action leads to an unrecorded transition, we mark it as UNKNOWN rather than assuming an unverified successor state, ensuring reliable evaluation. For fair comparison, all methods are evaluated on the same resolvable subset under each backbone and planning protocol, with the retained sample sizes reported below.

BackboneOne-shot RetainedReplanning Retained
Qwen3.5-9B71/10049/100
Qwen3.5-27B86/10064/100
Qwen3.5-35B-A3B73/10061/100

To further examine the impact of UNKNOWN filtering, we also analyzed all 100 original test cases. Across all three backbones under both planning protocols, CausGround achieves the best or tied-best success rates on the shared subsets and the highest number of confirmed successes on the original test cases. Notably, under 35B-A3B one-shot planning, CausGround achieves 39 confirmed successes, while the strongest baseline can achieve at most 37, even if all its UNKNOWN cases are counted as successful. This demonstrates that CausGround's advantage in this setting remains robust regardless of UNKNOWN outcomes. We have included these statistics and the corresponding analysis in Appendix A.2.3 of the revised manuscript.

W2 & Q2: Quantitative Real-World Evaluation

Thank you for your valuable suggestion. To complement the two real-world demonstrations in Section 4.6 with quantitative evidence, we further conducted 10 trials for each task. CausGround achieved 70% (7/10) success on the multi-object placement task and 60% (6/10) on the microwave-and-bread task. Notably, CausGround consistently generated the correct high-level action sequences across the evaluated trials, while most execution failures arose from low-level manipulation limitations, particularly unstable grasping under variations in object positions. These results highlight the stability of CausGround's high-level task decomposition and its applicability to real-world manipulation. We have included these quantitative results and failure analyses in Appendix E of the revised manuscript.

Dear Reviewer nRta,

We sincerely thank you for your positive assessment and constructive feedback. We greatly appreciate your recognition of our work and hope that our additional experiments and analyses have further addressed your concerns. If you have any remaining questions or suggestions, we would be more than happy to discuss them and provide further clarifications.

Thank you again for your time and consideration!

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.