acceptodds
Under review as a conference paper at ICLR 2027

Planning Takes More Than Token Prediction: Causal Plan for Benchmarking and Building Physically Grounded Embodied Reasoners

Abstract

Current benchmarks for embodied vision-language planning inadvertently favor linguistic next-token prediction over physically grounded next-state reasoning. This rewards models that mimic statistical language priors rather than track true causal dependencies, reducing complex physical planning to shallow sequence modeling. Hence, achieving genuine physical autonomy requires a fundamental shift from linguistically grounded token prediction toward physically grounded causal reasoning. To this end, we introduce Causal-Plan-Bench, a high-fidelity diagnostic suite spanning four causal dimensions, curated via multi-stage verification. To endow models with this capability, a four-stage annotation pipeline extracts structured interaction records from egocentric videos to construct Causal-Plan-1M, a dense million-scale corpus of explicit causal reasoning traces. Extensive evaluation reveals a striking gap: leading models struggle to demonstrate genuine physical agency — even GPT-6-astra scores only 43.04. In contrast, our tailored training recipe enables Causal Planner to internalize the complex physical logic required for accurate next-state estimation. Built upon Qwen3-VL-8B, Causal Planner raises its backbone’s score from 33.23 to 45.28, a 36.3% relative gain, and improves on three external benchmarks without benchmark-specific adaptation. We further observe an empirical Causal-Supervision Scaling Trend. Paired no-vision controls also reveal substantial visual dependence, while cross-judge comparisons and human scoring assess the reliability of automated evaluation. More importantly, we initiate the first effort to turn agents from superficial token predictors into physically grounded causal reasoners, bridging language modeling and world modeling.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Embodied planning, causal reasoning, vision-language models, egocentric video, physical reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_W4sBOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that embodied-planning benchmarks reward linguistically plausible next-step prediction rather than physical causal reasoning. Validation also includes no-vision controls, a cross-judge comparison, human calibration of the judge, and a data-scaling trend.

Critical strengths
  1. Unusually thorough evaluation-reliability analysis. No-vision controls drop MCQ accuracy to near chance. Rankings are stable across three judges (Spearman 0.90–1.00). Human calibration over 2,700 ratings quantifies both judge agreement (Spearman 0.835) and judge bias (+3.04). This makes the benchmark’s measurements more trustworthy than those of typical LLM-judged benchmarks.
  2. A reusable, well-documented resource. The structured annotations cover preconditions, effects, counterfactuals and failure recovery, with source-level train/test separation, full prompts, rubrics and provenance. They offer value to the community independent of the model results.
Critical weaknesses
  1. Circular, in-distribution evidence. GPT-5.6-sol annotates the data, generates the questions, writes the reasoning traces, judges the open-ended answers and supplies the RL reward. Training and test data come from the same pipeline. Beating the teacher in-domain therefore likely reflects learning the format and the judge’s preferences, not “physical causal reasoning.” External gains are modest (+3–5 points, still below several closed models), and RoboVQA is both a training source and an external test set.
  2. Claims outrun the evidence. The paper frames itself as moving beyond token prediction toward world modeling and physical agency. Yet the method is next-token SFT plus GRPO, the evaluation is entirely offline QA with no closed-loop execution, and ablations come from single training runs with differences of only 2–3 points.
Questions for authors

See the weakness.

3 comments
AuthorsOpenReview ↗

Thank You for Your Insightful Assessment and Valuable Guidance on Evaluation Reliability, Training Stability, and the Scope of Our Claims

We sincerely thank you for your thoughtful review and recognition of our resources. Your feedback motivated further experiments and helped us clarify our findings.

W1. Shared pipeline and external overlap. GPT-5.6-sol generates annotations, QA pairs, traces, RL rewards, and open-ended scores. We examined this potential bias using matched training, expert review, exact-match scoring, visual controls, and alternative judges.

For benchmark curation, Qwen3.5-397B-A17B ranks candidates, Gemini 3.7 Flash audits grounding, and two experts independently inspect the evidence, answers, traces, distractors, and rubrics. Expert agreement on 2,400 admission decisions is 96.29% (α=0.940; B.5–B.6). Table 11 rejects items answerable from language priors alone; plausible MCQ distractors change decisive physical details, such as the faucet outlet versus its neck (Figure 5).

Six MCQ tasks use exact matching without an LLM judge: Causal Planner scores 46.67% versus 35.83% for its backbone and 37.83% for GPT-5.6-sol (Table 16). Open-ended tasks use expert-designed, task-specific rubrics that assess state changes, dependencies, and recovery, penalize unsupported claims, and accept equivalent wording (B.18). Three experts scored 900 answers (2,700 ratings), with ICC(A,3)=0.929; GPT-5.6-sol correlates with their mean (Spearman 0.835; MAE 5.58; offset +3.04; B.11/Table 18). GPT-5.6-sol, Qwen3.5-9B, and Seed2.1 Pro also rescored the same blinded five-model answers. Causal Planner ranks first under each (+13.26/+11.16/+13.09 over its backbone; rank ρ 0.90–1.00; Table 4). Two alternative GPT-5.6-sol prompts, using fixed answers and rubrics, preserve its lead (+14.58/+13.94; rank ρ 1.00/0.90). GRPO uses separate categorical reward rubrics, distinct from continuous benchmark scoring (B.19/B.18).

We trained a matched general-CoT control with the same filtered QA pairs, visuals, GPT-5.6-sol teacher, initialization, filtering, and SFT/RL settings. Both retain rationales; only guidance on preconditions, state changes, and dependencies differs. Despite 5.6% more reasoning tokens, general CoT scores 42.46 versus 45.28 for Causal Planner (−2.82) and trails on all three external benchmarks (−3.58 on Cosmos-Reason1). With the teacher fixed, this supports causal guidance beyond trace length.

Visual controls offer further evidence. Removing videos, keyframes, and visual-derived metadata while keeping questions and scoring fixed lowers Causal Planner Overall from 45.28 to 22.90 and MCQ accuracy from 46.67% to 26%, near four-choice chance (25%). GPT-5.6-sol and Gemini 3.1 Pro also lose 16.92 and 18.76 points (Table 3). Task 10 tests how lid removal enables washing; Tasks 18–20 examine flawed-plan repair (30.50→41.80), blocked-pot counterfactuals (33.40→46.50), and kettle–faucet recovery (35.50→47.20; Table 15).

We screened SFT/RL data against all four benchmark evaluation sets using QA hashes, MinHash, embedding similarity, and available source/video/trajectory IDs, removing detected overlaps. Causal-Plan-Bench source videos and derivatives are excluded from training, and we preserve RoboVQA's official episode-disjoint split (B.24). Without adaptation, Causal Planner improves on EgoPlan-Bench2 (+3.45), RoboVQA (+4.88), and Cosmos-Reason1 (+4.60). Its external mean is 57.35 (rank 4/15), close to Seed2.1 Pro's 57.83. The same backbone trained separately on external QA data yields means of 53.81/53.31/56.43, versus 57.35 for Causal Planner. These results support broader transfer.

W2. Training stability and claim scope. To assess ablation stability, we repeated each variant across three independent seeds. Paired gains (mean±sample SD):

ComparisonS1S2S3Mean±SD
Staged − one-stage SFT (Overall)+2.94+2.84+3.23+3.00±0.20
Full − SFT w/o RL (Overall)+2.98+2.70+3.80+3.16±0.57
Full − SFT w/o RL (Robustness)+6.04+5.52+7.18+6.25±0.85
Full − RL w/o SFT traces (Overall)+4.34+4.03+5.45+4.61±0.75

All four gains remain positive across seeds (Table 6; B.22).

For the full Causal Planner, the three training scores are 45.28, 45.06, and 46.32 (45.55±0.67), distinct from fixed-checkpoint evaluation variation (45.28±0.82, B.13). A paired bootstrap over 1,200 instances gives a +2.24-point advantage over GPT-6-astra (95% CI [1.30, 3.10]). Figure 4(c) also shows gains as supervision grows.

Your point about the scope of our claims is well taken. Our primary contributions are Causal-Plan-Bench and Causal-Plan-1M. Causal Planner uses next-token SFT and GRPO to test whether structured supervision improves reasoning beyond plausible action sequences. Our results support improved offline reasoning about preconditions, effects, dependencies, and hypothetical recovery. Our revised discussion emphasizes this offline scope; we also plan VLA-based closed-loop tests of manipulation, long-horizon planning, and recovery.

We sincerely appreciate your careful and constructive feedback.

Thank you for the detailed rebuttal and the additional experiments. Given that the main contributions are the benchmark and the dataset, I am raising my score to 3.

AuthorsOpenReview ↗

Thank You for Your Thoughtful Reconsideration and Constructive Feedback

Dear Reviewer W4sB,

We sincerely thank you for taking the time to carefully read our rebuttal and consider the additional experiments and analyses. We are truly grateful for your encouraging response and thoughtful reassessment.

Your constructive feedback has been invaluable in helping us strengthen our experimental evidence and more clearly articulate the central contributions of Causal-Plan-Bench and Causal-Plan-1M. We deeply appreciate your careful consideration of our responses and your willingness to revisit your evaluation.

Thank you again for your valuable feedback and support. Your feedback has greatly contributed to improving the quality and clarity of our manuscript, and we are sincerely grateful for your support and encouragement.

Reviewer_RQLrOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Causal Plan, a framework aimed at shifting embodied models from statistical sequence predictors to causally grounded reasoners. It formalizes physical planning into four dimensions: executability, effects, composition, and robustness. The authors contribute Causal-Plan-Bench (1,200 expert-verified instances across 12 tasks) and Causal-Plan-1M (a million-scale corpus of reasoning traces from egocentric videos). Using a four-stage annotation pipeline and a progressive SFT+RL training recipe on Qwen3-VL-8B, the resulting Causal Planner achieves an overall score of 45.28, outperforming frontier models like GPT-6-astra (43.04). The work is validated through extensive experiments, including cross-benchmark transfer, visual-dependence controls, cross-judge comparisons, human calibration, and ablations demonstrating a causal-supervision scaling trend.

Critical strengths
  1. Novel and Actionable Problem Formalization: The decomposition of physical planning into four distinct causal dimensions (executability, effects, composition, robustness) provides a rigorous diagnostic framework. This shifts the evaluation beyond superficial sequence plausibility and directly targets the core deficiencies in current embodied models.

  2. Strong Resource Contribution and Empirical Validation: The creation of Causal-Plan-1M and the 1,200-instance Causal-Plan-Bench is a substantial contribution. The empirical demonstration that an 8B model (Causal Planner) trained with targeted causal supervision can outperform massive frontier models (e.g., GPT-6-astra, Kimi K2.5) strongly supports the paper's central thesis.

Critical weaknesses
  1. Overclaiming "Causality" and Lack of Anti-Shortcut Controls: The benchmark predominantly evaluates state recognition and sequence prediction rather than true causal reasoning (interventions/counterfactuals). Without rigorous anti-shortcut controls, the model may succeed by exploiting statistical visual-linguistic priors (e.g., a knife implies cutting) rather than internalizing actual physical mechanisms. This undermines the claim of learning "grounded causal reasoning."

  2. Circular Evaluation and Reward Hacking Risk: Both the SFT reasoning traces and dense RL rewards are generated by GPT-5.6-sol. Without closed-loop robotic execution, the 8B model may merely learn to align with the LLM judge's stylistic biases rather than internalizing physical laws. The "causal" supervision risks degrading into sophisticated text-pattern matching optimized for a specific LLM evaluator.

  3. Zero-Shot Generalization Contradicts the "Exceptional Transferability" Claim: While the model achieves SOTA on the authors' own benchmark, Table 5 reveals it underperforms on external benchmarks. On EgoPlan-Bench2, Causal Planner (45.32) lags behind Seed2.1 Pro (49.36), Gemini 3.1 Pro (47.49), and GPT-6-astra (47.26), ranking only 4th overall. This suggests the targeted training recipe may lead to overfitting on the specific task formulation rather than yielding superior general physical intelligence.

Questions for authors
  1. How do you rule out the possibility that Causal-Plan-Bench rewards models for imitating the stylistic and structural priors of GPT-5.6-sol, rather than evaluating genuine physical causal reasoning?

  2. Given the heavy reliance on LLM judges (GPT-5.6-sol, Qwen3.5, Seed2.1 Pro), how robust are the rankings under different prompting strategies? Did proprietary baselines receive equal optimization effort (e.g., prompt retries) compared to your model?

  3. Since there is no closed-loop execution, what specific evidence or planned future experiments can you provide to bridge the gap between high benchmark scores and reliable real-world robot policies?

5 comments
AuthorsOpenReview ↗

Thank You for Your Careful Review and Constructive Feedback on Physical Reasoning, Evaluation Reliability, and Cross-Benchmark Generalization

We sincerely thank you for your careful review and for recognizing the value of our problem formulation and resources. We have conducted additional experiments and analyses to address your concerns.

Q1 / W1. Physical reasoning and shortcut controls. We designed Causal-Plan-Bench to test why actions are feasible, what they change, and how plans adapt when conditions change.

We trained a matched general-CoT control with identical QA pairs, visuals, GPT-5.6-sol teacher, initialization, filtering, and SFT/RL settings. Both retain rationales; only guidance on preconditions, state changes, and dependencies differs. Despite 5.6% more reasoning tokens, general CoT scores 42.46 versus 45.28 for Causal Planner (−2.82) and trails on all three external benchmarks (−3.58 on Cosmos-Reason1). This supports causal guidance beyond trace length.

We also limit stylistic shortcuts in scoring. Six MCQ tasks use exact matching, not LLM judging (Causal Planner 46.67%; backbone 35.83%; GPT-5.6-sol 37.83%). Distractors remain plausible but violate physical conditions (e.g., faucet outlet versus neck; Figure 5); Table 11 rejects language-prior-only items. Candidates are ranked by Qwen3.5-397B-A17B, audited by Gemini 3.7 Flash, and reviewed by two experts (2,400; 96.29% agreement, α=0.940; B.5–B.6). Open-ended tasks use expert-designed, task-specific rubrics that assess physical relations, penalize unsupported claims, and accept equivalent wording (B.18). Human ratings show ICC(A,3)=0.929, and GPT-5.6-sol correlates with their mean (900 answers, 2,700 ratings; Spearman 0.835; B.11). Causal Planner ranks first under three judges (five-model ρ 0.90–1.00; Table 4).

Beyond action order, Task 10 asks how lid removal enables washing; Task 18 tests plan repair (30.50→41.80); Task 19 predicts a blocked-pot outcome (33.40→46.50); and Task 20 tests kettle–faucet recovery (35.50→47.20; Table 15, Figures 13–16). Removing videos, keyframes, and visual-derived metadata while keeping questions and scoring fixed lowers Causal Planner Overall 45.28→22.90 and MCQ 46.67%→26% (25% chance). GPT-5.6-sol and Gemini 3.1 Pro also lose 16.92 and 18.76 points (Table 3). Together, these results argue against style-only or language-only shortcuts.

Q2 / W2. Judge-prompt robustness and baseline fairness. To test prompt robustness, we used GPT-5.6-sol to rescore the same fixed answers from five models under the original and two alternative evaluation prompts, keeping the task-specific rubrics and scoring scale unchanged:

Judge promptCausal Planner rankOpen-ended gainRank ρ vs. original
Original1/5+13.261.00
Alternative A1/5+14.581.00
Alternative B1/5+13.940.90

Causal Planner remains first across all three prompts. For fair baseline comparisons, all models, including proprietary APIs, received the same predefined per-item prompts, visual evidence, answer constraints, and temperature 0.2. None received model-specific prompt tuning, selective retries, or best-of-N selection.

We also checked whether the ranking depends on the judge. GPT-5.6-sol, Qwen3.5-9B, and Seed2.1 Pro rescored the same blinded answers from five models under shared task rubrics. Causal Planner ranks first under each, with gains of +13.26/+11.16/+13.09 over its backbone and rank Spearman 0.90–1.00 (Table 4). Human calibration offers another check: across 900 answers scored by three experts (2,700 ratings), GPT-5.6-sol correlates with the expert mean (Spearman 0.835; MAE 5.58), with an average +3.04-point offset (B.11/Table 18).

Q3. Bridging offline reasoning and robot execution. Tasks 18–20 show gains in offline plan repair, counterfactual prediction, and recovery. We will prioritize VLA experiments to test whether Causal Planner's learned causal reasoning improves robot manipulation, long-horizon planning, and closed-loop execution. We will compare Causal Planner with its Qwen3-VL-8B backbone as high-level planners using the same VLA policy. Both will use post-action observations to update their plans, allowing us to assess task success and recovery when conditions change.

W3. Cross-benchmark transfer and claim scope. We appreciate your point about the scope of transfer. Without external adaptation, Causal Planner improves over its Qwen3-VL-8B backbone on EgoPlan-Bench2 (41.87→45.32; +3.45), RoboVQA (58.55→63.43; +4.88), and Cosmos-Reason1 (58.70→63.30; +4.60). Averaging 57.35, it ranks fourth among 15 models and approaches several frontier proprietary systems. We also trained the same backbone separately on each external benchmark's QA data; these controls yield means of 53.81/53.31/56.43, versus 57.35 for Causal Planner. Together, these results support consistent zero-adaptation gains from Causal-Plan-1M supervision beyond our benchmark.

We sincerely appreciate your suggestions, which have strengthened our controls and clarified our claims.

AuthorsOpenReview ↗

Our Sincere Thanks for Your Constructive Feedback and Further Consideration of Our Responses

Dear Reviewer RQLr,

Thank you very much for your careful review and constructive suggestions. We sincerely appreciate the attention you have given to the physical reasoning capabilities, evaluation fairness, and cross-benchmark generalization of our work.

We have carefully considered your concerns and conducted additional experiments and analyses, including matched supervision controls, judge-prompt robustness evaluations, and further cross-benchmark comparisons. These results have helped us strengthen our evidence and clarify the scope of our claims. We genuinely hope that our detailed responses have addressed the key questions you raised.

When you have an opportunity, we would be very grateful for your further consideration of these additional results. We would especially appreciate hearing whether the new evidence has helped resolve your original concerns, and we would be happy to provide any further clarification.

Once again, thank you sincerely for your time, thoughtful feedback, and valuable guidance. We look forward to your thoughts and further discussion.

AuthorsOpenReview ↗

Our Heartfelt Thanks for Your Constructive Feedback and Continued Consideration of Our Additional Experiments and Clarifications

Dear Reviewer RQLr,

We sincerely thank you again for your careful review and constructive feedback. Your suggestions have been very helpful in strengthening our experimental evidence and clarifying the scope of our work.

With the discussion period nearing its end, we wanted to respectfully follow up on our rebuttal and the additional analyses addressing your concerns about physical reasoning, evaluation reliability, and cross-benchmark generalization.

We sincerely hope that the new experiments and clarifications have provided a clearer and more convincing picture of our contributions. If you have an opportunity to revisit our responses, we would greatly appreciate hearing your updated perspective, particularly whether the additional evidence has addressed your main concerns. We remain happy to discuss or clarify any outstanding questions.

Thank you once again for your valuable time, thoughtful consideration, and guidance. We truly appreciate your feedback.

Reviewer_3AYYOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Causal-Plan-Bench and Causal-Plan-1M for physically grounded reasoning. It also trains a vision-language model through staged SFT and GRPO. The model improves over its backbone on the proposed benchmark and three external benchmarks. The resource contribution is substantial. However, several claims about causal planning go beyond the evidence.

Critical strengths
  1. The dataset and benchmark reflect substantial construction and curation effort.
  2. The four evaluation dimensions cover useful aspects of physical reasoning: preconditions, effects, dependencies, and recovery.
  3. The evaluation includes no-vision controls, multiple judges, human calibration, and external benchmarks. These checks provide meaningful support for the reported gains.
Critical weaknesses

I appreciate the substantial effort behind the dataset, benchmark, and evaluation. My main concerns are about the scope of the conclusions and a few gaps in experimental rigor.

  1. The link to planning could be stronger. Several tasks focus on describing observed outcomes or explaining demonstrated steps. These are useful capabilities, but they provide limited evidence of prospective planning. Direct evaluation of future-state prediction or plan generation would strengthen the paper.
  2. The contribution of causal supervision remains difficult to separate. A matched comparison with ordinary video QA or descriptive explanations would help. The current trace-removal ablation also reduces supervision volume, so it does not fully isolate the value of causal structure.
  3. Some claims would benefit from more cautious wording. The results support improved physical reasoning QA. They provide less direct support for a transition from token prediction to causal reasoning, or for reliable physical agency. Narrowing these claims would better reflect the demonstrated contribution.
  4. A few additional details would improve confidence in the results. The small margin over the strongest baseline would benefit from paired uncertainty estimates. A complete reward specification and clearer reporting of external data decontamination would also improve reproducibility.
Questions for authors
  1. Do the authors have a matched ordinary-QA or descriptive-rationale baseline that could clarify the specific benefit of causal supervision?
  2. Could the authors include a small evaluation where future outcomes are withheld or a key physical condition is changed?
  3. How stable is the margin over the strongest baseline across evaluation runs or judge choices?
  4. Could the authors clarify the numerical reward mapping and the coverage of external data overlap checks?
7 comments
AuthorsOpenReview ↗

Thank You for Your Thoughtful Review and Valuable Suggestions on Causal Supervision, Physical Reasoning, and Evaluation Robustness

We sincerely thank you for your thoughtful review and recognition of our resources. Your questions motivated targeted experiments and helped us clarify the contribution.

Q1 / W2. Isolating causal supervision. We conducted two complementary controls to distinguish the contribution of structured causal supervision from ordinary video QA and descriptive rationales.

First, we trained a matched general-CoT control with identical filtered video–QA pairs, visuals, GPT-5.6-sol teacher, initialization, filtering, and SFT/RL settings. Unlike the trace-removal ablation, both variants retain rationale supervision; only the guidance differs. General CoT gives ordinary explanations, whereas our traces explicitly link preconditions, state changes, and inter-step dependencies. Despite containing 5.6% more reasoning tokens, general CoT scores 42.46 versus 45.28 for Causal Planner (−2.82), and performs worse on all three external benchmarks (−3.58 on Cosmos-Reason1). This comparison isolates the benefit of causal structure beyond trace presence or length.

Second, for ordinary QA training controls, we trained the same Qwen3-VL-8B backbone on each external benchmark's training data:

Training data/modelCausal-Plan-BenchEgoPlan-Bench2RoboVQACosmos-Reason1
EgoPlan-Bench232.6646.4860.3254.62
RoboVQA33.5741.0562.5256.35
Cosmos-Reason134.5843.4859.0066.82
Causal Planner45.2845.3263.4363.30

Causal Planner achieves the highest external mean (57.35 versus 53.81/53.31/56.43) and outperforms the QA-trained baselines in all six cross-benchmark comparisons. Together, these experiments provide complementary evidence: the matched general-CoT control isolates the contribution of causal structure, while the ordinary-QA controls further demonstrate the transfer benefits of our supervision.

Q2 / W1. Prospective reasoning and changed conditions. We test how models predict consequences and propose plan revisions under changed physical conditions. Task 19 includes 100 held-out cases requiring the model to predict what happens if a lid blocks sausage pieces from entering a pot; Task 20 includes 100 recovery cases, such as realigning a kettle beneath a faucet (Figures 15–16). Causal Planner improves over Qwen3-VL-8B from 33.40 to 46.50 on Task 19 and from 35.50 to 47.20 on Task 20 (Table 15; 0–100 rubrics). Task 18 further requires diagnosing and repairing a flawed plan (30.50 to 41.80). Tasks 5/6 assess spatial and functional postconditions, with MCQ accuracy rising from 32% to 42% and 37% to 55% (Table 16). Together, these evaluations test action effects, counterfactual consequences, and corrective planning.

Q3 / W4. Stability against the strongest baseline. We evaluated each fixed model three times, regenerating answers and rejudging open-ended responses (mean±SD):

MetricQwen3-VLGPT-6-astraCausal Planner
Overall33.23±1.1843.04±1.2145.28±0.82
Executability38.67±1.4246.33±1.4748.00±0.96
Effects33.00±1.3639.67±1.3145.33±0.88
Composition28.10±1.5442.00±1.5842.60±1.03
Robustness33.13±1.2744.17±1.3545.17±0.84

Two further independent training runs yielded 45.06 and 46.32, alongside the original 45.28 (45.55±0.67, mean±sample SD). To quantify the margin over GPT-6-astra, we conducted a paired bootstrap analysis across the same 1,200 benchmark instances, accounting for task balance and shared video sources. Causal Planner achieves a +2.24-point advantage (95% CI [1.30, 3.10]). Three judges also rescored fixed answers from five models; Causal Planner ranked first under each (Spearman 0.90–1.00; Table 4).

Q4 / W4. Reward mapping and overlap checks. In GRPO (Tasks 18–20), the judge rates each criterion Yes, Partially, or No, mapped to 1, 0.5, or 0, respectively. These criterion scores enter training as separate reward components, without hand-tuned weighting. Samples with missing or invalid judgments are excluded from the reward and GRPO loss. This mapping is documented in B.19, distinct from continuous benchmark scoring (B.18).

To prevent training–evaluation overlap, we screened our SFT/RL data against all four benchmark evaluation sets using QA hashes, MinHash, embedding similarity, and available source/video/trajectory IDs, removing detected overlaps. We also excluded Causal-Plan-Bench source videos and their derivatives from training and preserved RoboVQA's official episode-disjoint split (B.24).

W3. Claim scope. Your point about the scope of our claims is well taken. Our aim is to examine whether plausible action sequences also account for the physical conditions governing execution—preconditions, effects, dependencies, and recovery. Causal-Plan-Bench evaluates these through offline reasoning tasks. We have clarified this distinction when discussing world modeling and physical agency.

We appreciate your constructive guidance in strengthening the evidence and its presentation.

AuthorsOpenReview ↗

Our Sincere Thanks for Your Thoughtful Review and Continued Consideration

Dear Reviewer 3AYY,

We sincerely appreciate your thoughtful review and the valuable questions you raised. Your feedback has been particularly helpful in guiding us to strengthen the evidence for causal supervision, physical reasoning, and evaluation reliability.

In response, we have conducted additional controlled experiments, examined training and evaluation stability, and provided further clarifications in both our rebuttal and revised manuscript. We believe these additions directly address your main concerns and offer a more comprehensive picture of our contributions.

We would be truly grateful if you could take another look at our responses when you have a chance. We sincerely hope the additional evidence helps resolve your concerns, and we would greatly value your updated perspective on our work.

Thank you again for your time, careful consideration, and constructive guidance. We look forward to hearing your thoughts and continuing the discussion.

AuthorsOpenReview ↗

Our Sincere Appreciation for Your Thoughtful Review and Hope to Hear Your Further Thoughts on Our Additional Evidence

Dear Reviewer 3AYY,

Thank you again for your thoughtful review and valuable suggestions. We sincerely appreciate the time and care you have devoted to evaluating our work.

As the discussion period approaches its conclusion, we wanted to kindly follow up on our detailed rebuttal and the additional experiments, particularly the matched supervision controls, cross-benchmark comparisons, and stability analyses conducted in response to your concerns.

We genuinely hope these new results have helped clarify our contributions and address your original questions. If your schedule permits, we would be very grateful for your further thoughts on our responses and whether any concerns remain. We would be happy to provide additional clarification during the remaining discussion period.

Thank you once again for your careful consideration and constructive guidance. We truly value your feedback.

Comments on the paper

2 comments

Dear reviewers, the authors have posted their responses to your reviews. Pls have a look and discuss.

Thanks, AC for #37015

AuthorsOpenReview ↗

Our Sincere Gratitude to the Area Chair and Reviewers for Their Thoughtful Guidance and the Opportunity for Further Discussion

Dear Area Chair,

We sincerely thank you for your kind coordination and for bringing our responses to the reviewers' attention. We have carefully considered each reviewer's concerns and provided detailed point-by-point responses, supported by additional experiments, analyses, and clarifications. We believe these efforts have thoroughly addressed the questions raised and substantially strengthened our work. We would be truly grateful for any further feedback from the reviewers and look forward to continuing a constructive and meaningful discussion.

Once again, we would like to express our heartfelt appreciation to you and all the reviewers for your time, thoughtful evaluations, and invaluable suggestions. Your guidance has been instrumental in improving our manuscript, and we sincerely look forward to hearing your thoughts.

Thank you very much for your continued support and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.