- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Counterfactual Contract Learning (CCL), a framework for training language models to generate local recovery programs for embodied multi-agent collaboration. The authors construct PARTNR-FS, a benchmark of execution failure states with verified recovery programs. CCL first initializes a repair model through supervised fine-tuning, then generates direct and alternative repair programs from the same failure state. Successful alternatives are identified using simulator replay and fixed recovery contracts. The learning objective emphasizes the first divergent action between unsuccessful and successful programs, while retaining weaker supervision on subsequent actions and a separate preservation objective.
Experiments evaluate four language model backbones, several repair baselines, and integration with cooperative controllers on TDW-MAT.
Critical strengths
-
Meaningful recovery evaluation.The paper distinguishes executable actions from repairs that actually restore the affected task goal while preserving previously completed work. The contract-based replay protocol provides a useful way to evaluate recovery outcomes beyond action validity.
-
Reasonably broad empirical evaluation.The authors evaluate multiple language model backbones, include relevant learning-based repair baselines, and demonstrate integration with two cooperative controllers. The appendix also provides additional supervision-scope controls that help examine the proposed training mechanism.
Critical weaknesses
- Limited methodological novelty and insufficient evidence for the proposed credit-assignment mechanism.
The overall approach primarily combines established components: supervised fine-tuning, sampling alternative programs, simulator-based success verification, and token-level loss reweighting. The main methodological distinction is emphasizing the first divergent action between unsuccessful and successful repairs. However, this divergence is a syntactic alignment heuristic rather than an identified causal contributor to recovery.
More importantly, Appendix I.1 shows that ordinary full-response cross-entropy training on verified alternatives already achieves 43.90% strict recovery, compared with 47.54% for CCL. Supervising all modified actions achieves 46.27%. Thus, while verified repair training is useful, the incremental benefit attributable to the proposed localization strategy is relatively modest. The paper does not provide sufficiently strong statistical evidence establishing the robustness of these small differences.
Consequently, I am not convinced that the proposed training objective represents a sufficiently substantial methodological advance over existing verification-guided imitation or repair learning.
- Limited evidence for robustness and generalization beyond the constructed recovery setting.
PARTNR-FS contains 183 test failure states, selected from cases with verified recovery programs within a five-skill-call limit. The main improvements are uneven across failure categories, with relatively small gains on reachability failures and larger percentage gains on categories containing only 20 and 9 test states.
In addition, the primary backbone experiments do not establish training-seed robustness across all four models. The complete-task TDW-MAT evaluation uses only 24 episodes, and the CCL repair model is separately updated using TDW-MAT training data. Therefore, this evaluation demonstrates adaptation to a second environment rather than zero-shot cross-environment transfer.
The improvements are encouraging, but the current evidence is insufficient to establish that the proposed mechanism generalizes reliably to more diverse execution failures or that its incremental gains persist across independent training runs.
Questions for authors
Q1. Can the authors provide paired, episode-aware statistical comparisons between CCL, full-response CE, and the modified-action supervision variant in Appendix I.1? In particular, how stable is the improvement specifically attributable to first-divergence supervision across independent training seeds?
Q2. Can the authors clarify the practical advantage of training a specialized repair model over inference-time repair using stronger general-purpose planners? Although larger proprietary models are included as direct-generation references, the experiments do not establish a matched online-cost comparison against stronger planners using additional feedback or refinement. How does the offline collection and verification cost affect this tradeoff?
Q3. Can the authors provide uncertainty estimates or paired episode-level results for the TDW-MAT improvements, especially the 2.50-percentage-point gain over PCE with SFT? This would help establish whether the complete-task advantage is robust.
7 comments
Rebuttal by Authors
Question 1: Can the authors provide paired, episode-aware statistical comparisons between CCL, full-response CE, and the modified-action supervision variant in Appendix I.1? In particular, how stable is the improvement specifically attributable to first-divergence supervision across independent training seeds?
Response: Thank you for this suggestion. We provide additional comparisons using the same Qwen3-8B Repair-SFT initialization and training seeds 181, 193, and 211. All variants use the same update budget and unconstrained evaluation protocol.
| Training seed | Full-response CE | Modified-actions | CCL |
|---|---|---|---|
| 181 | 83/183 (45.36%) | 87/183 (47.54%) | 87/183 (47.54%) |
| 193 | 79/183 (43.17%) | 82/183 (44.81%) | 86/183 (46.99%) |
| 211 | 79/183 (43.17%) | 85/183 (46.45%) | 88/183 (48.09%) |
The following intervals use 5,000 episode-cluster bootstrap resamples. We resample the source episodes with replacement and keep all states from each sampled episode together, recomputing the recovery-rate difference over the resampled states.
| Training seed | CCL − Full-response CE, 95% CI (pp) | CCL − Modified-action, 95% CI (pp) |
|---|---|---|
| 181 | +2.19 [+0.83, +5.59] | 0.00 [−2.99, +3.14] |
| 193 | +3.83 [+1.09, +6.56] | +2.19 [+0.83, +5.59] |
| 211 | +4.92 [+2.19, +8.20] | +1.64 [−1.72, +5.14] |
CCL improves over full-response CE in all three runs, with paired 95% bootstrap confidence intervals excluding zero for each seed. Compared with modified-action supervision, CCL has higher point estimates for most seeds.
We have added the per-seed results and training-seed variability to Section 4.4 and Table 4. Section 4.1 and Appendix H.1 describe the episode-cluster bootstrap procedure, and Appendix I reports the paired intervals and supports the distinction between the two comparisons.
Question 2: Can the authors clarify the practical advantage of training a specialized repair model over inference-time repair using stronger general-purpose planners? How does the offline collection and verification cost affect this tradeoff?
Response: Thank you for raising this practical comparison. The intended advantage is to move recovery search and verification offline, allowing deployment through one local model call. To test whether a stronger planner can instead recover through additional inference-time computation, we evaluated GPT-5.6 Sol with both self-refinement and actual execution feedback on Test183.
| Method | Recovery (%) | Mean calls per repair request |
|---|---|---|
| GPT-5.6 Sol | 34.97 | 1.00 |
| GPT-5.6 Sol + self-refinement | 44.26 | 3.00 |
| GPT-5.6 Sol + execution feedback | 41.53 | 1.89 |
| Qwen3-8B + CCL | 47.54 | 1.00 |
| Qwen3.5-9B + CCL | 53.55 | 1.00 |
Self-refinement uses three calls: an initial program, a critique, and a revised program. The execution-feedback variant receives feedback from its executed actions and can make up to three decisions within five cumulative skill calls. Both reported scores require success under all three replay seeds.
These results show that the specialized repair model remains competitive with a stronger planner given additional refinement or execution feedback, while requiring fewer online model calls.
The offline cost is a genuine part of the tradeoff. Accordingly, our practical claim is lower online interaction overhead after training, rather than universally lower total cost. Whether training is economical depends on how often the repair model is subsequently used.
We have added the stronger-planner comparison to Section 4.3 and Table 3, with the inference protocols specified in Appendix H.2.
Question 3: Can the authors provide uncertainty estimates or paired episode-level results for the TDW-MAT improvements, especially the 2.50-percentage-point gain over PCE with SFT?
Response: Thank you for this suggestion. We now supplement the aggregate transport rates with paired results on the same 24 test episodes. For PCE, we compare Repair-SFT and CCL under the same controller protocol.
| Metric | PCE + Repair-SFT | PCE + CCL |
|---|---|---|
| Mean transport rate | 210/240 (87.50%) | 216/240 (90.00%) |
| Fully successful episodes | 11/24 (45.83%) | 13/24 (54.17%) |
CCL increases the mean transport rate from 87.50% to 90.00% compared with Repair-SFT. It also increases the number of fully successful episodes from 11/24 to 13/24, indicating an observed improvement on the same 24 test episodes.
We have added the paired episode-bootstrap 95% interval of [+0.83, +4.58] percentage points to Section 4.5. Appendix H.3 and Table H.2 provide the bootstrap procedure and all 24 paired episode outcomes.
Response to the Authors' Rebuttal
Thank you for the detailed rebuttal and the additional experiments. I appreciate the authors' efforts to address my concerns, particularly the new paired statistical comparisons, multi-seed evaluations, stronger-planner baselines, and end-to-end results on TDW-MAT.
These additions substantially strengthen the empirical evaluation, and several of my original concerns have been meaningfully addressed.
1. Credit assignment and methodological contribution
The new three-seed comparisons provide useful evidence that CCL consistently outperforms full-response cross-entropy supervision. In particular, the paired bootstrap confidence intervals for CCL versus full-response CE exclude zero across all three training seeds. I acknowledge that this supports the effectiveness of the proposed supervision strategy relative to conventional full-response training.
However, my main concern regarding the independent contribution of first-divergence supervision remains.
Compared with modified-action supervision, the improvements across the three training seeds are 0.00, 2.19, and 1.64 percentage points. Two of the three paired confidence intervals include zero.
Similarly, the new random modified-action control achieves competitive performance. Although CCL obtains higher point estimates, the reported confidence intervals for the differences do not strictly exclude zero.
These results suggest that emphasizing modified actions within verified counterfactual repair pairs is useful, but they do not yet establish that selecting the first divergent action provides a consistently meaningful advantage over more general modified-action weighting strategies.
I appreciate the authors' clarification that the first divergence is intended as a practical supervision anchor rather than a complete causal explanation of recovery. This is a reasonable interpretation, but it also narrows the methodological claim.
2. Robustness, practical utility, and generalization
The additional training-seed results across four backbones substantially address my concern about training stability.
I also find the new GPT-5.6 Sol comparisons valuable. They provide evidence that a specialized repair model can achieve competitive or better recovery performance with fewer online model calls under the evaluated protocols. The authors' distinction between reduced online overhead and total amortized training cost is appropriate.
The additional TDW-MAT paired results and bootstrap confidence interval further strengthen the evidence for end-to-end utility in a second environment.
I appreciate the clarification that TDW-MAT involves environment-specific training rather than zero-shot transfer. The corresponding results should therefore be interpreted as evidence of applicability to another environment, not direct cross-environment generalization.
Overall, I consider these concerns substantially addressed within the stated evaluation scope.
3. Remaining assessment
My remaining reservation is primarily methodological rather than experimental.
The rebuttal provides stronger evidence that the overall CCL pipeline is useful: verified counterfactual repair pairs, contract-preserving replay, and targeted supervision together improve local recovery.
However, the results still do not convincingly isolate a substantial, consistent advantage specifically attributable to first-divergence credit assignment over generic modified-action supervision.
The distinction matters because the latter already captures much of the observed improvement, while the additional gain from first-divergence weighting remains relatively small and statistically uncertain across training seeds.
I therefore acknowledge the significant improvements made in the rebuttal, but maintain my overall rating of 2 (Weak Rejection).
My recommendation is based on the remaining uncertainty about the independent methodological novelty, rather than a lack of evidence for practical effectiveness or training stability.
Thank you again for the careful and constructive response.
Rebuttal by Authors
We thank the reviewer for the valuable feedback. We have prepared a point-by-point response to the comments below. All modifications in the manuscript have been highlighted in blue.
Response:
The contribution of CCL is not simply to train on successful trajectories. We pair a failed repair with a verified successful alternative from the same saved state, and use their difference to supervise recovery decisions. Verification requires completing the local goal while preserving previously completed work, rather than merely making the failed action executable.
The motivation for emphasizing the first divergence is straightforward. Before that action, the two programs share the same prefix. This is the first decision at which the successful alternative differs under the same preceding context. Analogous to autoregressive token generation, changing this decision can change subsequent predictions and execution states. We therefore emphasize this action without discarding the remaining corrective steps: the completion term supervises them with weight 0.25. We do not assume that the first divergent action alone explains the entire recovery.
The additional supervision controls examine this design:
The random-position result indicates that the first divergence provides a simple, reproducible anchor for emphasizing a corrective decision while retaining supervision on the rest of the verified repair.
We have clarified the supervision rationale in Section 3.5 and added the matched controls to Section 4.4 and Table 4. Appendix I.1 provides the full supervision comparisons and the additional random modified-action control, making the evidence for this design explicit.
Response: Thank you for identifying these distinctions.
Benchmark scope and category sizes. PARTNR-FS evaluates bounded local recovery within the five skill-call budget. This makes the evaluation target feasible and comparable across methods. However, it does not imply coverage of every possible execution failure. The category imbalance reflects the failure frequencies observed during our Train_2k collection process, rather than a deliberately balanced allocation. We agree that small categories require count-level interpretation. More importantly, its overall improvement also includes reachability (47 to 53/109), holding (23 to 25/45), and unsupported actions (10 to 15/20). Table 1 reports the category-wise recovery rates and subset sizes.
Training-seed variability. Our Qwen3-8B CCL runs use the same protocol with three independent training seeds. They achieve 87, 86, and 88 successes out of 183, respectively, with a mean strict recovery of 47.54%. The table below additionally reports three CCL training-seed runs for each of the other three backbones.
Generalization to TDW-MAT. This experiment evaluates adaptation to a second environment, not zero-shot transfer. We train the repair modules on TDW-MAT training data and evaluate complete-task performance on its 24 test episodes within CoELA and PCE. This tests whether the recovery-training approach remains useful in another environment and with different controllers.
We demonstrate that our method is not only useful for offline repair but also enhances performance during the execution of complete tasks.
We have added the analysis excluding Relation to Section 4.2 and the cross-backbone results to Appendix I.5. Section 4.5 now explicitly describes training on TDW-MAT data, and Table 5 reports complete-task performance. Appendix H.3 details the training data and controller integration.