- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TWILL addresses proactive household-robot decision making when an autonomous, scene-driven proposal must be reconciled with a human request. A scene-only proposer generates potential tasks, SHUTTLE separately reads scene context and both intent sources, and a frozen GR00T N1.6 executor acts in a closed loop. TWINE evaluates proposals, 360 decisions in 180 controlled pairs, and simulated execution; reported complete-decision accuracy is 71.85% for SHUTTLE versus 68.15% for Direct, with 81.48% request-free closed-loop success.
Critical strengths
- Dual-intent arbitration addresses an important problem in proactive robot assistance. Separating robot-generated proposals from human requests captures conflicts missed by instruction-only systems. Examples involving conditional requests, unsafe instructions, and additional goals make the formulation practically meaningful.
- TWILL provides a coherent perception-to-action design. A scene-only proposer, SHUTTLE decision module, and frozen GR00T N1.6 executor create a clear division of labor. Closed-loop replanning allows decisions to respond to new observations or requests.
- TWINE gives a useful diagnostic benchmark for dual-intent decisions. Its 180 controlled pairs change one consequential factor, distinguishing relevant reasoning from superficial instruction matching. Separate understanding probes and physical-obligation checks support focused error analysis.
- Experimental controls are stronger than simple backbone comparisons. Matched backbones, LoRA settings, data, and parameter-matched fusion baselines help isolate SHUTTLE. Three seeds, bootstrap confidence intervals, and human checks of automatic scoring improve credibility.
- SHUTTLE shows measurable gains in decision quality. Complete-decision pass increases from 68.15% for Direct to 71.85%, with a reported 95% CI of [+1.76, +5.65] points. Correctness on both members of a controlled pair also improves from 47.96% to 53.33%.
- Proposer shows strong restraint in scenes needing no action. Restraint improves from 23.16% to 96.84%, while false proposals fall from 88 to 3 relative to the untuned model. Avoiding unnecessary interventions is especially important for proactive agents.
Critical weaknesses
- Novelty of source-specific attention remains somewhat incremental. SHUTTLE adapts established multi-source attention and residual fusion, while its 1.94-point advantage over merged-text reads has Holm-adjusted p = 0.0604. Evidence for a distinct benefit from separate attention normalization is therefore less conclusive than the Direct comparison suggests.
- Closed-loop gains attributable to SHUTTLE cover only a narrow set of situations. Request-free success is identical to Direct (81.48%) because SHUTTLE is bypassed, and matched-request success is lower (88.15% versus 96.30%). Improvement on absent-object requests (42.22% versus 21.48%) is promising but does not yet demonstrate broad dual-intent advantages.
- Proposer experiments do not isolate the pairwise ranking objective. Reported joint accuracy of 89.88% is compared with an untuned VLM, rather than an equally fine-tuned model without the ranking term. Consequently, benefits beyond task-specific supervised adaptation remain unclear.
- Task coverage and physical generalization are limited. Proposals involve six predefined on/off or closure problems, and closed-loop tests use only three fixtures in simulated RoboCasa365 kitchens. Such a narrow action space restricts evidence for open-ended proactive behavior and real-world robustness.
- External closed-loop baselines do not fully match task-specific training. TWILL uses a supervised scene-state proposer, whereas the VLM commander is based on an untuned Qwen3-VL model. A task-adapted commander under a comparable training budget would better isolate the contribution of system decomposition.
Questions for authors
- Can authors report SHUTTLE-versus-Direct closed-loop results for conditional, unsafe, and added-goal requests, beyond the absent-object setting?
- How much does the pairwise ranking term improve the proposer over matched supervised fine-tuning without that term?
- What explains SHUTTLE’s lower matched-request success (88.15% versus 96.30%) despite higher offline decision accuracy?
- Would a task-adapted VLM commander narrow the closed-loop gap, and do the findings extend to unseen fixture types or physical robots?
3 comments
Thanks for the detailed response and additional experiments. The new comparisons, ablations, and generalization results address most of my concerns and strengthen the paper. Real-world validation remains an open limitation. I will maintain my rating for accept support.
Thank You for Your Constructive Feedback and Support
We sincerely thank you for the thoughtful feedback and support for our work. Your suggestions directly helped us strengthen the comparisons, ablations, and generalization analyses in the revision. We also agree on the importance of real-world validation, and evaluating TWILL on physical robots will be a key focus of our future work.
Response to Reviewer 7Myw (7-Seed Significance, Ranking Loss, Commander & Closed-Loop)
We thank Reviewer 7Myw for the supportive and constructive comments.
Response to W1: Source-Specific Attention & 7-Seed Significance vs. Merged-text
To rule out seed noise in the original 3-seed comparison against Merged-text (Holm p=0.0604, Table 12), we retrained all four variants across 7 seeds with a fixed final step and the Qwen3.8-Flash judge (Table 3, App. C.4):
In this 7-seed evaluation:
Response to W3 & Q2: Isolating the Pairwise Ranking Term
To isolate the ranking term, we compared λ=0 and λ=0.1 across 3 seeds on the same paired images (App. C.6, Table 28):
Cross-entropy already supervises positive detection; contrasting on/off states at λ=0.1 raises restraint on every seed and cuts false proposals by 34% while preserving joint accuracy.
Response to W2, Q1 & Q3: Closed-Loop Arbitration & Matched-Request Analysis
Response to W4, W5 & Q4: Equal-Budget Tuned Commander & Scope