- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces EgoProactive, a dataset with scripted task deviations and recovery guidance, and Pro²Bench, which combines it with re-annotated public datasets. It also proposes PWR, where an interaction model decides whether to stay silent or give guidance. After an intervention, a separate planner updates the completed, current, and remaining steps. This lets the plan change when execution differs from the original instructions.
Critical strengths
-
Recording mistakes together with recovery and intervention guidance addresses an important problem. Combining multiple sources gives the benchmark broader coverage. I see the data as the main contribution.
-
The method improves across several backbones. The same-data, no-plan comparisons help show the benefit of keeping an explicit task plan.
Critical weaknesses
-
Recovery is tested only on the authors' scripted dataset. Testing on other benchmarks like Ego-MC-Bench would help show whether the gains extend to independently collected mistakes and corrective guidance. It would also be useful to discuss which real mistakes the scripts do not cover.
-
The method is fairly incremental. Updating a plan as execution progresses is a natural extension of using a fixed plan. This concern does not outweigh the value of the data for me.
-
The evaluation uses recorded actions and reference dialogue, so it does not directly measure successful task completion under the model's own guidance.
Questions for authors
-
Can you test the trained system on Ego-MC-Bench?
-
How were the scripted mistakes selected, and how well do they cover mistakes that users make naturally?
-
With the trained interaction model fixed, how does performance change when the planner is used without fine-tuning? This would help clarify what planner training adds.
-
Since no OOP sessions are used in training, how does the planner learn to generate recovery steps? Does this mainly rely on the pretrained model, or do the training annotations contain corrective guidance even without OOP labels?
-
Should a task completion rate or success rate be reported?