- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
In this paper, the authors introduced SOLPA, a method for training video-based proactive assistants that can select parallel helpful actions without waiting for explicit human requests. The authors used task-graph supervision during training while making graph input optional at inference time. The method introduces two training techniques: completion-conditioned QA, which creates counterfactual supervision by hypothetically marking steps as complete to encourage action reselection, and graph-context dropout, which randomly omits graph text during training to reduce inference-time dependence on structured input like graphs. The system tracks perception (identifying task state from video) and planning (selecting assistance actions) separately, and is trained via LoRA-based supervised finetuning on the ProAct-75 benchmark. The 8B configuration achieves 33.37% step-success and 24.10% parallel-action rate, outperforming the same-backbone Direct baseline by 10 and 2.5 percentage points, respectively.
Critical strengths
- The task is well established and of good interest.
- Whatever experiments were done were done in a mostly sound way.
- The completion-conditioned QA and graph-context dropout insights are interesting.
Critical weaknesses
-
The writing could be significantly improved, especially the introduction and motivation sections. The current introduction jumps into explaining what the current methods do and what the authors are proposing. However, the exact motivation and research questions could have been explained more explicitly and clearly. Currently, the introduction reads more like: we have a task and a dataset and here's what we do to improve the performance.
-
One of the paper's main pitch is removing graph dependency at inference, but the key advantage of inference-time graphs was domain adaptability without retraining. But I didn't find any analysis on the generalizability of the proposed approach. No cross-domain or structurally diverse graph experiments are done.
-
The training approach seems pretty much SFT-only; no RL-based mechanism was explored for the model to learn from planning mistakes. For a task requiring structural reasoning about parallel workflows, SFT likely teaches pattern matching over labels rather than genuine task-graph reasoning.
-
The authors studied only a single dataset (ProAct-75). No evidence of transfer to other proactive assistance settings, real-world deployment, or benchmarks with different task structures. This significantly limits the scope of the method and contribution.
-
The analysis is insufficient across model scales. The 8B model gets the headline results, but there's thin analysis of why different backbones (3B, 7B, 8B) respond differently to the training. Base model capabilities before finetuning are under-explored as they could have been explored with more test-time approaches.
-
[Minor] The related work section on structured planning is underexplored. The authors cited mostly earlier work but more recent studies on constrained generation and structured planning with LLMs are missing. A more thorough survey of this space would better contextualize the proposed approach.
Questions for authors
Please refer to the weaknesses section.