- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes LAmp, which identifies unresolved assumptions, enumerates their interpretations, and assesses whether different interpretations change task outcomes to decide what to clarify. The framework is evaluated on BIRD and SWE-bench, with an additional study on distilling the clarification policy into a 7B model.
Critical strengths
Deciding whether to clarify based on task outcomes is practically relevant. The paper evaluates downstream task success in addition to assumption recovery, and the improvements on the main BIRD cohort suggest that the approach has potential.
Critical weaknesses
-
The contribution and novelty are limited. LAmp mainly organizes assumption identification, interpretation enumeration, and clarification decisions into a simple framework, offering an incremental methodological advance.
-
The evaluation lacks simple yet effective baselines, such as a general-purpose agent equipped with clarification skills, which might achieve similar results. Evidence for reducing unnecessary questions and generalizing across tasks also remains limited.
Questions for authors
The proposed steps of identifying assumptions, enumerating interpretations, and deciding what to clarify appear straightforward to standardize as a reusable workflow. Could similar results be achieved simply by using agentic skills? Please clarify what additional benefit LAmp provides over such an implementation.