- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CurioPO measures feedback productivity with restricted next-action belief shift and context-adjusted observation surprise. These signals guide sandboxed candidate resampling, trajectory restarts, and intermediate-turn advantage weighting. Experiments on WebShop, ALFWorld, and search QA report task performance, stability diagnostics, counterfactual observation masking, matched-call controls, and compute costs.
Critical strengths
- The distinction between valid interaction and useful feedback is valuable. Observation masking, usefulness labels, and matched-environment-call controls offer evidence beyond the method's own spurious-turn classification.
- The task gains are accompanied by component ablations and training-stability diagnostics across two backbone sizes. The appendix also reports the extra compute, allowing a realistic reading of the exploration-efficiency claim.
Critical weaknesses
- The belief-shift proxy uses only first tokens of admissible actions. It can miss useful confirmation, distinctions later in an action string, and delayed bridge information. The reported Heat regression and delayed-usefulness analysis show that this is an actual limitation of the default intervention, not only a hypothetical edge case.
- Fewer committed turns do not mean lower total training cost. The reported WebShop comparison increases GPU-hours from 38.2 to 49.5 and wall-clock time from 9.7 to 12.6 hours. Matched-call controls are helpful, but a matched-total-compute learning curve is still needed to assess the efficiency benefit.
Questions for authors
- How does the default method compare with a confirmation-aware or richer action representation under matched computation?
- At equal GPU-hours or wall-clock budget, how do success and collapse frequency compare with T2PO and GiGPO?