- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
AutoValueWorld proposes a framework for choosing assistant responses based on their consequences for later decisions. It separates a user’s fixed priorities over ten human values from a changing state describing how the interaction supports or violates those values. The authors jointly train Qwen2.5-Instruct 3B and 7B models using a hand-designed simulator to supervise value-state transitions and simulated reactions, and PRISM human ratings to supervise immediate-rating prediction. Their main training contribution is a candidate-difference loss that explicitly teaches the model how alternative responses differ in their value consequences, reducing relative-state prediction error by 13.1–13.8%. They separately train a small tree-based selector to choose between immediate response selection and three-step planning using predicted consequences; across 5,000 synthetic episodes, selective planning produces small return improvements for two model–objective combinations. They also evaluate rating prediction on held-out PRISM participants and test transfer to HelpSteer2 human helpfulness labels.
Critical strengths
- The paper provides a new angle to candidate response selection in multi-turn conversations. It separates stable user value priorities from the interaction’s changing support or violation of those values. Their trained predictor could predict value-support state in longer horizon and provides the planner a signal to choose potentially based on long term value rather than immediate.
- The paper introduces a new candidate-difference loss and the control run comparing candidate-difference weights demonstrates that it is useful for consequence prediction.
Critical weaknesses
- The method's empirical performance improvement seems small and inconsistent. Based on the results, selective planning improves return for only two of four model–objective combinations and the gain is small. Therefore, this experiment results show that the method helps with consequence prediction (improve quite a bit) but only translate little to selection regret. This reads as an evidence that this selective planning is not necessary.
- The advantage over simpler baselines is not established. Reading from Table 8 , 9 and 10, structured ridge models seem to perform strongly on some dynamics and planning measures even outperforming AutoValueWorld. The paper also does not compare against preference or reward training that already uses multi-turn outcomes, leaving unclear when explicit value-state prediction is necessary.
Questions for authors
- See weakness 1. How do authors interpret that the method improves on consequence prediction but not selection itself.
- See weakness 2. If the simpler method already outperforms language-based predictor on several simulator metrics, what is the benefit of using this more complex method?