- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies how to handle player actions that violate world rules in LLM-driven interactive stories, and proposes StoryHARN, which checks each action against the rules before it takes effect and, on a violation, rewrites, defers, or branches it. On 12 Chinese story worlds built by the authors, the method violates rules significantly more often than simply rejecting violating actions (67.5% vs. 90.8% state validity), and the two perform on par after the authors fix problems in their own rules.
Critical strengths
None. The authors honestly report their negative results and the problems in their own implementation, but this does not substitute for a contribution.
Critical weaknesses
- The method underperforms the simplest baseline. Directly rejecting violating actions reaches 90.8% state validity, while StoryHARN reaches only 67.5%; after the rules are fixed, the two perform on par. The branching and deferral mechanisms that the paper emphasizes are almost never triggered in the experiments.
- The method lacks novelty. In Section 2 the authors themselves acknowledge that checking and replacing a violating action before it takes effect "is not itself a new contribution."
- The comparison is unfair. The methods give the model different prompts, so the differences in results cannot be attributed to the methods themselves.
Questions for authors
- If all methods used the same prompt, would the gap between StoryHARN and direct rejection still exist?
- Is there any example from natural runs where branching or deferral was triggered and led to a better outcome than direct rejection? If not, where does the value of these two mechanisms lie?