- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ACTOR proposes an action-centered framework for improving literary role-playing agents by organizing character behavior into chronological Thought–Action–Observation (T-A-O) trajectories. The framework extracts character actions and contextual evidence from 143 literary works, reconstructs story chronology, maintains dynamic character profiles and memories, and generates plausible but character-inconsistent counterfactual behaviors. The resulting data are used for preference-based training with Relative Preference Optimization (RPO), combining DPO with an auxiliary negative log-likelihood objective.
The authors construct 270,856 training preference records and evaluate on 17,560 four-way next-behavior selection instances, including temporal evaluation on training works and cross-work evaluation on 28 additional works. The main contribution is an end-to-end data construction and training pipeline that emphasizes temporally grounded character actions rather than static character descriptions.
Critical strengths
-
The paper goes beyond static character profiles by organizing behavior into chronological T-A-O records, reconstructing non-linear story timelines, and maintaining dynamic profiles. The resulting dataset is substantial, and the authors provide detailed prompts, schema descriptions, and structural validation statistics. This is a useful resource and engineering contribution for research on literary role-playing agents.
-
Consistent improvements over the corresponding base models. ACTOR improves four-way next-behavior selection accuracy by approximately 8–9 percentage points for the evaluated Qwen-2.5 and Llama-3.1 backbones. The paper also reports gains in open-ended CoSER evaluations. These results suggest that the constructed supervision can improve model behavior under the authors' evaluation settings, even though the generality of the measured improvements remains uncertain.
Critical weaknesses
The main benchmark asks models to select the original behavior from four candidates: one source-text behavior and three automatically generated counterfactuals. Although the authors instruct the generator to make the alternatives plausible, character-inconsistent, and similar in length and form, this setup may still allow models to exploit artifacts of counterfactual generation or differences in how the options are phrased. High accuracy therefore does not necessarily demonstrate robust modeling of a character's motivations, long-range development, or behavior in genuinely unseen situations. Structural validation of the records does not establish the semantic validity of the counterfactuals. The open-ended evaluation provides additional evidence, but it relies on CoSER scores and LLM-based judging, and the qualitative examples still exhibit speaker-attribution and character-fidelity failures. Stronger independent validation is needed to support the central capability claim.
The optimization objective is a combination of standard DPO and an auxiliary likelihood loss. The main novelty consequently lies in the data construction and supervision design rather than a new learning algorithm. The reported gains are encouraging, but it remains unclear how much comes specifically from chronological T-A-O trajectories, counterfactual preference supervision, additional training exposure, or the auxiliary likelihood objective. In particular, the ablations indicate that removing long-term memory has little effect on Qwen models and can slightly improve Llama performance, which weakens the claim that long-range behavioral history is a central driver of the improvement. More controlled comparisons are needed to establish which components provide the substantive contribution.
Questions for authors
-
Did the authors conduct a human evaluation of the generated counterfactuals to assess both their contextual plausibility and their inconsistency with the target character? If so, please report the annotation protocol, sample size, and agreement. Can the authors also provide results under controls that reduce the possibility of exploiting option-level artifacts, such as evaluating on independently authored alternatives or using a generation process different from the one used to construct the training data?
-
How robust are the CoSER results to the choice of evaluator and judging model? Are there human evaluations or independent evaluation protocols that confirm the reported improvements, particularly in character fidelity and speaker attribution? This matters because the main multiple-choice benchmark may not fully measure open-ended role-playing ability.
-
Can the authors provide controlled comparisons that separate the effects of T-A-O trajectory supervision, counterfactual preference training, and the auxiliary NLL objective? Given that removing long-term memory has little effect on some models, what evidence supports the claim that reconstructing and preserving long-range behavioral history is an important source of the gains?
-
What is the provenance and licensing status of the 143 literary works, and what assessment was made of whether publicly releasing the derived behavior records is permitted under the applicable copyright and data-use terms?
4 comments
Weaknesses 1: The open-ended evaluation provides additional evidence, but it relies on CoSER scores and LLM-based judging, and the qualitative examples still exhibit speaker-attribution and character-fidelity failures. Stronger independent validation is needed to support the central capability claim. & Questions 2:
- How robust are the CoSER results to the choice of evaluator and judging model? Are there human evaluations or independent evaluation protocols that confirm the reported improvements, particularly in character fidelity and speaker attribution? This matters because the main multiple-choice benchmark may not fully measure open-ended role-playing ability.
To reduce scoring variability, we used a strong frontier model as the judge and averaged the results across repeated evaluation runs.
We also conducted a human evaluation by randomly sampling 10 shared scenarios and anonymously pairing each of the three ACTOR models with its corresponding backbone. Three annotators independently rated the conversations on a 1–5 scale across four dimensions: storyline consistency, anthropomorphism, character fidelity, and storyline quality. They also identified errors such as character identity confusion and speaking on behalf of other characters, and indicated their overall preferences:
| Model | Non-empty Character Turns | Error Rate | Empty Turns After Cleaning |
|---|---|---|---|
| Qwen-2.5-7B | 41/156 | 26.28% | 5 |
| ACTOR-Qwen-2.5-7B | 16/143 | 11.19% | 0 |
| Qwen2.5-14B | 18/111 | 16.22% | 3 |
| ACTOR-Qwen-2.5-14B | 7/97 | 7.22% | 0 |
| Llama-3.1-8B | 64/140 | 45.71% | 0 |
| ACTOR-Llama-3.1-8B | 18/154 | 11.69% | 0 |
Given substantial variation in annotators’ use of the rating scale, we did not report absolute human scores. Instead, we used their judgments to qualitatively assess whether ACTOR models generally outperformed their backbones and included illustrative case analyses in the appendix.
Weaknesses 2: The optimization objective is a combination of standard DPO and an auxiliary likelihood loss. The main novelty consequently lies in the data construction and supervision design rather than a new learning algorithm. The reported gains are encouraging, but it remains unclear how much comes specifically from chronological T-A-O trajectories, counterfactual preference supervision, additional training exposure, or the auxiliary likelihood objective. In particular, the ablations indicate that removing long-term memory has little effect on Qwen models and can slightly improve Llama performance, which weakens the claim that long-range behavioral history is a central driver of the improvement. More controlled comparisons are needed to establish which components provide the substantive contribution. & Questions 3:
- Questions 3.1: Can the authors provide controlled comparisons that separate the effects of T-A-O trajectory supervision, counterfactual preference training, and the auxiliary NLL objective?
- Questions 3.2: Given that removing long-term memory has little effect on some models, what evidence supports the claim that reconstructing and preserving long-range behavioral history is an important source of the gains?
3.1:
We first explored SFT by fine-tuning Qwen2.5-7B-Instruct solely on the ground-truth behavior text. The results on the anonymous evaluation set are shown below:
| Model | Selection accuracy | Proportion of A predictions |
|---|---|---|
| Qwen2.5-7B-Instruct | 71.34% | 27.57% |
| SFT step500 | 70.31% | 27.93% |
| SFT step1000 | 63.93% | 39.65% |
| SFT step1500 | 58.25% | 47.23% |
SFT did not improve behavior-selection accuracy under this configuration. A possible explanation is that SFT encourages the generation of correct behavior without explicitly distinguishing it from incorrect alternatives in relation to the character context. Continued fine-tuning may also interfere with the generation behavior inherited from the backbone.
DPO and RPO instead introduce comparative supervision through counterfactual negative examples. Both achieved substantially higher behavior-selection accuracy than the backbone:
| Model | Selection accuracy |
|---|---|
| Backbone | 71.34% |
| Qwen2.5-7B-DPO | 87.34% |
| Qwen2.5-7B-RPO | 80.20% |
Although pure DPO achieved higher selection accuracy, its performance under the CoSER evaluation framework was considerably lower than that of RPO. Some DPO simulations also exhibited repetitive generation and outputs that reached the token limit. One possible explanation is that DPO optimizes the relative preference between chosen and rejected responses without directly maximizing the likelihood of the chosen response. Improved preference discrimination therefore does not necessarily translate into better open-ended generation.
| Model | CoSER mean score |
|---|---|
| Backbone | 10.43 |
| Qwen2.5-7B-DPO | 10.7375 |
| Qwen2.5-7B-RPO | 21.99 |
Considering both behavior selection and open-ended interaction, we chose RPO as our final training objective. Its auxiliary NLL term directly supervises the generation of correct behavior while retaining counterfactual preference learning.
3.2:
As discussed in the ablation section of our paper, dynamic profiles summarize a character’s past experiences and personality and may partially compensate for the absence of long-term memory. To further examine this explanation, we conducted an additional evaluation using static profiles. We selected a subset of questions for which the characters’ initial profiles could be recovered and used these initial versions as the static profiles. The results are shown below:
| Model | All memory | Without long-term memory | Change (percentage points) |
|---|---|---|---|
| ACTOR-Qwen-2.5-7B | 80.31% | 80.13% | −0.18 |
| ACTOR-Qwen-2.5-14B | 88.37% | 87.09% | −1.28 |
| ACTOR-Llama-3.1-8B | 77.57% | 76.71% | −0.86 |
With static profiles, removing long-term memory reduced accuracy across all three models.
Questions 4:
- What is the provenance and licensing status of the 143 literary works, and what assessment was made of whether publicly releasing the derived behavior records is permitted under the applicable copyright and data-use terms?
All the book texts we use are sourced from publicly available versions. We do not publish the original texts, and we will reconfirm copyright permissions before releasing the constructed dataset.
We hope that our explanations above can clarify your doubts and you can consider our work more favorably.
We thank the reviewer for the thoughtful feedback. We address the comments below.
1.1:
We selected 400 evaluation questions through stratified random sampling. Two annotators independently reviewed all questions, with a third adjudicating disagreements. The assessment covered four dimensions: source fidelity, checking whether the original behavior matched the source text; contextual validity, checking consistency between the character context and the current scene, as well as potential future-information leakage; counterfactual quality, assessing the contextual plausibility of each alternative; and answer discriminability, checking whether the provided character information clearly supported the original behavior more strongly than the other options.
The final results are as follows:
1.2:
We added an options-only evaluation to assess model performance based solely on the answer options. The results are shown below:
Models may learn differences between the options during training. However, the gains observed when sufficient character information is provided are substantially larger than those obtained from the options alone.