acceptodds
Under review as a conference paper at ICLR 2027

Scaling Long-Form Story Generation via Narrative State Tracking

15 views

Abstract

LLMs have demonstrated strong capabilities in creative writing. However, scaling them to full-length novels remains challenging, as maintaining narrative consistency becomes increasingly difficult. Existing story-generation methods typically focus on stories of up to about ten thousand words, leaving their ability to scale to full-length novels underexplored. In this work, we introduce Narrative State Tracking Agent (), a training-free agentic framework that allows LLMs to track a structured narrative state including characters, past events and future requirements. We extend an existing benchmark to compare narrative consistency across lengths, and use it together with a writing-quality benchmark to systematically evaluate stories ranging from 10K to 100K words. We show that achieves better narrative consistency and writing quality as stories grow longer, and neither of them degrades noticeably as length increases, suggesting that it provides an effective approach to scaling story generation toward full-length novels.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
long context, long-form generation, narrative consistency, LLM agents

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_BvZTOpenReview ↗
Rating
4(+2)
Confidence
4
Informativeness
2(+1)
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper shows how to scale story up to 100K length while maintaining consistency and writing quality. It also suggest NSTAgent, a training free framework that keeping the character information, past events, and unresolved narrative requirements. The NSTAgent dynamically adapt to the current story, so the content inside NSTAgent changes during the story. This framework also allows the writer to retrieve and revise earlier passages. The paper also propose ConStory-Bench to evaluate contradictions within story, using the preceding story as evidence.

Critical strengths
  1. The paper shows how to evaluate long-form stories for contradictions, even at substantial lengths.

  2. Tracking unresolved narrative requirements is the paper’s greatest strength, and this approach could be widely adopted in other works.

  3. The paper is very well organized and easy to understand. I could easily follow its central contributions, including the generation process and evaluation metrics. The experimental results also show clear improvements over the baselines.

  4. The appendix shows that NSTAgent’s performance can be improved through GRPO.

  5. The paper provides an online link to the generated stories.

Critical weaknesses
  1. I understand why the authors choose to check for contradictions only in the final 10K tokens, but is this the best approach? What if the distribution of contradictions differs from that of the full text? The paper does not compare contradictions in the full text with those in the final 10K tokens. (I later found that Appendix B.2 addresses this issue, but it examines only one type of error: “abandoned plot elements.”)

  2. The paper does not include a human evaluation. Story generation should be evaluated by humans after all, since automatic metrics are still unreliable for assessing a story’s creativity, quality, and engagement. The authors should conduct a human evaluation, at least on a subset of shorter samples.

  3. Related to the second weakness, the paper relies heavily on LLM-as-a-judge evaluation, which is vulnerable to several biases. This raises serious concerns about the reliability of the experimental results.

  4. The paper states that the main experimental results do not align with those obtained using a smaller judge in Appendix B.1. However, it does not compare the results with other strong judges, such as GPT-5.6 or Claude, or provide further analysis of this discrepancy.

Questions for authors
  1. I wonder whether the experimental results would hold when using different LLM judges. I strongly recommend evaluating the method with at least two additional LLMs.

  2. I also wonder how human readers perceive the generated stories. I expect that stories produced using your method may be more engaging and interesting to readers. Even if this is not the case, analyzing the reasons would still be valuable.

  3. I recommend comparing the method with other story-generation baselines on stories of up to 10K tokens, such as DOC or following works building on it.

If you can show that your evaluation results are stable across different LLM judges or human evaluations, and that your method yields significant improvements over other strong baselines, I would give you a higher overall score.

7 comments
AuthorsOpenReview ↗

Response to Reviewer BvZT (1/3)

We thank the reviewer for the detailed review and for naming the key concerns. We answer in the order of the review.

W1. Contradictions in the final window versus the full text

For 20 prompts we scored windows of the same size at the beginning, middle and end of the 50K and 100K DeepSeek-V4-Flash stories (Differences = NstAgent - RollSum):

Target lengthWindowSubtype CED ↓, NstAgentSubtype CED ↓, RollSumDifferenceInstance CED ↓, NstAgentInstance CED ↓, RollSumDifference
50KBeginning4.095.75−1.665.848.81−2.97
50KMiddle5.076.90−1.837.5811.48−3.89
50KEnd5.256.75−1.517.2311.76−4.52
100KBeginning4.336.78−2.455.7010.67−4.96
100KMiddle4.697.70−3.027.6313.68−6.05
100KEnd4.877.50−2.637.5912.44−4.85

The terminal-window result holds at every position. The beginning window scores lower for both methods, as expected for text with nothing earlier to contradict. Apart from the opening, the terminal window is therefore representative of the story. We chose the terminal window to avoid "lost-in-the-middle" as much as possible.

W2. Human evaluation

We ran a blind pairwise study on 20 pairs of 10K DeepSeek-V4-Flash stories, NstAgent against RollSum. The two stories are shown side by side, following the pairwise design of DOC's own human evaluation [1]. Annotators do not know which method wrote which story.

DimensionNstAgent betterTieRollSum betterSign test p
Overall preference14060.12
Consistency1802<0.001
Engagement11090.82
Prompt adherence21800.50

These results suggest that NstAgent produces more favored stories. We know this evaluation is a small subset of samples and are working to expand it.

W3. Reliance on LLM-as-a-judge

LLM judges can carry biases, including a preference for their own model family. We added several checks to address this.

  1. Three judges from other providers re-scored the stories (Q1).

  2. An extended injection study. The original injection study had only 24 template controls. We raised template controls to 30 per length, added 15 natural contradictions and 15 consistent controls per length written in the story's own voice.

Target lengthNatural contradictions recalledNatural controls: judge false alarmsTemplate controls: judge false alarms
10K12 of 151 of 154 of 30
20K15 of 151 of 159 of 30
50K15 of 152 of 156 of 30
100K14 of 152 of 158 of 30
  1. Manual review of consistency errors. 200 judge alerts, 25 per combination of method and length, were labeled blind to the method:
Alerts by target length / methodRealFalse alarmUnsurePrecision, decided alertsPrecision, unsure counted as false
10K366886%72%
20K2771679%54%
50K2921994%58%
100K3221694%64%
NstAgent, all lengths58113184%58%
RollSum, all lengths6662892%66%
All124175988% [82%, 92%]62% [55%, 68%]

Precision does not differ significantly between the methods (Fisher p=0.20) and does not fall with length. Scaling each setting's CED by its own precision (unsure counted as false) leaves the gap almost unchanged:

Target lengthSubtype CED gap, RollSum - NstAgentAfter precision correction
10K1.911.69
20K1.261.55
50K1.371.71
100K2.382.31

We will add these checks into the revision.

W4. The smaller judge in Appendix B.1, and strong judges

That judge was used only in an early calibration of recall on injected contradictions, never to compare methods. Its recall dropped from 93% to 53% at long lengths, which is why we chose DeepSeek-V4-Pro.

We compared and analyzed our results against strong judges including Claude Opus 5.5, Gemini 4 Argon and gpt-6.1-sol. (see Q1).

AuthorsOpenReview ↗

Response to Reviewer BvZT (2/3)

Q1. Do the results hold under different LLM judges?

We re-scored the 20 10K DeepSeek-V4-Flash stories with three representative judges: Claude Opus 5.5, Gemini 4 Argon and gpt-6.1-sol. (Difference = NstAgent - RollSum)

Narrative Consistency:

JudgeSubtype CED ↓, NstAgentSubtype CED ↓, RollSumDifference [95% CI]Instance CED ↓, NstAgentInstance CED ↓, RollSumDifference [95% CI]
DeepSeek-V4-Pro (paper judge)3.875.79−1.91 [−3.00, −0.78]4.878.72−3.85 [−5.57, −2.10]
claude-opus-5-57.629.56−1.94 [−3.11, −0.90]13.0219.61−6.58 [−9.36, −3.87]
gemini-4-argon5.006.65−1.65 [−2.80, −0.59]7.5412.22−4.68 [−7.06, −2.44]
gpt-6.1-sol4.104.48−0.38 [−1.24, +0.41]6.647.38−0.74 [−2.76, +1.17]

Writing quality:

JudgeWritingBench ↑, NstAgentWritingBench ↑, RollSumDifference [95% CI]
DeepSeek-V4-Pro (paper judge)9.188.87+0.31 [+0.12, +0.50]
claude-opus-5-56.726.53+0.19 [+0.02, +0.37]
gemini-4-argon9.419.24+0.17 [−0.03, +0.37]
gpt-6.1-sol6.856.92−0.08 [−0.21, +0.06]

How the judges differ. We compared the judges error by error. The judges differ mainly in how much they report and they largely agree on what an error is. GPT differs in one specific way. In NstAgent stories it files the same passage under several error types more often: 2.07 entries per passage, against 1.36 in RollSum stories. Both CED metrics count such repeats, which hides the gap under GPT. Counting each erroneous passage once, all four judges find significantly fewer in NstAgent stories:

JudgeDistinct erroneous passages per 10K words ↓, NstAgentDistinct erroneous passages per 10K words ↓, RollSumDifference [95% CI]Entries per passage, NstAgentEntries per passage, RollSum
DeepSeek-V4-Pro (paper judge)2.735.82−3.10 [−4.04, −2.17]1.671.44
claude-opus-5-57.6811.29−3.60 [−5.28, −2.05]1.651.64
gemini-4-argon4.327.76−3.44 [−4.82, −2.17]1.751.57
gpt-6.1-sol2.954.67−1.72 [−2.75, −0.75]2.071.36

The gain also holds for passages on which judges agree. Passages flagged by at least two judges number 3.76 per 10K words for NstAgent against 6.52 for RollSum (−2.75 [−4.05, −1.55]). Passages flagged by all four number 0.93 against 2.10 (−1.17 [−1.86, −0.48]).

Both stories in each comparison come from the same backbone, so family preferences affect both sides equally.

All three judges reproduce the consistency gain.

Q2. How do human readers perceive the stories?

We conducted preliminary human evaluation (see W2) and results suggest that NstAgent produces more favored stories.

Qualitatively, some readers say they are amazed that LLM can generate such lengthy stories, and it is cool that they can use their own prompt. Our simple but effective state design also helps readers to better keep up the plot. However, some feedback indicates that the stories lacks in variety compared to human stories due to the limit of backbone model.

AuthorsOpenReview ↗

Response to Reviewer BvZT (3/3)

Q3. Comparison with DOC and follow-up work at up to 10K

We ran the official code of DOC [1] (specifically DOC Story Generation V2) and of ConWriter [2], a recent state-tracking writer with verification and repair. Both use our prompts and ±20% length gate, and we changed only configuration. For ConWriter we also turned off its option to stop the whole story when a scene still fails a check after repair, because most stories then ended early.

Consistency at 10K under the paper judge (difference = NstAgent - baseline):

BaselineBackboneSubtype CED ↓, NstAgentSubtype CED ↓, baselineDifference [95% CI]Instance CED ↓, NstAgentInstance CED ↓, baselineDifference [95% CI]
DOCDeepSeek-V4-Flash3.8510.47−6.62 [−7.67, −5.53]4.7619.60−14.84 [−17.56, −12.06]
ConWriterDeepSeek-V4-Flash3.949.12−5.18 [−6.53, −3.91]4.9916.16−11.17 [−14.22, −8.50]
ConWriterGPT-5.6 Luna4.425.54−1.12 [−2.40, +0.07]6.248.00−1.76 [−3.53, −0.09]

Writing quality at 10K under the paper judge:

BaselineBackboneWritingBench ↑, NstAgentWritingBench ↑, baselineDifference [95% CI]
DOCDeepSeek-V4-Flash9.225.12+4.11 [+3.61, +4.61]
ConWriterDeepSeek-V4-Flash9.185.59+3.59 [+3.01, +4.18]
ConWriterGPT-5.6 Luna9.196.91+2.28 [+1.64, +3.00]

Note: DOC was run only at 10K on DeepSeek-V4-Flash. It is designed for stories of a few thousand words, one 10K story takes several hours, and its candidate scoring needs token log-probabilities, which GPT-5.6 Luna does not return.

The same comparison under three other judges. claude-opus-5-5, gemini-4-argon and gpt-6.1-sol re-scored the DOC and ConWriter stories written with DeepSeek-V4-Flash, using unchanged prompts and rubrics.

Consistency at 10K:

JudgeBaselineSubtype CED ↓, NstAgentSubtype CED ↓, baselineDifference [95% CI]Instance CED ↓, NstAgentInstance CED ↓, baselineDifference [95% CI]
claude-opus-5-5DOC7.5714.06−6.50 [−7.08, −5.94]12.6246.19−33.57 [−38.36, −28.81]
claude-opus-5-5ConWriter7.6211.66−4.04 [−4.99, −3.12]13.0229.40−16.38 [−20.62, −12.26]
gemini-4-argonDOC5.0212.45−7.43 [−8.48, −6.38]7.9132.29−24.38 [−27.17, −21.19]
gemini-4-argonConWriter5.1611.38−6.22 [−7.24, −5.28]8.0323.04−15.01 [−17.67, −12.55]
gpt-6.1-solDOC4.116.83−2.71 [−4.05, −1.30]6.7913.88−7.10 [−10.69, −3.72]
gpt-6.1-solConWriter4.106.22−2.11 [−3.22, −1.03]6.6414.83−8.19 [−11.67, −5.09]

Writing quality at 10K:

JudgeBaselineWritingBench ↑, NstAgentWritingBench ↑, baselineDifference [95% CI]
claude-opus-5-5DOC6.793.69+3.09 [+2.85, +3.35]
claude-opus-5-5ConWriter6.723.99+2.73 [+2.46, +3.01]
gemini-4-argonDOC9.433.08+6.35 [+5.98, +6.67]
gemini-4-argonConWriter9.413.39+6.02 [+5.53, +6.47]
gpt-6.1-solDOC6.934.08+2.84 [+2.65, +3.03]
gpt-6.1-solConWriter6.854.34+2.51 [+2.20, +2.82]

All four judges find NstAgent significantly better than both baselines in consistency and in writing quality.

DOME [3], which the paper already compares with at 10K, follows DOC's evaluation setting and reports improvements over DOC. We also compare StoryWriter, which reports improvements over DOC. (Table 2 in paper)

Thank you again for your time and insightful comments. We would be glad to address any further questions or concerns.

[1] Kevin Yang et al. DOC: Improving Long Story Coherence With Detailed Outline Control. ACL 2023.

[2] Jindong Li et al. ConWriter: Transition-Constrained Stateful Long-Form Story Generation with Lightweight Neuro-Symbolic Consistency Control. Findings of EMNLP 2026.

[3] Qianyue Wang et al. Generating Long-form Story Using Dynamic Hierarchical Outlining with Memory-Enhancement. NAACL 2025.

[4] Haotian Xia et al. StoryWriter: A Multi-Agent Framework for Long Story Generation. CIKM 2025.

Reviewer_VUUVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MULTIRAG-BENCH, a benchmark for evaluating retrieval across multiple knowledge graphs (KGs). The authors point out that in real-world applications, different knowledge graphs may be built from the same documents using different models or extraction settings. This creates a challenge when a question requires information from multiple graphs. To study this problem, the authors build five domain-specific knowledge graphs from around 21K Wikipedia documents and generate 2,348 cross-domain questions. They compare two main approaches: Graph Fusion, which merges multiple graphs into one, and Joint Retrieval, which searches the graphs separately and combines the retrieved results. The authors also propose an entity-cache mechanism to help share information across retrieval steps. For validation, the authors evaluate different retrieval methods using LightRAG and PG-Direct, and analyze performance across different question difficulties. The results show that searching a single graph is often insufficient for cross-domain questions, while combining information from multiple graphs improves retrieval. However, they also find that most of the gains from Joint Retrieval come from simply combining search results, rather than the proposed cache mechanism. The paper also shows that merging graphs can introduce noisy connections, but remains helpful for questions that genuinely require cross-domain reasoning.

Critical strengths

I think the main strength of this paper is the structured narrative state design. Unlike traditional summarization-based methods that mainly record what happened before, NSTAGENT tracks character information, past events, and future plot requirements. This seems particularly useful for long story generation, where the model needs to both remember earlier details and maintain consistency with future plot developments. The experiments show that this design helps reduce consistency errors as stories become longer. I find this a meaningful contribution since maintaining long-term consistency is one of the main challenges in generating full-length stories. The paper evaluates story generation at lengths up to 100K words, which is much longer than what most previous works have studied. The results show that NSTAGENT maintains relatively stable consistency and writing quality as the story length increases, while the rolling-summary baseline becomes less reliable. The authors also provide ablation studies and cost analysis, showing that the framework can scale to longer stories without a large increase in cost per generated word. These results make the practical value of the proposed framework more convincing.

Critical weaknesses
  1. One major concern is that for stories longer than 10K words, NSTAGENT is mainly compared with RollSum, which is a relatively simple rolling-summary method.
  2. The paper mainly relies on an LLM judge to evaluate narrative consistency and writing quality. However, the injection study shows that the judge incorrectly identifies consistency errors in around 25% of the control cases. This raises concerns about the reliability of the reported consistency error density.
  3. The paper lacks human evaluation of the generated stories, especially at longer lengths. Since writing quality is subjective and difficult to measure automatically, it is unclear whether the reported improvements actually translate into better reading experiences. This is an important limitation given that generating high-quality long stories is the main goal of the paper.
Questions for authors
  1. The paper reports that writing quality remains stable as story length increases. However, LLM judge prefers a longer response.
  2. In Appendix B.1, the injection study shows that the LLM judge reports consistency errors in around 25% of the control cases, even when no errors are injected. How much does this false positive rate affect the reported CED results?
5 comments
AuthorsOpenReview ↗

Response to Reviewer VUUV (1/3)

We thank the reviewer for the feedback. We note respectfully that the Paper Summary seems to describe another submission (MULTIRAG-BENCH). We kindly ask the reviewer to verify if any other parts of the review were also inadvertently mixed up. The Strengths and Weaknesses clearly refer to our paper, and we respond to weaknesses and questions in order.

W1. Beyond 10K, NstAgent is compared only with RollSum

Most baselines are incapable of generating a complete story beyond 10K. We added ConWriter [1], a recent writing agent with verification and repair. We originally intended to compare with DOC [2], but found it is designed for stories of a few thousand words, one 10K story takes several hours.

(Difference = NstAgent - baseline)

Narrative Consistency:

BaselineBackboneTarget lengthSubtype CED ↓, NstAgentSubtype CED ↓, baselineDifferenceInstance CED ↓, NstAgentInstance CED ↓, baselineDifference
ConWriterDeepSeek-V4-Flash10K3.949.12−5.184.9916.16−11.17
ConWriterDeepSeek-V4-Flash20K4.729.55−4.837.3319.14−11.81
ConWriterGPT-5.6 Luna10K4.425.54−1.126.248.00−1.76
ConWriterGPT-5.6 Luna20K5.134.93+0.207.847.62+0.22

Writing quality:

BaselineBackboneTarget lengthWritingBench ↑, NstAgentWritingBench ↑, baselineDifference
ConWriterDeepSeek-V4-Flash10K9.185.59+3.59
ConWriterDeepSeek-V4-Flash20K9.155.53+3.62
ConWriterGPT-5.6 Luna10K9.196.91+2.28
ConWriterGPT-5.6 Luna20K9.296.92+2.37

The results demonstrate that NstAgent overall outperforms the baseline beyond 10K as well. We will add these results to the revision.

AuthorsOpenReview ↗

Response to Reviewer VUUV (2/3)

W2. The judge flags about 25% of control cases

LLM judges can carry biases, so we measured the effect in four ways.

  1. Paired design. Both methods are scored by the same judge with the same prompts and windows, so a false-alarm rate that does not depend on the method cancels in the difference.

  2. An extended injection study. The 25% figure comes from 24 template controls, whose inserted text did not match the story's style. We raised template controls to 30 per length, added 15 natural contradictions and 15 consistent controls per length written in the story's own voice.

Target lengthNatural contradictions recalledNatural controls: judge false alarmsTemplate controls: judge false alarms
10K12 of 151 of 154 of 30
20K15 of 151 of 159 of 30
50K15 of 152 of 156 of 30
100K14 of 152 of 158 of 30

On natural text the judge's false-alarm rate is 6 of 60 (10%), against 27 of 120 (22%) on template text. Recall does not fall with length, and false alarms on natural controls stay at 1 or 2 of 15 at every length.

  1. Independent judges. Scores on the 10K stories (difference = NstAgent - RollSum):
JudgeSubtype CED ↓, NstAgentSubtype CED ↓, RollSumDifference [95% CI]Instance CED ↓, NstAgentInstance CED ↓, RollSumDifference [95% CI]
DeepSeek-V4-Pro (paper judge)3.875.79−1.91 [−3.00, −0.78]4.878.72−3.85 [−5.57, −2.10]
claude-opus-5-57.629.56−1.94 [−3.11, −0.90]13.0219.61−6.58 [−9.36, −3.87]
gemini-4-argon5.006.65−1.65 [−2.80, −0.59]7.5412.22−4.68 [−7.06, −2.44]
gpt-6.1-sol4.104.48−0.38 [−1.24, +0.41]6.647.38−0.74 [−2.76, +1.17]

We compared the judges error by error. The judges differ mainly in how much they report and they largely agree on what an error is. GPT differs in one specific way. In NstAgent stories it files the same passage under several error types more often: 2.07 entries per passage, against 1.36 in RollSum stories. Both CED metrics count such repeats, which hides the gap under GPT. Counting each erroneous passage once, all four judges find significantly fewer in NstAgent stories.

  1. Manual review of consistency errors. 200 judge alerts, 25 per combination of method and length, were labeled blind to the method:
Alerts by target length / methodRealFalse alarmUnsurePrecision, decided alertsPrecision, unsure counted as false
10K366886%72%
20K2771679%54%
50K2921994%58%
100K3221694%64%
NstAgent, all lengths58113184%58%
RollSum, all lengths6662892%66%
All124175988% [82%, 92%]62% [55%, 68%]

Precision does not differ significantly between the methods (Fisher p=0.20) and does not fall with length. Scaling each setting's CED by its own precision (unsure counted as false) leaves the gap almost unchanged:

Target lengthSubtype CED gap, RollSum - NstAgentAfter precision correction
10K1.911.69
20K1.261.55
50K1.371.71
100K2.382.31

We will add these checks into the revision.

W3. Human evaluation

We ran a blind pairwise study on 20 pairs of 10K DeepSeek-V4-Flash stories, NstAgent against RollSum. The two stories are shown side by side, following the pairwise design of DOC's own human evaluation [2]. Annotators do not know which method wrote which story.

DimensionNstAgent betterTieRollSum betterSign test p
Overall preference14060.12
Consistency1802<0.001
Engagement11090.82
Prompt adherence21800.50

These results suggest that NstAgent produces more favored stories. We know this evaluation is a small subset of samples and are working to expand it.

AuthorsOpenReview ↗

Response to Reviewer VUUV (3/3)

Q1. Writing quality remains stable with length, but LLM judges prefer longer responses

LLMs tend to favor longer responses. However, in story writing tasks, complete full-length novels are indeed considered higher quality than shorter ones due to their richer plots and other elements. Therefore, the key is to distinguish metric gains caused solely by increased length. Longer output does not by itself earn higher scores in WritingBench: In the WritingBench [3] experiments, increasing output length did not yield gains in writing quality; instead, it exposed quality degradation issues in long-text generation across models.

Combining these with the results below, we conclude that writing quality remains stable with length.

BackboneMethod10K20K50K100K100K − 10K
DeepSeek-V4-FlashNstAgent8.9989.1409.2409.204+0.206
GPT-5.6 LunaNstAgent9.2589.2169.2109.244−0.014

Q2. How much does the false-positive rate affect CED?

Little, for the comparisons we draw. False alarms common to both methods cancel in the paired difference, the false-alarm rate is about 10% at every length (See W2).

Thank you again for your time and insightful comments. We would be glad to address any further questions or concerns.

[1] Jindong Li et al. ConWriter: Transition-Constrained Stateful Long-Form Story Generation with Lightweight Neuro-Symbolic Consistency Control. Findings of EMNLP 2026.

[2] Kevin Yang et al. DOC: Improving Long Story Coherence With Detailed Outline Control. ACL 2023.

[3] Yuning Wu et al. WritingBench: A Comprehensive Benchmark for Generative Writing. NeurIPS 2025.

Reviewer_xnrDOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper frames long-form story generation as a length-scaling problem and makes two contributions. First, it shows that ConStory-Bench's consistency error density (CED) is not comparable across lengths, because a judge reading an entire 50K–100K-word story reports far fewer errors than length-proportionality would predict; it repairs this with fixed-size terminal windows (a scope marker, so that only errors whose later manifestation falls in the final ~10K words are counted, with the whole preceding narrative as evidence), and validates the protocol with a contradiction-injection study showing no recall decay over 100K-word prefixes. Second, it proposes NSTAGENT, a training-free agentic framework that writes chapter by chapter from a frozen outline while maintaining a structured narrative state (character snapshots, keyed past events, open future requirements) via typed update tools, plus read/search/correct lookback tools. On two backbones (DeepSeek-V4-Flash, GPT-5.6 Luna), NSTAGENT outperforms a rolling-summary baseline from 20K words upward with margins that grow with length, keeps consistency and writing quality roughly flat across 10K–100K words, and scales approximately linearly in cost (about USD 1.32 per 100K-word story under prefix caching).

Critical strengths

1: The method is coherent, and each component maps to an explicit failure mode:

The structured, typed, keyed state (overwritten character snapshots, appended keyed events, explicitly listed and resolved future requirements) contrasts cleanly with RollSum's rewritten free-text summary, and the prospective requirements are the main design delta over prior structured memories. The case studies (Appendix E) make the mechanism concrete — summary paraphrase drift in RollSum (Cases 1–2) — while honestly presenting a failure mode in which the state locks in and propagates a wrongly extracted fact (Case 3), noting that correct is invoked only 0.1–2.3 times per story and that no verification step checks new state entries against the source text.

2: The paper identifies and concretely documents a real measurement problem.

Length-normalized consistency metrics such as ConStory-Bench's CED assume that the number of judge-reported errors grows in proportion to story length. The paper shows this breaks down beyond ~10K words: judge recall saturates on long inputs, so full-story CED falls with length even when the writing does not improve — for an earlier agent, Subtype CED dropped from 1.47 to 0.87 to 0.50 across 20K/50K/100K words, with reported counts growing less than 3x under a 5x length increase (Appendix A.1) — and a human-adjudicated probe at 10K found that free enumeration covered only ~38% of confirmed root causes. This drift silently invalidates cross-length consistency comparisons, which are exactly what scaling studies of book-length generation need. Diagnosing the failure concretely, and repairing it with a protocol whose enumeration load is length-invariant, has standalone value beyond the proposed method.

Critical weaknesses

W1. The terminal window is never validated as representative of the whole story. The extended protocol counts only contradictions whose later manifestation falls in the final ~10K words; at 10K the window is the whole story, while at 20K–100K it is only the ending. Cross-length comparisons in Table 3 therefore assume that error density is uniform across story positions, and this assumption is never tested — no early or middle window is ever scored. Position bias is plausible a priori (terminal chapters must agree with the most accumulated history) and could differ across methods: for example, if longer stories call back to earlier facts less often, the terminal window offers fewer contradiction opportunities, and a flat CED would then reflect shallower interconnection rather than preserved consistency. Please score at least one additional window position (e.g., first/middle) at some lengths, and report how often window text invokes pre-window facts across lengths.

W2. No direct evidence that the protocol restores cross-length comparability. The validation provided is instrument-level, not metric-level: Appendix A.1 demonstrates the disease (full-story CED falling ~40% from 20K to 100K), the marker experiment covers a single length (10K), and the injection study (Table 8) shows flat per-length recall only for explicit, conspicuous contradictions at ceiling, with precision probed on merely 6 controls per length (0/6, 2/6, 2/6, 2/6). Nothing aligns the metric's readings on natural stories against a length-invariant reference — and the flat CED trend in Table 3 cannot serve as evidence, since that is the result to be validated (circular). Cheap direct tests exist: score multiple window positions within the same story; run the protocol on published human fiction at matched window sizes (readings should be length-invariant if the instrument is stable); extend the injection study from recall to recovered density with more controls; or human-audit a sample of windows per length. Absent any such anchor, "CED is comparable across lengths" remains an assumption rather than a demonstrated property.

Questions for authors

see Weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer xnrD (1/2)

We thank the reviewer for an exceptionally precise review. We answer in the order of the review.

W1. Is the terminal window representative?

For 20 prompts we scored windows of the same size at the beginning, middle and end of each DeepSeek-V4-Flash story, with the same judge and prompts. (Differences = NstAgent - RollSum):

Target lengthWindowSubtype CED ↓, NstAgentSubtype CED ↓, RollSumDifferenceInstance CED ↓, NstAgentInstance CED ↓, RollSumDifference
50KBeginning4.095.75−1.665.848.81−2.97
50KMiddle5.076.90−1.837.5811.48−3.89
50KEnd5.256.75−1.517.2311.76−4.52
100KBeginning4.336.78−2.455.7010.67−4.96
100KMiddle4.697.70−3.027.6313.68−6.05
100KEnd4.877.50−2.637.5912.44−4.85

The terminal-window result holds at every position. The beginning window scores lower for both methods, as expected for text with nothing earlier to contradict. Apart from the opening, the terminal window is therefore representative of the story. We chose the terminal window to avoid "lost-in-the-middle" as much as possible.

We also measured how often window text invokes pre-window facts: the share of named characters mentioned in the terminal window who were introduced before it.

Method20K50K100K
NstAgent0.680.770.86
RollSum0.610.670.75

This share rises with length. Longer stories do not refer back less, so a flat CED is not explained by shallower interconnection.

We will add these results to the revision.

AuthorsOpenReview ↗

Response to Reviewer xnrD (2/2)

W2. Direct evidence that the protocol restores comparability

We agree that the paper validated the instrument (judge recall) but not fully prove the metric's comparability across lengths on natural stories. We ran all four tests the reviewer suggested.

  1. Multiple window positions. Covered by the W1 table.

  2. Published human fiction. We ran the unchanged protocol on 10K, 20K, 50K and 100K prefixes of 20 public-domain novels, split into the same number of pseudo-chapters as our stories. The last two columns give the paper's scores for generated stories on DeepSeek-V4-Flash.

Prefix lengthNovelsSubtype CED ↓, mean (SD)Instance CED ↓, mean (SD)NstAgent, Subtype CEDRollSum, Subtype CED
10K201.48 (1.74)2.32 (3.53)4.545.98
20K202.51 (2.04)4.05 (3.65)5.386.76
50K202.40 (1.75)4.69 (3.65)5.797.16
100K202.69 (2.09)4.13 (3.28)4.867.24
Paired comparisonSubtype CED difference [95% CI]Instance CED difference [95% CI]
50K - 20K−0.11 [−0.91, +0.68]+0.64 [−0.84, +1.99]
100K - 20K+0.19 [−0.73, +1.09]+0.09 [−1.24, +1.34]
100K - 50K+0.30 [−0.35, +0.95]−0.56 [−1.81, +0.79]
20K - 10K+1.02 [+0.24, +1.86]+1.73 [−0.04, +3.33]

From 20K to 100K the metric is flat on human fiction. The 10K value is lower because a 10K prefix is the novel's opening, with nothing earlier to contradict.

  1. An extended injection study. Table 8 probed precision on only 6 template controls per length. We raised template controls to 30 per length, added 15 natural contradictions and 15 consistent controls per length written in the story's own voice.
Target lengthNatural contradictions recalledNatural controls: judge false alarmsTemplate controls: judge false alarms
10K12 of 151 of 154 of 30
20K15 of 151 of 159 of 30
50K15 of 152 of 156 of 30
100K14 of 152 of 158 of 30
Distance between the two contradicting passagesRecalled
about 5.5K words20 of 20
about 26K words19 of 20
about 45K words on average, about 100K at the 100K length17 of 20 (4 of 5 at about 100K)

Recall does not fall with length, and false alarms on natural controls stay at 1 or 2 of 15.

  1. Manual review of consistency errors. 200 judge alerts, 25 per combination of method and length, were labeled blind to the method:
Alerts by target length / methodRealFalse alarmUnsurePrecision, decided alertsPrecision, unsure counted as false
10K366886%72%
20K2771679%54%
50K2921994%58%
100K3221694%64%
NstAgent, all lengths58113184%58%
RollSum, all lengths6662892%66%
All124175988% [82%, 92%]62% [55%, 68%]

Precision does not differ significantly between the methods (Fisher p=0.20) and does not fall with length. Scaling each setting's CED by its own precision (unsure counted as false) leaves the gap almost unchanged:

Target lengthSubtype CED gap, RollSum - NstAgentAfter precision correction
10K1.911.69
20K1.261.55
50K1.371.71
100K2.382.31

We will add these checks into the revision.

Thank you again for your time and insightful comments. We would be glad to address any further questions or concerns.

Dear Reviewer xnrD,

As the discussion period closes soon, we would like to briefly summarize how our response addresses your two weaknesses.

W1. Is the terminal window representative?

  • Additional window positions. At 50K and 100K we scored same-size windows at the beginning, middle and end of each story. At every position NstAgent has fewer errors than RollSum. Apart from the opening, which has nothing earlier to contradict, the terminal window gives the same picture as the other positions.
  • How often the window invokes pre-window facts. The share of characters in the terminal window who were introduced before it rises with length. Longer stories do not refer back less, so a flat CED does not reflect shallower interconnection.

W2. Direct evidence that the protocol restores cross-length comparability. You listed four cheap direct tests. We ran all of them:

  1. "Score multiple window positions within the same story." See W1: the gain holds at the beginning, middle and end.
  2. "Run the protocol on published human fiction at matched window sizes." On 10K–100K prefixes of 20 public-domain novels, the unchanged protocol is flat from 20K to 100K (all paired differences n.s.). The 10K prefix is lower because it is the novel's opening.
  3. "Extend the injection study … with more controls." We raised template controls from 6 to 30 per length and added 15 natural contradictions and 15 consistent controls per length, written in the story's own voice. Recall of natural contradictions is 12–15 of 15 at every length and does not fall with the distance between the two passages. False alarms on natural controls stay at 1–2 of 15.
  4. "Human-audit a sample … per length." We labeled 200 judge alerts, 25 per method and length, blind to the method. Precision does not fall with length and does not differ between methods. Correcting CED by each setting's precision leaves the gap almost unchanged.

We hope these results address your concerns about terminal window representativity or cross-length comparability. We would be glad to clarify any potential concerns before the discussion closes. Thank you again for your time.

Reviewer_nKeHOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents NstAgent, a training-free framework for generating stories up to 100K words while maintaining consistency and quality. It uses a fixed outline and an evolving narrative state to track characters, past events, and unresolved plot points. The authors also introduce a fixed-size evaluation window for ConStory-Bench. Results on two LLMs show better consistency than rolling-summary baselines, especially for longer stories. The experiments are fairly thorough, though I still have some concerns about evaluation validity, the role of individual components, and scalability.

Critical strengths
  1. The paper evaluates story generation from 10K to 100K words across two model families, with paired statistical tests. NstAgent shows clear consistency improvements at 100K words. I also appreciate the discussion of how longer stories can lead to underestimated contradiction rates. The terminal-window evaluation and contradiction-injection tests are useful contributions.

  2. The structured narrative state tracks characters, past events, and unresolved plot points, which seems useful for maintaining long-term consistency. The ablations show that both narrative state and lookback tools help, while the qualitative examples highlight problems with rolling summaries.

Critical weaknesses
  1. The consistency evaluation mainly focuses on the final 10K words, so it is unclear whether the results hold across the entire story. I also have some concerns about the LLM judge. Appendix B.1 reports that 79% of consistent control pairs are flagged under at least one subtype, suggesting possible false positives. Writing quality is also judged without human evaluation, and some improvements are not statistically significant. These issues make the broader claims about full-length story quality less convincing.

  2. The explicit tracking of unresolved future plot points seems to be the most interesting part of the framework. However, the ablations remove the entire narrative state rather than testing its components separately. It is therefore hard to tell how much this specific design contributes. The main comparison with RollSum also changes several things at once. A more controlled baseline and clearer discussion of related work like ConWriter would help establish the novelty of the approach.

Questions for authors

Q1. Could the authors add an ablation that removes only future requirements while keeping the other state components and tools? This would help clarify how much prospective tracking actually contributes. A baseline with similar lookback capabilities would also be useful.

Q2. Does NstAgent still perform better if evaluation windows are sampled from different parts of the story? Could the authors also manually check some naturally occurring contradictions to estimate false-positive rates and verify the rankings?

Q3. Could the authors report completion rates, the number of evaluated stories, and actual terminal-window lengths for each setting? Since incomplete stories are excluded and generated lengths vary, these details would help assess possible selection bias.

5 comments
AuthorsOpenReview ↗

Response to Reviewer nKeH (1/3)

We thank the reviewer for the positive assessment and the concrete suggestions. We answer in the order of the review.

W1. Evaluation scope, judge false positives and human evaluation

The consistency evaluation mainly focuses on the final 10K words, so it is unclear whether the results hold across the entire story.

We scored same-size windows at the beginning, middle and end of the 50K and 100K DeepSeek-V4-Flash stories. NstAgent is significantly better than RollSum at every position (table under Q2).

Appendix B.1 reports that 79% of consistent control pairs are flagged under at least one subtype, suggesting possible false positives.

That figure counts a flag under any subtype. Most were style-shift reports caused by the inserted template text, and 25% were flagged under the targeted subtype. To better control false positives, we raised template controls to 30 per length, added 15 natural contradictions and 15 consistent controls per length written in the story's own voice.

Target lengthNatural contradictions recalledNatural controls: judge false alarms
10K12 of 151 of 15
20K15 of 151 of 15
50K15 of 152 of 15
100K14 of 152 of 15
All56 of 606 of 60 (10%)

The results show false positives are rare and have little impact on the overall assessment. We also conducted a manual review of 200 judge alerts. The results show that the majority of alerts are real, and the precision is high (table under Q3).

Writing quality is also judged without human evaluation, and some improvements are not statistically significant.

We ran a blind pairwise study on 20 pairs of 10K DeepSeek-V4-Flash stories, NstAgent against RollSum. The two stories are shown side by side, following the pairwise design of DOC's own human evaluation [1]. Annotators do not know which method wrote which story.

DimensionNstAgent betterTieRollSum betterSign test p
Overall preference14060.12
Consistency1802<0.001
Engagement11090.82
Prompt adherence21800.50

These results suggest that NstAgent produces more favored stories. For significance, in the paper, the gain over RollSum is significant on DeepSeek-V4-Flash at 20K, 50K and 100K and on GPT-5.6 Luna at 100K. We will describe writing quality gain more specifically in the revision.

W2. Individual components, a more controlled comparison, and ConWriter

We conducted the ablation study which just removes the future requirements and results show that the removal of future requirements leads to a decrease in narrative consistency (table under Q1).

ConWriter [2] keeps evolving story states, checks transitions symbolically and repairs flagged sentences. However, its state has no explicit category of open obligations, its sample size is only five and its performance at longer length underexplored. We ran its official code with ±20% length gate (difference = NstAgent - ConWriter):

Narrative Consistency:

BackboneTarget lengthSubtype CED ↓, NstAgentSubtype CED ↓, ConWriterDifferenceInstance CED ↓, NstAgentInstance CED ↓, ConWriterDifference
DeepSeek-V4-Flash10K3.949.12−5.184.9916.16−11.17
DeepSeek-V4-Flash20K4.729.55−4.837.3319.14−11.81
GPT-5.6 Luna10K4.425.54−1.126.248.00−1.76
GPT-5.6 Luna20K5.134.93+0.207.847.62+0.22

Writing quality:

BackboneTarget lengthWritingBench ↑, NstAgentWritingBench ↑, ConWriterDifference
DeepSeek-V4-Flash10K9.185.59+3.59
DeepSeek-V4-Flash20K9.155.53+3.62
GPT-5.6 Luna10K9.196.91+2.28
GPT-5.6 Luna20K9.296.92+2.37

On DeepSeek-V4-Flash, NstAgent is far better on both metrics. On GPT-5.6 Luna, consistency is close, and NstAgent's writing quality is much higher than ConWriter's. These results show it is crucial to combine both metrics to evaluate generated stories rather than only consistency. We will discuss the difference with ConWriter and add results to the revision.

AuthorsOpenReview ↗

Response to Reviewer nKeH (2/3)

Q1. Removing only future requirements

We conducted the ablation study which just removes the future requirements and results show that the removal of future requirements leads to a decrease in narrative consistency. WritingBench is 9.18 for both variants.

Consistency metricNstAgentWithout future requirementsRollSum (reference)Difference [95% CI]
Subtype CED ↓3.874.775.79+0.90 [+0.00, +1.80]
Instance CED ↓4.876.638.72+1.76 [+0.31, +3.06]
Abandoned-plot errors per 10K words ↓0.500.810.66+0.32 [+0.05, +0.58]

The stories become less consistent by removing the future requirements, most clearly in the error type this component targets: the judge reports 18 abandoned plot elements across the 20 ablation stories, against 11 for NstAgent, which indicates it is a critical component.

Q2. Other window positions, and a manual check of natural contradictions

For 20 prompts we scored windows of the same size at the beginning, middle and end of the 50K and 100K DeepSeek-V4-Flash stories (Differences = NstAgent - RollSum):

Target lengthWindowSubtype CED ↓, NstAgentSubtype CED ↓, RollSumDifferenceInstance CED ↓, NstAgentInstance CED ↓, RollSumDifference
50KBeginning4.095.75−1.665.848.81−2.97
50KMiddle5.076.90−1.837.5811.48−3.89
50KEnd5.256.75−1.517.2311.76−4.52
100KBeginning4.336.78−2.455.7010.67−4.96
100KMiddle4.697.70−3.027.6313.68−6.05
100KEnd4.877.50−2.637.5912.44−4.85

For the manual check, 200 judge alerts, 25 per combination of method and length, were labeled blind to the method:

Alerts by target length / methodRealFalse alarmUnsurePrecision, decided alertsPrecision, unsure counted as false
10K366886%72%
20K2771679%54%
50K2921994%58%
100K3221694%64%
NstAgent, all lengths58113184%58%
RollSum, all lengths6662892%66%
All124175988% [82%, 92%]62% [55%, 68%]

Precision does not differ significantly between the methods (Fisher p=0.20) and does not fall with length. Scaling each setting's CED by its own precision (unsure counted as false) leaves the gap almost unchanged:

Target lengthSubtype CED gap, RollSum - NstAgentAfter precision correction
10K1.911.69
20K1.261.55
50K1.371.71
100K2.382.31

We will add these checks into the revision.

AuthorsOpenReview ↗

Response to Reviewer nKeH (3/3)

Q3. Completion rates, evaluated stories and words scored

The table below gives these numbers for the main comparisons in the paper: NstAgent and RollSum at all lengths, and the other 10K baselines.

BackboneMethodTarget lengthPromptsCompletedScoredWords scored, mean (min–max)
DeepSeek-V4-FlashDirect10K10099999,376 (7,973–11,925)
DeepSeek-V4-FlashDOME10K2020209,843 (9,685–10,117)
DeepSeek-V4-FlashStoryWriter10K10098989,735 (7,900–11,854)
DeepSeek-V4-FlashRollSum10K10010010010,172 (9,125–11,509)
DeepSeek-V4-FlashNstAgent10K10010010011,364 (10,620–13,351)
DeepSeek-V4-FlashRollSum20K10010010010,684 (9,464–12,115)
DeepSeek-V4-FlashRollSum50K100969610,032 (8,504–11,611)
DeepSeek-V4-FlashRollSum100K5050509,691 (7,761–11,940)
DeepSeek-V4-FlashNstAgent20K10010010010,524 (9,518–11,516)
DeepSeek-V4-FlashNstAgent50K10010010010,853 (9,447–12,316)
DeepSeek-V4-FlashNstAgent100K50505010,564 (8,015–13,006)
GPT-5.6 LunaDirect10K10097979,568 (7,865–11,682)
GPT-5.6 LunaDOME10K2020209,982 (9,891–10,108)
GPT-5.6 LunaStoryWriter10K10010010010,170 (7,977–11,426)
GPT-5.6 LunaRollSum10K10010010010,724 (9,438–12,056)
GPT-5.6 LunaNstAgent10K10010010010,985 (10,320–12,355)
GPT-5.6 LunaRollSum20K1001001009,619 (8,622–10,526)
GPT-5.6 LunaRollSum50K10010010010,143 (8,411–12,526)
GPT-5.6 LunaRollSum100K5050509,955 (8,690–11,329)
GPT-5.6 LunaNstAgent20K10010010010,433 (9,708–11,269)
GPT-5.6 LunaNstAgent50K1001001009,744 (8,491–11,253)
GPT-5.6 LunaNstAgent100K5050509,245 (8,300–10,445)

This shows that the completion rates, evaluated stories and words scored are largely consistent, with little impact on the final results.

Thank you again for your time and insightful comments. We would be glad to address any further questions or concerns.

[1] Kevin Yang et al. DOC: Improving Long Story Coherence With Detailed Outline Control. ACL 2023.

[2] Jindong Li et al. ConWriter: Transition-Constrained Stateful Long-Form Story Generation with Lightweight Neuro-Symbolic Consistency Control. Findings of EMNLP 2026.

Comments on the paper

2 comments

Dear reviewers,

The authors have submitted their rebuttals. Please take a look and interact with the authors if necessary.

Thanks a lot.

AC

AuthorsOpenReview ↗

Rebuttal Summary

We thank the Area Chair and all four reviewers for their time and insightful feedback. Below we summarize the paper, the concerns and evidence, and the feedback.

1. Paper summary and strengths

We frame long-form story generation as a length-scaling problem. We make consistency comparable across lengths by scoring fixed-size terminal windows, and we propose NstAgent, a training-free agent that tracks a structured narrative state of characters, past events and unresolved future requirements, with tools to read and search earlier chapters. From 10K to 100K words, NstAgent is more consistent than a rolling-summary baseline, with margins that grow with length, while its writing quality stays high and its cost grows almost linearly. The reviewers highlighted the following strengths:

  • Narrative state tracking. The structured state "seems useful for maintaining long-term consistency" (nKeH). The prospective requirements are "the main design delta over prior structured memories" (xnrD). The state design tracks "character information, past events, and future plot requirements" and is "a meaningful contribution" (VUUV). "Tracking unresolved narrative requirements is the paper's greatest strength, and this approach could be widely adopted in other works" (BvZT).
  • Evaluation at book length. "The terminal-window evaluation and contradiction-injection tests are useful contributions" (nKeH). The paper "identifies and concretely documents a real measurement problem", and the repair "has standalone value beyond the proposed method" (xnrD). It evaluates "lengths up to 100K words, which is much longer than what most previous works have studied" (VUUV), and "shows how to evaluate long-form stories for contradictions, even at substantial lengths" (BvZT).
  • Thorough experiments. The experiments "from 10K to 100K words across two model families" (nKeH), "each component maps to an explicit failure mode" (xnrD), and the paper "also provide ablation studies and cost analysis" (VUUV).
  • Clarity, Provided stories and Effective GPRO (BvZT).

2. Concerns and new evidence

  1. Reliability of the LLM judge (all reviewers)
  • Other judges. We use Claude Opus 5.5, Gemini 4 Argon and GPT-6.1 Sol re-scored the stories. All judges reproduce the consistency gain.
  • Passages flagged by several judges. Passages flagged by at least two judges number 3.76 per 10K words for NstAgent against 6.52 for RollSum.
  • Injected contradictions. The judge recalls 12–15 of 15 natural contradictions at every length, and recall does not fall with length. False alarms on natural controls are 6 of 60.
  • Blind audit of 200 judge alerts. Precision is 88% among decided alerts, or 62% if the unsure alerts are counted as false. Correcting CED for precision leaves the gap almost unchanged.
  1. Human evaluation (nKeH, VUUV, BvZT)
  • We ran a blind pairwise study on 20 pairs. NstAgent was judged more consistent in 18 of 20 pairs. Overall preference favours NstAgent in 14 of 20 pairs.
  1. Representativeness of the window and comparability across lengths (xnrD, nKeH, BvZT)
  • Window position. NstAgent is significantly better than RollSum in windows at the beginning, middle and end of 50K and 100K stories.
  • References to earlier text. We found longer stories refer back to earlier material more.
  • Human-written novels. On 20 public-domain novels, the unchanged protocol is flat from 20K to 100K words.
  1. Stronger baselines and components (nKeH, VUUV, BvZT)
  • ConWriter and DOC. We ran the official code of ConWriter and DOC. On DeepSeek-V4-Flash, NstAgent outperforms both significantly in consistency and writing quality, under all four judges and NstAgent remains far more consistent than ConWriter at 20K. On GPT-5.6 Luna, the consistency of the two is close, and NstAgent writes much better.
  • Future requirements. Removing only the future requirements makes stories less consistent, most clearly in abandoned plot elements, the error type this component targets.
  1. A frontier generation model (BvZT)
  • GPT-6.1 Sol. NstAgent still makes fewer consistency errors than RollSum, ConWriter and DOME and writing quality remains much higher than ConWriter and DOME.

3. Feedback from the discussion

  • Reviewer BvZT found that most of the questions were resolved, asked for a frontier generation model, which we added, and then found the work "valuable enough to be accepted" and raised the rating.
  • Reviewer nKeH found that the new ablation, window-position and manual-validation results "substantially address" the main concerns of the review, and noted two remaining limitations: the uncertainty in manual contradiction judgments and the non-significant overall human preference.
  • Reviewer VUUV thanked us for the detailed answers.

We thank all reviewers again for their engagement, which has clearly improved the paper. The revised PDF incorporates these, with all changes marked in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.