acceptodds
Desk rejected at ICLR 2027

MindCompletion: Learning Latent Human Intent from Behavioral Continuations

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Language models are increasingly deployed as proactive writing assistants, suggesting continuations as the writer drafts. Effective writing assistance demands more than fluency: the assistant must anticipate what the writer intends to say next, a behavioral judgment that standard language model training is not designed to capture. To address this, we propose process-continuation learning, a paradigm that uses naturally-occurring expert writing traces as behavioral reward for reinforcement learning, operationalized through a structured user agent without per-instance annotation. To stabilize optimization, we schedule a curriculum reward progressing from local coherence to semantic alignment, and construct context-variant trajectories covering prefix-only and history-conditioned settings. The trained policy doubles acceptance rate on a co-writing benchmark, reducing the writer's coordination burden, and matches strong closed-source assistants. These gains further generalize beyond interactive writing, with emergent context-aware reasoning and consistent improvements on mathematical reasoning and instruction following, reflecting a stronger capacity to anticipate user intent.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human-AI Collaboration

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_3cQaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes process-continuation learning for generating text continuations based on the current context. The authors argue that standard SFT is not ideal because it treats one observed continuation as the unique target, although multiple continuations may express the same intent. They therefore train a policy with GRPO, using GPT-5.6 Luna as a user agent that evaluates the generated continuations.

Critical strengths

(S1) The paper tackles the interesting problem of text suggestion.

(S2) The paper provides an intuitive training framework.

(S3) The two-stage curriculum Stage I: local validity and Stage II: semantic alignment is a clever way to tackle the problem.

Critical weaknesses

(W1) Could the authors also compare against stronger baselines than SFT, such as direct preference optimization methods?

(W2) The method relies on GPT-5.6 Luna to evaluate generated suggestions during GRPO training. This dependence on a closed model makes the method difficult to reproduce. Could the authors run experiments using a fully open-weight pipeline?

(W3) The authors only provide experiments in which Qwen3-8B is the trainable policy updated with GRPO. It would be beneficial to evaluate another model family, since recent work indicates that GRPO behavior can be highly model-family-dependent and that Qwen models may improve even under spurious rewards [1].

References

[1] Shao, Rulin, et al. "Spurious Rewards: Rethinking Training Signals in RLVR." arXiv preprint arXiv:2506.10947 (2025).

Questions for authors

see weaknesses

Reviewer_NfNJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces PCL - process continuation learning, where a writing style is "learnt" by using the continuations from human written articles as reward mechanisms for a RL setup. Local coherence and semantic alignment are used in a GRPO setup. The paper also investigates training with prefixes, synthetic interaction histories, and a mix of both. The results show improvements over SFT approaches across several writing benchmarks. Human evaluations, external domain tests, and supplementary analysis provide additional support for the approach.

Critical strengths
  1. This is a relevant problem. A continuation can be fluent without matching the direction the writer wants to take. The paper makes a reasonable case for using human-written continuations as evidence of that direction while allowing different wording.

  2. The reward choices make intuitive sense, and the ablations help explain them. In particular, the comparison between local validity, semantic alignment, and holistic acceptance makes a coherent case.

  3. PCL Mix improves macro HAR from 28.41% for the matched SFT Mix baseline to 36.10% which is quite significant!

  4. The supplementary analysis addressed several concerns that came up while reading. These include agreement between judges and humans, robustness to alternative judges, and whether post-training reduces performance on other tasks. These checks make the results more convincing.

Critical weaknesses
  1. Concerns about the data: The training data could be elaborated more thoroughly. The authors describe the collection and filtering process, but the choice of the 30 "high-quality" authors is left open-ended. More information about author selection, topic and style distributions, and the effects of filtering would help explain what kind of writing behavior the model learns. Since the improvements rely directly on the quality of the dataset, this analysis is necessary.

  2. Thirty authors is a narrow basis for modeling human intent in general, since the policy effectively learns the continuation habits of 30 people. The paper does not report an author-level split or diversity analysis.

  3. Zhihu articles are finished, revised text, so the sentence that follows a breakpoint is the product of editing and may differ from the writer's next move during drafting. In contrast, the evaluation sets (CoAuthor, ScholaWrite) come from actual drafting sessions. This mismatch between training supervision and evaluation is not discussed.

  4. Improvements after training on math solutions are not evidence that writing training transfers to math. The distinction between writing only training and additional math training should be clearer. The writing only gains on AIME are small and should be interpreted cautiously. The supplementary reasoning trace analysis is useful, but individual examples do not establish that structured internal planning causes these improvements.

Questions for authors
  1. How were the 30 authors selected? Could you provide author, topic, and style distributions and explain the effects of filtering?

  2. How do you account for the difference between continuations in finished, revised articles and a writer's next move during drafting? Have you examined whether this affects the suitability of the training supervision?

  3. Does PCL's advantage over matched SFT persist under alternative judges? The current alternative-judge experiments establish improvements over the starting model but do not include SFT.

  4. Please clarify the initialization and training procedure for the AoPS trained model. AoPS hosts AIME 2025 problems and solutions, and the Fig. 14 case study is an AIME 2025 problem. Was the AoPS training data analysed for presence of AIME 2025 problems?

Reviewer_KQpcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes process-continuation learning (PCL), which turns existing articles into reward signals for training large language models with a reinforcement learning curriculum. The goal is to improve the models’ ability to generate text continuations. The authors evaluate PCL on MindCopilot, CoAuthor, and ScholaWrite, and provide further evidence of its effectiveness through generalization experiments and human evaluations.

Critical strengths
  1. The paper turns static data, in the form of human-written articles, into a simulated training environment in which a model can generate continuations and receive feedback. This approach could make better use of the large amount of existing human-written text and may inform future work on model training.
  2. The authors report detailed results from ablation and generalization experiments and validate their findings in a study with people using the models as they write.
Critical weaknesses
  1. MindCopilot [1] already used suggestions that writers accepted or text that they edited in real writing sessions as reference continuations. It trained Qwen3-4B with GRPO using hierarchical acceptance rate (HAR) as the reward and compared training with and without interaction history. The proposed method overlaps substantially with MindCopilot. The authors should explain more clearly what PCL contributes beyond MindCopilot and compare it with MindCopilot’s direct HAR optimization under the same mixed-trajectory training conditions. This would help establish both the novelty and the effectiveness of PCL.
  2. The claim of “learning latent human intent” is broad. Judging a generated continuation’s quality and whether the author would accept it by comparing it with what the author actually wrote primarily captures similarity in meaning and line of thought. It provides limited evidence about which other continuations the author would have welcomed. For example, a model might generate a continuation that the author would prefer to their original version because it better expresses what they wanted to convey. If that continuation differs sufficiently from the reference in meaning, it may still receive zero semantic reward. It is also worth investigating whether a model that learns human intent can accommodate the diversity of human preferences in practice.

[1] "MindCopilot: Towards Formalizing and Evaluating Granular Human-LLM Co-Writing", Fang, Tang, et.al.

Questions for authors

None

Reviewer_W9xvOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Process-Continuation Learning (PCL), which uses continuations from human-written documents as behavioral references for reinforcement learning. An external LLM provides rewards for local coherence and semantic alignment, and the policy is optimized with GRPO using a two-stage reward curriculum. The method also incorporates synthetic interaction histories. Experiments report improvements over SFT baselines across several continuation benchmarks, supported by human preference evaluation and a small user study.

Critical strengths

1. Effective use of naturally occurring supervision.

The method converts existing documents into reference-conditioned reward signals without requiring per-instance human preference annotation. This is a practical approach to constructing supervision for open-ended generation.

2. A relevant problem setting.

The paper addresses a meaningful setting for extending reinforcement learning beyond tasks with directly verifiable outcomes.

3. Substantial gains under the reported evaluation protocol.

PCL-Mix achieves 36.10% Macro HAR, compared with 28.41% for the matched SFT-Mix baseline. Evaluation spans multiple datasets and context configurations.

Critical weaknesses

1. Optimization and primary evaluation share the same judge, limiting the strength of the empirical claim.

Appendices C and F explicitly state that GPT-5.6 Luna provides both the training rewards and the primary HAR/KED evaluation. Different prompts do not eliminate the possibility that the policy learns preferences specific to GPT models. Consequently, the reported improvements may partly reflect increased agreement with the reward provider, rather than a corresponding improvement in human acceptance.

2. The computational efficiency is insufficiently characterized. The method samples 16 candidates per prompt and uses a strong model for reward computation, in addition to preprocessing and synthetic-history generation. Avoiding manual annotation does not by itself establish cost-effective scalability. The paper should report rollout token counts, judge usage and API costs, and compare alternatives under matched resource budgets.

3. Relation to prior works (minor concern).

The discussion of reducing annotation costs could also include approaches that derive training signals without preference labels or simulated user feedback, such as Self-Rewarding Language Models [1], and EMPO [2]. A Brief discussion on this line of work would help clarify PCL’s supervision assumptions, particularly its use of human continuations and an external judge.

[1] Self-Rewarding Language Models. ICML, 2024.

[2] Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization. NeurIPS, 2025.

Questions for authors

Please refer to weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.