acceptodds
Under review as a conference paper at ICLR 2027

Future-Grounded Supervision: Improving LLM Intelligence with Real-World User Behavior Data

Abstract

Large language models are increasingly deployed in real-world applications, from personal assistants that take actions based on users' long-term behavior histories to recommendation and advertising systems that predict what users will do. These applications demand the user dimension of LLM intelligence: the ability to infer a person's preferences, intent, and likely future actions from weeks to years of user behavior data. Yet current models remain limited in this user dimension of intelligence: they are trained on static text or synthetic data, neither of which captures how user behavior evolves over time. Real-world user behavior data, by contrast, is already available at scale on content platforms and in personal-assistant agents. These records capture sequences of user actions, making the user's actual next action a natural supervisory signal with the potential to improve LLM intelligence. To this end, we propose *future-grounded supervision*, a paradigm in which the training signal for LLMs comes from the user's real future behavior. We further introduce **Vela**, a framework that operationalizes this paradigm by converting real user behavior data into trainable tasks and trajectories. From the de-identified behavior records of 100,000 users, Vela generates 100,000 tasks across 91 real-world task templates. Difficulty-based filtering and trajectory acceptance criteria jointly retain approximately 75,000 tasks with selected training trajectories. Fine-tuning Qwen3.6-35B-A3B on these training trajectories improves performance over the base model by up to 11.5 points on user-related tasks and by an average of 4 points on general benchmarks. Scaling experiments further show that training on more user behavior data improves user-related performance. Reinforcement learning after fine-tuning outperforms reinforcement learning from the base model by up to 7.7 points on user-related tasks. These gains suggest that future-grounded supervision on real-world user behavior data can improve not only user-centered capabilities but also the broader intelligence of LLMs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, User Behavior

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TQvaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Future-Grounded Supervision, which uses real-world user behavior histories and observed future actions as supervision signals to improve LLMs' understanding of users' long-term preferences, intentions, and behavior patterns. The authors introduce Vela, a framework that converts de-identified user behavior histories into various natural language tasks, followed by difficulty filtering and teacher-generated trajectory selection. The resulting data is used to fine-tune Qwen3.6-35B-A3B, and the authors further investigate its effectiveness as an initialization for reinforcement learning.

Experiments show that Vela improves performance on the proposed VelaBench, several user-related benchmarks, and some general capability benchmarks. The paper also examines the effects of data scaling and RL initialization.

Critical strengths
  1. Interesting and practically relevant problem. The paper addresses how real-world user behavior can be used to improve LLMs' understanding of users. The problem is clearly motivated and relevant to personalized AI agents.

  2. Meaningful data construction effort. The authors convert real-world user behavior into diverse training tasks and develop a fairly complete data construction pipeline. However, its long-term value to the community will depend on the reproducibility and broader adoption of the data and construction process.

  3. Positive empirical results. Vela achieves improvements on several user-related tasks and general benchmarks, demonstrating the value of the constructed data for LLM post-training.

Critical weaknesses
  1. The contribution of Future-Grounded Supervision is not sufficiently isolated. The proposed approach mainly consists of task construction, data filtering, teacher-generated trajectories, and standard SFT, with limited methodological novelty. While the experiments demonstrate that the overall pipeline is effective, they do not establish whether Future-Grounded Supervision provides additional benefits over conventional user-domain SFT. Therefore, the evidence is not yet sufficient to support presenting it as a new supervision paradigm.

  2. The independence and reliability of some evaluations need further validation. VelaBench is constructed using the same task templates as the training data. Although the training and evaluation users do not overlap, the model may still benefit from familiarity with the task formats. In addition, Qwen3.8-2.4T-A95B is used both to generate training trajectories and to evaluate some benchmarks, which may introduce evaluation bias. The authors should clarify the sources and independence of the user-related OOD benchmarks and provide validation with an independent evaluator.

Questions for authors
  1. What additional benefits does Future-Grounded Supervision provide over conventional user-domain SFT?

Could the authors include a controlled SFT baseline that does not use future-behavior feedback, while keeping the teacher model, training data size, and optimization settings the same? This would help establish the specific contribution of Future-Grounded Supervision.

  1. How much does each task type contribute to the performance gains?

The four task types (Objective, Open-prediction, Open, and Attribution) use different forms of supervision. Could the authors provide an ablation study to clarify the contribution of each type, particularly the role of actual future-behavior supervision?

  1. How are the independence and reliability of the user-related benchmarks ensured?

Are the four user-related OOD benchmarks existing public datasets or newly constructed by the authors? How is their independence from the training data ensured? Also, could the authors validate the main results using an evaluator different from the teacher model?

Reviewer_XqpfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces future-grounded supervision for improving LLM understanding of individual users. Vela converts longitudinal behavior records from 100,000 users into composite tasks using 91 templates, with future actions supplying targets for predictive questions. It filters tasks by student difficulty and selects teacher-generated trajectories for fine-tuning Qwen3.6-35B-A3B. The resulting model gains 11.5 points on VelaBench, 6.2 points on average across four user-related OOD benchmarks, and 4.0 points across five general benchmarks.

Critical strengths

The problem is relevant, and long-horizon real behavior logs are a distinctive supervision source.

Task construction is careful. For example, each future event gets either a Predictive or an Attribution question, never both, which avoids leakage. The composer and scorer prompts are fully disclosed.

The evaluation is broad, covering in-domain, user-related OOD, and general benchmarks, plus scaling and RL analyses.

Critical weaknesses

The gains are confounded with teacher distillation. All Vela-Instruct trajectories come from a much larger teacher, and future behavior acts only as an acceptance filter. Vela-RFT, which uses self-generated trajectories, shows 0.0 average gain on general benchmarks. This suggests the “broader intelligence” gains come from distillation rather than from user behavior data. A control that keeps the same teacher but removes future-grounded selection is needed.

The scorer and teacher are the same model. Qwen3.8 composes the tasks, generates the trajectories, scores most benchmarks, and provides the RL reward. Gains are large on LLM-scored benchmarks (ProfileSummary +10.1, AdRecommendation +11.2). They are near zero on PurchaseIntent (+0.2), the rule-based benchmark closest to actual future prediction.

Much of the signal is not future-grounded. 41 of the 91 templates are Open or Attribution questions scored by rubric. No per-question-type breakdown is given.

Questions for authors

How do results compare with a control that distills the same teacher on the same tasks without future-grounded filtering?

Do the gains hold with a scorer from a different model family, or under human evaluation on a subset?

1 comment
Reviewer_XqpfOpenReview ↗

Thank you for the detailed responses

Thank you for the detailed responses and for conducting the additional experiments. I appreciate the authors' efforts to address my concerns.

The evaluation with independent judges helps alleviate my concern about potential evaluation bias, and the additional ablation provides useful evidence that future-grounded selection contributes to the observed improvements beyond teacher distillation alone. However, the improvements attributable specifically to future-grounded selection remain relatively modest compared with the overall gains, leaving some uncertainty about its contribution beyond teacher distillation.

Overall, while the additional experiments and clarifications have addressed several of my concerns, I still have some reservations regarding the strength of the central claim. I therefore intend to maintain my original rating.

Reviewer_6hJVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Vela converts real user histories into composite prediction, summarization, and attribution tasks. Future behavior supplies predictive targets, while other tasks use evidence-grounded rubrics. The model is fine-tuned on approximately 75Kselected trajectories generated by a larger teacher. The paper reports considerable improvements on user-related and general benchmarks, alongside experiments on dataset scaling, difficulty filtering, and subsequent reinforcement learning.

Critical strengths
  • A practical way to use diverse user behaviours for language-model training
    The pipeline covers six behaviour categories and supports both candidate-based and open-ended prediction. Importantly, reference answers are extracted from recorded future actions using deterministic rules, and are not generated by the teacher model. Thus I think it can be a way for evaluating predictions and is a useful contribution to data construction.

  • Improvement on OOD tasks beyond behaviour prediction
    Training on Vela-Instruct supposedly improves GPQA-Diamond by 4.6 points and IFBench by 6.7 points (Table 1) which is a strong result. Both of these tasks use a rule-based evaluation, so the gains do not depend on the paper's rubric-based judge. Although the prsented experiments in the paper do not investigate the source of these improvements, they suggest that the proposed training approach can benefit other capabilities beyond behaviour prediction.

  • Efficiency in data size
    VelaBench reaches 61.8 with approximately 9.4K examples, compared with 51.5 for the base model and 63.0 with 75K examples (Figure 5). This suggests that much of the reported improvement is achieved using roughly one-eighth of the full dataset which is a useful finding.

Critical weaknesses
  • The experiments do not isolate the benefit of future grounding over vanilla teacher distillation.
    The core idea of Vela-Instruct is essentially a teacher-to-student trajectory distillation: a larger model generates candidate trajectories and the best-scoring one is selected if it outperforms the base-model reference. The student model is subsequently trained on the selected outputs. However, distillation and even reasoning traces distillation is already established. RLPF also uses supervision derived from future user activity. The core contribution is the specific data construction and selection strategy, and the benefits it provides. None of the ablations concretely test whether future-dependent selection is necessary, or whether similar gains can be obtained using history-only tasks or non-user long-context tasks using any form of traces generated by the same teacher. Further, the roughly size-matched comparison between Vela-RFT (54.2 general-benchmark average) and the 9.4K teacher-trained model (58.9) does not establish a match in task identities or training-token budgets (Table 1; Figure 5). Thus, the experiments support the overall training recipe but do not establish the incremental benefit of future grounding, which is central to the paper's proposed supervision paradigm.

  • The selection criterion and aggregate results do not establish whether future prediction improves.
    Each composite task is scored by averaging across its questions. This combines future-prediction scores with rubric-based scores for historical summaries and explanations of disclosed outcomes. Trajectories are then selected based on this combined score. Therefore, I worry whether a trajectory could pass the selection criterion by improving its rubric-based answers while not at all improving on its future predictions. The paper does not report accepted-trajectory quality by question type or separate evaluation gains for Predictive and Rubric questions. Hence, the 11.5-point improvement on VelaBench does not, by itself, demonstrate better future prediction. Per-family results and training ablations are needed to distinguish improvements in forecasting from those in historical summarisation and outcome-conditioned explanation, which the current aggregate results combine.

  • The strongest user-related gains need independent evaluation.
    Qwen3.8 generates teacher trajectories, scores and selects them, constructs rubrics, evaluates outputs, and provides RL rewards. Therefore, much of the evaluation relies on the same model that shapes the training targets and their selection. The paper reports no independent-judge or human validation. Prior work has demonstrated preference leakage between generators and judges. My concern is particularly relevant to the main user-related results. The judge-scored ProfileSummary and AdRecommendation improve by 10.1 and 11.2 points, respectively, while rule-scored PurchaseIntent and PersonaBench improve by 0.2 and 3.2 points (Table 1). Therefore, independent evaluation is needed to establish whether the largest gains hold beyond the model used throughout the pipeline. This concern does not undermine the rule-based general-benchmark gains which I appreciated in the strengths.

Questions for authors

Please check weaknesses section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.