acceptodds
Under review as a conference paper at ICLR 2027

GLARE: Generative Learning via Adversarial Reward Estimation For Social Dynamics Forecasting

Abstract

Understanding human communication is an important aspect of social reasoning. Meeting continuation requires tracking the agenda, speaker roles, participant intentions, and disagreement across long multi-party discussions. We introduce the Meeting Dynamic Forecasting Benchmark (MDFB), constructed from 2,207 real-world meetings and 24,794 future-facing queries. Given a transcript prefix and an active question, a model generates a plausible multi-turn continuation in one call. We evaluate utility—progress toward the question—and human-likeness—plausible conversational flow and role consistency—without requiring exact reproduction of the observed future. We further present GLARE, an adaptation of adversarial imitation learning to conditional language generation. A discriminator ranks the observed continuation above samples from the current actor, and its score supplies a KL-regularized policy reward; retraining on current-policy negatives allows the reward landscape to evolve with the actor. GLARE attains average human-evaluated win rates of 0.66 on utility and 0.70 on human-likeness, outperforming SFT and SPIN while remaining below the observed human continuation. We also demonstrate MDFB as a social reasoning arena for comparing general-purpose models, including closed-source systems, through reference-assisted judgments. Together, these studies illustrate the benchmark's use for both task-specific learning and output-based evaluation of meeting behavior.

Primary area:
reinforcement learning
Keywords:
Human-centric AI, Reward Modeling, Post-trainning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_hcPMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces meeting continuation forecasting and the Meeting Dynamic Forecasting Benchmark (MDFB), comprising 24,794 queries derived from 2,207 real-world meetings. It proposes GLARE, an adaptation of adversarial imitation learning where a discriminator ranks observed human continuations above samples from the current policy, and an actor optimizes the resulting reward with KL regularization. The experiments compare GLARE against SFT and other baselines, with reference-assisted human evaluation reporting direct GLARE–SFT win rates of 0.62 for utility and 0.69 for human-likeness.

Critical strengths
  1. The focus on naturally occurring meetings provides a highly relevant and challenging setting for studying multi-speaker continuation. MDFB represents a substantial and valuable resource for this task. Furthermore, the strict splitting at the source-meeting level ensures a clean separation between training and evaluation, giving the benchmark lasting value independent of the specific training method proposed.

  2. The human evaluation provides credible, grounded evidence that goes beyond learned reward metrics or model-based judging. The evaluation protocol is well-structured. Crucially, the direct comparison with the SFT baseline offers convincing evidence of improved perceived utility and human-likeness within the reference-assisted evaluation framework.

Critical weaknesses
  1. The core training procedure and attribution of gains are insufficiently specified. While Appendix A.4 provides details on the SFT initialization and discriminator architecture, key hyperparameters for the actor update are missing. Specifically, the concrete actor-update algorithm, the numerical KL regularization coefficient, the candidate-set size, the actor learning rate and update schedule, and the exact SPIN tuning configuration are not clearly stated. These details are highly consequential, especially given that the SPIN baseline appears to substantially underperform SFT in Figure 6. Although the paper discusses unsuccessful fixed-reward/PPO/DPO attempts and provides frozen-discriminator diagnostics, this does not cleanly isolate the benefit of the refreshing mechanism. To properly attribute the gains, a comparison of final output quality is needed under strictly matched actor optimization and training budgets.

  2. The significance analysis incorrectly treats repeated ratings as independent observations. Each system pair is evaluated on 50 instances by three raters, yet the analysis in Figure 6 (and related text) applies exact binomial tests to all 150 annotations under an independence assumption. In reality, these ratings share instance content and inherent difficulty, and queries may also share source meetings. While the point estimates of the win rates remain informative, these nominal statistical tests do not accurately establish uncertainty over new instances or meetings. Grouped confidence intervals (e.g., bootstrapping at the instance or meeting level) are necessary to assess the robustness of the human-evaluation gains.

Questions for authors
  1. Clarification and focused validation: Can you provide the missing actor/SPIN configurations and a frozen-versus-refreshed discriminator comparison using the same actor optimizer, training budget, and final output-quality evaluation?

  2. Focused reanalysis: What are the direct GLARE–SFT confidence intervals when ratings are grouped by instance and, where applicable, source meeting? Please clarify how instances and raters are reused across system pairs.

  3. The paper reports unsuccessful fixed-reward training attempts and frozen-discriminator diagnostics. Under the same actor optimizer, training budget, and checkpoint-selection procedure, how do frozen and refreshed discriminators compare on the reported final output-quality measures? This would isolate the contribution of reward adaptation from differences in optimization.

Reviewer_s3HPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces the Meeting Dynamic Forecasting Benchmark (MDFB) and GLARE (Generative Learning via Adversarial Reward Estimation) for generating plausible multi-turn continuations of real-world meetings. MDFB contains 2,207 meetings and 24,794 future-facing queries built through an LLM-assisted pipeline. GLARE adapts adversarial imitation learning: a discriminator ranks the observed continuation above current-policy samples, and its score is used as a reward for KL-regularized policy optimization. Human evaluations show improvements over SFT, vanilla Qwen, and SPIN in utility and human-likeness, and the paper also uses MDFB as a "social reasoning arena" to compare frontier models with LLM judges. The benchmark is a promising contribution, while the methodological novelty and the robustness of the evaluation remain open concerns.

Critical strengths
  1. MDFB offers a challenging setting with long-horizon, multi-party meetings from several corpora, potentially useful for studying socially grounded generation (Section 2). Its diversity is somewhat limited, since MeetingBank city-council meetings provide about half of the queries.
  2. The task acknowledges that multiple futures can be plausible and evaluates agenda progression and conversational coherence rather than exact match (Sections 2.1-2.2).
  3. GLARE's average utility and human-likeness win rates are 0.66 and 0.70, compared with SFT's 0.58 and 0.55, and direct GLARE-vs-SFT comparisons favor GLARE at 0.62 and 0.69 (Section 5.1, Appendix E). A machine judge agrees in direction (Appendix F).
  4. GLARE learns from observed continuations and current-policy negatives, and the authors are transparent about what is and is not novel (Sections 1, 3, Appendix C).
Critical weaknesses
  1. The authors explicitly do not claim the actor-discriminator paradigm as new. The remaining adaptation, a multi-candidate InfoNCE-style discriminator over whole sequences, is close to adversarial dialogue generation (Li et al., 2017) and to SPIN. The contribution therefore rests mainly on the benchmark, and the method must be justified by strong evidence, which is limited (Weaknesses 4-5).

  2. Judges see the LLM-generated "overall direction" of the observed future, and the human-likeness prompt explicitly asks them to prefer responses aligned with it (Figure 10, Appendix H). Because GLARE's discriminator is trained to score the observed continuation highest, this protocol structurally favors GLARE over baselines not trained toward the realized trajectory. The results therefore do not isolate conversational realism, and the "human-likeness" label overstates what is measured.

  3. Queries are generated from the full transcript, including the future, and then rewritten in future tense. The audit's "insufficient" category (16 of 100) is defined as queries that introduce issues or entities absent from the prefix, which suggests queries can leak future content. The audit covers only 100 validation instances, checks only prefix sufficiency (not the correctness of queries, spans, or references), and does not report the number of annotators or their agreement (Section 2.3, Appendices A, B).

  4. Only one actor-discriminator configuration is trained. The frozen-discriminator and fixed-reward alternatives are described only as failing to converge, with a reward-margin plot (Figure 8) rather than end-task comparisons. There is no analysis of output length or diversity, although adversarial training may reduce diversity in a task framed around multiple plausible futures. SPIN hyperparameters and SFT checkpoint selection are not reported.

  5. Each system pair uses 50 instances and three evaluators, and the significance tests treat the 150 annotations as independent although they share instances (Figure 6). Evaluator expertise and recruitment are not described. The actor optimization algorithm, KL coefficient, candidate-set size, and number of iterations are not specified (Appendix A.4). The social-reasoning arena includes self-judging, its rankings vary by judge, and it reports no number of prompts, confidence intervals, or human agreement (Section 5.2).

Questions for authors
  1. Can you report human evaluations without the observed direction, or with a direction-neutral human-likeness rubric, and show whether GLARE's gains persist?
  2. How often do future-facing queries reveal information absent from the prefix? Can you annotate the evaluated test instances for sufficiency and leakage and report results on the clean subset?
  3. Can you provide end-task comparisons against a frozen discriminator and other configurations, plus output length and diversity statistics?
  4. Can you report confidence intervals or tests that account for repeated judgments on the same instances, and results across multiple seeds?
  5. Can you specify the actor optimization algorithm, KL coefficient, candidate-set size, iteration count, and SPIN settings, and describe the human evaluators?
Reviewer_M8v9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces the Meeting Dynamic Forecasting Benchmark (MDFB) and Generative Learning via Adversarial Reward Estimation (GLARE).

MDFB is constructed from 2,207 real-world meetings and 24,794 future-facing queries. The paper formulates a task in which a model generates a multi-turn dialogue continuation conditioned on a preceding conversation history and an active question. The generated continuations are evaluated along two dimensions: utility and human-likeness.

GLARE employs an adversarial reward-learning approach in which a discriminator learns to rank observed human continuations above continuations generated by the current policy. The discriminator's output is then used as a reward signal to optimize the generator. The experiments use Qwen2.5-7B as the actor and Qwen2.5-3B as the discriminator. The reported results show that GLARE outperforms the SFT, vanilla Qwen, and SPIN baselines on the paper's main human-evaluation metrics.

Critical strengths
  1. MDFB combines multiple meeting corpora and introduces future-facing queries to study multi-turn dialogue continuation in meeting settings. The task is practically relevant and the dataset could be a useful resource for related research.

  2. The idea of GLARE, training the discriminator against negative continuations sampled from the current policy is reasonable.

  3. The reported human-evaluation results indicate that GLARE improves over the SFT baseline in both utility and human-likeness. This is encouraging given GLARE does not specifically use utility and human-likeness in the discriminator score.

Critical weaknesses
  1. A central challenge in this task is that multiple substantially different continuations may all be plausible and useful. However, GLARE trains its discriminator to prefer the single observed human continuation over policy-generated alternatives. Although this objective is understandable given the available supervision, it may favor reference-specific characteristics rather than capture the broader set of plausible futures. It is useful to report the rank correlations between discriminator scores and independent human judgments.

  2. Section 5.1 uses 50 test instances per system pair, with three evaluators. The paper would benefit from human evaluation on a larger and more diverse test set.

  3. It reports that 84 out of 100 validation instances have at least partially sufficient context. This is a useful preliminary diagnostic, but it does not establish that the benchmark is sufficiently well-defined, or at least the test set.

  4. The main experiments compare GLARE with SFT, vanilla Qwen (which Qwen?), and SPIN. The paper notes that PPO and DPO experiments did not converge reliably. This is understandable as a practical limitation, but it leaves an important question unresolved: does GLARE offer a substantive advantage over established preference-optimization approaches?

Questions for authors

Please see the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.