acceptodds
Under review as a conference paper at ICLR 2027

Grammar-Weighted Futures: Modeling Human Expectation with Open Goals

16 views

Abstract

Behavioral and neural evidence suggests that human expectations in language and music are not driven by surface statistics alone: readers and listeners also anticipate how open goals will be completed, as a subject awaits its verb and a chord progression awaits its return to the home chord. We present Grammar- Weighted Futures, a model in which such anticipation constrains the probability of the next event. A fixed surface sequence model, which predicts events well without exposing open goals, samples continuations; a probabilistic grammar, which represents open goals explicitly but predicts events poorly, scores how well each continuation completes the goals of the observed prefix; and these scores reweight the surface prediction of the next event. Tested against listeners’ surprise ratings for chord progressions with Transformer, LSTM and variable-order surface models, our proposed grammar weighting improves the prediction of held-out ratings with all three. In language, an exploratory implementation improves the prediction of word-predictability ratings over GPT-2 alone. These results indicate that expectations about how open goals will be completed shape human expectation in abstract cognitive domains such as language and music beyond surface statistics.

Primary area:
applications to neuroscience & cognitive science
Keywords:
human expectation, language, music, hierarchical prediction, computational cognitive modeling, probabilistic grammars

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AHxFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

“Open goals” are defined as parts of a sequence that have not yet occurred but can be confidently predicted (and expected) by a human, given the context (for example, a chord progression going back to the root chord).

The paper asks whether explicitly accounting for open goals improves the modeling of human expectation.

The method consists of using a probabilistic grammar on top of a frozen sequence model (the surface model) to reweigh its sampled continuations, which combines two pre-existing techniques in a new way.

The resulting predictions are more aligned with human surprise ratings, even without improving the prediction accuracy.

Critical strengths

The paper addresses an interesting question and does so in a pretty rigorous and scientific way. The three hypotheses are well defined, the method builds on sensible baselines, and the experiments are well designed and properly executed.

Critical weaknesses

I find the paper interesting and overall well written, but unfortunately I believe that its contribution is not relevant or impactful enough for the ICLR community. The technique of reweighting a sequence model’s sampled continuations with an external scoring function is, as the authors point out, well known, and the same holds for using a grammar’s probabilities on top of a frozen sequence model (the baseline defined as “one-step”). The author’s addition to score sampled futures in terms of the open goals inferred from the prefix seems to be, unfortunately, of no use for modern neural sequence models, and I don’t see a clear insight that would advance the understanding of the community on the topics that are of interest for ICLR.

Reading the discussion, I seem to understand that most of what the method did was to increase the overall probability assigned to the root chord (which is almost always a non-surprising continuation). It is therefore pretty unsurprising that this would result in sequences less probable (-10%) but also less surprising to the listener.

Even though I think it is strongly out of scope for ICLR, I believe that this paper constitutes good research and I would encourage the authors to resubmit to a more appropriate conference, possibly in the fields of music or neural sciences.

Questions for authors

Are there any considerations that I did not understand that made me undervalue the impact and/or relevance of this result?

I can see some extension of this work feeding into the topic of making LMs generate more human-like text (hinted at in lines 439-444), but I think that was not elaborated enough to be considered a contribution of the paper.

2 comments

Thank you for recognizing the rigor of the study and the value of the research question. And thank you for your thoughtful question about relevance. We see this work as using deep learning to make a cognitive theory of anticipation computationally testable, rather than proposing a generative rescoring method. This places the work within ICLR's scope of applications to neuroscience and cognitive science. The question is how a learned sequence model's predictions interact with expectations about structures that remain to be completed.

For example, a listener may anticipate a return to the home chord without expecting it immediately. An intervening chord can feel appropriate because of the possible paths it opens toward that return. Our framework makes this account testable: the sequence model supplies plausible futures, while the grammar identifies open goals and evaluates their completion. Keeping the trained model fixed allows us to investigate how its learned knowledge is used to model human expectation.

This perspective also matters for interpreting the baselines. One-step already integrates possible completions of open goals through the grammar's next-event probabilities; its success therefore does not negate structural anticipation. Full condition tests the additional role of the sequence model's own futures. That contribution is model-dependent, but detectable for IDyOM and in the LSTM whole-song analysis. The improvement is reduced but remains positive after adding tonal controls in the auxiliary LSTM whole-song analysis.

The broader contribution is to turn a theory of structural anticipation into a computational question that can be tested using learned sequence models, while distinguishing what comes from explicit structural goals and what comes from the models' own possible futures.

Thank you for your reply. This doesn't substantially change my assessment towards acceptance, and I still believe that the applications and the scope of this work are too limited for a conference such as ICLR.

Reviewer_Qv9JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes propose using a grammar model to model human expectations beyond next-event probabilities. The probabilistic grammar identifies open goals for a surface sequence and evaluates how well sampled futures complete them. The score is used to reweighted the next-event probabilities. Experiments on music chord progression show the propose method improves predicting human surprise.

Critical strengths
  1. The research question and motivation s quite interesting. The paper is motivated by a psychology phenomenon that human tends to have a long-term expectation beyond a low-level surface prediction. The long-term structural goal assumption is a bit similar to planning in the world model, which suggests hierarchy thinking improves intelligence.
  2. Interesting results that better next-event prediction does not imply better human surprise.
  3. The grammar-model involved framework is intuitive.
Critical weaknesses
  1. The grammar-involved framework is very similar to conventional grammar-based rescoring or fusion methods, e.g., language model rescoring (n-gram) in ASR, HMM/CRF-based post-hoc in chord recognition. At least, they should be the baseline.
  2. The paper claims the contribution of a long-term goal expectation, but the on-step baseline already covers most of the improvement.
  3. For the chord progression experiment, both the surface model and the grammar model are trained on chord sequences, which means they have the same observation level. It is unclear whether the method still works when dealing with hierarchical structural knowledge. And the experimental results might simply reflect chord statistics learned by another rule-based model, making the improvement a result of model fusion.
  4. The extra tonal prior provided by the grammar model already explains part of the improvement.
  5. The next-chord prediction task is a bit toy. It is unclear how it goes with more complicated music task such symbolic music generation.
Questions for authors

See the weakness above.

1 comment

Thank you for the review.

  1. We agree that our method shares its inference framework with conventional rescoring and fusion, including the ASR and chord-recognition methods you mention. Reweighting candidate sequences is not a new operation. Our contribution concerns the relation being modeled and the target of evaluation. Our score asks whether a continuation completes the harmonic goals inferred from the heard prefix. We test the resulting next-chord probabilities against listeners' surprise ratings, that is, against what a listener expects before the next chord sounds. Comparing this goal-completion score with conventional statistical scores tests whether that relation helps explain human expectation.

    The HMM/CRF applications you mention concern chord recognition. Adapting them to predict from the heard prefix would provide relevant additional baselines. For this revision, we will focus on n-gram rescoring and grammar fusion, which allow us to compare scoring methods while retaining the same sequence model.

    We will follow your suggestion and compare goal-based scoring with n-gram rescoring of the same model-generated continuations. We will make this comparison for both the submitted model and the extension in point 2.

  2. We agree that One-step recovers most of the improvement for the neural models. The reason it remains relevant to our hypothesis is that it already integrates possible future completions. Our label One-step, which we may rename in a later draft, was a little misleading. This condition retains the sequence model's next-chord probabilities and corrects them using the grammar's integrated completions of the open goals. It is a condition built around our hypothesis that anticipated completions affect immediate expectation. The full model additionally samples the sequence model's own paths and scores how they complete those goals within a window.

    Whether those paths add value remains our central question. The submitted model scores completions within the window but does not credit compatible progress toward completions beyond it. We are testing an extension: It keeps the grammar fixed and also credits compatible unfinished paths, together with the grammar's probability of completing them later. A matched marginal applies the same boundary treatment without the sampled paths.

  3. We agree that both models observe the same chords, and that useful chord statistics from a second model could explain the gain. The structural claim concerns how those observations are represented: the grammar infers unfinished harmonic constituents and local tonicizations without supplied phrase or section annotations. This allows us to ask whether an inferred hierarchy helps predict human expectation from the same observations.

  4. Tonal preferences can account for part of the improvement. The question is whether expectations inferred from the prefix add anything beyond those preferences. All models already receive the song's key through key-relative chord symbols; the grammar adds preferences among chords and inferred harmonic structure. In the primary analysis, fixed chord preferences (a static grammar tilt and the unigram tilt) are not distinguished from the full model for the neural models. In the whole-song analysis reported in the appendix of the submission, we added tonic-root and dominant-root indicators and the static prior's surprisal to both response models. This reduced the LSTM gain over Baseline from +50.27 to +22.22 [4.77, 40.63]. These tonal covariates reduce the measured gain without eliminating it. We will revise the introduction, discussion and conclusion to distinguish the measured gain from its unresolved goal-based explanation.

  5. We agree that chord progressions provide a limited test of musical modeling, and we have not evaluated symbolic generation. We chose this task to test human expectation directly: the models' next-chord probabilities predict listeners' surprise ratings. This target shows something a prediction benchmark would miss. For the LSTM and Transformer, the full model makes the chords that occurred about 10% less probable while predicting held-out ratings better. Selecting by next-event likelihood and selecting by fit to listeners therefore favor different versions of the same network. The corrected next-chord distribution could support a future study of symbolic generation, but the present contribution concerns modeling listeners' expectations.

Reviewer_6Je1OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper assesses whether augmenting standard sequence prediction models (e.g., LSTMs, causal transformers) with information pertaining to higher-level structured goals (e.g., returning to the tonic chord in a musical piece, completing phrases in language) will improve modeling of human surprisal ratings across music and language domains. The work specifically produces a hybrid model that mixes the probability distributions of standard models with probability distributions supplied by a grammar with a gate function.

Overall, results are mixed. Their hybrid model beats many baselines comfortably, including the sequence-model only and ablations of the hybrid model. However, the hybrid model fails to beat the one-step baseline, indicating that the hybrid model is no better than marginalizing over the grammar. Similarly, the hybrid model fails a baseline that combines the sequence model with a static bias toward particular types of chords. Together, this suggests that the hybrid model contains far more machinery than is necessary to model the surprise ratings in the music domain --- it appears that much of the gain is due to a theoretically motivated bias toward goal-relevant continuations.

The language results are too underdeveloped to properly assess, and labeling the analysis as "exploratory" does not relieve the the responsibility of running enough experiments/baselines/comparisons to support scientific conclusions.

Critical strengths

The work attempts to address an interesting question: do structure-sensitive "goals" inform human surprise ratings over and above "shallow" next token/chord prediction? The resolution to this question would integrate or adjudicate between surprisal based accounts and structural accounts of expectation. The work also analyzes both music and language, which are two interesting (and meaningfully different!) domains that are both amenable to this sort of analysis.

The hybrid model also broadly makes sense, and several controls and baselines are logically presented and evaluated in the music analysis.

Critical weaknesses

First, the empirical results do not seem conclusive. The hybrid model neither outperforms the one-step nor static tilt baseline, and these two baselines appear to point in broadly opposite directions. The static tilt baseline provides a context independent push towards theoretically motivated categories, which questions the necessity of a complex grammar. The one-step baseline questions the necessity of the surface model predictions, rather than just marginalizing through the parser. Together, we are left without a clear understanding of what exactly is necessary for predicting surprise ratings.

Second, the language section is substantially under-analyzed. The work would clearly benefit from a greater set of LM evaluations, of datasets, etc. Additionally, calling even a small LM like GPT2 a "surface model" is likely underselling the syntactic and prospective prediction capabilities of these models. One can investigate whether the model itself maintains open goals using techniques like FutureLens, and attempt to understand how open goals vs immediate next-token statistical prediction contribute to an LMs output probabilities. Investigating neural architectures that learn (or learn to use) structured grammars, such as Recurrent Neural Network Grammars or Tree-Structured Neural Networks, would serve as invaluable alternative hybrid models.

Minor: 3.5 is a bit difficult to understand.

Questions for authors

Why do you choose the LSTM as the primary test? It appears that transformers and IDyOM are not less appropriate for analysis.

2 comments

Thank you for the review.

  1. Our central hypothesis is that expecting a later structural completion changes the expectation of the next event. For the LSTM, Transformer and IDyOM, a grammar-derived correction predicts listeners' surprise ratings better than the uncorrected sequence model (Baseline in the paper). The open question is which parts of the construction this result requires.

    Our label "One-step," which we may rename in the later draft, was a little misleading because it suggests a condition that sees only the next chord. It is a condition built around our hypothesis rather than just a local model. It keeps the sequence model's next-chord probabilities and corrects them using the grammar's integrated completions of the goals left open by the prefix. The submitted full model (Ours in the paper) adds one element: it samples the sequence model's own continuations and scores how they complete those goals within a window.

    This is why Static tilt and One-step are closer than they may appear. Both keep the sequence model and the gate, and both add a chord preference. In Static tilt the preference is fixed; in One-step it is derived from the prefix. In the primary analysis, the full model's advantage over One-step is uncertain for the LSTM, near zero for the Transformer and positive for IDyOM. Whether the sequence model's paths add value is the central question we continue to pursue.

    The submitted model scores completions within the window but does not credit compatible progress toward completions beyond it. We are testing an extension for music that keeps the grammar fixed and also credits unfinished paths using the grammar’s probability of completing those goals later. A matched marginal control applies the same treatment without the sequence model’s sampled paths.

  2. We agree that the language evaluation is limited, and that calling it exploratory does not replace comparisons across models and datasets. Our current evidence comes from one GPT-2/PCFG configuration and 205 rated sentences: the full model improves on GPT-2 alone (ΔELPD +5.08, 95% interval [1.19, 9.09]), but its additional predictive value beyond the released IAS score remains uncertain. We will make clear that the language results apply to GPT-2 on this dataset. Testing other models and datasets remains future work.

    We also agree that “surface model” can wrongly suggest that GPT-2 lacks syntactic or prospective capabilities. We use it to describe the model's role in providing the initial predictions, not an absence of structure. Our question is whether explicitly scoring goal completion adds value to a neural model that can itself learn structure. “Sequence model” better describes this role.

    Your FutureLens suggestion addresses whether GPT-2 already represents the open goals that our grammar makes explicit. Testing whether those representations influence next-word probabilities could indeed be a useful further step for us. RNNGs and incremental tree-structured networks would also provide relevant baselines because they incorporate learned structure directly into prediction. We have not run these comparisons. We will discuss them, as directions for extending the language evaluation.

  3. We agree that Section 3.5 needs a clearer explanation of how the language model and grammar work together. GPT-2 supplies possible continuations, and the grammar evaluates their syntactic structure through part-of-speech distributions. Many goals remain unfinished within a short continuation, so the language score gives compatible continuations a base score and increases it with the expected number of tracked goals completed. These scores are normalized and used to reweight next-word probabilities. The gate controls the strength of this correction according to how informative the first open goal is about the next syntactic category. We will clarify these steps and consider adding a worked example showing how they change a next-word prediction.

  4. We agree that the Transformer and IDyOM are equally appropriate tests. The LSTM became the primary test because we developed the method with it, including a comparison of gate forms on these ratings. We will present the three models together.

Thank you for your reply. Including these revisions to the prose will improve the manuscript. I will be maintaining my score.

Comments on the paper

1 comment

Thank you again for the discussion. We made the following revisions in the rebuttal draft.

  • Model comparisons. Section 4.3 presents all three models together and introduces “Grammar marginal”, previously called “One-step”. We now use the term "sequence model" instead of "surface model". Appendix C explains how we selected the gate during LSTM development.
  • Rescoring controls. Related Work explains the connection to conventional rescoring. Section 4.3 and Appendix E report the new controls, including the requested n-gram comparison.
  • Interpretation. The Introduction clarifies that both components start from the same chords and song key. The Discussion highlights that, with each neural network held fixed, grammar weighting improves prediction of listeners’ surprise even as it lowers the likelihood of the chords they heard.
  • Language. Section 3.5 clarifies the GPT-2 implementation. The Discussion states the scope of the language result and outlines follow-up tests.
  • Extension. We tested the extension that gives credit to compatible continuations that leave goals unfinished. For both LSTM and Transformer, it outperformed bigram rescoring, but its corresponding grammar marginal retained most of the gain in rating prediction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.