acceptodds
Under review as a conference paper at ICLR 2027

Teacher Model-Guided Bayesian Decision Support for Structured Choice Prediction in JEV

99 views

Abstract

Structured choice prediction is central to decision support systems in economics, transportation, and recommendation. Classical discrete choice models (DCM) provide interpretable utilities and well-defined likelihoods, but they often rely on handcrafted attributes that miss semantic context. Recent large language models (LLMs) can provide low-cost preference signals from textual descriptions, yet such signals are noisy, miscalibrated, and difficult to use directly as decision rules. In this paper, we propose TG-BC, a Teacher LLM-Guided Bayesian decision support framework for structured Choice prediction. The framework treats the LLM as an teacher se- mantic scorer and integrates its signal into a Bayesian dis- crete choice model through calibrated utility guidance and lightweight latent steering. Posterior inference then produces calibrated choice probabilities, credible intervals, and action- level uncertainty for downstream policy simulation. We in- stantiate the framework in an urban mobility mode-choice setting, where a decision maker compares interventions such as fare discounts, service-frequency improvements, parking surcharges, and informational nudges. The experimental pro- tocol evaluates prediction, calibration, decision quality, in- terpretability, and robustness against DCM-only, LLM-only, naive-fusion, and deterministic-steering baselines

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Discrete Choice Model

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_xiXCOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper attempts to incorporate an LLM to amplify the structured attributes with context-aware semantic terms in discrete choice modelling. The author proposes a teacher-informed posterior to balance the weight between conventional observed utility and additional information brought by the LLM. They applied on four travel mode choice dataset and claimed their method has the best fitness performance on test dataset.

Critical strengths

A promising idea is to consider both the score and its uncertainty of the LLM when modelling the weight of the LLM's contribution.

Critical weaknesses
  1. The introduction is hard to follow, with inaccurate statements and disrupted writing flow. For example, the key concept "teaching model" is never introduced; Figure 1's title and descriptions are inconsistent. Meanwhile, the figure introduces many concepts and terms without prior definition at that point. In the related work, I can't find a discussion of how the proposed TG-BC differs from the literature.

  2. Given the unclear statements in the research gaps, I feel the contribution here is overstated and less persuasive.

  3. Misuse of terminology. I am concerned that the authors’ use of "context-dependent" and "context effects" differs from their established meanings in the discrete choice modelling literature [10.1016/j.jmp.2023.102790]. Here, these terms seem to refer to contextual information omitted from the specified attributes, rather than to effects arising from the composition of the choice set. "context-aware" is more appropriate.

  4. The methodology part is problematic in its framing. The teacher-informed posterior seems unable to identify ρ\rho. The log-likelihood is decreasing as ρ\rho increases. If the sample size is large, ρ\rho shrinks to 0. That means no contribution is made by the LLM teacher model based on its definition. Unclear how the LLM generates semantic signals zz. What are the dimensions of zz and γ\gamma?

  5. Results are overstated.

  • Only fitness metrics are compared without considering the complexity of the model.
  • The counterfactual experiment setting is unclear.
  • No LLM specification or its prompt on semantic terms is reported.
Questions for authors
  1. Would the authors please clearly identify the research gap with literature evidence that motivates this study?
  2. Would you report the parameter recovery results of your proposed teacher-informed posterior when ρ\rho has non-zero values? Also, please report the estimated ρ\rho value on four empirical datasets.
  3. How to distinguish the interpretation of ρ\rho and γ\gamma?

Unrelated to my final judgement but worth noting here, i) the title here is different from that in the manuscript; ii) the manuscript isn't following the ICLR template; and iii) the in-line citations aren't aligned with academic paper format.

Reviewer_7KjFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Summary

This paper proposes TG-BC, a framework that integrates semantic preference signals from an LLM teacher with a Bayesian discrete choice model (DCM). The framework uses entropy-based confidence weighting and Bayesian posterior inference to dynamically adjust the influence of the teacher signals. The authors conduct experiments on four travel choice datasets and claim that the proposed method improves predictive accuracy, probability calibration, and downstream decision quality, while demonstrating some robustness to teacher bias.

Critical strengths

The paper addresses a practically relevant problem: how to incorporate semantic information provided by LLMs into traditional DCMs while preserving model interpretability and uncertainty quantification capabilities.

Critical weaknesses

Weaknesses

W1. Limited methodological novelty. The proposed framework appears to be primarily a combination of existing components. DCMs, LLM teacher guidance, and entropy-based weighting are not new techniques, and the overall methodological novelty seems limited. The authors also do not sufficiently clarify what fundamentally distinguishes their approach from existing adaptive fusion or uncertainty-aware distillation methods.

W2. Questionable assumption regarding teacher confidence and reliability. Normalized entropy measures the concentration of the teacher's predictive distribution, which does not necessarily reflect its reliability. A teacher with systematic bias can still produce highly confident but incorrect predictions. However, the paper implicitly equates confidence with reliability and uses this connection as a core motivation for the proposed method.

W3. Limited generalizability of the experimental evaluation. All experiments are conducted on transportation choice tasks, which limits the generalizability of the findings. The authors should either narrow their claims accordingly or provide additional evaluations across different populations and tasks.

W4. Insufficient explanation of Latent Steering. The Latent Steering component is not clearly explained, which affects both the reproducibility of the method and the reader's understanding. Given that the authors conduct a separate ablation study for this component, it is unclear why it is not sufficiently defined in the formulation of the complete method.

Questions for authors

N/A

Reviewer_GrDGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes integrating an LLM with a classical discrete choice modeling framework (MNL). The core idea is that LLMs are good for processing context, and therefore can capture nuances (e.g. heterogeneity) that a classical statistical model cannot. The paper uses an LLM to provide a "teacher" specialized in a mode-choice modeling context (based on a description of the choice problem, characteristics of the options, etc.). This teacher provides a probability vector corresponding to the choice probabilities. The (bayesian DCM) student then uses these probabilities, weighed by confidence in the teacher (dependent on the entropy properties of the generated probability vector), as a "prior" for the inference of the final ("posterior") choice probabilties.

The experiments are based on classical DCM open datasets (e.g. swissmetro), and show some benefits of the approach.

Critical strengths

The idea of using LLMs to incorporate new contextual knowledge in a choice problem makes a lot of sense. The general framework is sound, the evaluation metrics are well-chosen. The use of open datasets, together a classical model (MNL) and a well-debated problem (Mode choice), makes this a very reproducible research.

Critical weaknesses

Main concerns:

  • "...so a description that never becomes a column can never enter the utility." From a representation learning perspective, (any) context can be mapped into a “column” (in practice, an embedding vector). Therefore, it’s unclear to me what the benefit of this approach is. In fact, one can use an LLM precisely to create those embeddings in a dynamic way, and incorporate them in an MNL, similarly to other existing research (e.g. Sifringer paper, Han paper)…

  • LLM prompting has virtually infinite degrees of freedom, which complicates the evaluation process.

  • I do not know all these datasets, but if all are open, it is quite questionable whether the LLM has had access to them during training (i.e., data contamination). Can the authors verify this? If this were true, then the whole experiment would be compromised.

  • Baselines: I think we are missing a model that integrates representation learning embeddings (e.g. Sifringer’s) and Bayesian Model averaging. The naive approach is too weak a baseline.

I actually like this idea, but I'm not convinced that using an LLM truly adds new information, except for the (training) knowledge of the LLM itself (for which we have no control or even a clear understanding). Wouldn't it be much more useful to use an LLM on genuinely new information (e.g., POIs, dynamic trip context, such as weather, events, etc.)?

Clarity:

  • In the introduction, the concept of “teacher” appears abruptly (actually, not introduced at all!). What are we talking about? The model architecture is very unclear in the beginning.
  • Motivation figure (Fig 1) is supposed to support the preceding paragraph, but there is no clear connection between the two, except for the first couple of sentences in that paragraph…
  • The research question is really long! It took me a while to absorb it...
  • “First, we define a teacher semantic signal z^T_{nj} for utility guidance.” very fuzzy sentence. One is forced to find it in the pseudo code to understand before following
  • “and recommends the action with the largest expected value.” Why using argmax and not sampling? Treating DCM as a deterministic inference problem is classically wrong and defeats the purpose of the Bayesian framework. Later in the experiments, I understood that the full posterior distribution is, in fact, evaluated. Then why this sentence?
Questions for authors

See my comments above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.