acceptodds
Under review as a conference paper at ICLR 2027

Learning Dynamic Belief Graphs for Theory-of-mind Reasoning

Abstract

Theory of Mind (ToM) reasoning in highly uncertain environments often involves complex, dynamic, and interdependent belief evolution that jointly shape the process of information seeking and sequential decision-making – especially in high-stakes settings such as disaster response, emergency medicine, and human-in-the-loop autonomy. However, prior approaches for ToM reasoning in LLMs often treat beliefs as static and independent, producing incoherent mental models over time and weak reasoning in dynamic contexts. We introduce a structured cognitive trajectory model for LLM-based ToM that represents mental state as a dynamic belief graph, jointly inferring latent beliefs, learning their time-varying dependencies, and linking belief evolution to information seeking and decisions. Our model contributes (i) a novel projection from textualized probabilistic statements to consistent probabilistic graphical model updates, (ii) an energy-based factor graph representation of belief interdependencies, and (iii) an ELBO-based objective that captures belief accumulation and delayed decisions. Across multiple real-world disaster evacuation datasets, our model significantly improves action prediction and recovers interpretable belief trajectories consistent with human reasoning, providing a principled module to augment LLMs using ToM reasoning with dynamic and interdependent beliefs in highly uncertain environment.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Theory of Mind, Latent Belief Modeling, Cognitive Modeling, Variational Inference, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_nvj8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a structured Theory-of-Mind framework that models human mental states as dynamic belief graphs to capture multiple interdependent beliefs and their temporal evolution. It combines LLM-derived semantic embeddings with an energy-based factor graph to model belief dependencies, employs variational learning with an ELBO objective to infer latent beliefs from observation-action trajectories without belief-level supervision, and uses action-specific self-attention for action prediction. Experiments on real-world wildfire evacuation datasets and standard ToM benchmarks demonstrate improvements in action prediction and alignment with human-reported belief states.

Critical strengths

The paper addresses a practically important problem: human decisions are influenced by multiple interdependent beliefs that evolve with new observations. The proposed framework provides a reasonable connection between this motivation and its technical components, using factor graphs for belief dependencies, temporal transitions for belief evolution, and variational learning to address the lack of belief-level supervision.

Critical weaknesses
  1. The paper emphasizes learning time-varying belief interactions, but the pairwise potential in Eq. (5) is computed from LLM embeddings of belief pairs without explicitly conditioning on current observations or previous belief states. It is therefore unclear whether the model learns genuinely dynamic interaction strengths or primarily captures time-varying marginal beliefs through the unary potentials.

  2. Although the factor graph models a joint distribution over interdependent beliefs, the action-specific self-attention module in Eq. (6) receives only marginal belief probabilities. Different joint distributions can share identical marginals, potentially losing dependency information relevant to decisions.

Questions for authors
  1. How are time-varying belief dependencies explicitly modeled? In Eq. (5), the pairwise potential appears to depend only on the semantic representations of belief pairs. Could the authors clarify how these interactions vary with observations and previous belief states, and distinguish changes in pairwise potentials from changes in the resulting joint belief distribution?
  2. Why does the action prediction module use only marginal belief probabilities rather than the joint belief distribution? Since the factor graph explicitly models belief dependencies, marginalization may discard information about joint configurations. Could the authors explain this design choice and evaluate whether incorporating joint belief conf?
2 comments

We thank the reviewer for the thoughtful questions and constructive feedback. We address each point below and will incorporate the corresponding clarifications into the revised manuscript.

For weakness 1. We would like to clarify that our belief interaction modeling explicitly considers observations, and the pairwise interaction strengths change over time as new observations arrive. In our survey setup, each time step corresponds to a new observation, such as observing nearby fire cues, receiving an official evacuation warning, or receiving a warning from a familiar person. Specifically, in Eq. (5), the current observation oto_t and previous belief state bt−1b_{t-1} are the conditioning inputs defined in Eq. (1), while the pairwise potential ϕij\phi_{ij} corresponds to the belief interaction term in Eq. (2). These equations define how the current observations and previous belief states are incorporated into the belief distribution, together with the pairwise interactions. We will update the explanation of Eq. (5) to explicitly clarify how these terms are connected and where the conditioning takes place.

We further provide empirical evidence that the pairwise interaction strengths ϕij\phi_{ij} change over time as new observations arrive. For example, when residents receive an official warning after observing nearby fire cues, the interaction between home damage and death of others increases by 14.35%, while the interaction between personal injury and personal death decreases by 11.74%. The interaction between home damage and neighborhood damage also increases by 11.21%. This suggests that official warnings strengthen the connections between property-related and broader community risks, while the relationship between personal injury and death becomes less pronounced. After receiving a warning from a familiar person, these interactions change again by −2.22%, +1.78%, and −0.89%, respectively, suggesting that additional social information further adjusts these belief relationships, although the changes are smaller than those following official warnings.

Belief relationshipNearby fire cues (t0 initial phi_ij)Fire cues → Official warning (t0 → t1)Official warning → Familiar-person warning (t1 → t2)
Home damage ↔ Death of others1.61614.35%-2.22%
Personal injury ↔ Personal death1.484-11.74%1.78%
Home damage ↔ Neighborhood damage1.58611.21%-0.89%
Home damage ↔ Injury to others1.5989.33%-1.53%
Personal injury ↔ Injury to others1.456-7.87%0.05%

For weakness 2. In our action predictor, the action-specific self-attention mechanism captures how multiple beliefs jointly influence actions by learning the dependencies among individual beliefs' contributions to action prediction. As described in Section 3.2 (Eq. 6), our action-specific self-attention mechanism learns how the contribution of one belief to an action depends on the activation of other beliefs. Specifically, the attention weight At,j[i,k]A_{t,j}[i,k] captures how belief kk influences the contribution of belief ii when predicting action jj. By learning these interdependencies directly through self-attention, the action predictor can capture how beliefs jointly influence decisions using marginal belief probabilities, without evaluating all 2K2^K configurations of the full joint belief distribution, substantially reducing computational costs. We will further explain this design choice of action predictor in Section 3.2 (Action Generation), explaining why we use marginal belief probabilities with self-attention instead of the full joint distribution for action prediction.

For question 1. Please see our response to Weakness 1, where we address this concern in detail. We will update Section 3.2 (Action Generation)

For question 2. Please see our response to Weakness 2, where we address this concern in detail.

Thank you to the author for the clarification. These explanations have made some technical details of the paper clearer, and I believe it is a standard paper; therefore, I tend to maintain my score(3).

Reviewer_doCdOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a structured cognitive trajectory model for LLM-based ToM that represents mental state as a dynamic belief graph, jointly inferring latent beliefs, learning their time-varying dependencies, and linking belief evolution to information seeking and decisions. According to the experimental results, this method improves action prediction and recovers interpretable belief trajectories consistent with human reasoning across multiple real-world disaster evacuation datasets.

Critical strengths
  1. This paper identifies the limitations of "explicit belief supervision," "independence/mutual exclusivity assumptions," and "static single-step" approaches in LLM-based Theory of Mind (ToM). It proposes a dynamic multi-belief graph as an alternative, which is a direction that holds both theoretical significance and practical application value.
  2. By using LLM-generated semantic embeddings as evidence—rather than directly outputting belief probabilities—and projecting them into energy-based models and factor graphs, this approach avoids the inherent fragility of pure prompting. Furthermore, it models belief evolution through the lens of Deep Markov models, offering a more systematic structural framework than inference-time prompting methods.
  3. In experiments, it uses post-disaster survey data, which encompasses multi-step, multi-belief, and authentic decision-making trajectories. It is more representative of real-world decision making scenarios than synthetic grid-world environments or single-step Theory of Mind QA tasks.
Critical weaknesses
  1. Experiments supporting the "dynamic multi-belief" claim are almost exclusively focused on a single dataset involving wildfire evacuations. On standard Theory of Mind benchmarks, the proposed model shows only marginal advantages. Although the authors assert in their conclusion that the approach can be extended to reasoning in domains such as healthcare and law, they provide no experimental evidence to support this.
  2. Training does not utilize belief labels. However, the evaluation employs self-reported beliefs—collected via surveys—as a proxy ground truth. A potential issue arises: do the prompts used for extracting LLM embeddings contain the belief statements, specific question phrasings, or relevant free-text descriptions found in the surveys? If they do, the model might simply "read out" information from its semantic embeddings that is highly correlated with the evaluation targets, rather than genuinely inferring latent beliefs from behavioral trajectories. The paper fails to adequately rule out this form of evaluation leakage.
  3. During training, the posterior has access to the action. However, during testing, one must infer the belief solely from the prior and observations before predicting the action. There might be a situation that gives rise to exposure bias.
Questions for authors
  1. Were the self-reported beliefs from the evaluation survey used during training, hyperparameter selection, prompt design, or the extraction of LLM embeddings? Did the prompts used for extracting LLM embeddings contain the belief statements or the original survey questions? How was label leakage rigorously prevented?
  2. Please clarify the specific meanings of "Inter" and "Final" in Table 1.
  3. During ELBO training, where exactly does bt−1b_{t-1} in the KL term come from? How is b0b_0 initialized during testing? Is any information regarding the "true" belief utilized?
  4. How can a belief score comparable to a 1–5 Likert scale be derived from a binary latent belief P(bt,i=1)P(b_{t,i} = 1)?
  5. Can the "belief intervention" experiment be further validated using human counterfactual data or independent behavioral data?
4 comments

We thank the reviewer for the thoughtful questions and constructive feedback. We address each point below and will incorporate the corresponding clarifications into the revised manuscript.

For Weakness 1. Collecting real human data that tracks multiple beliefs over time is extremely difficult. To our knowledge, there are no other existing open-access datasets that record how multiple human beliefs evolve across time steps during real-world decision-making, apart from the Kincade Fire dataset [1] and Marshall Fire dataset [2] used in this paper. Both studies required extensive efforts to recruit people affected by real disasters and collect detailed survey responses about their observations, changing beliefs, and decisions at different stages of the events, unlike many existing ToM datasets that rely on synthetic scenarios or LLM-generated data. Our improvements on standard ToM benchmarks are also substantial, rather than marginal. As shown in Table 2, our model improves action accuracy from 0.446 to 0.991 over AutoToM on SimpleToM (Qwen3-8B), and belief accuracy from 0.695 to 0.949 on BigToM (Qwen3-8B).

Finally, healthcare and law are mentioned as possible directions for future work, not as domains where we claim to have already validated the model. We will revise the conclusion to explicitly present these domains as potential future applications rather than experimentally supported findings.

For Weakness 2. There is no evaluation leakage in our setup. We use the original belief statements from the survey without rephrasing or adding any descriptions, as shown in Appendix K, Table 12, and Appendix L, Table 13. These statements only specify which beliefs the model should infer, without providing their ground-truth ratings. More importantly, we do not use any free-text responses that could reveal participants' beliefs. The observation inputs come exclusively from structured multiple-choice survey questions (Appendix K, Table 12), where participants select from predefined options about factual events, such as whether they observed flames or smoke or received evacuation warnings. The LLM prompts contain neither the participants' self-reported belief ratings nor any additional descriptions of their beliefs. We will add these clarifications to the revised manuscript.

For weakness 3. This is a standard setting in variational latent-variable modeling, rather than an unaddressed exposure bias [3,4,5]. During training, the inference posterior uses both observations and observed actions to infer latent beliefs, while the generative prior learns to infer beliefs using only observations and previous belief states. The KL regularization in Eq. (9) minimizes the discrepancy between the two distributions, allowing the generative prior to learn action-relevant belief information from the posterior without directly observing actions. At test time, we exclusively use the learned generative prior to infer beliefs from observations and previous belief states, without access to ground-truth actions.

For question 1. No self-reported belief ratings were used during training, hyperparameter selection, prompt design, or LLM embedding extraction. The prompts contain the original, pre-fixed belief statements from the survey, which are identical for all participants, as shown in Appendix K, Table 12, and Appendix L, Table 13. They did not include participants' responses or ground-truth belief ratings. There is no label leakage in our setup. Belief ratings are used exclusively for evaluation. The model inputs contain only observations from structured multiple-choice survey questions and observed actions, without any free-text responses that could reveal participants' beliefs.

For question 2. “Inter” refers to intermediate action prediction (4-way classification), including (1) preparing and waiting, (2) acting immediately (e.g., evacuating, defending property, or taking shelter), (3) seeking additional information, and (4) no reaction. “Final” refers to final evacuation prediction (binary classification): evacuate or stay. These action categories are described in Section 4.2 and Appendix G. We will also clarify them in the caption of Table 1.

For question 3. During ELBO training, bt−1b_{t-1} comes from the model's prediction at the previous time step, based on the environmental observations received so far. It is not taken from the participants' survey-reported beliefs. During testing, b0b_0 is initialized using the learned model's own prediction, and subsequent beliefs are updated sequentially through the generative prior. No ground-truth belief information is used for initialization, training, or inference. The self-reported belief ratings from the surveys are used exclusively for evaluation. We will clarify this in Section 3.4 Variational Learning Objective.

For question 4. In our model, while the belief​ is defined as binary (indicating whether a proposition holds), it will infer a continuous subjective probability in [0,1] that represents how strongly that proposition is believed to be true (e.g, "My house will be damaged"). The 1–5 Likert ratings collected in the wildfire surveys provide ordinal measurements of the same underlying subjective belief strength (e.g., "How much did you believe your house would be damaged on a scale of 1 to 5?"). This interpretation is consistent with behavioral decision theory, where beliefs are represented as degrees of confidence in propositions [6–8]. The predicted probabilities in [0,1] can therefore be directly compared with the 1–5 Likert ratings using Spearman correlation. Since Spearman correlation is rank-based, it does not require the two measures to share the same numerical scale or assume a linear relationship. It measures whether higher predicted belief probabilities correspond to higher self-reported belief ratings.

For question 5. Validating belief interventions using human counterfactual data is practically infeasible. It would require reproducing the similar scenarios (like similar wildfire events), individual's psychological state, observations, and environmental conditions during a real disaster, while changing only one belief and observing how their decision changes. However, our intervention results show patterns consistent with existing psychological and behavioral studies. For example, reducing the belief in personal injury from above 0.8 to 0 decreases the probability of acting immediately by 35.22 percentage points and increases preparing and waiting by 31.84 points. This aligns with [9], who found that differences in risk attitudes help explain why some residents act early while others wait before deciding. Similarly, increasing the belief in home damage from below 0.2 to 1 increases the probability of preparing and waiting by 18.44 points and evacuating by 11.66 points. This is consistent with [10], who found that perceived safety and risk perception are important predictors of wildfire evacuation decisions.

BeliefInterventionPrepare and waitAct immediatelySeek/wait for informationNo reactionEvacuateStay
Home damage<0.2 → 1+18.44 ± 3.06+9.14 ± 0.91−22.05 ± 2.50−5.53 ± 0.98+11.66 ± 2.10−11.66 ± 2.10
>0.8 → 0+10.51 ± 2.54−27.75 ± 1.29+16.75 ± 1.44+0.48 ± 0.16−8.21 ± 1.52+8.21 ± 1.52
Neighborhood damage<0.2 → 1+21.20 ± 3.25+9.28 ± 0.93−24.96 ± 2.62−5.52 ± 0.97+19.48 ± 3.28−19.48 ± 3.28
>0.8 → 0+6.72 ± 2.81−27.49 ± 1.22+20.09 ± 1.95+0.67 ± 0.26−38.84 ± 1.88+38.84 ± 1.88
Personal injury<0.2 → 1−3.62 ± 1.75+10.19 ± 1.09−2.04 ± 1.13−4.54 ± 0.86+7.17 ± 1.72−7.17 ± 1.72
>0.8 → 0+31.84 ± 1.49−35.22 ± 1.26+2.94 ± 0.43+0.45 ± 0.19−11.60 ± 1.40+11.60 ± 1.40
Personal death<0.2 → 1−0.33 ± 1.42+6.80 ± 0.81−2.17 ± 1.07−4.29 ± 0.83+0.92 ± 0.92−0.92 ± 0.92
>0.8 → 0+28.86 ± 2.47−33.81 ± 1.98+3.81 ± 0.56+1.14 ± 0.41+1.31 ± 0.98−1.31 ± 0.98
Injury to others/pets/livestock<0.2 → 1+13.98 ± 3.07+10.35 ± 1.12−19.69 ± 2.29−4.64 ± 0.82+5.36 ± 2.13−5.36 ± 2.13
>0.8 → 0+21.74 ± 3.32−41.49 ± 1.80+19.32 ± 1.59+0.44 ± 0.18−12.15 ± 2.13+12.15 ± 2.13
Death of others/pets/livestock<0.2 → 1+13.97 ± 2.90+9.08 ± 0.90−19.58 ± 2.21−3.47 ± 0.60+7.24 ± 2.24−7.24 ± 2.24
>0.8 → 0+23.28 ± 2.43−39.76 ± 1.68+16.11 ± 1.02+0.36 ± 0.17−23.23 ± 2.06+23.23 ± 2.06

References

[1] Kuligowski, E. D. (2022). Modeling evacuation decisions in the 2019 Kincade fire in California.

[2] Forrister, A. (2024). Analyzing Risk Perception, Evacuation Decision and Delay Time: A Case Study of the 2021 Marshall Fire in Colorado.

[3] Sohn, K. (2015). Learning Structured Output Representation using Deep Conditional Generative Models.

[4] Chung, J. (2015). A Recurrent Latent Variable Model for Sequential Data.

[5] Kohl, S. A. A. (2018). A Probabilistic U-Net for Segmentation of Ambiguous Images.

[6] Ramsey, F. P. (1931). Truth and Probability.

[7] Kyburg, H. E. (1980). Studies in Subjective Probability.

[8] Li, H. H. (2020). Confidence reports in decision-making with multiple alternatives violate the Bayesian confidence hypothesis.

[9] McCaffrey, S. M. (2018). Should I stay or should I go now? Or should I wait and see? Influences on wildfire evacuation decisions.

[10] Kuligowski, E. D. (2022). Modeling evacuation decisions in the 2019 Kincade fire in California.

Thank you again for your thoughtful review. We have prepared detailed responses to the questions and concerns raised in your review and would greatly appreciate it if you had a chance to look them over. We would be happy to clarify any remaining questions or concerns.

I appreciate the authors for the effort and clarification. I have carefully read the authors' rebuttal and other reviewers comments. The response basically address my concerns. Therefore, I decide to raise the score to 3.

Reviewer_5C5MOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies Theory-of-Mind reasoning in sequential, uncertain settings where an agent holds multiple beliefs that evolve and interact. The authors point out that existing ToM methods often treat beliefs as static or independent, and thus fail to connect evolving beliefs to sequential actions. To address this gap, the authors propose a temporal latent-variable model that combines frozen LLM embeddings, a belief graph, an action predictor, and ELBO training to infer predefined beliefs from observation–action trajectories without belief labels. From the improved action prediction and correlations with human survey responses, the authors claim that the model learns interpretable, temporally evolving belief structures and provides more behaviorally grounded ToM reasoning.

Critical strengths
  1. The framework combines frozen LLM semantic representations with explicit belief variables, temporal updating, probabilistic normalization, and a belief-conditioned action model. This makes the reasoning process more inspectable and controllable.

  2. The evaluation is broader than standard end-task prediction. Although some metrics have limitations, this is an encouraging attempt to test whether the latent variables are meaningful.

Critical weaknesses
  1. The paper presents the factor graph, particularly the pairwise belief interactions, as the key addition beyond existing temporal latent-variable models. However, Table 8 shows that removing the ELBO reduces action accuracy from 0.8443 to 0.4794, whereas additionally removing the factor graph changes it only from 0.4794 to 0.4802. Since the paper omits the crucial ELBO-without-factor-graph condition, it does not show that the interaction structure improves action prediction when variational learning is retained. The reported gains may therefore come primarily from standard ELBO-based temporal latent-variable training, with limited or unestablished benefit from the proposed factor graph. This may substantially weaken the claim that modeling belief interactions is responsible for the model’s superior performance.

  2. The motivating gap is not demonstrated clearly. The paper argues that prior ToM methods perform poorly because they model beliefs as static and independent, and that temporal, interacting beliefs are therefore necessary. However, Table 1 compares the proposed end-to-end ELBO-trained model primarily against prompting or inference-time methods that differ in training, architecture, and access to action supervision. Their lower performance therefore does not establish that static or independent belief representations are the cause of failure.

Therefore, I would suggest the authors provide evidence that the proposed temporal and interaction components address a genuine limitation. For example, through controlled ablations or matched comparisons that isolate the contribution of temporal dependence and pairwise interactions while keeping the main training setup and supervision as comparable as possible. Without such evidence, the results support the benefit of task-specific end-to-end training, but not the paper’s claimed diagnosis of the limitations of prior ToM methods.

  1. The action predictor sees all individual belief probabilities, but not their joint distribution. This means some information learned by the factor graph is discarded before action prediction. The graph matters only when its relationships change the individual marginals. Therefore, the paper’s claim that actions depend on the joint belief configuration is not fully supported by its decoder design.
Questions for authors

See weaknesses. Additionally:

  1. The paper claims time-varying, observation-conditioned belief interactions. However, equation 5 defines ϕij\phi_{ij} only from the two belief descriptions, suggesting static edges. Could the authors clarify whether ϕij\phi_{ij} is conditioned on oto_t and bt−1b_{t-1} and, if so, provide the correct formulation and evidence that the interactions change over time?

  2. Equations 1 and 9 use pθ(at∣bt)p_\theta(a_t\mid b_t), while Equation 6 constructs the predictor exclusively from P(bt,i=1)P(b_{t,i}=1). If only marginals are used, how can the predictor distinguish different joint belief relationships with identical marginals, and how is the stated ELBO consistent with the implemented decoder?

3 comments
AuthorsOpenReview ↗

Reply to Reviewer 5C5M part 1

We thank the reviewer for the thoughtful questions and constructive feedback. We address each point below and will incorporate the corresponding clarifications into the revised manuscript.

For Weakness 1. We would like to clarify that the primary contribution of the factor graph is to explicitly model and recover the interdependencies among latent human beliefs, thereby improving the interpretability of the inferred cognitive states. While a standard ELBO-based latent-variable model identifies how different beliefs contribute to observed actions but does not explicitly characterize their dependencies, our factor graph provides an interpretable structure that reveals which beliefs interact and whether they reinforce or suppress each other. As shown in Figure 3(b), removing pairwise belief interactions reduces the Spearman correlation between predicted and human-reported belief dependency structures from 0.664 to 0.564, demonstrating the importance of the factor graph in capturing how human beliefs reinforce or weaken one another. This allows us to move beyond simply predicting what people do toward explaining how different beliefs influence one another and jointly shape human decisions.

For Weakness 2. We would like to clarify that Figure 3 presents controlled ablation results that demonstrate the contributions of temporal modeling and belief interactions. In Figure 3, by comparing three ablation settings (Full Model, No Temporal, and No Pairwise & Time), we show that removing temporal dependence worsens the recovery of human belief trajectories, measured by Dynamic Time Warping (DTW) distance between predicted and human-reported belief trajectories (0.394 → 0.421, lower is better), and reduces the alignment between belief updates and behavioral changes (Cohen's d: 0.538 → 0.352). We also show that further removing pairwise interactions reduces the correlation with human belief dependency structures (Spearman: 0.625 → 0.564). In addition, removing temporal dependence reduces intermediate and final action accuracy from 0.8443 to 0.8180 and from 0.9157 to 0.8670, respectively, while further removing pairwise interactions reduces intermediate and final action F1 from 0.7730 to 0.7320 and from 0.8788 to 0.8181.

AblationIntermediate Action AccIntermediate Action F1Final Action AccFinal Action F1
No Pairwise & Time0.8250.73200.86730.8181
No Temporal0.81800.77300.86700.8788
Full Model0.84430.7960.91570.9023

For Weakness 3. In our action predictor, the action-specific self-attention mechanism captures how multiple beliefs jointly influence actions by learning the dependencies among individual beliefs' contributions to action prediction. As described in Section 3.2 (Eq. 6), our action-specific self-attention mechanism learns how the contribution of one belief to an action depends on the activation of other beliefs. Specifically, the attention weight At,j[i,k]A_{t,j}[i,k] captures how belief kk influences the contribution of belief ii when predicting action jj. By learning these interdependencies directly through self-attention, the action predictor can capture how beliefs jointly influence decisions using marginal belief probabilities, without evaluating all 2K2^K configurations of the full joint belief distribution, substantially reducing computational costs. We also want to clarify that the joint belief relationships learned by the factor graph describe how beliefs reinforce or suppress one another to form the joint belief distribution, which is different from how multiple beliefs jointly influence actions in the action predictor.

We will further explain this design choice of action predictor in Section 3.2 (Action Generation), explaining why we use marginal belief probabilities with self-attention instead of the full joint distribution for action prediction.

AuthorsOpenReview ↗

Reply to Reviewer 5C5M part 2

For Question 1 We would like to clarify that pairwise belief interactions are time-varying and conditioned on observations and previous belief states. We will add the time-step subscript tt to the pairwise interaction terms in Eq. (5), explicitly writing ϕij,t\phi_{ij,t} and hij,th_{ij,t} to clarify their time-varying nature and maintain consistency with Eq. (2).

We further provide empirical evidence that the pairwise interaction strengths ϕij,t\phi_{ij,t} change over time as new observations arrive. For example, when residents receive an official warning after observing nearby fire cues, the interaction between home damage and death of others increases by 14.35%, while the interaction between personal injury and personal death decreases by 11.74%. The interaction between home damage and neighborhood damage also increases by 11.21%. This suggests that official warnings strengthen the connections between property-related and broader community risks, while the relationship between personal injury and death becomes less pronounced. After receiving a warning from a familiar person, these interactions change again by −2.22%, +1.78%, and −0.89%, respectively, suggesting that additional social information further adjusts these belief relationships, although the changes are smaller than those following official warnings. We will also include a detailed explanation of these results in the revised manuscript.

Belief relationshipNearby fire cues (t0 initial phi_ij)Fire cues → Official warning (t0 → t1)Official warning → Familiar-person warning (t1 → t2)
Home damage ↔ Death of others1.61614.35%-2.22%
Personal injury ↔ Personal death1.484-11.74%1.78%
Home damage ↔ Neighborhood damage1.58611.21%-0.89%
Home damage ↔ Injury to others1.5989.33%-1.53%
Personal injury ↔ Injury to others1.456-7.87%0.05%

For Question 2 In our action predictor, the action-specific self-attention mechanism captures how multiple beliefs jointly influence actions by learning the dependencies among individual beliefs' contributions to action prediction. As described in Section 3.2 (Eq. 6), our action-specific self-attention mechanism learns how the contribution of one belief to an action depends on the activation of other beliefs. Specifically, the attention weight At,j[i,k]A_{t,j}[i,k] captures how belief kk influences the contribution of belief ii when predicting action jj. By learning these interdependencies directly through self-attention, the action predictor can capture how beliefs jointly influence decisions using marginal belief probabilities, without evaluating all 2K2^K configurations of the full joint belief distribution, substantially reducing computational costs. We also want to clarify that the joint belief relationships learned by the factor graph describe how beliefs reinforce or suppress one another to form the joint belief distribution, which is different from how multiple beliefs jointly influence actions in the action predictor.

Regarding the ELBO, the action predictor computes the action likelihood from individual belief probabilities through self-attention, as shown in Eq. (6), without evaluating each belief configuration separately. We will update Eq. (9) and the corresponding explanation to accurately reflect how the action likelihood is computed in our implementation.

Thank you again for your thoughtful review. We have prepared detailed responses to the questions and concerns raised in your review and would greatly appreciate it if you had a chance to look them over. We would be happy to clarify any remaining questions or concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.