acceptodds
Under review as a conference paper at ICLR 2027

Inferring Cognitive Profiles with Cognition-Conditioned Deep Inverse Planning

Abstract

Bounded rationality proposes that humans think and act as best they can given limited cognitive resources. For example, to solve complex problems, people routinely engage in *subtasking*, in which they chain together solutions to simpler subtasks. But the boundedly rational solution to a task depends on a person's *cognitive profile*, the individual factors that determine what they find rewarding or cognitively effortful. This makes it hard to use bounded rationality to analyze human behavior in naturalistic tasks, as it requires computing optimal solutions under many different candidate profiles. Our method, *cognition-conditioned deep inverse planning* (CCDIP), addresses this by training a single hierarchical policy that takes cognitive parameters as input and yields tractable action likelihoods. By inverting this forward model, we can then recover people's subtasking strategies and cognitive profiles from their behavior. In simulation, CCDIP recovers parameters of agents with known profiles. On a new dataset of participants performing a driving task (, timesteps), CCDIP shows that attentional factors are needed to explain behavior and reveals stable individual differences in cognitive parameters. CCDIP thus offers a route to theory-driven, interpretable cognitive modeling in naturalistic tasks where traditional approaches do not scale.

Primary area:
applications to neuroscience & cognitive science
Keywords:
bounded rationality, cognitive science, bayesian inference, reinforcement learning, inverse reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_Ef9jOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Cognition-Conditioned Deep Inverse Planning (CCDIP), a framework for inferring individual cognitive profiles from behavior in complex planning tasks. The key idea is to pretrain a hierarchical reinforcement learning policy conditioned on cognitive parameters, including outcome preferences, attentional costs, and goal-selection biases, thereby amortizing the computational cost of solving planning problems for different parameter settings. The pretrained policy is then used for likelihood-based parameter inference from observed behavior. Using a 2D driving task, the authors demonstrate parameter recovery on synthetic data and evaluate the model on human behavioral data. Their results show that incorporating attentional costs and heuristics improves model fit, while several inferred parameters exhibit stable individual differences across trials.

Critical strengths
  • Well-motivated problem. The paper addresses an important computational bottleneck in cognitive modeling: fitting planning-based models becomes computationally expensive in complex environments, particularly when optimal policies must be recomputed for different cognitive parameters.

  • A promising and scalable methodological framework. By pretraining a cognition-conditioned hierarchical policy, CCDIP amortizes the cost of planning across different parameter settings. Although the underlying techniques are not entirely new, their integration offers a practical approach to fitting resource-rational cognitive models in complex tasks.

  • Meaningful empirical validation. The authors evaluate the framework using both synthetic and human behavioral data. The parameter recovery experiments demonstrate that several cognitive parameters can be reasonably recovered, while the ablation studies show that attentional costs and heuristics contribute to explaining human behavior.

  • Potential for broader applications. The framework provides a flexible approach to cognitive model fitting that could potentially be extended to other planning tasks and cognitive model specifications. This makes it a potentially useful methodological contribution beyond the particular driving task investigated in the paper.

Critical weaknesses
  • Insufficient comparison with existing methods. The main limitation is the lack of direct comparisons with established approaches to cognitive model fitting, such as simulation-based likelihood estimation (e.g., inverse binomial sampling) or traditional inverse planning methods. While the proposed framework is promising, it remains unclear how much methodological advancement CCDIP provides over existing approaches. In particular, comparisons in computational efficiency, parameter recovery accuracy, and fitting stability would help establish its practical advantages. The current experiments primarily focus on internal model ablations rather than comparisons with alternative inference methods.

  • Limited evaluation of data efficiency. Although the authors report parameter recovery on synthetic data, the recovery accuracy varies substantially across parameters. It would be useful to examine how parameter recovery changes with the amount of available behavioral data. This would help clarify how much data is required for reliable cognitive parameter inference, particularly in complex planning tasks where different parameters may produce similar behavioral patterns.

Questions for authors
  1. Scope and limitations of the methodological contribution. Could the authors further clarify the specific bottleneck addressed by CCDIP and its broader implications? My understanding is that CCDIP primarily addresses the computational tractability of fitting predefined planning-based cognitive models, while challenges related to parameter identifiability and hypothesis-space constraints remain. Importantly, collecting more behavioral data and incorporating qualitatively different process-level evidence may address different aspects of these challenges. For example, eye-tracking data have been used to reveal human planning strategies (Callaway et al., 2024), while think-aloud data have been shown to reshape the structures of automatically discovered cognitive models (Xie et al., 2026). These studies provide examples of how additional process-level evidence can help constrain cognitive model inference beyond behavioral choices alone. It would be interesting to discuss such complementary directions and how they might extend the proposed framework toward broader cognitive model identification.

  2. Inconsistency between the main text and Appendix. Section 4.2 describes four cognition-conditioning parameters (collision cost, cognitive cost, goal proximity cost, and goal-flag distance cost), whereas Appendix A.2.1 (Eq. 8 and Tables 1–2) specifies only two (collision cost and cognitive cost). Could the authors clarify which parameterization was used in the actual training and experiments, and how the remaining parameters were incorporated?

References

Callaway, F., Yu, M., & Mattar, M. G. (2024). Revealing human planning strategies with eye-tracking. Proceedings of the 46th Annual Conference of the Cognitive Science Society, 4343–4349. https://escholarship.org/uc/item/8sm0k8z2

Xie, H., Jagadish, A. K., Pan, L., & Wilson, R. C. (2026). Think-Aloud Reshapes Automated Cognitive Model Discovery Beyond Behavior. arXiv preprint, arXiv:2605.05091. https://arxiv.org/abs/2605.05091

3 comments

We thank the reviewer for their interest in our work and their helpful comments. The reviewer raises some important questions about the novelty and contribution of our work. We address the author’s concerns and provide clarifications below.

Q1: Scope and limitations of the methodological contribution. Could the authors further clarify the specific bottleneck addressed by CCDIP and its broader implications? My understanding is that CCDIP primarily addresses the computational tractability of fitting predefined planning-based cognitive models, while challenges related to parameter identifiability and hypothesis-space constraints remain. Importantly, collecting more behavioral data and incorporating qualitatively different process-level evidence may address different aspects of these challenges. For example, eye-tracking data have been used to reveal human planning strategies (Callaway et al., 2024), while think-aloud data have been shown to reshape the structures of automatically discovered cognitive models (Xie et al., 2026). These studies provide examples of how additional process-level evidence can help constrain cognitive model inference beyond behavioral choices alone. It would be interesting to discuss such complementary directions and how they might extend the proposed framework toward broader cognitive model identification.

The reviewer brings up a fascinating connection to work that uses process-level evidence in cognitive modeling. While we primarily see CCDIP as a general scalable technique for modeling choice behavior at a computational level of analysis (in the sense of Marr), it is conceptually straightforward to also have the forward model generate process outputs, such as gaze and even verbal reports, under varying cognitive constraints. We could then perform inference over both kinds of data, while leveraging process-level evidence in the model development phase, which could help with the identifiability issues the reviewer raises. This is an important point that we believe underscores the generality and extensibility of our approach, which we now note in the discussion on future work (section pasted below).

“Third, inferring latent profiles from behavioral choices presents challenges with parameter identifiability and hypothesis-space constraints. To address this, the forward model used in CCDIP could be extended to generate secondary behavioral measures, such as reaction times, eye movements, or even verbal reports, that vary with cognitive profiles but are not optimized. This extended model could then be inverted and used to better infer cognitive parameters by conditioning on the joint pattern of choices and process measures such as reaction-time (Bogacz et al., 2006), eye-tracking (Callaway et al., 2024), or think-aloud (Xie et al., 2026) data. Relatedly, because the trained model can cheaply simulate behavior under any profile in its training range, it could support simulation-based power analyses for experimental design.”

Q2: Inconsistency between the main text and Appendix. Section 4.2 describes four cognition-conditioning parameters (collision cost, cognitive cost, goal proximity cost, and goal-flag distance cost), whereas Appendix A.2.1 (Eq. 8 and Tables 1–2) specifies only two (collision cost and cognitive cost). Could the authors clarify which parameterization was used in the actual training and experiments, and how the remaining parameters were incorporated?

This is an important comment that echoes the concerns expressed by Reviewer 2. We thank the reviewer for bringing this oversight to our attention. We have revised the appendix, which now correctly mentions the four context parameters the model was conditioned on during training in Appendix A.2.1. We also added an additional section C in the Appendix detailing how the heuristic parameters were incorporated in our model, separate from training.

Comment: Insufficient comparison with existing methods. The main limitation is the lack of direct comparisons with established approaches to cognitive model fitting, such as simulation-based likelihood estimation (e.g., inverse binomial sampling) or traditional inverse planning methods. While the proposed framework is promising, it remains unclear how much methodological advancement CCDIP provides over existing approaches. In particular, comparisons in computational efficiency, parameter recovery accuracy, and fitting stability would help establish its practical advantages. The current experiments primarily focus on internal model ablations rather than comparisons with alternative inference methods.

We thank the reviewer for making the connection to other advanced cognitive modeling methods. Simulation-based likelihood estimation approaches such as inverse binomial sampling (IBS) require the ability to simulate behavior under candidate parameter settings. In our task this is itself the bottleneck, and CCDIP allows us to amortize the cost of computing the choice model. Thus, the two are complementary since IBS could be run with our trained model. However it would provide a noisy estimate of the likelihood that our model gives directly and can be differentiated through, so we use the exact likelihood.

Traditional inverse planning requires a solution to the planning problem under different candidate profiles, and we are not aware of an alternative method that provides this for our task. Exact approaches (e.g., Baker et al., 2009; 2017) rely on dynamic programming over an enumerable state space, which is not available in a continuous, high-dimensional task. Approaches based on online search (e.g., Zhi-Xuan et al., 2020) have been developed for discrete, symbolic domains, where inference is over a finite set of goals with the agent’s bounded-rationality parameters held fixed; they do not directly apply to a continuous, stochastic task in which those parameters are the target of inference. The remaining alternative is to use deep RL to find solutions, but it would be impractical to do this without a cognition-conditioned policy: based on our experience training policies on this task, we conservatively estimate that training a subtask-selection policy for a single fixed cognitive profile would take at least 5 GPU-hours, so even an efficient search procedure that evaluated 100 cognitive profiles would take 500 GPU-hours. In contrast, in our implementation of CCDIP, the subtask-selection policy took 22 GPU-hours to train, works for a range of cognitive parameters, and is differentiable.

Comment: Limited evaluation of data efficiency. Although the authors report parameter recovery on synthetic data, the recovery accuracy varies substantially across parameters. It would be useful to examine how parameter recovery changes with the amount of available behavioral data. This would help clarify how much data is required for reliable cognitive parameter inference, particularly in complex planning tasks where different parameters may produce similar behavioral patterns.

We agree that this is a valuable analysis, and the reviewer’s point that different parameters may produce similar behavior is well taken. The current manuscript reports recovery at a single amount of data (six episodes per profile) and does not characterize how recovery depends on the amount of data. One advantage of CCDIP is that the trained model can cheaply simulate behavior under any cognitive profile in the trained range. This makes it possible to run simulation-based power analyses that inform experimental design, by estimating in advance whether a given design allows parameters to be recovered precisely and distinguished from one another. We now note this in the discussion as a direction for future work.

Thanks for the authors' rebuttal and clarification. I don't have any remaining concerns, and since it's already a positive score, I would keep the current score. Good luck!

Reviewer_wxFwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Cognition-conditioned deep inverse planning (CCDIP) estimates individual cognitive parameters from sequential behavior. A hierarchical policy selects local goals and attentional scopes, while a low-level policy executes the resulting subtasks. Conditioning the policy on cognitive parameters allows planning computations to be shared across candidate profiles. The authors evaluate the approach through synthetic parameter recovery and a driving study with 39 retained participants, examining model fit, collision behavior, and the consistency of parameter estimates across episodes.

Critical strengths
  • A shared policy handles different cognitive profiles without requiring a separate planning solution for each set of candidate parameters. This is useful when fitting individual differences, where many parameter settings need to be evaluated. The hierarchy also allows goals and attentional scopes to change during a trial, which suits the changing demands of driving.

  • The model fits human actions well, with a substantial improvement in BIC over the ablated models. The authors also examine parameter recovery, collision behavior, and consistency across episodes.

Critical weaknesses
  • Sections 4.2–4.3 describe four conditioning parameters, whereas Appendix A.2.1, Equation 8, and Tables 1–2 specify only two. The relationship between these configurations is not explained, leaving the model used for the reported results unclear. The implementation of Equation 5 is also underspecified, particularly the treatment of persistent subtasks and irregular observation intervals in the trajectory likelihood. These omissions make the parameter estimation procedure difficult to reproduce.

  • The "no-scopes" ablation sets attention cost and heuristic weights to zero without explicitly stating that all vehicles remain visible. The distinction between removing attention costs and removing restrictions on attention is therefore unclear in this comparison. The collision analysis also lacks a clear separation between episodes used for fitting and evaluation, leaving performance on new episodes insufficiently documented.

Questions for authors
  • Could authors clarify which conditioning parameters were included in the models used for the main results, and how those configurations relate to Equation 8 and Tables 1–2? If the appendix describes a different configuration, please identify where the four-parameter implementation differs.

  • Could authors outline the recursion used to evaluate Equation 5, including how persistent goals, object identities, and intervals between recorded observations are handled? Which low-level action distribution is used when evaluating human trajectories?

  • In the "no-scopes" condition, are all candidate vehicles explicitly kept visible? If mask selection remains active, could authors report how often the policy selects a restricted scope?

  • Was the human–agent collision correlation evaluated on episodes separate from those used for fitting? Were participant parameters held fixed between fitting and evaluation routes? Could response delay or action persistence influence the estimated attention costs?

3 comments

We thank the reviewer for their interest in our work and thoughtful comments!

Q1: Could authors clarify which conditioning parameters were included in the models used for the main results, and how those configurations relate to Equation 8 and Tables 1–2? If the appendix describes a different configuration, please identify where the four-parameter implementation differs.
Related Comment: Sections 4.2–4.3 describe four conditioning parameters, whereas Appendix A.2.1, Equation 8, and Tables 1–2 specify only two. The relationship between these configurations is not explained, leaving the model used for the reported results unclear.

We thank the reviewer for pointing out this oversight, and we now describe how all four context parameters (two reward and two cognitive cost) were used to condition the model during training, in the Appendix. The heuristic parameters were integrated separately; they were not used to train the model. Specifically, these parameters reweighted subtask choices by adding a weighted bias to the scope-head’s output score for each candidate mask and renormalizing the resulting distribution, as described in detail in Appendix C.

Q2: Could authors outline the recursion used to evaluate Equation 5, including how persistent goals, object identities, and intervals between recorded observations are handled?
[...]

The reviewer raises an important reproducibility concern with regard to the lack of clarity about the implementation of the parameter inference process. Appendix D now provides the details of the inference process, which we summarize below.

Goal persistence: We remap goals in a subtask from each timestep tt to the next timestep t+1t+1 by shifting their relative positions to the agent by the agent’s displacement between the two timesteps. The subset of goals whose shifted relative position at time t+1t+1 fall behind the agent are categorized as reached/missed goals based on the agent’s interpolated path between the two timesteps, and subtasks which include those goals are consequently considered terminated. For all other subtasks, previous subtasks are now remapped to goal bins with reduced relative distances, and a new set of subtasks are defined on a new row of goal bins furthest from the agent.

Object identities Getting object tracking to work correctly across frames was critical to the success of our experiments since the central premise of our study is the dynamic allocation of attentional resources to objects which move in and out of the observation window. During training and simulation, we have access to the internal simulator states, which allows us to uniquely identify individual objects in our environment. For recorded human trajectories however, we do not assume the availability of such information. Each frame records the position, object types of each object in view, among other features. The goal was to develop a simple yet accurate procedure for tracking objects in the environment. To achieve this, we matched objects of the same type across consecutive frames based on spatial proximity, treating the nearest neighbors as the same object.

Intervals between observations: Since the simulator runs at 30 frames per second, in the worst case, there is at most a ~100ms (three 33ms frames) lag between any two recorded frames. Since average human response times range between 200 and 250ms (Jain et al, 2015), a 100ms lag can at most contain 1 human decision, which would be captured by our model.

Reference: Jain, A., Bansal, R., Kumar, A., & Singh, K. D. (2015). A comparative study of visual and auditory reaction times on the basis of gender and physical activity levels of medical first year students. International journal of applied & basic medical research, 5(2), 124–127.

Which low-level action distribution is used when evaluating human trajectories?

We use the trained subtask-conditioned policy to evaluate low-level actions (marginalizing over possible subtasks according to Eq 5)

Related Comment: The implementation of Equation 5 is also underspecified, particularly the treatment of persistent subtasks and irregular observation intervals in the trajectory likelihood. These omissions make the parameter estimation procedure difficult to reproduce.

We thank the reviewer for pointing out how the implementation of Eq 5 is underspecified. Many of the details of our specific implementation, such as handling persistence and irregular observations, were intentionally left out of this section in order to provide a clear general statement of the cognitive parameter likelihood that could apply to other tasks. Of course, we completely agree that our experiments should be reproducible and so have added an extensive appendix D (Parameter Estimation) that maps the general formulation in the main text to our specific experiments.

Q3: In the “no-scopes” condition, are all candidate vehicles explicitly kept visible? If mask selection remains active, could authors report how often the policy selects a restricted scope?
Related Comment: The “no-scopes” ablation sets attention cost and heuristic weights to zero without explicitly stating that all vehicles remain visible. The distinction between removing attention costs and removing restrictions on attention is therefore unclear in this comparison.

The reviewer is absolutely right that we overlooked the distinction between removing attention costs and removing restrictions on attention. The no-scopes model does not force all vehicles to be visible: the subtask-selection policy still chooses scopes, conditioned on zero attentional cost (θatt=0\theta_{att}=0; Appendix D.2). Even at zero cost, the policy hides objects that are not relevant to the current subtask. To address this, we now include an additional comparison with a full-visibility model in the paper, in which all objects are forced to remain visible to the subtask policy (Section 5, Human Data Experiments). The no-scopes model fits the human data better than the full-visibility model by a large margin (ΔBIC=−862.9\Delta\text{BIC} = -862.9; 95% CI [−953.5,−775.4];p<.001[-953.5,-775.4]; p<.001). This shows that even without an attentional cost, a tendency to ignore task-irrelevant objects helps explain human behavior. We thank the reviewer for their close reading of our analyses, which has improved the quality of our work.

Q4: Was the human–agent collision correlation evaluated on episodes separate from those used for fitting? Were participant parameters held fixed between fitting and evaluation routes?
[...]
Related Comment: The collision analysis also lacks a clear separation between episodes used for fitting and evaluation, leaving performance on new episodes insufficiently documented.

We thank the reviewer for bringing this to our attention, and have now revised our explanation of the experimental process for greater clarity. Human-agent collision correlation was evaluated on the same episodes used for fitting, and parameters were estimated separately for each episode rather than held fixed across episodes. To be clear, this is not a re-description of the fit since the model is fit to actions, whereas collisions are an outcome that arises only when the model is simulated freely on the map. The analysis is therefore a check that the model reproduces behaviorally meaningful patterns that it was not directly fit to (Wilson & Collins, 2019; Palminteri et al, 2017) and complements analyses based on relative fit (e.g., BIC). We agree that it does not establish performance on new episodes, and state this in the revised Results (section 5).

Wilson, R. C., & Collins, A. G. (2019). Ten simple rules for the computational modeling of behavioral data. elife, 8, e49547.

Palminteri, S., Wyart, V., & Koechlin, E. (2017). The importance of falsification in computational cognitive modeling. Trends in cognitive sciences, 21(6), 425-433.

Could response delay or action persistence influence the estimated attention costs?

Without running actual experiments, we can only hypothesize that both response delay or action persistence would influence attention cost estimates. We believe that adding more constraints to the agent's ability to react to certain conditions would impose greater attentional demands, and attentional strategies would become more important under resource constraints. We note this as a direction for future work.

Comment: Sections 4.2–4.3 describe four conditioning parameters, whereas Appendix A.2.1, Equation 8, and Tables 1–2 specify only two. The relationship between these configurations is not explained, leaving the model used for the reported results unclear. The implementation of Equation 5 is also underspecified, particularly the treatment of persistent subtasks and irregular observation intervals in the trajectory likelihood. These omissions make the parameter estimation procedure difficult to reproduce.

See responses to questions 1 and 2

Comment: The “no-scopes” ablation sets attention cost and heuristic weights to zero without explicitly stating that all vehicles remain visible. The distinction between removing attention costs and removing restrictions on attention is therefore unclear in this comparison. The collision analysis also lacks a clear separation between episodes used for fitting and evaluation, leaving performance on new episodes insufficiently documented.

See responses to questions 3 and 4

Thank authors for the detailed responses, which have addressed my major concerns. I will raise my score accordingly.

Reviewer_F9gkOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper creates a framework for cognitive-conditioned deep inverse planning (CCDIP) that unlike most forms of IRL or inverse planning, includes cognitive parameters (e.g., attentional cost, heuristics) to better capture real human behavior. Their method of capturing cognitive limitations is rooted in hierarchal RL. Unlike a lot of the bounded rationality literature, their paper also uses modern deep learning with a wider action space than a lot of prior work in cogsci. They implement their approach in a driving simulator environment, both with synthetic data and human data, and demonstrate that the bounded-rational "heuristics" and "scoping" aspects of their model are key for capturing human behavior.

Critical strengths
  1. New framework for "cognition-conditioned deep inverse planning (CCDIP)" that formalizes the role of cognitive parameters through hierarchical RL. I thought the way the authors captured scoping (through masking) and heuristics was very conceptually clean.
  2. Their method learns a network policy that is conditioned on cognitive parameters, and the uses that fixed network policy to do inverse inference of cognitive parameters, for a driving simulator environment. This to my understanding is much more realistic than a lot of work on bounded rationality in cognitive science. Their approach can scale is because they learn a network policy that is conditioned on cognitive parameters, so this basically amortizes the problem of finding the optimal policy given a specific setting of cognitive parameters, whereas prior work is often solving the expensive optimal policy search as an inner loop of the inverse planning.
  3. The experiments are a nice validation of their method. They do a solid human evaluation, and driving even in these simulations is quite a tricky / complex domain to capture human behavior in, but they nonetheless show that the heuristics and scoping of their model significantly improve the BIC score compared to a vanilla model.
  4. Paper is very well-written and clear. Major appreciation among all the slop these days.
Critical weaknesses

No critical weaknesses I see

Questions for authors
  1. How important was hyperparameter selection for getting this method to work? In practice, how much iteration did it take & what were the hardest parts of implementation?
1 comment

We are grateful to the reviewer for their interest in our work and their thoughtful comments. Below we address the main questions that were brought up and make some clarifications:

Q1: How important was hyperparameter selection for getting this method to work? In practice, how much iteration did it take & what were the hardest parts of implementation?

We thank the reviewer for raising these important questions about the implementation details, which are crucial to the reproducibility of our approach. For most hyperparameters, we retained the CleanRL PPO default values. Aside from the standard batch, mini-batch, and network size parameters, the learning rate had to be tuned to train the subtask policy (Appendix B.2). The most significant improvement in the policy performance resulted from incorporating a custom state encoder that feeds into the multi-layer perceptron (MLP) policy network. Training the subtask selection policy was much more challenging, however. The default MLP network used in CleanRL’s implementation does not provide the structural biases necessary for modeling attention. Consequently, we turned to resource-allocation approaches in the literature, specifically the Deep Sets architecture, given its inherent property of permutation invariance, which allows us to allocate attentional resources to objects regardless of their input ordering. For the subtask-selection policy, we tuned the same set of PPO hyperparameters while additionally disabling learning-rate annealing (Appendix B.1).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.