acceptodds
Under review as a conference paper at ICLR 2027

mini-psych-agent: Evaluating Human-Like Behavior in LLM Agents through Interactive Psychological Environments

22 views

Abstract

A Large Language Model (LLM), its harness, and the test-time compute budget jointly determine its behavior. Consequently, the same underlying model, embedded in different agentic workflows or allocated different amounts of test-time compute, can exhibit markedly different performance. Psychologists face an analogous challenge when evaluating human performance: the same person may behave differently under different incentives and levels of deliberation. Psychological tasks therefore provide an ideal testbed for studying such behavioral variation. However, when adapted to LLMs, these tasks are typically text-transcribed, replaying fixed, language-rendered experiments as static prompts. This format is ill-suited to agent evaluation, where actions can alter future observations and task states. Conversely, although dynamic environments are prevalent in the evaluation of LLM agents, they rarely ask whether the resulting behavior is human-like. Here, we bridge this gap by transforming psychological tasks into dynamic environments in which agents can interact and by using a learned human-policy model to evaluate the human-likeness of the resulting behavioral trajectories. Across these environments, we find that the human-likeness of a given LLM varies with its agentic configuration and the psychological task. These findings motivate a more holistic evaluation of LLM agents in psychology-inspired environments, accounting for both the backbone model and its agentic scaffold.

Primary area:
applications to neuroscience & cognitive science
Keywords:
LLM agents, Psychological tasks, Interactive environments, Human behavioral alignment, Agentic scaffolds

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HABFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces MindGym, a suite of 25 interactive psychological tasks, and mini-psych-agent, a framework for comparing LLM agents with different memory configurations. Unlike evaluations based on replaying human histories, agents act autonomously and experience the consequences of their own decisions. Task-specific RNNs trained on human data score the resulting trajectories. The experiments suggest that providing memory generally improves task performance and agreement with the learned human policies, although the effects vary across tasks and models.

Critical strengths

The emphasis on closed-loop evaluation is well motivated. Allowing agents to generate their own histories captures behavioral differences that teacher-forced replay can miss. The collection of interactive tasks provides a useful platform for studying these effects across different memory configurations.

Critical weaknesses
  1. The main metric does not reliably establish human-likeness. Average log-likelihood under a human-policy model can favor an agent that repeatedly chooses the most probable action over one that reproduces human variability. The paper’s N-back example makes this concern concrete: an almost constant “non-match” policy receives a higher score than held-out humans despite chance-level balanced accuracy. The authors acknowledge this limitation, but it still weakens the interpretation of improved scores as improved human-likeness. More systematic trajectory-level validation is needed to support that interpretation.

  2. The evaluator is not sufficiently validated on the histories it is used to score. The motivation for the RNN evaluator is to assess agent-generated histories that may be rare or absent in the human dataset. However, its predictive validation primarily uses held-out human trajectories. This does not establish that its predictions remain reliable under the distribution shift induced by autonomous agents. Consequently, differences between memory configurations could partly reflect evaluator extrapolation errors rather than differences in human behavioral compatibility.

Questions for authors
  1. How do simple policies, such as always selecting one action or greedily choosing the RNN’s most probable action, rank against humans and LLM agents?
  2. Do trajectory-level measures support the same conclusions about memory?
  3. Could the authors validate the RNN on human responses collected after exposure to agent-generated histories, particularly those far from the training distribution?
4 comments
AuthorsOpenReview ↗

Response to Reviewer HABF (1/3)

W1. Average log-likelihood under a human-policy model can favor repeatedly choosing the most probable action over reproducing human variability (e.g., an almost constant non-match policy in N-back scores above held-out humans), so higher scores alone do not establish human-likeness without trajectory-level validation.

We agree that a higher likelihood does not by itself establish human-likeness; we have also noted this in the Eq. 13 of our original submission. We now calculated an alternative KL divergence (see Table 2 and Appendix K): most of the memory gain is lower entropy, and KL falls modestly, most under Cumulative. Under KL the four memory configurations keep their original order (Cumulative, Surprise, Window, Retrieval); only Stateless moves ahead of Retrieval, and no configuration differs from Stateless after Holm correction.

The KL divergence also penalizes repetitive choices. For example, in the N-back (Table 13; Table R7 below): the always non-match policy has KL 0.249, held-out humans about 0, and the memory configurations of DeepSeek and Qwen 0.04 to 0.20; Section 4 now states this. KL has limitations, which the revised paper states (Section 4, Appendix K, Discussion). It needs the agent's action probabilities, so we could compute it only for the two backbones whose APIs return token log probabilities (DeepSeek and Qwen); many closed models, including GPT-5.6 Luna, do not return them. Even for these two it is approximate: the probabilities were recovered afterwards by re-submitting the saved requests, so they are not the probabilities the model used when it made the original choices.

Trajectory-level validation on 44 pre-specified features and on the 16 behavioral effects of Figure 9 (Appendix M, Figures 10 to 12): memory brings history-dependent features into the human range but also makes agents more likely to repeat previous decisions than humans. The Discussion now gives this interpretation.

Table R7 (revised paper: Table 13). The N-back example under the KL divergence (five seeds; KL is the plug-in estimate, and Table 13 also gives a sample-based estimate; n/a: no token probabilities).

PolicyThe original likelihood score, SKL divergence
Held-out humans-0.458-0.008 (calibration gap)
Always non-match-0.2490.249
Always match-0.3020.302
Human-policy model, greedy-0.2480.248
Uniform random-0.9500.259
DeepSeek Stateless-0.4860.265
DeepSeek Cumulative-0.4670.042
DeepSeek Window-0.4510.129
DeepSeek Retrieval-0.5390.197
DeepSeek Surprise-0.4810.083
Qwen Stateless-0.9570.310
Qwen Cumulative-0.6470.157
Qwen Window-0.4930.096
Qwen Retrieval-0.5990.087
Qwen Surprise-0.5360.101
GPT Luna Stateless-0.365n/a
GPT Luna Cumulative-0.472n/a
AuthorsOpenReview ↗

Response to Reviewer HABF (2/3)

W2. The evaluator is validated mainly on held-out human trajectories, not on the agent-generated histories it scores, so differences between memory configurations could partly reflect extrapolation errors.

This is a fair concern. It cannot be fully resolved without new human data, and the limitations now say so. In the new Appendix L, we added:

(1) the RNNs reproduce behavioral signatures of held-out participants in 16 tasks, and their autonomous rollouts match held-out humans on 14 trajectory features (Fig. 9, Table 14);

(2) scoring every policy on the same histories (the human-policy RNN first plays the task on its own, and this sequence is shown to every agent; no agent-generated history is involved) gives the same advantage of memory over Stateless (Table 15; Table R3 below);

(3) simple-policy checks in closed loop (Table 12). We did not compute a re-analysis restricted to human-covered histories.

Table R3 (revised paper: Table 15). Mean log score S on the same human-like histories (teacher-forced replay, five seeds per task): a trajectory sampled from the human-policy model is imposed and every policy's choice is scored on it. The greedy row is the maximum attainable; no KL here because the replay runs recorded no token probabilities.

PolicyIowa gamblingCastle two-stepSocial banditWeatherMean
RNN greedy (maximum)-0.431-0.536-0.301-0.443-0.428
RNN sample-0.799-0.763-0.524-0.686-0.693
DeepSeek Stateless-0.738-1.902-1.015-0.753-1.102
DeepSeek Cumulative-0.741-0.927-0.446-0.718-0.708
DeepSeek Window-0.736-1.058-0.423-0.708-0.731
DeepSeek Retrieval-0.756-0.930-0.544-0.707-0.734
Qwen Stateless-1.209-1.900-0.974-0.712-1.199
Qwen Cumulative-1.081-0.977-0.425-0.729-0.803
Qwen Window-0.809-1.191-0.434-0.704-0.785
Qwen Retrieval-0.975-1.021-0.507-0.722-0.806
Best constant action-2.042-1.710-0.626-0.744-1.281
Uniform random-2.233-1.864-0.987-0.772-1.464

Q1. How do simple policies, such as always selecting one action or greedily choosing the RNN’s most probable action, rank against humans and LLM agents?

We ran this on 20 tasks with the same RNNs and seeds (Table 12; Table R2 below). In the original S (the higher the better) the concern is confirmed: the greedy policy (-0.13) and the best constant action (-0.30) score above held-out humans (-0.44) and above the LLM agents (-0.38 to -0.71). In KL (the lower the better) the order is: held-out humans about 0, greedy 0.13, LLM agents 0.14 to 0.24, constant and random policies about 0.30. On shared histories, agents with memory are close to a sample from the RNN and well above constant and random policies (Table 15). We now report these results in the revised paper.

Table R2 (revised paper: Table 12). Evaluating held-out human participants, simple RNN policies and LLM agents (closed loop, same seeds, equal-task means over 20 tasks). KL is the plug-in estimate (Table 12 also gives a sample-based estimate); for held-out humans the calibration gap is shown; n/a: no token probabilities.

PolicyThe original likelihood score, SKLTasks with S ≥ humans
Held-out humans-0.438-0.002 (calibration gap)-
Human-policy model, sampled-0.4520 (by definition)7/20
Human-policy model, greedy-0.1300.13020/20
Best constant action-0.2970.29717/20
Worst constant action-0.6390.63914/20
Uniform random-1.1510.3020/20
DeepSeek Stateless-0.6370.2005/20
DeepSeek Cumulative-0.4090.14612/20
DeepSeek Window-0.4780.19211/20
DeepSeek Retrieval-0.4930.2418/20
DeepSeek Surprise-0.4300.18611/20
Qwen Stateless-0.7130.1662/20
Qwen Cumulative-0.5550.1367/20
Qwen Window-0.5960.1484/20
Qwen Retrieval-0.6190.1863/20
Qwen Surprise-0.5760.1394/20
GPT Luna Stateless-0.523n/a8/20
GPT Luna Cumulative-0.380n/a14/20
GPT Luna Window-0.437n/a11/20
GPT Luna Retrieval-0.481n/a9/20
GPT Luna Surprise-0.413n/a13/20
AuthorsOpenReview ↗

Response to Reviewer HABF (3/3)

Q2. Do trajectory-level measures support the same conclusions about memory?

Only in part (Appendix M: Figs. 10 to 12, Tables 16 and 17; Tables R5 and R5b below). We developed several qualitative features for each task. Figs. 10 to 12 compare each backbone's own episodes with held-out participants on the 16 behavioral effects of Fig. 9, without any RNN: memory brings some history-dependent effects to the human level (for example repeating after a reward in the reward bandit, DeepSeek and Qwen), while in five tasks every configuration stays far from people. Features that require using past outcomes move into the human range with memory: in the two-step task the effect of the previous reward on repeating a choice is absent under Stateless and matches the RNN value under Cumulative. But memory also makes agents more likely to repeat previous decisions than humans (they switch less), most under Cumulative, so the share of 44 features inside the human-model 95 percent interval falls from 53 percent to 34 to 47 percent. In the risk tasks every configuration is far from humans (BART: 2 to 5 pumps per balloon against 37). The Discussion now states this.

Table R5 (revised paper: Table 16). Trajectory-level behavior by memory configuration: number of pre-specified behavioral features (44 features, nine tasks) inside the central 95% interval of the human-policy model's own rollouts, and paired change of log distance to the human-model median relative to Stateless (positive = farther; 95% task-bootstrap interval).

ConfigurationAll backbonesGPT LunaDeepSeekQwenDistance to human model vs Stateless
Stateless67/126 (53%)21/38 (55%)24/44 (55%)22/44 (50%)-
Cumulative43/127 (34%)13/39 (33%)11/44 (25%)19/44 (43%)+0.44 (+0.23 to +0.66)
Window55/127 (43%)15/39 (38%)18/44 (41%)22/44 (50%)+0.22 (-0.01 to +0.47)
Retrieval57/122 (47%)17/39 (44%)15/39 (38%)25/44 (57%)+0.17 (-0.01 to +0.34)
Surprise54/126 (43%)15/38 (39%)16/44 (36%)23/44 (52%)+0.30 (+0.07 to +0.55)

Table R5b (revised paper: Table 17). Examples of behavioral features by memory configuration (range over the three backbones). Memory brings the history-dependent signature in the first row into the human range, makes agents more likely to repeat previous decisions than humans in rows two to five, and leaves risk taking far from human levels (last two rows).

FeatureHuman-policy model (95%)StatelessCumulativeWindowRetrievalSurprise
Two-step: reward effect on repeating0.050 (-0.002 to 0.105)-0.013 to 0.0070.038 to 0.0710.029 to 0.058-0.084 to 0.0290.047 to 0.103
Two-step: repeats first choice0.49 (0.39 to 0.63)0.36 to 0.480.67 to 0.960.62 to 0.880.46 to 0.800.58 to 0.97
Two-step: entropy of first choices0.89 (0.72 to 0.97)0.77 to 0.970.15 to 0.560.44 to 0.910.36 to 0.850.09 to 0.79
Iowa: switch rate0.49 (0.27 to 0.72)0.14 to 0.330.03 to 0.200.02 to 0.170.05 to 0.420.05 to 0.22
Iowa: advantageous decks0.51 (0.36 to 0.67)0.53 to 0.610.70 to 0.910.55 to 0.890.37 to 0.610.71 to 0.78
BART: pumps per balloon37.4 (30.7 to 43.2)2.4 to 3.61.5 to 4.31.9 to 5.32.1 to 3.71.5 to 4.3
Columbia: cards per round5.1 (3.7 to 6.3)0.9 to 1.80.6 to 0.90.9 to 1.01.0 to 1.40.7 to 0.9

Q3. Could the authors validate the RNN on human responses collected after exposure to agent-generated histories, particularly those far from the training distribution?

We agree that collecting additional human data after exposure to agent-generated histories would be the most direct test. We did not collect new human data in the discussion period; the limitations now state this and describe the experiment that a direct test would require. However, we provided indirect evidence in Appendix L: behavioral signatures on held-out participants, autonomous rollouts and shared histories.

Thank you for the detailed response and new analyses, which directly address some of my questions and are candidly reported.

However, the new results largely confirm my main concern. Most of the memory gain in S (76–95%) is lower entropy. No configuration differs from Stateless in KL after Holm correction. On trajectory-level features, memory moves agents away from the human-policy model (53% → 34–47% of features inside the human interval). This supports “memory makes agents more predictable and perseverative,” not “more human-like.” The KL used, KL(agent‖human), is also mode-seeking (the greedy RNN policy beats every LLM agent) and covers only two backbones. Finally, the evaluator checks in Appendix L use only human-like histories, so the off-distribution concern remains, though it is now acknowledged.

The benchmark and closed-loop framing are valuable, but the abstract, introduction, and conclusions should be reframed around predictability/compatibility rather than human-likeness, with KL and trajectory measures reported alongside S. I will keep my current rating.

Reviewer_tZE6OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  1. This paper introduces MindGym, which turns 25 psychological tasks into interactive environments where LLM agents can act autonomously and receive feedback. The authors train an RNN for each task using human behavioral data to evaluate how compatible agents’ choices are with human behavior.
  2. The study compares five configurations across three LLMs: stateless, full-history, sliding-window, retrieval, and surprise-based event memory. And the results show that adding memory generally improves task performance and human-policy fit, while compact memory can retain most of the gains with fewer tokens.
Critical strengths
  1. Closed-loop evaluation is closer to how autonomous agents operate: It makes models bear the consequences of their own choices, allowing the observation of accumulating errors and behavioral changes.
  2. Broad task coverage and a unified interface: It integrates multiple cognitive domains, making it easier to systematically compare different models and memory strategies.
  3. Actively examines the limitations of its metrics: It reports counterexamples with high scores but abnormal behavior and provides comparisons of memory gains and token usage.
Critical weaknesses
  1. The human-likeness metric has a bias: Repeatedly choosing high-probability actions may receive high scores while failing to reproduce the diversity of human behavior.
  2. Insufficient experimental control and repetitions: Each setting usually has only five environment seeds, and different memory configurations have unequal compute budgets, limiting the strength of the conclusions.
Questions for authors

None

1 comment
AuthorsOpenReview ↗

Response to Reviewer tZE6

W1. The human-likeness metric has a bias: Repeatedly choosing high-probability actions may receive high scores while failing to reproduce the diversity of human behavior.

We agree, and we calculated an alternative metric based on KL divergence (see Eq. 13; revised paper: Table 2 and Appendix K). This KL is approximate and available only for models that return token probabilities (here DeepSeek and Qwen). We also scored simple policies with the same RNNs (Table 12; Table R2 in our response to Reviewer HABF): a policy that always chooses the same option exceeds held-out humans in S in 17 of 20 tasks, but under KL held-out humans are at about 0, constant and random policies at about 0.30, and the LLM agents at 0.14 to 0.24. The revised paper reports KL next to the original score and states that most of the gain reflects more predictable choice.

W2. Insufficient experimental control and repetitions: Each setting usually has only five environment seeds, and different memory configurations have unequal compute budgets, limiting the strength of the conclusions.

Budgets differ by design and are reported in Table 7 (Table R6 below). As suggested, we regressed each episode's outcome on the number of prompt tokens per decision, within task and backbone (Appendix M, Figure 13). We find that a tenfold increase in tokens goes with higher task performance (+0.40 SD, 95% interval 0.17 to 0.69), higher likelihood score S (+0.11, 0.06 to 0.17) and lower KL divergence (-0.029, -0.044 to -0.013). Because each memory configuration has a fixed budget, token count and configuration are almost collinear, so the regression cannot fully separate the amount of context from the memory mechanism. On seeds: seed-to-seed variation is about 2 percent of the variance of the pooled memory effect, which has the same sign in every seed.

Table R6 (token accounting in the revised paper: Table 7). Prompt tokens of the action request per decision (geometric mean over 20 tasks; reasoning tokens are zero in every run), with equal-task mean S and KL, ordered by increasing budget.

ConfigurationTokens: GPT LunaTokens: DeepSeekTokens: QwenS: GPT LunaS: DeepSeekS: QwenKL: DeepSeekKL: Qwen
Stateless1,996411370-0.523-0.637-0.7130.2000.166
Retrieval3,6512,0802,085-0.481-0.493-0.6190.2410.186
Window3,6532,0822,096-0.437-0.478-0.5960.1920.148
Surprise6,0634,9405,564-0.413-0.430-0.5760.1860.139
Cumulative20,12218,81419,323-0.380-0.409-0.5550.1460.136
Reviewer_m9kXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces MindGym, a suite of 25 interactive psychological tasks in which agents' actions determine later observations. It evaluates three LLM backbones under five memory configurations. Agents are scored on task performance and on how likely their choices are under GRUs trained on human trajectories. Memory generally improves performance and human-policy compatibility, with effects varying across tasks and models. Replay comparisons and trajectory checks examine what happens when agents encounter the consequences of their own actions. The analysis also shows that repetitive choices can receive high likelihood without reproducing human behavioral variability.

Critical strengths

[+] The 25-task suite evaluates autonomous agents whose choices affect later observations.

[+] Matched seeds, grouped human-data splits, and token accounting make the memory comparisons reasonably controlled.

[+] The authors examine failures of their own likelihood score and add replay and trajectory checks.

Critical weaknesses

[-] The likelihood score can reward repetitive choices, so higher completeness alone does not show closer behavioral similarity.

[-] The GRUs have not been validated on unusual agent-generated histories. The autonomous reference checks use the same fitted models.

[-] The text adaptations omit Stroop interference and response timing. No human validation uses these same interfaces.

Questions for authors
  1. Do memory rankings hold with alternative scorers and trajectory measures that penalize repetitive choices?
  2. How sensitive are the conclusions to task-interface changes? Could a small human comparison check this?
3 comments
AuthorsOpenReview ↗

Response to Reviewer m9kX (1/2)

W1. The likelihood score can reward repetitive choices, so higher completeness alone does not show closer behavioral similarity.

Indeed, we have noted this issue in Eq. 13. Alternatives for behavioral similarity between agent trajectories and human-policy RNNs could include (i) the KL divergence shown in the Eq. 13, or (ii) checking qualitative features. We calculated both and found no qualitative changes in our conclusion.

For the KL, we need the agent’s action probabilities in addition to the original likelihood scores. We can recover these for the two open-weight LLMs who return token log probabilities (DeepSeek and Qwen). For each decision in the original trajectories, we re-submitted the saved request and read the log probabilities of the valid action tokens at the decision position; the action probabilities are these values renormalized over the valid actions. Because the requests were re-submitted afterwards, the probabilities are an approximation (Appendix K). See the new Table 2 in Section 4; Appendix K, Table 11; Table R1 below.

Lower action entropy accounts for 76 to 95 percent of the memory gain in the likelihood score (i.e., S) under Cumulative, Window and Surprise, and for all of it under Retrieval. KL itself is still lowest under Cumulative for both backbones (-0.054 [-0.092, -0.019] and -0.031 [-0.072, 0.009] relative to Stateless) and highest under Retrieval, so Cumulative remains the closest configuration, with a much smaller effect than under S. The Discussion now says that most of the gain reflects more predictable choices. GPT-5.6 Luna is closed-source and returned no log probabilities, so KL covers two backbones and 20 tasks.

As a complement to the two quantitative metrics, we also analyzed qualitative behavioral patterns in our tasks (see Figures 10 to 12 in Appendix M). We find that memory brings some history-dependent effects to the human level (for example, repeating a rewarded option in the reward bandit), but it also makes agents more likely to repeat previous decisions than humans (e.g., in Castle two-step, after the goal is reversed, GPT and DeepSeek with memory repeat a rewarded first-stage choice 81% to 99% of the time, whereas human participants repeat only 25% of the time).

Table R1 (revised paper: Table 11; the KL column matches Table 2). Equal-task means over the 20 tasks with action probabilities, in nats per decision. S is the mean log score of the submission, which Table 1 rescales to completeness over all 23 tasks. KL is the plug-in estimate, KL = -S - H, where H is the entropy of the agent's action distribution recovered from token log probabilities; Table 11 (Appendix K) also gives a sample-based estimate, which uses the probability of the action the agent actually took, and both put Cumulative lowest and Retrieval highest. Δ: change from Stateless.

BackboneConfigurationOriginal likelihood score, SAction entropy: H(agent)KLΔSΔHΔKL
DeepSeekStateless-0.6370.4360.200---
Cumulative-0.4090.2630.146+0.228-0.174-0.054
Window-0.4780.2860.192+0.159-0.150-0.008
Retrieval-0.4930.2530.241+0.143-0.183+0.040
Surprise-0.4300.2440.186+0.207-0.192-0.014
QwenStateless-0.7130.5470.166---
Cumulative-0.5550.4200.136+0.158-0.127-0.031
Window-0.5960.4480.148+0.117-0.099-0.018
Retrieval-0.6190.4320.186+0.095-0.115+0.020
Surprise-0.5760.4370.139+0.137-0.110-0.028
GPT LunaStateless-0.523n/an/a---
Cumulative-0.380n/an/a+0.143n/an/a
Window-0.437n/an/a+0.086n/an/a
Retrieval-0.481n/an/a+0.042n/an/a
Surprise-0.413n/an/a+0.110n/an/a
AuthorsOpenReview ↗

Response to Reviewer m9kX (2/2)

W2. The GRUs have not been validated on unusual agent-generated histories. The autonomous reference checks use the same fitted models.

We added checks of RNN models against human participants (revised paper, Appendix L).

(1) Compared to the held-out participants, the RNNs reproduce task-specific behavioral signatures in 16 tasks (Fig. 9): the largest gap is at most 0.06 in 14 of them and at most 0.09 in the other two.

(2) Autonomous RNN rollouts match held-out humans on all 14 trajectory features that can be computed for both (Table 14; Table R4 below).

(3) On the same histories for every policy (the human-policy RNN first plays a task on its own, and this exact sequence of choices and outcomes is then shown to every agent and simple policy), LLM agents with memory (-0.71 to -0.81) score close to the RNN's own sampled choices (-0.69) and far above always choosing the same option (-1.28) or choosing at random (-1.46) (Table 15). Nevertheless, we could not test histories that have no human visits; the limitations now state this.

Table R4 (revised paper: Table 14). Held-out humans against autonomous rollouts of the human-policy model (the evaluator acting on its own). The held-out human mean lies inside the rollout 95% interval for all 14 features.

TaskBehavioral featureHeld-out humansModel rollouts, median (95% interval)
BARTAttempted pumps per balloon36.7037.43 (30.67 to 43.20)
BARTAdjusted pumps cash only40.2439.71 (31.37 to 47.78)
BARTPop rate0.3350.340 (0.253 to 0.420)
BARTLate-minus-early pumping1.6752.980 (-5.140 to 11.540)
Columbia cardCards attempted per round4.9745.121 (3.738 to 6.312)
Columbia cardLoss-card endpoint rate0.5890.586 (0.443 to 0.712)
Columbia cardPoints per round32.2835.08 (11.31 to 59.89)
Columbia cardHigher-loss minus lower-loss cards-1.357-1.145 (-2.682 to 0.273)
Columbia cardLate minus early attempted cards-0.716-0.721 (-1.779 to 0.357)
Iowa gamblingAdvantageous deck share0.5640.514 (0.358 to 0.672)
Iowa gamblingChange in advantageous deck share0.2040.110 (-0.150 to 0.380)
Iowa gamblingSwitch rate0.5170.491 (0.273 to 0.719)
Iowa gamblingShift after net loss0.7190.711 (0.426 to 0.893)
Iowa gamblingShift after net win0.4740.449 (0.231 to 0.685)

W3. The text adaptations omit Stroop interference and response timing. No human validation uses these same interfaces.

We agree and now state this in the limitations (Discussion). The text adaptations preserve the choice structure of each task but not perceptual interference or response timing, and no human data were actually collected using the text interface, which is an important direction for future research that wishes to conduct even closer comparisons between human and LLM agents.

Q1. Do memory rankings hold with alternative scorers and trajectory measures that penalize repetitive choices?

Partly. Under KL, which could penalize repetition, Cumulative is still lowest (i.e., best memory config) and Retrieval highest for both backbones and under both estimates. Retrieval exceeds Cumulative by 0.094 [0.051, 0.136] (DeepSeek) and 0.051 [0.013, 0.088] (Qwen); Window and Surprise do not differ reliably from Stateless (Table 2, Appendix K). Trajectory-level measures give a different picture: memory moves history-dependent features toward the human range but also makes agents more likely to repeat previous decisions than humans, most under Cumulative (features inside the human-model 95% interval: 53% under Stateless, 34 to 47% with memory; Appendix M, Tables 16 and 17). The revised paper now reports both.

Q2. How sensitive are the conclusions to task-interface changes? Could a small human comparison check this?

Most of our tasks are symbolic choice tasks whose original interface already consists of labelled options and numeric feedback, so the text version changes the presentation rather than the information available; the tasks that depend on timing or perceptual conflict are the exception. We now note this limitation of task-interface explicitly in the Discussion.

Thanks for the new analyses. I’m keeping my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers. The reviews share two common concerns: (i) the likelihood score can reward repetitive choices, and (ii) the human-policy RNNs are not validated on agent-generated histories. The revised paper (changes in blue) addresses both with new analyses of the original episodes; the results of the submission are unchanged. We highlight the following items:

(1) Behavioral compatibility (i.e., per-decision log-likelihood assigned by the human-policy RNNs) versus KL divergence (i.e., per-decision KL divergence between the agent trajectories and the learned RNN model): for DeepSeek and Qwen models, we recovered each agent's action probabilities from token log probabilities and split the score into action entropy and the KL divergence from the human-policy RNN (see new Table 2; Appendix K). Lower action entropy accounts for 76 to 95 percent of the memory gain in the score; consistent with our original rankings, KL is lowest under Cumulative and highest under Retrieval for both backbones. We couldn’t calculate the KL with GPT-5.6 Luna because it is closed-source and returned no log probabilities.

(2) Validating human-policy RNNs in the same environments: we find that the RNNs reproduce behavioral signatures of held-out participants in 16 tasks, and their rollouts match held-out humans on 14 trajectory features (see Figure 9 in Appendix L). The human-policy RNNs of weather classification and range learning are now trained with a longer schedule (Appendix B, Table 5); Table 1 and every number that depends on them are updated, and no qualitative changes in our original conclusion.

(3) Trajectory-level behavior described in qualitative features, prompt budgets and seeds: Appendix M.

(4) The Discussion now states the limitations: extrapolation to agent-generated histories, a text-only interface without response times, and an approximate KL for two of three backbones.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.