acceptodds
Under review as a conference paper at ICLR 2027

ESim: Long-Horizon Simulation of Everyday Human Behavior with Evolving States

Abstract

AI agents are increasingly expected to support users over long periods, across projects, work, and daily life. This requires modeling not only who users are, but also how their goals, interests, activities, and interactions evolve over time. A central challenge in longitudinal user simulation is maintaining consistency between these evolving states and the consequences of actions in a shared environment. We introduce ESim, a longitudinal user simulation framework that jointly models evolving actor states and a dynamic shared environment. ESim connects actor-state updates with shared-world changes through a feedback loop linking immediate actions, delayed outcomes, and actor-specific observations across work and life. This allows the consequences of earlier actions to shape later behavior across work and life, while preserving differences in what each actor knows. We further propose a suite of longitudinal trajectory diagnostics tailored to longitudinal simulation, measuring dynamic consistency, coherent event progression and collaboration, and non-degenerate trajectory development rather than static consistency alone. In one 60-day simulation, ESim produced more extracted event threads judged meaningful by our evaluator and fewer near-duplicate actions than four direct-generation baselines. Together, the framework and evaluation methods provide a basis for studying longitudinal user simulation, with potential applications in training and evaluating AI agents that assist users over extended periods.

Primary area:
generative models
Keywords:
longitudinal user simulation, LLM agents, user modeling, long-horizon agents, personalized AI, agent simulation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_gPV6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents E2Sim, a framework for long-horizon human behavior simulation that couples evolving actor states with a shared environment, actor-specific observations, and delayed consequences. It also introduces longitudinal diagnostics for consistency, event progression, collaboration, and non-degeneracy. Evaluation includes one 60-day run per method against four direct-generation baselines and 14-day continuation ablations. The clearest observed gains are reduced repetition and higher meaningful-event density; consistency results are mixed.

Critical strengths
  1. The framework addresses an important challenge: preserving persistent consequences and differences in actors’ knowledge across extended interactions. Its explicit treatment of shared-world updates and observation-dependent state changes provides a useful design for longitudinal simulation.

  2. The evaluation recognizes that locally coherent behavior can still become repetitive or inactive. Assessing non-degeneracy alongside consistency is consequential because conventional plausibility scores may overlook this failure mode.

Critical weaknesses
  1. Each method is evaluated only once. The main comparison uses one 60-day run per method from a single initial scenario. The bootstrap intervals reflect variation across actors, not independent runs. It is therefore unclear whether the reported gains would persist with different random seeds or scenario configurations.

  2. The baseline comparison is limited. All four baselines are direct-generation variants. The paper does not compare against existing simulation frameworks with persistent memory or environment state. It also does not report comparable inference costs, leaving unclear how much of the improvement comes from the proposed design versus additional model calls.

  3. The evaluation needs stronger validation. Most semantic judgments come from an LLM evaluator, without human agreement measurements. The perturbation tests cover selected collaboration errors but provide limited validation for the other metrics. In addition, the IDC procedure sometimes exposes the response when judging whether a state change was expected, which may bias the assessment.

  4. Event extraction may affect the main findings. The extractor links 1,276 actions for E2Sim, compared with 1,906–2,298 for the baselines. Event-quality scores thus cover different subsets of the trajectories. Meaningful-event density also depends on how actions are grouped into threads. The paper does not establish whether its conclusions hold under alternative grouping and abstention decisions.

  5. The ablations do not test several central mechanisms. They examine outcome sampling, external events, and goal creation, but not shared-world persistence, actor-specific visibility, or delayed consequences. The experiments therefore do not show what these mechanisms individually contribute.

  6. The practical value remains unclear. E2Sim reduces repetition and increases meaningful-event density, but does not consistently improve dynamic consistency or collaboration. The exploratory QA study shows that the generated history supports questions, but does not establish whether E2Sim is more useful than other simulators for training or evaluating assistants.

Questions for authors

I would ask the authors to primarily respond to the points raised in Critical Weaknesses, as these most strongly determine my assessment. Beyond those points.

4 comments
AuthorsOpenReview ↗

Response to Reviewer gPV6, Part 1

Thank you for the careful review and for recognizing that E²Sim addresses persistent consequences and differences in what actors know, that its shared-world updates and observation-dependent state changes are a useful design, and that assessing non-degeneracy alongside consistency matters because plausibility scores can miss repetitive or inactive trajectories. Our main responses:

  • Run variance: Within our budget, we reran three 10-day trajectories from each of two starting points (day 0 and day 13). All stay within IDC 92.8–98.3 and ECD 88.7–91.4.
  • Baselines and cost: We added Concordia and token usage for all methods. E²Sim makes far more calls than the prompting baselines and uses 91.9% fewer tokens than Concordia.
  • Evaluator validity: The IDC judge no longer sees the target day when forming its expectation, the ECD judge catches injected errors in controlled tests, and in a small blinded study (19 participants), readers more often picked E²Sim for clearer development and did not judge it less consistent.
  • Event extraction: Grouping varies, so we report meaningful-event density only as a supporting measure.
  • Ablations and practical value: The three highlighted mechanisms act on one shared record, so we evaluate them jointly. Downstream assistant gains are not shown.

We have revised the paper to address your comments. Changes are marked in blue, and we cite line numbers in the revised paper below.

Response to Weakness 1:

We agree that with one run per method, the actor-bootstrap intervals do not measure run variance. A full 60-day rerun of every method exceeded our budget. Instead, we restarted E²Sim from two points of the original run, the initial scenario (day 0) and a mid-run state (day 13), and generated three new 10-day trajectories from each. Restarting mid-run also tests whether stability depends on starting from the initial scenario. The revised metrics need seven days of history, so the scored windows are short. Run 2 from day 0 uses a later code version, and the original run is shown for reference only.

Table R1. Repeated runs from two starting points (same scenario and model).

StartMetricScored daysOriginalRun 1Run 2Run 3
Day 0IDC ↑9–1089.2998.3396.2893.41
Day 0ECD ↑8–1093.8589.6391.3790.23
Day 13IDC ↑14–2393.3095.7295.7292.84
Day 13ECD ↑14–2390.8689.8889.0088.74

Runs from both starting points stay within IDC 92.8–98.3 and ECD 88.7–91.4. These windows are not comparable to the full-run results, and IDC excluded 3/80 and 18/400 invalid judge outputs. We did not rerun the baselines or test other scenarios and models. The IDC spread across these reruns (about 5.5 points) is larger than the IDC gaps among E²Sim, Guided, Per-Actor and Summary, so we do not read those gaps as differences. The ECD spread (about 2.7 points) is smaller than E²Sim's ECD lead over the strongest baseline (6.8 points).

Reading the day-13 runs, each keeps the unfinished tasks from the shared history, but they end differently: the open item carries over, and its outcome is not scripted (Table R2).

Table R2. A day-13 open item in three new runs: a study whose evaluation jobs had produced no results.

RunDays 14–23State on day 23
OriginalMeasurements recorded and claims narrowed.Submitted on day 17.
Run 1Access stays unavailable. The student drafts other sections, and a collaborator joins after receiving a request.Submitted, with venue receipt recorded.
Run 2A data mismatch and resource errors force reruns, and each claim is checked against the results.Claims cleared, but the submission site fails.
Run 3Repeated failed contacts, then a collaborator joins and finds a working way to run jobs on the cluster.Some jobs launched, with no results yet.

Schedule changes also come with recorded reasons: in two runs, a confirmed Tuesday accessibility test moves to Thursday after a failed sign-in or a missing auditor confirmation.

Response to Weakness 2:

To separate the effect of the design from that of extra model calls, we compared with Concordia (v2.4.0, its planning agent and a game master built from its own components), using the same model, scenario and ten actors. In a traced case, Concordia resolved a bank sign-off and launch in one action, while E²Sim kept them open.

Table R3. Usage over seven simulated days.

MethodAPI callsTotal tokens (M)Estimated API cost
E²Sim4,06934.634$6.83
Concordia22,978429.697$102.69

E²Sim makes far more calls than the prompting baselines (4,069 vs. 7–71) and uses more tokens (34.6M vs. 0.17–0.32M), so part of its gain may come from additional computation. Costs use reference prices, not invoices.

AuthorsOpenReview ↗

Response to Reviewer gPV6, Part 2

Response to Weakness 3

The reviewer asks for stronger validation of the LLM evaluator, and points out that the IDC judge could see the response when forming its expectation. Following this point, we made a minor change to IDC, which checks whether changes in an actor’s state are supported by its history. The judge now (L324–327, L730–733). E²Sim scores 93.36, above Direct (82.49) and Guided (91.57) but below Per-Actor (97.22) and Summary (98.24). Where state stays the same, it matches them (3.94 vs. 3.90–3.93 out of 4).

We made a similar minor change to ECD, which checks whether decisions involving other actors respect dependencies. It now scores sampled decisions that name another actor, using both actors’ records, so it no longer depends on extracted threads (Weakness 4). It also flags repeated requests, overly long waits, and unsupported claims (L337–338, L758–762). E²Sim scores 87.85 vs. 72.22–81.03.

Table R4. Controlled checks of the ECD judge.

Judge promptInjected errors caughtLong waits caughtHarmless edits not penalized
Revised9 / 10 valid4 / 43 / 3

These checks show that the judge catches injected errors, not that it agrees with people. We did not repeat such checks for revised IDC. The exact-repetition rate in our ablations is computed directly from the text, without a judge. The near-duplicate rate is judged by the evaluator. For human judgment, 19 participants compared anonymous short segments from E²Sim and each baseline in a blinded study, choosing the one more consistent with the history, the one with clearer development, and optionally the more believable one (Table R5).

Table R5. Blinded reading study with 19 participants (E²Sim / baseline / no clear difference).

E²Sim vs.More consistent with historyClearer developmentMore believable (optional)
Direct12 / 7 / 014 / 3 / 213 / 5 / 1
Guided10 / 6 / 315 / 4 / 012 / 7 / 0
Per-Actor11 / 5 / 313 / 6 / 07 / 7 / 5
Summary9 / 8 / 213 / 2 / 413 / 2 / 4
Total42 / 26 / 855 / 15 / 645 / 21 / 10

Across 76 judgments (19 participants × 4 pairs), readers more often picked E²Sim for clearer development against every baseline. On consistency, they did not favor the baselines, including Per-Actor and Summary, which score higher on IDC. The study is small, so we treat it as a sanity check on the automatic metrics rather than a full human evaluation.

Response to Weakness 4

We agree that event grouping differs across methods, and will state that event-quality scores cover different action subsets (1,276 vs. 1,906–2,298). We asked the extractor to split 30 sampled actions per method into events twice with the original prompt (A and B), and once each with instructions for coarser and finer splits (Table R6).

Table R6. Segmentation of 30 sampled actions per method.

MethodEvents found (A)CoarserFinerSame split in A and BOverlap of split points ↑
Direct97831108 / 300.46
Guided987810410 / 300.53
Per-Actor100771398 / 300.42
Summary857310711 / 300.58
E²Sim59408912 / 300.65

How actions are grouped varies for every method. We did not recompute meaningful-event density under these alternatives, so we report it only as a supporting measure and base the collaboration claim on revised ECD, which does not group actions into threads.

Response to Weakness 5

The reviewer asks what shared-world persistence, visibility, and delayed consequences each contribute. Our ablations target random selection, external events and new Stage Goals (Table R7), and we evaluate these three mechanisms differently. Shared-world persistence, visibility and delayed consequences operate on the same shared record (Figure 4), and the baselines lack all three, so the baseline gap reflects their combined effect. We therefore evaluate them jointly, through what they should produce. On handling what others must deliver, E²Sim scores 74.3 vs. 34.3–47.8 for the baselines, and on building on earlier steps 83.6 vs. 37.3–58.2 (sub-scores of the revised ECD). In the Concordia comparison, E²Sim keeps a bank sign-off pending for days, while Concordia resolves it in one action. Runs restarted from day 13 keep the unfinished tasks from the shared record (Table R2). We will state in the paper that these three mechanisms are evaluated jointly.

Table R7. Ablations over days 61–74, rerun from one day-60 state (one run per setting).

MeasureFull E²SimNo random selectionNo external eventsNo new Stage Goals
Actions that did not go as planned49%19%46%47%
Responses to outside events per day1.141.860.000.93
Time with no active project7%13%11%26%
Exact repeated actions2.0%8.0%3.3%5.6%
AuthorsOpenReview ↗

Response to Reviewer gPV6, Part 3

Response to Weakness 6

We agree that the practical value for assistants is the key question. E²Sim improves collaboration and reduces repetition, while IDC is mixed. Our QA study (Table R8) shows that the trajectories require long-horizon memory, but not that they improve an assistant or are more useful than other simulators' outputs. We will narrow this claim. The 80 questions come from the 60-day trajectory: half can be answered from one record, and half need information across long periods or several actors. In the notes condition, the model reads the records in order, keeps its own compressed notes, and answers from those notes plus the last three days.

Table R8. QA accuracy on 80 four-option questions about the E²Sim history (chance 25%).

Reader modelFull record ↑Own notes + last 3 days ↑
GPT-5.6 Luna92.1%51.7%
GPT-6 Luna96.7%50.0%
AuthorsOpenReview ↗

Response Follow-up

Dear reviewer, thank you again for your detailed review. As the discussion period is coming to an end, we would like to kindly check whether our response has addressed your concerns. We would be happy to answer any further questions.

Reviewer_qoaGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a framework for long-horizon simulation of everyday human behavior using LLMs. It jointly models actor states and a shared environment, allowing actions and their consequences to affect subsequent behavior.

Critical strengths
  • This paper targets an important question of social simulation, i.e., how to simulate huam behavior in a long horizon.
  • This paper handles a failure mode in long-horizon agent simulation: locally plausible behavior can still be repetitive and stagnant.
Critical weaknesses
  • Limited technical contributions. The proposed framework is mainly based on prompting using the LLM API (small models and models with minimal thinking capabilities in OpenAI GPT series). It remains unclear whether the proposed framework would still offer substantial benefits over baseline agents driven by stronger models and greater context windows,
  • Insufficient baseline comparisons. All four baselines are direct-generation variants, without comparisons with closely related social simulation frameworks.
  • Missing details behind the performances. The framework would introduce extra computational costs, which should be parts of the evaluation.
Questions for authors
  • What if the underlying LLM switches to those with greater capabilities and context windows?
  • How much additional computational cost does the proposed framework introduce, and do the observed gains justify this additional cost?
4 comments
AuthorsOpenReview ↗

Response to Reviewer qoaG, Part 1

Thank you for your feedback and for recognizing that long-horizon simulation of human behavior is an important question, and that E²Sim targets a key failure mode: individual actions can be plausible even as the trajectory becomes repetitive and stagnant. Our main responses:

  • Technical contribution: The LLM proposes possible outcomes, and the framework picks one and controls when and to whom its consequences arrive.
  • Model choice: The same seven-day E²Sim run would cost USD 170.86 on GPT-5.6 Sol instead of USD 6.83, so we build the framework on a low-cost model.
  • Existing simulators: We added a seven-day comparison with Concordia.
  • Cost: E²Sim uses more tokens than the prompting baselines and 91.9% fewer than Concordia.

We have revised the paper to address your comments. Changes are marked in blue, and we cite line numbers in the revised paper below.

Response to Weakness 1 The reviewer asks what the framework contributes beyond prompting, and whether this still holds with stronger models. Each component does call an LLM, but our contribution is what the framework does around these calls:

  • Outcomes drawn in code. The model lists possible outcomes and the code draws one at random, because a model left to choose picks the smooth outcome.
  • Recorded state changes. Goals, plans and projects change only through recorded events.
  • Shared world with per-actor visibility. Each actor's context contains only what it can observe.
  • Delayed consequences. Pending outcomes, such as a bank's reply, resolve in later time slots (Table R2).
  • Long-horizon evaluation that checks whether trajectories stay consistent or become repetitive over weeks. A stronger model or a longer context window could improve the baselines and narrow the gap, but we have not tested this. Cost also matters, because it grows with every simulated day (Table R1).

Table R1. Estimated API cost of seven simulated days at the same token usage.

MethodTotal tokens (M)GPT-5.6 Luna (as run)GPT-5.6 Sol (estimated)
E²Sim34.6$6.83$170.86
Concordia429.7$102.69$2,567.18

Sol's per-token rates are 25 times Luna's. With this low-cost model, E²Sim already reduces repetition and improves collaboration over the prompting baselines (ECD 87.85 vs. 72.22–81.03).

Response to Weakness 2 To compare with an existing social simulation framework rather than only direct-generation baselines, we ran Concordia, which, like E²Sim, resolves actions through a game master over a shared world. We compared with Concordia (v2.4.0, its planning agent and a game master built from its own components, with details in the Appendix), using the same model, scenario, ten actors and seven days. With one run per system, we trace one case (Table R2).

Table R2. A product launch that requires partner-bank sign-off.

TimeConcordiaE²Sim
Day 2, 12:00A single action resolves as "the product received the required sign-offs and launched safely."The bank integration test fails with an authorization error, and the rerun waits for new credentials.
Days 2–4One actor records monitoring "the stable live product" in six consecutive slots.Actors list credential checks and keep following up with the bank. Several bank channels return "service unavailable."
Days 6–7The same request for compliance acknowledgments repeats in five records.The rerun passes. The sign-off request is delivered but still unanswered.
AuthorsOpenReview ↗

Response to Reviewer qoaG, Part 2

Response to Weakness 3:

We agree that the added computational cost belongs in the evaluation. Table R3 reports it for E²Sim and Concordia over seven simulated days.

Table R3. Seven-day efficiency, same scenario and model.

E²SimConcordia
API calls4,06922,978
Total tokens (M)34.6429.7
Input cache hit rate48.4%6.3%
Estimated API cost$6.83$102.69
Time from first to last API call (min)71.25605.54

Costs use the same list prices for both systems and are not invoices. Concordia runs actors one at a time while we ran ten in parallel, and reasoning settings differed, so not all of the time difference comes from the design. E²Sim's higher cache hit rate (48.4% vs. 6.3%) also lowers its cost. The prompting baselines use far fewer tokens (0.17–0.32M over seven days).

Response to Question 1:

On stronger models and longer context windows, see our response to Weakness 1. A stronger model could narrow the gap, which we have not tested. Running E²Sim itself on GPT-5.6 Sol would cost about 25 times more.

Response to Question 2:

On how much the framework adds in cost and what it buys, Weaknesses 1 and 3 give the details. In short, E²Sim uses about 110–210 times more tokens than the prompting baselines (34.6M vs. 0.17–0.32M over seven days), because it also generates candidate outcomes, judges them and updates actor and world states. In return, it cuts the near-duplicate rate from 60.5–73.5% to 22.3% and raises ECD from 72.22–81.03 to 87.85, while consistency (IDC) is mixed. The full seven-day run with ten actors costs $6.83. It uses 91.9% fewer tokens than Concordia at 93.3% lower estimated API cost.

AuthorsOpenReview ↗

Response Follow-up

Dear reviewer, thank you again for your detailed review. As the discussion period is coming to an end, we would like to kindly check whether our response has addressed your concerns. We would be happy to answer any further questions.

Thank you for the response. The proposed method uses roughly 110–210 times as many tokens as the prompting baselines. Given that gap, my main question remains whether a baseline using a stronger model or a longer context could achieve comparable trajectory quality in the same budget. Additionally, the Concordia comparison is not robust or convincing enough. I therefore maintain my original score.

Reviewer_PakZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

the paper studies the generation of long-horizon everyday action trajectories from an initial scenario that specifies actors, their relationships, and their shared world. the trajectories of interest can span multiple weeks because the goal of the trajectories is to serve for training and evaluating personal assistants that could serve a user for multiple months. it is a challenging problem and worth studying.

the proposed method is called e2sim, a framework that initializes from a scenario an actor state and shared environment. the actor state includes both static and dynamic states. the shared environment is a representation of facts, communication and follow-up events. world changes are also considered in the framework.

the paper further introduces a suite of longitudinal evaluation metrics for benchmarking the generated trajectories.

Critical strengths
  1. the problem is interesitng for sure. personal assitants are becoming more and more popular and serving users for months and perhaps years. the action trajectories need to span at least weeks to be useful for training and evaluation purposes.

  2. the suite of evaluation metrics proposed in the paper are helpful as well.

  3. i also appreciate the transparency of reporting evaluation results in the paper since the proposed method doesn't seem to be outperforming in major metrics.

Critical weaknesses
  1. the propose metrics do not really favor e2sim. consistency is an important metric however table 1 shows a direct baseline method beats e2sim on IDC metrics. i understand that e2sim wins on near-duplicate rate however it doesn't justify holistically that e2sim is a better approach given it's additional complexity.

  2. a bigger problem is there is no human eval in this paper. this is quite critical from my perspective. the purpose of the paper is to generate action trajectories to simulate human users. however, without human evaluation, how can we tell if the generated trajectories make sense?

  3. the variance of table 1's results is unknown. the results are generated by one single run...

  4. the goal of generated trajectories is to improve personal assistant. ideally, the best evaluation is to put these trajectories into a model and see how the model improves. if authors include those experiments, the paper could've be much stronger.

Questions for authors
  1. what's the token usage for e2sim compared to baseline methods?

  2. please respond to my questions above about the evaluations.

  3. do you plan to release the code?

5 comments
AuthorsOpenReview ↗

Response to Reviewer PakZ, Part 1

Thank you for your feedback and for recognizing that long-horizon action trajectories are an important problem for personal assistants that serve users over months, that our evaluation metrics are helpful, and for valuing our transparent reporting. Our main responses:

  • Table 1: With the clarified IDC and ECD (see Weakness 1), E²Sim leads all baselines on collaboration (ECD), matches the strongest baselines on consistency (IDC) when an actor's state stays the same, and scores above Direct on overall IDC.
  • Human evaluation: In a small blinded study (19 participants, 4 pairs each), readers more often picked E²Sim for clearer development (55 vs. 15 of 76 judgments) and did not judge it less consistent (42 vs. 26).
  • Run variance: Within our budget, we reran three 10-day trajectories from each of two starting points (day 0 and day 13). All stay within IDC 92.8–98.3 and ECD 88.7–91.4.
  • Assistant evaluation: Not run. A QA study shows that the trajectories require long-horizon memory.
  • Tokens and code: E²Sim uses more tokens than the prompting baselines and 91.9% fewer than Concordia. We will release the code upon acceptance.

We have revised the paper to address your comments. Changes are marked in blue, and we cite line numbers in the revised paper below.

Response to Weakness 1

We agree with the reviewer that lower repetition alone would not justify the added complexity, and that the consistency results in Table 1 deserve a closer look. We aim to keep plausible single actions, which the base model already handles well, while improving behavior over weeks.

IDC checks whether changes, or their absence, in an actor’s communication, behavior, goals, and interests are supported by its history. ECD checks whether a decision involving another actor respects dependencies and builds on earlier steps. Following the reviewer’s comments, we made two minor changes to what the judge reads, without changing what either metric measures, and recomputed both for every method. IDC now scores a whole actor-day from the records, forming its expectation from the previous seven days before seeing that day. ECD now scores sampled decisions that name another actor, using both actors’ records, and flags repeated requests, overly long waits, and unsupported claims (L324–338, L730–733, L758–762). The evaluation now separates the methods more clearly: ECD spans 72.22–87.85, and its judge catches 9 of 10 injected errors and 4 of 4 long waits (L349–351, L801–803). Since a judge tends to accept unchanged items, we also report IDC on unchanged items (Table R1, Table 1 at L378–389).

Table R1. Revised metrics on the 60-day runs (one run per method, 10 actors).

MethodIDC ↑Unchanged state itemsIDC on unchanged items (0–4) ↑ECD ↑
Direct82.4995%3.3472.22
Guided91.5788%3.7081.03
Per-Actor97.2291%3.9079.81
Summary98.2489%3.9375.78
E²Sim93.3639%3.9487.85

E²Sim now scores above Direct on IDC but below Per-Actor and Summary, which rarely change an actor's state. Where state stays the same, E²Sim is as consistent as they are. On ECD, it leads every baseline. Its share of near-duplicate actions, as judged by the evaluator, is also lower (22.3% vs. 60.5–73.5%).

AuthorsOpenReview ↗

Response to Reviewer PakZ, Part 2

Response to Weakness 2:

We agree that human evaluation is needed to judge whether the trajectories make sense. A 60-day trajectory has 3,600 actions, too many to read in full, so we evaluate short sampled segments in a blinded study. Readers see an actor's previous day and two anonymous two-day continuations (E²Sim vs. one baseline), and pick the one more consistent with the history and the one with clearer development. The pairs cover all ten actors over days 6–54, and an optional third question asks which is more believable.

Table R2. Blinded reading study with 19 participants (E²Sim / baseline / no clear difference).

E²Sim vs.More consistent with historyClearer developmentMore believable (optional)
Direct12 / 7 / 014 / 3 / 213 / 5 / 1
Guided10 / 6 / 315 / 4 / 012 / 7 / 0
Per-Actor11 / 5 / 313 / 6 / 07 / 7 / 5
Summary9 / 8 / 213 / 2 / 413 / 2 / 4
Total42 / 26 / 855 / 15 / 645 / 21 / 10

Across 76 judgments (19 participants × 4 pairs), readers more often picked E²Sim for clearer development against every baseline. On consistency, they did not favor the baselines, including Per-Actor and Summary, which score higher on IDC. The study is small, so we treat it as a sanity check on the automatic metrics rather than a full human evaluation.

Response to Weakness 3:

We agree that a single run leaves the variance of Table 1 unknown. A full 60-day rerun of every method exceeded our budget. Instead, we restarted E²Sim from two points of the original run, the initial scenario (day 0) and a mid-run state (day 13), and generated three new 10-day trajectories from each. Restarting mid-run also tests whether stability depends on starting from the initial scenario. The revised metrics need seven days of history, so the scored windows are short. Run 2 from day 0 uses a later code version, and the original run is shown for reference only.

Table R3. Repeated runs from two starting points (same scenario and model).

StartMetricScored daysOriginalRun 1Run 2Run 3
Day 0IDC ↑9–1089.2998.3396.2893.41
Day 0ECD ↑8–1093.8589.6391.3790.23
Day 13IDC ↑14–2393.3095.7295.7292.84
Day 13ECD ↑14–2390.8689.8889.0088.74

Runs from both starting points stay within IDC 92.8–98.3 and ECD 88.7–91.4. These windows are not comparable to the full-run results, and IDC excluded 3/80 and 18/400 invalid judge outputs. We did not rerun the baselines or test other scenarios and models. The IDC spread across these reruns (about 5.5 points) is larger than the IDC gaps among E²Sim, Guided, Per-Actor and Summary, so we do not read those gaps as differences. The ECD spread (about 2.7 points) is smaller than E²Sim's ECD lead over the strongest baseline (6.8 points).

Reading the day-13 runs, each keeps the unfinished tasks from the shared history, but they end differently: the open item carries over, and its outcome is not scripted (Table R4).

Table R4. A day-13 open item in three new runs: a study whose evaluation jobs had produced no results.

RunDays 14–23State on day 23
OriginalMeasurements recorded and claims narrowed.Submitted on day 17.
Run 1Access stays unavailable. The student drafts other sections, and a collaborator joins after receiving a request.Submitted, with venue receipt recorded.
Run 2A data mismatch and resource errors force reruns, and each claim is checked against the results.Claims cleared, but the submission site fails.
Run 3Repeated failed contacts, then a collaborator joins and finds a working way to run jobs on the cluster.Some jobs launched, with no results yet.

Schedule changes also come with recorded reasons: in two runs, a confirmed Tuesday accessibility test moves to Thursday after a failed sign-in or a missing auditor confirmation.

AuthorsOpenReview ↗

Response to Reviewer PakZ, Part 3

Response to Weakness 4

We agree that testing whether the trajectories improve an assistant is the most direct test, and we have not run it. Our QA study (Table R5) shows that the trajectories require long-horizon memory: for the two stronger readers, accuracy drops by about half (from 92–97% to 50–52%) when they keep only their own notes plus the last three days. It does not show training gains or an advantage over other simulators. We built the 80 questions from the 60-day trajectory: 40 can be answered from one record, and 40 need information across long periods or several actors. We checked that every answer is in the trajectory. In the notes condition, the model reads the records in order, keeps its own compressed notes, and answers from those notes plus the last three days. Each question is answered three times. On the 40 harder questions, notes-only accuracy is 32.5–35% for every model.

Table R5. QA accuracy on 80 four-option questions about the E²Sim history (chance 25%).

Reader modelFull record ↑Own notes + last 3 days ↑
GPT-5.4 nano56.3%43.8%
GPT-5.6 Luna92.1%51.7%
GPT-6 Luna96.7%50.0%

For GPT-5.4 nano and GPT-6 Luna, a backup summarization setting was sometimes used, which slightly helps their notes-only scores.

Response to Question 1

Table R6 compares E²Sim's token usage with the prompting baselines and Concordia over seven simulated days.

Table R6. Token usage over seven simulated days (ten actors, same initial scenario).

MethodAPI callsInput (M)Output (M)Total (M)
E²Sim4,06932.7221.91334.634
Direct70.1370.0290.166
Guided70.1400.0290.169
Per-Actor710.2810.0390.320
Summary80.1410.0350.176
Concordia22,978428.1721.525429.697

The prompting baselines write each day in one or a few calls, while E²Sim also generates candidate outcomes, judges them, and updates actor and world states. E²Sim used 91.9% fewer tokens than Concordia, with an estimated API cost of USD 6.83 vs. USD102.69.

Response to Question 2

The evaluation points raised above are covered in our responses to Weaknesses 1–4. Briefly, E²Sim leads on collaboration and matches the strongest baselines where state stays the same. In a small blinded study, readers more often picked E²Sim for clearer development and did not judge it less consistent. Runs restarted from day 0 and day 13 stay within IDC 92.8–98.3 and ECD 88.7–91.4. Downstream assistant training remains untested.

Response to Question 3

Yes, we will release the code and make it publicly available upon acceptance.

Reviewer_HpEhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces E2Sim, a framework for longitudinal long horizon simulation of human behaviour in everyday task scenarios. The framework jointly models evolving actor states and a dynamic shared environment. The paper distinguishes between the stable and dynamic aspects of the actor and the actions they can take. The framework also sets up an actor-environment interaction model and a shared space for multi-actor interaction. The authors also propose longitudinal evaluation metrics in E2Sim that measure individual dynamic consistency, event progression, collaboration, and trajectory non-degeneracy. In experiments across a 60-day setup with 10 actors, E2Sim shows fewer repetitive actions and newer event threads than baselines.

Critical strengths
  1. The paper addresses a relevant problem in long-horizon simulation on considering the actor and the environment and their influence on each other.
  2. The paper introduces a conceptually useful set of evaluation metrics for long-horizon simulation, by covering how new events, collaborations and repetitive actions develop in simulation.
Critical weaknesses
  1. I am not convinced that the experiments sufficiently establish the contribution of the proposed framework relative to existing simulation approaches.
  • The experiments contain only one scenario, one generator model and one set of runs per condition. Apart from the gap in repetition metrics, I am unable to interpret how the method performs against the baselines on the other dimensions given that they are so close. E2SIM is in fact worse on consistency metrics, so what was the point of the structured action and environment coupling?
  • I am unable to understand which parts of the framework are actually working and contributing to the claims of the method. The ablation only looks at random draw among candidate paths, external events, or new Stage Goal creation. More insight into how mechanisms like shared-world feedback, actor-specific visibility, and delayed consequences are contributing is required.
  • Several of the core mechanisms, including persistent environments, evolving agent states, and environment-mediated interactions, have been explored in prior work such as Generative Agents, Concordia, and LifeSim. Why not use these as baselines? I am unable to discern the novel contributions of E2SIM apart from it being a conceptually useful integration of the above concepts.
  1. The evaluation setup, while thoughtful, lacks human validation or calibration. All metrics employed are LLM-judge based. Although the authors introduce perturbation based checks, these are not actually useful in understanding whether the judges are in fact aligned with individual human judgements of the trajectory quality. More importantly, the paper motivates its framework through consistency between actor states and the shared environment, but does not sufficiently evaluate whether these mechanisms work correctly, and whether this affects realism of the framework at all.
Questions for authors
  1. Can the authors provide more targeted ablation isolating the central mechanisms they propose as a contribution of their framework?
  2. With just a single scenario/model/run, it is very hard to say if the results generalise or are reproducible. Could the authors run more variations to support the robustness of their results?
  3. Could the authors validate their proposed evaluation framework with independent human raters?
  4. Could you try other baselines that I have listed in the weaknesses?
  5. Please see weaknesses for other concerns
6 comments
AuthorsOpenReview ↗

Response to Reviewer HpEh, Part 1

Thank you for your detailed feedback and for recognizing that E²Sim addresses a relevant problem in how actors and their environment influence each other over long horizons, and that our metrics usefully cover how new events, collaboration, and repetition develop. Our main responses:

  • Gains beyond repetition: With the clarified IDC and ECD (see Weakness 1), E²Sim leads all baselines on collaboration (ECD) and matches the strongest baselines on consistency (IDC) when an actor's state stays the same.
  • Components: Removing a component mainly hurts what it targets. For example, without random selection, actions that do not go as planned drop from 49% to 19%.
  • Existing simulators: Compared with Concordia, E²Sim keeps a pending bank approval open instead of resolving it at once, at 93.3% lower estimated API cost.
  • Robustness: Within our budget, we reran three 10-day trajectories from each of two starting points (day 0 and day 13). All stay within IDC 92.8–98.3 and ECD 88.7–91.4.
  • Human validation: In a small blinded study (19 participants, 4 pairs each), readers more often picked E²Sim for clearer development (55 vs. 15 of 76 judgments) and did not judge it less consistent (42 vs. 26).

We have revised the paper to address your comments. Changes are marked in blue, and we cite line numbers in the revised paper below.

Response to Weakness 1:

The reviewer asks what E²Sim gains beyond lower repetition, which parts of the framework drive these gains, and how it differs from Generative Agents, Concordia, and LifeSim. We first note that we do not claim E²Sim wins on every dimension. We aim to keep plausible single actions, which the base model already handles well, while improving behavior over long-term horizon. IDC checks whether changes, or their absence, in an actor’s communication, behavior, goals, and interests are supported by its history. ECD checks whether a decision involving another actor respects dependencies and builds on earlier steps. Following the reviewer’s comments, we made two minor changes to what the judge reads, without changing what either metric measures, and recomputed both for every method. IDC now scores a whole actor-day from the records, forming its expectation from the previous seven days before seeing that day. ECD now scores sampled decisions that name another actor, using both actors’ records, and flags repeated requests, overly long waits, and unsupported claims (L324–338, L730–733, L758–762). The evaluation now separates the methods more clearly: ECD spans 72.22–87.85, and its judge catches 9 of 10 injected errors and 4 of 4 long waits (L349–351, L801–803). Since a judge tends to accept unchanged items, we also report IDC on unchanged items and at an equal change rate (Table R1, Table 1 at L378–389). Table R1. Revised metrics on the 60-day runs (one run per method, 10 actors).

MethodIDC ↑Unchanged state itemsIDC on unchanged items (0–4) ↑IDC at equal change rate ↑ECD ↑ECD: dependencies ↑ECD: building on earlier steps ↑
Direct82.4995%3.3480.6372.2234.337.3
Guided91.5788%3.7090.6781.0347.758.2
Per-Actor97.2291%3.9097.3579.8147.853.7
Summary98.2489%3.9398.1075.7841.546.6
E²Sim93.3639%3.9496.8287.8574.383.6

IDC scores four state items per actor-day (communication, behavior, goals, interests). The baselines rarely change them, and the judge usually accepts an unchanged item. On unchanged items, E²Sim matches the strongest baselines (3.94 vs. 3.90–3.93 out of 4). If all methods changed state equally often, the IDC gap would shrink to 0.5–1.3 points. On ECD, E²Sim leads every baseline, mainly in handling what others must deliver and building on earlier steps. It is slightly lower on consistency with recorded events and roles (events 94.5, roles 91.2, vs. 96.3–99.8 for the baselines). Its share of near-duplicate actions, as judged by the evaluator, is also lower (22.3% vs. 60.5–73.5%).

AuthorsOpenReview ↗

Response to Reviewer HpEh, Part 2

Response to Weakness 1 (continue):

For the ablations, each removes one component and reruns days 61–74 from the same day-60 state (Table R2).

  • Random selection. Real attempts often stall or fail, and these setbacks change what people do next. A model that chooses outcomes itself picks the smooth one. In a preliminary run, a project that typically takes 3–5 months finished in about a week. Without the random draw, setbacks largely disappear, and the only obstacle left is other actors not replying, which turns into repeated follow-up checks.
  • External events. Real people keep reacting to news and chance events. Without them, no actor reacted to anything outside its own plans in 14 days.
  • New Stage Goals. Without them, actors have no active project a quarter of the time.

Table R2. Ablations over days 61–74 (one run per setting).

MeasureFull E²SimNo random selectionNo external eventsNo new Stage Goals
Actions that did not go as planned49%19%46%47%
Responses to outside events per day1.141.860.000.93
Time with no active project7%13%11%26%
Exact repeated actions2.0%8.0%3.3%5.6%

The three mechanisms you highlight work together (Figure 4): the shared record stores a request, a delayed reply arrives later, and visibility decides who receives it. The baselines lack all three, so their gap reflects the combined effect. Two ECD scores measure what this combination should produce: handling what others must deliver (74.3 vs. 34.3–47.8) and building on earlier steps (83.6 vs. 37.3–58.2, Table R1). The Concordia comparison shows a delayed consequence directly: E²Sim keeps a bank sign-off pending for days, while Concordia resolves it in one action (Table R3). Runs restarted from day 13 also keep the unfinished tasks from the shared record (Table R7). On prior work, E²Sim builds on Generative Agents, Concordia and LifeSim, but focuses on two problems: consequences that arrive later or reach actors at different times, and steady progress across days. Our contribution is what the framework does around the LLM calls: the model lists possible outcomes and code draws one, goals and plans change only through recorded events, each actor sees only what it can observe, and pending outcomes resolve in later time slots. We also contribute metrics that check whether long trajectories stay consistent and keep developing. Within our budget, we could run one existing framework and chose Concordia, which, like E²Sim, resolves actions through a game master over a shared world. We compared with Concordia (v2.4.0, its planning agent and a game master built from its own components, with details in the Appendix), using the same model, scenario, ten actors and seven days.

Table R3. A product launch that requires partner-bank sign-off.

TimeConcordiaE²Sim
Day 2, 12:00A single action resolves as "the product received the required sign-offs and launched safely."The bank integration test fails with an authorization error, and the rerun waits for new credentials.
Days 2–4One actor records monitoring "the stable live product" in six consecutive slots.Actors list credential checks and keep following up with the bank. Several bank channels return "service unavailable."
Days 6–7The same request for compliance acknowledgments repeats in five records.The rerun passes. The sign-off request is delivered but still unanswered.

Table R4. Seven-day efficiency, same scenario and model.

E²SimConcordia
API calls4,06922,978
Total tokens (M)34.6429.7
Estimated API cost$6.83$102.69
Time from first to last API call (min)71.25605.54
Costs use the same list prices for both systems and are not invoices. Concordia runs actors one at a time while we ran ten in parallel, and reasoning settings differed, so not all of the time difference comes from the design.
AuthorsOpenReview ↗

Response to Reviewer HpEh, Part 3

Response to Weakness 2:

The reviewer is concerned that all metrics rely on an LLM judge, and that the paper does not directly check whether the coupling between actors and the shared world works. For the judge, a 60-day trajectory has 3,600 actions, too many to check by hand, so, as in SOTOPIA, it should be validated on samples. Our perturbation checks only test whether the judge catches injected errors. To check whether the mechanisms work, ECD's scores for dependencies and for building on earlier steps measure what they should produce. E²Sim leads on both (Table R1), and Table R3 traces one case. In our blinded study, readers see an actor's previous day and two anonymous two-day continuations (E²Sim vs. one baseline), and pick the one more consistent with the history and the one with clearer development. The pairs cover all ten actors over days 6–54, and an optional third question asks which is more believable.

Table R5. Blinded reading study with 19 participants (E²Sim / baseline / no clear difference).

E²Sim vs.More consistent with historyClearer developmentMore believable (optional)
Direct12 / 7 / 014 / 3 / 213 / 5 / 1
Guided10 / 6 / 315 / 4 / 012 / 7 / 0
Per-Actor11 / 5 / 313 / 6 / 07 / 7 / 5
Summary9 / 8 / 213 / 2 / 413 / 2 / 4
Total42 / 26 / 855 / 15 / 645 / 21 / 10

Across 76 judgments (19 participants × 4 pairs), readers more often picked E²Sim for clearer development against every baseline. On consistency, they did not favor the baselines, including Per-Actor and Summary, which score higher on IDC. The study is small, so we treat it as a sanity check on the automatic metrics rather than a full human evaluation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Revisions

We thank all reviewers for their constructive feedback. Since the discussion period is ongoing, we provide a brief summary of the rebuttal below for the AC's reference. The reviewers agreed that long-horizon simulation of everyday behavior is an important problem, that the coupling between actor states and a shared world is a useful design, and that our metrics capture failure modes that plausibility scores miss. Below we summarize the main concerns and how we addressed them. All changes are marked in blue in the revised paper, with line numbers.

  • Results beyond less repetitive behavior (HpEh, PakZ, gPV6)

    • Question: A key problem in long-horizon simulation is that actors fall into repeating the same actions day after day. E²Sim clearly reduced this in the original paper. Does it also do better than the baselines on consistency and collaboration?
    • Our response: Following the reviewers' comments, we made two minor changes to how the IDC and ECD judges read evidence, without changing what the metrics measure. IDC now scores a whole actor-day and forms its expectation before seeing that day. ECD now scores sampled decisions that name another actor, using both actors' records (L324–351). With the same judge for all methods, E²Sim leads on ECD (87.85 vs. 72.22–81.03) and matches the strongest baselines on IDC where actor state stays the same (3.94 vs. 3.90–3.93 out of 4). It also keeps far fewer near-duplicate actions (22.3% vs. 60.5–73.5%) (Table 1, L378–389, L397–399).
  • Human validation (HpEh, PakZ, gPV6)

    • Question: Do people agree with the LLM judge that the trajectories make sense?
    • Our response: We added a blinded user study. Nineteen participants compared anonymous two-day continuations from E²Sim and each baseline, giving 76 judgments. Readers more often picked E²Sim for clearer development (55 vs. 15) and did not judge it less consistent (42 vs. 26) (Section 5.3, L471–481).
  • Comparison with existing simulators and cost (HpEh, qoaG, gPV6)

    • Question: How does E²Sim compare with an existing simulation framework, and how much does it cost to run?
    • Our response: We added Concordia (v2.4.0) with the same model, scenario, ten actors, and seven days. In a traced case, Concordia resolves a pending bank sign-off in a single action, while E²Sim keeps it open until the outcome arrives. E²Sim uses 91.9% fewer tokens than Concordia (estimated USD 6.83 vs. 102.69). We also report token usage for all methods (L416–417, L1005–1016).
  • Robustness across runs (HpEh, PakZ, gPV6)

    • Question: With one run per method, do the results hold when the simulation is run again?
    • Our response: We restarted E²Sim from day 0 and day 13 and generated three new 10-day trajectories from each starting point. All six runs stay within IDC 92.8–98.3 and ECD 88.7–91.4. The ECD spread across runs (about 2.7 points) is smaller than E²Sim's lead over the strongest baseline (6.8 points) (Appendix H, L1151–1159).
  • Contribution of each component (HpEh, gPV6)

    • Question: Which parts of the framework actually matter?
    • Our response: Removing random outcome selection drops the share of actions that do not go as planned from 49% to 19%, and raises the share of exactly repeated actions from 2.0% to 8.0%. Removing external events reduces responses to outside events from 1.14 per day to zero. Removing new Stage Goals raises the time actors spend without an active project from 7% to 26%. Shared-world persistence, visibility, and delayed consequences act on one shared record, so we evaluate them jointly. They show up in ECD's dependency and step-building scores (74.3 vs. 34.3–47.8, and 83.6 vs. 37.3–58.2) and in the Concordia case (L513–514).
  • Practical value and stronger models (PakZ, qoaG, gPV6)

    • Question: Are the trajectories useful for assistants, and do the benefits hold with stronger models?
    • Our response: We have not trained an assistant on the trajectories. Our QA study shows that they require long-horizon memory: accuracy drops from 92–97% with the full record to 50–52% when a model keeps only its own notes plus the last three days. We did not test stronger generation models, because running E²Sim on GPT-5.6 Sol would cost about 25 times more. We will release the code upon acceptance (L573).

We hope this summary helps. The revisions are limited to clarifying the evaluation and adding the experiments above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.