- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CogMob, an LLM-based agent framework for simulating individual daily mobility without training on trajectory data. Each agent keeps a personal cognitive map of the places it has visited or heard of, its routes and the regions it knows. The map is updated through experience and slowly forgets places that are not revisited. Candidate destinations are retrieved from the map (plus new POIs from a database) according to the agent's spatial intent, and the LLM picks one. Experiments on Beijing and NYC compare CogMob with training-free LLM simulators and trained generative models, with ablations, a plug-in experiment with LLMob and a segregation analysis.
Critical strengths
Current LLM mobility simulators do not reproduce how differently individuals explore new places and return to familiar ones (Fig. 2), and giving each agent its own evolving map of what it knows is a natural way to address this. The ablations support it. Removing the map clearly hurts the exploration and return metrics (Table 3), and adding it to LLMob helps as well (Fig. 5). I find this part convincing, and it's the main reason I'm positive about the paper.
The evaluation also goes further than most papers in this line, with metrics that test known mobility regularities and data-fitted baselines such as TimeGeo.
Critical weaknesses
-
My main concern is that Table 1 overstates what the cognitive map itself contributes. On Beijing, the variant without the map (w/o CM-B) already beats the training-free baselines on most metrics, so much of the gain seems to come from the rest of the agent design, some of which the adapted baselines lack. The map clearly matters for exploration and return, and I would frame the claims around that.
-
It is not clear what the map's structure adds over simply remembering visited places. The visit-history baseline in App. D.4 looks like a separate, simpler simulator rather than CogMob with a flat memory.
-
No runtime or cost is reported. Each decision needs at least two LLM calls plus periodic reflection, and App. B.4 runs at most five agents at a time, so it is hard to judge how this would scale to realistic population sizes.
-
There are many hand-set parameters (decay rates, search radii, the clustering radius, candidate limits), and I could not find a sensitivity analysis or an explanation of how they were chosen.
Questions for authors
- Could you run CogMob with the cognitive map replaced by a flat list of visited places, keeping everything else the same? If the structured map still clearly helps on the dynamics metrics, that would address my main concern.
- How does the cognitive map relate to the per-agent environment trees in Generative Agents (Park et al., 2023), which also start partial, grow with exploration and are used to choose locations?
- For NYC, how are the simulated full-day trajectories made comparable to the sparse check-in data before computing the JSDs? Some of the NYC differences also look small relative to the run-to-run variation in Table 4.
- The segregation analysis doesn't seem to separate models much (CoPB also beats the shuffled baseline, Fig. 9), and for Beijing the reference data come from US cities. Unless I'm missing something, I would present it as a sanity check rather than as a contribution.
8 comments
Response to Reviewer QESN (2/6)
Q2: How does the cognitive map relate to the per-agent environment trees in Generative Agents (Park et al., 2023), which also start partial, grow with exploration and are used to choose locations?
Response: We thank the reviewer for this question. In terms of purpose, Park et al.’s environment tree helps agents choose where to carry out activities in a predefined town with only 19 top-level venues, with several categories represented by a single venue, such as a café, pub, or park. CogMob addresses how individuals choose among the many destinations available for the same activity in a city, and why they repeatedly visit only a small, changing subset of these alternatives. Its cognitive map models how spatial knowledge and experience mediate this selective use of urban space, allowing personal activity spaces to form and evolve through successive destination choices.
Methodologically, the two approaches differ in how spatial knowledge is formed and updated, and how it supports decisions at different spatial scales. In Park et al.’s approach, each agent starts with partial knowledge of a predefined “world–venue–subarea–object” containment hierarchy and expands it through direct observation. Given an intended activity, the LLM selects a known venue, then a subarea and a specific object within it. However, simply accumulating observations alone does not fully capture how individuals understand and use urban space. Even people who visit the same places may form different impressions, remember different nearby POIs, perceive accessibility differently, and group places into different personal regions. These differences shape which destinations they consider, revisit, or explore.
In contrast, CogMob organizes spatial knowledge into places, routes, and regions. It distinguishes visited places from those the agent only knows about and forms personal understanding of regions by clustering visited places. This knowledge evolves with new information exposure and new experience. Agents learn about unvisited places through indirect information relevant to their interests and current intent. Reflection revises personal impressions of visited places, and memory strength decays over time. These updates change what the agent remembers, how it perceives places, and how its spatial knowledge is organized. CogMob then uses this evolving knowledge to support destination choice among urban POIs. Depending on whether the agent has a specific place, a target region, or only preferences in mind, intent-guided retrieval generates destination candidates. This process links what an individual knows and remembers about the city to the destinations they consider visiting.
Experimentally, to compare CogMob with the environment-tree mechanism, we introduce a variant, CogMob with Perception-expanded Flat Memory (CogMob w PFM), which replaces the cognitive map with a place memory. After each visit, this memory adds the visited place and all nearby POIs, following the observation-based expansion used in Park et al.’s environment trees. All other components and experimental settings remain unchanged. As shown in the table, CogMob achieves an average improvement of 33.36% over this variant on the Beijing dataset.
| Dataset | Method | Radius | Jump | Duration | Entropy | Explore | Return | Visit-Freq | Capacity | LocFreq | OdSim |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Beijing | CogMob w PFM | 0.0325 | 0.0370 | 0.0373 | 0.1482 | 0.0543 | 0.0907 | 0.2028 | 0.0009 | 0.1878 | 0.3435 |
| CogMob | 0.0214 | 0.0292 | 0.0378 | 0.1331 | 0.0271 | 0.0295 | 0.0911 | 0.0003 | 0.1544 | 0.2943 | |
| Improvement | 34.15% | 21.08% | -1.34% | 10.19% | 50.07% | 67.47% | 55.07% | 64.79% | 17.78% | 14.32% |
Response to Reviewer QESN (3/6)
Q3.A: For NYC, how are the simulated full-day trajectories made comparable to the sparse check-in data before computing the JSDs?
Response: We thank the reviewer for this question. Check-in data are widely used to evaluate human mobility simulations in prior studies such as LLMob, CoPB, and MIRAGE. Despite being sparse compared to dense data like GPS trajectories, check-in data are favored considering its public availability and effectiveness in representing human mobility. Our evaluation compares the distributions of mobility statistics that characterize human mobility patterns, rather than individual full-day trajectories. Before using the NYC POI check-in data for evaluation, we verified that the patterns considered in our study remained observable despite the sparsity of the records (Figures 14–19). We then computed the corresponding statistics from the simulated trajectories and used JSD to assess how closely the simulated distributions match those observed in the check-in data.
Q3.B: Some of the NYC differences also look small relative to the run-to-run variation in Table 4.
Response: We thank the reviewer for this comment. Each experiment was repeated three times, and Table 4 reports the mean and standard deviation across runs. Among the methods listed, CogMob achieves the lowest mean JSD on 12 of the 13 NYC metrics. To examine the smaller margins relative to run-to-run variability, we inspected the results for user-level Explore, Return curve, Capacity, and LocFreq. For the first three metrics, CogMob achieved a lower JSD than the strongest baseline in all three runs, with relative reductions in mean JSD of 7.81%, 19.35%, and 11.54%, respectively. Thus, although the absolute differences are small, the improvements were consistent across the evaluated runs.
The advantage on LocFreq is more limited: CogMob outperformed the strongest baseline in two of the three runs, with a mean JSD reduction of 3.34%. This metric measures aggregate spatial visitation shares, so better individual exploration and return behavior need not produce an equally large improvement in LocFreq. NYC’s smaller study area may also allow distance-based selection to capture the main activity areas reasonably well, leaving less room for improvement. By comparison, cognitive-map-based retrieval may offer greater benefits in the larger Beijing study area by identifying relevant destinations outside agents’ immediate neighborhoods.
Response to Reviewer QESN (1/6)
Response: We thank the reviewer for this question. We have added an ablation variant, CogMob with Visited-only Flat Memory (CogMob w VFM), which replaces the cognitive map with a flat list of visited places while keeping the remaining components unchanged. Given the intent generated in Stage 1, Stage 2 retrieves candidates, ranked by visit frequency and distance. If fewer than nine candidates are available, it supplements them through a search of the full POI database.
As shown in the table, CogMob reduces JSD by an average of 31.52% compared with this variant. Flat visitation memory records where an agent has been, but does not fully capture its personal understanding of the city: which places it knows beyond those already visited, how places are connected through routes and organized into regions, and how this knowledge evolves over time. CogMob explicitly models these aspects and uses them to guide destination selection.