acceptodds
Under review as a conference paper at ICLR 2027

Rational Use of Time for Social Coordination in Humans and AI

Abstract

All social interactions are situated in time. Previous studies have shown that humans are able to use implicit temporal cues, such as the duration of a pause, to infer others' mental states. But how do humans use temporal cues for dynamic, fine-grained social coordination? And can AI systems do the same, without “experiencing” continuous real-time as humans do? We study these questions using a cooperative card game task called The Mind, which requires players to achieve a shared goal across repeated interactions but forbids explicit communication: the only available signal is the passage of time. We evaluate 150 humans, and two classes of AI models which reflect different ways of operationalizing time in artificial systems: computer-use agents (CUAs), and standard text-prompted LLMs. We find that human behavior is best captured by a Bayesian strategy that infers a partner's belief from their timing—and similarly for frontier models. However, unlike humans, none of the tested models seem to efficiently adapt their behaviors to their partners' over the course of the game. From a cognitive perspective, our findings suggest that humans perform implicit probabilistic reasoning about continuous-time signals in dynamic social settings, and that such reasoning mechanisms may emerge without real-time experience. Practically, our study highlights new challenges for AI systems to coordinate efficiently with other agents by learning to interpret and adapt to their implicit signals in dynamic settings.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Coordination, LLMs, CUA, temporal reasoning, social coorporation, social cognition, human study, silence

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_UNcYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work studies the using of time as an implicit coordination signal for social interaction. The authors use a game-"The Mind", where two players must play cards in ascending order with the passage of time as the only available signal. The experiments design two rule-based strategies and genereate five types of rule-based partner. Then, the authors let humans, computer-use agents, and text-based LLMs play with these partners, respectively. The results indicate that humans and stronger models often behave in ways consistent with Bayesian reasoning. Moreover, humans show much stronger online adaptation to their partners.

Critical strengths
  1. The problem being studied is very Interesting. The ability of using time as the coordination signal is overlooked by most of the agent research work.

  2. The authors have conduct comprehensive experiments. They compare humans, CUAs, and text-based LLMs under a shared framework. The results indicate that agents lack the online adapation ability and also raise a promising research direction.

Critical weaknesses
  1. The interaction setting is still quite controlled. Most experiments pair humans or models with fixed rule-based partners. This makes the analysis cleaner, but it is unclear whether the same conclusions hold when both players are adaptive, especially in Human-LLM or Human-CUA interaction.

  2. The timing setup is not fully comparable across humans and LLMs. Humans experience continuous wall-clock time, while text-based LLMs are explicitly given simulated elapsed time or WAIT counts. This makes some of the human-LLM comparison difficult to interpret.

Questions for authors
  1. Have the authors considered directly pairing humans with LLMs or CUAs? It would be interesting to see whether the temporal coordination strategies observed with rule-based partners also emerge in human-AI interaction.

  2. In the +Calibration conditions, both the human and the rule-based partner may adapt over time. Can the authors better disentangle how much of the performance improvement comes from human adaptation versus partner adaptation?

  3. In the human experiment, was the use of external timing tools such as a system clock or phone explicitly prohibited? Since the interface in Fig. 6 itself does not show elapsed time, this seems relevant to the claim that humans rely on implicit temporal perception.

3 comments

We thank the reviewer for their helpful comments and questions!

Weakness #1: but it is unclear whether the same conclusions hold when both players are adaptive, especially in Human-LLM or Human-CUA interaction.

Thank you for raising this point! We note that we do already consider settings where both players are adaptive: the rule-based players with calibration are adaptive, and we also include LLMs vs LLMs in our experiments.

We did not conduct Human-LLM interaction because it would be extremely unnatural (if not impossible), as LLMs and humans process time differently. LLMs can only respond to prompts and have no intrinsic “sense” of time, while humans directly experience the continuous passage of time. In this case, it is infeasible to make humans and LLMs use their “native” ability about time in a shared game.

Another reason why we did not include Human-CUA, Human-LLM, or Human-Human experiments is because identifying a player’s strategy and measuring adaptation requires the partner’s strategy to be known. When both players’ strategies are unknown and co-evolving, it becomes difficult to identify which player is adapting to whom, and the strategy-fitting analysis becomes much more noisy. Additionally, a shared set of partners makes humans, LLMs and CUAs comparable with each other in their adaption and fitting strategies. We will elaborate on these points in the camera-ready revision.

Weakness #2: The timing setup is not fully comparable across humans and LLMs.

Yes, we agree that absolute timing is not directly comparable between humans and text-based LLMs. Please note that we have already flagged this in Section 3.3.3 and the final limitations paragraph in the Discussion.

This incongruity is rooted in how LLMs operate: they can only respond to prompts and have no intrinsic “sense” of time. The distinction between human and LLM treatment of time is motivated by our broader question: i.e., how a non-time-embodied system can perform the time-based social coordination task. This is also one of the reasons why we additionally included CUAs, which can be evaluated under the same experimental conditions as humans.

Importantly, only RQ1 (but not RQs2-4) relies on the direct comparison of absolute timing between humans and LLMs. We will make the limitation more explicit when we discuss RQ1 in the camera-ready revision.

Question #1: Have the authors considered directly pairing humans with LLMs or CUAs?

Thank you for this question! In our current experiments, we limit the human/LLM/CUA players with rule-based players for a more clean analysis of the behavior patterns, as discussed in our response above.

Question #2: In the +Calibration conditions, both the human and the rule-based partner may adapt over time. Can the authors better disentangle how much of the performance improvement comes from human adaptation versus partner adaptation?

This is an interesting question! To answer this question, we collected the errors made by human players and their +Calibration partners separately and divided the play history into three bins. We find human error rates dropped significantly from the first to the last third with both Linear (18.8%->12.6%, paired t(29) = 2.73, p = 0.011) and Bayesian partners (14.6%->10.7%, paired t(29) = 2.60, p = 0.014). The partners’ error rates changed by less than 1.5 percentage points with no significant results (p>0.49). This suggests that the performance improvement in the +Calibration conditions comes mainly from human adaptation rather than partner calibration. We did a similar analysis with GPT6 CUAs and we did not find such adaptation. This is consistent with our analysis in Sec 4.3. We will add a brief analysis of the results to Sec 4.3.

Question #3: In the human experiment, was the use of external timing tools such as a system clock or phone explicitly prohibited?

Thanks for raising this question. We did not explicitly prohibit participants from using external timing tools. However, as the experiment is restricted to desktop/laptop users and our experiment setting is made on a timescale of seconds, we believe that the system clock would provide little useful information about time passage given that a system clock typically displays at minute-level resolution.

Thanks for the detailed responses. My concerns have been addressed.

Dear Reviewer UNcY,

Thanks again for your helpful suggestions and questions! We are glad our responses addressed your concerns. We really appreciate your thoughtful feedback!

Best,

Authors

Reviewer_anzpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether AI systems can infer other agent's mental states from implicit signals, specifically the duration of pauses. Humans can do this for instance when someone solves a complex task very fast and one infers that they must have familiarity with the task. The authors test this with a cooperative card game in which players must play all their cards in ascending order without any communication, so the only information players have is the passage of time. The authors compare humans, computer-use agents (CUAs), and LLMs across four research questions: performance, behavioral adaptation, the cognitive strategy that best captures each one’s behavior, and consistency across prompts. They model behavior with a linear strategy, a Bayesian strategy, and a random baseline. The authors find that the Bayesian strategy captures both humans and LLMs, and that LLMs can outperform humans in task success, but humans adapt more to their partner’s behavior while LLMs are more egocentric. They also find that LLM behavior depends on the prompt, and that prompts with no reference to time units can lead to similar playing times, which suggests the LLM might rely on numerical information instead of reasoning about real time.

Critical strengths
  1. This paper looks at how AI systems adapt to and understand others through implicit signals instead of explicit ones, which are more common in the literature.
  2. The authors clearly thought about the methods they used. They compare a linear strategy, where lower cards are played according to the duration of time, a Bayesian strategy, and a random baseline.
Critical weaknesses
  1. The authors mention mental states but they never actually get there. The LLM is basically acting as a classifier based on time information, so there is no real study of mental states. The paper does not map what those states are or how to measure them with the LLM, and the same happens with the CUAs.
  2. Human players never interact with each other. They play with rule-based partners instead. Because of that, we don’t know if the Bayesian fit appears because humans default to this strategy when their partner is non human, or if it would also fit a human-to-human interaction. This setup also lacks a mental state assessment, since there is no real partner to infer a mental state from. The authors also don’t collect self-reports from participants about the reasoning behind their decisions, which could show how that reasoning aligns with the actual durations. There are many variables involved in the game itself, and duration is probably not the only factor.
  3. It’s not clear how the authors infer the partner’s state, or what that state is.
  4. The authors prompt the LLM at simulated time steps with a fixed amount of time, which seems disjoint from the tempo used for the other models.
Questions for authors
  1. Is there a version of this study in which you define and map the mental states?
  2. You prompt the LLM at simulated time steps with a fixed amount of time. Is this disjoint from the tempo used for the other models?
  3. The CUAs reach human performance but take longer to play the game. Is this because of processing time? Are there pauses involved, and do you discount them?

Suggestions

  • The term “Bayesian partner” is hard to understand, and the paper doesn’t make clear early on that humans don’t play with each other. Stating this upfront would help.
  • The authors should describe the task with all of its details at the beginning, for example, whether every player has the same information about the task. When I first read it, I thought people didn't know what they were doing, or that only some people would receive the information and others would have to figure it out. Later on I understood it.
5 comments
AuthorsOpenReview ↗

Author response 1/2

We thank the reviewer for recognizing the strengths of our experimental design and for their constructive feedback! We address their main concerns and questions below:

Weakness #1.1: The authors mention mental states but they never actually get there.

Thank you for raising this concern. We acknowledge there may have been some ambiguity about the term “mental states”, which may be taken to mean goals, desires, and so on. In our study, the Bayesian player doesn’t represent these kinds of mental states, but rather their partner’s private knowledge of their lowest card. This information is known only to the partner, so it can be seen as a latent knowledge state that must be inferred. We will clarify what we mean by mental states in our experiment in the introduction.

Weakness #1.2: The LLM is basically acting as a classifier based on time information, so there is no real study of mental states.

We are not quite sure what you mean by the LLM acting as a classifier based on time information. If you mean that the LLM has no explicit symbolic representation of mental states, then we agree, and this is part of what makes the experiment interesting: we are testing the model’s emergent ability to engage in social cooperation without any built-in social reasoning structure. The same holds for CUAs.

Weakness #2.1: Human players never interact with each other…we don’t know if the Bayesian fit appears because humans default to this strategy when their partner is non human

Thanks for pointing this out! We would like to clarify that participants were never told that their partner was non-human. We have reason to believe the Bayesian fit is not a default strategy toward non-human partners: humans are more likely to adopt the constant waiting strategy when playing with random or Linear (- calibration) partners (Fig 3). As clarified in our response to Weakness #1.1, we still view a rule-based partner as holding private information, so inferring the partner’s hidden state from its timing is meaningful regardless of whether the partner is human. We will clarify the underlying logic at the beginning Section 4.

Turning to your broader point that human players never interact with each other, we agree that human-human experiments would provide valuable complementary data to our experiments. Please note that we already discuss this as the first limitation in our Discussion (lines 470-473).

With that said, we believe that human-human data is not needed to address our research questions. We paired players only with rule-based partners for several reasons. First, identifying a player’s strategy and measuring adaptation requires the partner’s strategy to be known. When both players’ strategies are unknown and co-evolving, it becomes difficult to identify which player is adapting to whom, and the strategy-fitting analysis becomes much more noisy. Second, a shared set of partners makes humans, LLMs and CUAs comparable with each other in their adaption and fitting strategies. We will elaborate on these points in the camera-ready revision.

We also note that we ran a brief pilot experiment that paired humans together (which was not included in the paper). In practice, we found that data collection was challenging because a single non-attentive participant would invalidate both players’ data. Therefore, for both theoretical and practical reasons, we believe that conducting human-human experiments warrants follow-up work that is beyond the scope of our study.

Weakness #2.2: no self-report in human experiments

We did in fact collect self-reports from participants about their gameplay strategies (not discussed in the paper), but they rarely described specific strategies such as counting-based or explicit Bayesian rules. This is consistent with previous literature in time perception bias (Jazayeri & Shadlen 2010) and limits of introspection to decision processes (Nisbett & Wilson, 1977). Therefore our analysis relies more on the behavior data. We will add a summary of the self reports to the appendix in the camera-ready revision.

Weakness #2.3: There are many variables involved in the game itself, and duration is probably not the only factor.

We totally agree that there are multiple factors that can potentially influence players’ decisions and we discussed this in the Discussion section (lines 473-477). We will add more elaboration to the discussion in the camera-ready version.

Weakness #3 It’s not clear how the authors infer the partner’s state, or what that state is.

Please see our response to Weakness #1.1

AuthorsOpenReview ↗

Author response 2/2

Weakness #4: The authors prompt the LLM at simulated time steps with a fixed amount of time, which seems disjoint from the tempo used for the other models.

Thank you for this question. If by “other models” you mean CUAs, we note again that LLMs are fundamentally different from humans and CUAs because they have no “native” sense of time. So, to evaluate them on the task, we have to make some commitment about how to map their prompts onto timing frequency.

Questions #1 & 2:

Thank you for these questions! Please see our response to Weakness #1 and #4.

Question #3: The CUAs reach human performance but take longer to play the game. Is this because of processing time? Are there pauses involved, and do you discount them?

Thank you for your question. We measure CUA play time in wall-clock time, as for human participants, without any discounting of potential processing time. However, we do not believe the longer hesitation times are due to processing time. CUAs adjust their hesitation times depending on the partner, and when playing with Random partners, their hesitation times are comparable to those of humans (Figure 2). This shows that processing time alone does not make CUAs substantially slower; their longer hesitation with other partners instead reflects longer waiting, which we interpret as a speed–accuracy trade-off. We will extend this discussion in the revision.

Suggestions Thank you for the very helpful suggestions. We will make the experiment design more clear in our updated version.

References: Jazayeri, M., Shadlen, M. Temporal context calibrates interval timing. Nat Neurosci 13, 1020–1026 (2010). https://doi.org/10.1038/nn.2590

Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259. https://doi.org/10.1037/0033-295X.84.3.231

I thank the authors for their detailed response. The clarifications about the rule-based partners, the self-reports, and the CUA timing address my concerns.

A few follow-ups. First, the authors clarify that the latent state is the partner’s private knowledge of its lowest card, which is much narrower than “mental states,” so I encourage the authors to use this more precise term in the paper. Second, my point about the LLM was that the paper does not look at how it reaches its decisions, for example through the reasoning text it produces, which would help show whether it actually infers the partner’s state. Third, are the results sensitive to the interval the authors choose between LLM prompts?

These points are not blockers, and I am raising my score. I encourage the authors to address them in the revision.

Reviewer_pP77OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper explores how AI systems, and humans, behave when communication is limited to the passage of time. It fundamentally is exploring how social interactions are often about nuanced things like time and other signals. They use a simplified game "The Mind" -- a game where players try to play privately-held cards in ascending order, and communication constrained to the time between plays -- to study how humans, computer-use agents, and LLMs behave. The three players are paired and evaluated against scripted players (linear, Bayesian, and random). Experimental results show that the Bayesian model is the best-in-class at explaining behavior of play and that some LLMs achieve human-level performance, amongst other observations.

Critical strengths
  • Temporal prompts are insightful. The experiments around temporal prompts are useful and insightful in broader contexts, showing how representation of time can have meaningful impact on models' ability to perceive and use time to their advantage.
  • Straightforward experiments. In a similar tone, the experiments are clear and do provide a useful setting for studying how existing models coordinate in time. This can have broader applications.
Critical weaknesses
  • There is a large gap in paper's ambitions to the findings and experiments. The experimental setting is relatively simple to the larger problem of nuance human social interactions and observing/exploring how agents/models perform at that task, and how they might improve at that task. Social inference and numerical timing of a gamified environment are very different things, and not necessarily correlated. The lack of human-human experiments also prevents grounding the experiments to a 'natural' baseline, and limits the interpretation of the results around human-like interactions.
Questions for authors
  1. How do human-human games play out, and what can they tell us about the human-machine interactions and machine-machine interactions?
  2. The game focuses on the perception of time in a gamified environment, which tends to have patterns matched to observable outcomes/rewards. Social interaction often lack this, so how can we interpret the results to the nuanced/non-verifiable domains of social human interaction?
3 comments

We thank the reviewer for their helpful feedback and we address their concerns and questions below:

Weakness #1: The experimental setting is relatively simple to the larger problem of nuance human social interactions and observing/exploring how agents/models perform at that task, and how they might improve at that task

Thank you for raising this point! We agree that our study focuses on coordination with implicit signals rather than social interactions in general, while our abstract/intro might be read as making broader claims. We will narrow the framing accordingly.

We note that there is an established tradition of using relatively simple games to study more complex cognitive processes (e.g., Allen et al. 2024). In our study, we deliberately chose to use a simple task which removes all other social cues (e.g., language, tone, gaze), so that timing can be studied in isolation. The simplified setup allows us to derive an exact Bayesian model of the inference, and fit it to each game record. As a result, we are able to determine what strategy a player is using, not just how well they performed.

Weakness #2: The lack of human-human experiments also prevents grounding the experiments to a 'natural' baseline

We agree that human-human experiments would provide valuable complementary data to our experiments. Please note that we already discuss this as the first limitation in our Discussion (lines 470-473).

With that said, we believe that human-human data is not needed to address our research questions. In our experiments, we paired players only with rule-based partners for several reasons. First, identifying a player’s strategy and measuring adaptation requires the partner’s strategy to be known. When both players’ strategies are unknown and co-evolving, it becomes difficult to identify which player is adapting to whom, and the strategy-fitting analysis becomes much more noisy. Second, a shared set of partners makes humans, LLMs and CUAs comparable with each other in their adaption and fitting strategies. We will elaborate on these points in the camera-ready revision.

We also note that we ran a brief pilot experiment that paired humans together (which was not included in the paper). In practice, we found that data collection was challenging because a single non-attentive participant would invalidate both players’ data. Therefore, for both theoretical and practical reasons, we believe that conducting human-human experiments warrants follow-up work that is beyond the scope of our study.

Question #1: How do human-human games play out, and what can they tell us about the human-machine interactions and machine-machine interactions?

Thank you for the question! It would be interesting to compare human-human data points with LLM-LLM and CUA-CUA ones. It would allow us to better understand how humans reason with implicit cues when the partner is more flexible than our rule-based players in both strategy and behavior. However, as discussed in our response to weakness #2, we believe human-human experiments are not necessary to answer our research questions and beyond the scope of the study. We will elaborate on these points further in the discussion section.

Question #2: Social interaction often lack [observable outcomes/rewards], so how can we interpret the results to the nuanced/non-verifiable domains of social human interaction?

Thank you for this important question! We agree that many social interactions lack the immediate outcome feedback present in our game and we do not claim that our results transfer directly to such domains. We also want to note that during the game, players in our game also cannot verify their inference about the partner’s silence because the feedback comes only afterwards. This is similar to how conversational inferences from pauses are confirmed or repaired in subsequent turns. We will narrow our narrative and focus more on the coordination and add a brief discussion of how our game can be potentially extended to broader social interactions.

References: Allen, K., Brändle, F., Botvinick, M. et al. Using games to understand the mind. Nat Hum Behav 8, 1035–1043 (2024). https://doi.org/10.1038/s41562-024-01878-9

The rebuttal points are acceptable, although I maintain that human-human evals would strengthen the work and enable greater conclusions. However I admit the difficulty of achieving clean results here. I will raise my score, and I am encouraged that some revision in text has already been made.

The paper questions and analyses some interesting questions, and follow-up questions from various reviews also add some interesting discussion I believe would be valuable to include in the paper, including the two questions I posed.

Dear Reviewer pP77,

Thank you so much for your questions and suggestions, and for taking the time to update your assessment! Your feedback has helped us strengthen the paper. In the updated version, we will expand our discussion of human-human experiments and the complexity of social interactions. We will also incorporate relevant points raised in Reviewer anzp's follow-up questions. Thank you again!

Best, Authors

Comments on the paper

1 comment

Dear all,

We have uploaded a revised PDF of the paper and its appendix, addressing all the concerns and questions raised by the reviewers.

The main changes are: (1) we clarified the scope of our study, key term definitions, and the rationale for excluding the human-human experiments (2) we expanded the discussion regarding the comparability between LLMs and humans, gaming setting, and human-human experiments (3) we added results from human participants’ self reports to the appendix (4) we added more analysis on the human and CUA adaptation (5) we added an anonymous repository link.

All the changes are highlighted in blue. We are happy to address any remaining issues. Thank you again for your time and helpful comments!

Best, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.