acceptodds
Under review as a conference paper at ICLR 2027

Beyond Latency: Benchmarking Real-Time Strategic Control for LLM Agents in Evolving Environments

Abstract

Large language models are increasingly used in Agent tasks that require continuous observation, reasoning, and action. In real-time environments, however, the world continues to evolve during model inference. Each decision therefore spans both wall-clock time and environment time, affecting the decision frequency achievable within a fixed horizon. We develop an OpenRA-based benchmark for real-time strategic control, comprising controlled Fixed-Force diagnostics and complete Full-Skirmish games. The benchmark jointly records wall-clock response time, observation-to-action gap, decision frequency, and control outcomes. Experiments show that added waiting widens the gap while reducing decision frequency. The same wall-clock waiting time corresponds to substantially different gaps under different world speeds; the mean gap at the fastest speed is roughly three times that at the slowest. Full-Skirmish experiments further reveal clear differences in computational cost and long-horizon control performance across observation representations and control architectures. The benchmark also supports evaluation across model backbones, Bot behaviors, and opponent difficulties in continuously running RTS environments.

Primary area:
datasets and benchmarks
Keywords:
LLM Agents, Real-Time Decision Making, Agent Benchmarking, Real-Time Strategy Games

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_cymXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies the behaviour of LLM agents in continuously changing environments, focusing on the impact of model completion latency described as the delta_env. The paper introduces a benchmark built on OpenRA, an open-source library for real-time strategy games, with 2 settings: Fixed-Force and Full-Skirmish. The benchmark measures response time, the game time gap between observation and resulting actions (delta_env), decision frequency and game outcomes. The main experiment add artificial delays before model actions are applied and varies game speed (accelerating or decelerating ticks) to study their effect on final metrics. The authors find that structured observations use fewer input tokens, while hierarchical orchestration control achieves better outcomes in their full-game experiments, but at the cost of using more tokens.

Critical strengths

The OpenRA RTS setting is a promising benchmark direction for agents that must both act at low-latency when the environment continues to evolve, as well as long-term planning to solve the game.

Critical weaknesses
  1. The main paper underspecifies the experimental setup. While high-level descriptions of the tasks and agents are given, the experimental setup omits critical information to better understand the paper. Specifically, very few details are given on the goals/horizons/interfaces and prompts of the OpenRA games. The hierarchical control scaffold is not explained either. This should be developed in appendix.
  2. The choice of artificially adding wait time before action is questionable. The authors study the impact of the model using more time to take an action by adding an artificial delay. This is a major confound: all the observations on downstream metrics (cf Figure 2) are the consequences of this choice rather than model behaviour. A cleaner setting would vary the reasoning strength of the model so more latency implies more compute.
Questions for authors
  • Can the experimental give more details on the games, victory rules, action space, Flat vs Hierachical orchestrations.

  • How does this benchmark compare to VideoGameBench, Gaia2 (time split of the benchmark), GameWorld-RT, TickingCollabBench, and RTSGameBench? Please clarify its positioning, novelty, and motivation in the introduction or related work.

  • In addition to the artificial wait times, could you report the effect of varying reasoning effort of a fixed model and report the actual latency and game outcomes? This would actually measure whether more time spent on compute improves or hinders downstream performance in such low-latency setup.

Reviewer_aLUoOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper studies the problem of an agent in a non-stationary environment i.e., the environment transitions while the agent deliberates and performs an action
  • To understand the consequence of this scenario, the authors provide a benchmark based on "OpenRA" (Open Red Alert, which provides an interface for the Red Alert game). The game itself is a real-time strategy (RTS) game, where the agent controls multiple units, gather resources, build bases/units, and fights an opponent.
  • The paper proposes a benchmark built on top of OpenRA, by doing the following:
    • developing an interface for LLM-based agents, where environment and agent can be configured independently
    • proposing two tasks (fixed force and full skirmish)
    • proposing an evaluation protocol to record certain metrics e.g., wall-clock response time
  • Within this benchmark, the papers analyzes various influences, such as:
    • effect of wait time on success
    • effect of observation space: raw vs. structured
    • effect of control architecture: flat vs. hierarchical
Critical strengths
  1. Well-justified problem: Latency of LLM-based agents can indeed be a bottleneck in environments requiring real-time responses (e.g., games, robotics). This paper takes a step towards understanding its influence on objective outcomes.
Critical weaknesses

Major concerns

1. Artificially introduced wait times

  • tl; dr: wait times are artificially introduced; unclear whether the agent a priori is aware of such a wait time.
  • Context: There are two measures to quantify the delay in the paper: (a) a "deliberation time" / "environment elapsed time" (Eq. 2) and (b) a "wait time" dd
    • (a) is natural, since there is some latency involved in converting observations-to-actions. I believe this is important to study.
    • (b) is artifically introduced (L141).
  • I find (b) strange, because it appears to handicap the LLM-agent and naturally puts the agent at a disadvantage (unlike the opponent who doesn't experience such a delay). Naturally, one would expect this affecting success rates, which the experiments find and quantify. It's unclear why (b) is a useful factor to study.
  • Furthermore, is the agent aware of the delay a priori (e.g., mentioned in the prompt, delay reported in observations)? Because if agent operates under the false assumption of no delay, this might further explain the success degradation.

2. Analysis: Heavily-grounded in game-specifics

  • tl; dr: Analysis almost entirely focuses on game-specific details (e.g., observation space, success rates) and broader impact of this is unclear.
  • The bulk of the analysis largely understands the influence of certain factors (e.g., wait times, observation space) on game-specific metrics (e.g., army value differential, win/loss rate).
  • While this is helpful of practioners of the game, the broader consequenes/understanding remains unclear. For instance:
    • Why does the delay cause decreased performance, and also introduce a larger gap (in ticks) - because of more reasoning tokens?
    • What makes the agent robust to the wait times? i.e., how can a practitioner generally improve performance of an agent in real-time environments?
    • Can the LLM inference be optimized (e.g., aggressive quantization, smaller models) to improve performance at delay = 0?
    • How to mitigate the observation-to-action gap (i.e., delay > 0)?

Minor concerns

3. Notation and Terminology

  • The paper assumes the reader is aware of many game-specific/OpenRA-specific details that a general reader might be unaware of e.g., "environment ticks?, "predefined forces and deployments", "army value differential"
  • Notation if often incomplete, or defined and never used again. For instance, horizon HH, wait time dd, speed ss. How does this related to Equation (2)?

4. Implementation Details

  • The paper does not provide sufficient implementation details e.g., what is the horizon length (in seconds)? what is the agent's prompt? how many ticks are in a second?
  • Many implementation details are further OpenRA-specific e.g., unclear what the precise difference is between "fixed force diagnostics" and "full skirmish". Explanations are incomplete.
Questions for authors

Clarification requests:

  1. wrt concern (1):
    • what is the motivation of artifically introduced wait times?
    • are wait times only applied to the agent and not the opponent?

Suggestions for future work (non-decisive):

  1. wrt concern (2)
    • It would be very helpful to extend the analysis to address LLM-specific insights that would provide a deeper understanding of why LLMs perform poorly in real-time scenarios and how to mitigate the gap.
Reviewer_9QP6OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors introduce an OpenRA-based benchmark for evaluating real-time strategic control by LLM agents in continuously evolving RTS environments. The benchmark combines controlled Fixed-Force combat scenarios with Full-Skirmish games and jointly measures wall-clock response time, observation-to-action gap, decision frequency, and control outcomes. The experiments investigate the effects of action delays and world speeds, as well as the impact of different observation representations and control architectures on agent performance.

Critical strengths

The paper raises a practical challenge in evaluating LLM agents operating in continuously evolving RTS environment.

Critical weaknesses
  • Limited novelty: The proposed benchmark substantially overlaps with existing OpenRA-based frameworks (e.g., OpenRA-RL [1] and OpenRA-Bench [2]), and its novel contributions are not clearly differentiated from prior work.
  • Limited empirical insights: The main experimental findings are largely predictable and provide limited new insights into real-time agent behavior. Longer delays increase observation-to-action gaps and reduce decision frequency, while faster environments advance more ticks during the same wall-clock delay. More importantly, although the authors demonstrate that observation-to-action gaps can differ under near-matched decision frequencies (Figure 2(c)), they do not report the corresponding control outcomes. Consequently, the independent impact of observation staleness on agent performance remains unclear.
  • Insufficient experimental controls: The system-level comparisons show that structured observations reduce input costs and hierarchical control achieves better outcomes with greater computation. However, the comparisons use different computational budgets without compute-matched baselines, making it difficult to attribute performance improvements to architectural choices rather than increased computation. These results remain largely descriptive and provide limited insight into the underlying mechanisms.
  • Limited experimental scale: The experiments rely on only 4–8 seeds per condition, with insufficient statistical analysis to establish the reliability of the reported differences. Given the variability in game outcomes, it is difficult to determine whether the observed improvements are robust or merely reflect seed-dependent variation.

References

[1] OpenRA-RL Contributors, "OpenRA-RL: Open Framework for AI Agents to Play Red Alert through Reinforcement Learning," GitHub repository, 2026. https://github.com/yxc20089/OpenRA-RL
[2] qpluslab, "OpenRA-Bench," Hugging Face Spaces, 2026. https://huggingface.co/spaces/qpluslab/OpenRA-Bench

Questions for authors
  • Choice of benchmark: What motivates the choice of OpenRA over well-established RTS benchmarks such as StarCraft II? What specific advantages does OpenRA offer for evaluating real-time LLM agents?

  • Fine-grained performance analysis: While the paper reports W/D/L and terminal ArmyValue differential, could the authors provide additional intermediate metrics, such as economic development, resource utilization, and unit production? Such analyses would help clarify whether the improvements from hierarchical control primarily stem from better economic management, production planning, or combat control.

  • Asynchronous control in hierarchical agents: Does the Hierarchical agent allow lower-level controllers to continue executing actions while the high-level LLM is reasoning? If not, could the authors compare against an architecture that explicitly decouples LLM planning from real-time execution, as in JEV-Star [1], to investigate whether such decoupling mitigates latency-induced performance degradation?

Reference

[1] Ma et al., "JEV-Star: Fast, Low-Cost StarCraft II Control with Language-Model Planning," arXiv, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.