- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
BusyWorld evaluates computer-use agents on multi-goal planning in shared environments. There are events that change shared state during execution and goals that compete for a limited action budget. Matched static/dynamic and importance variants show that competition lowers success, and agents often reach the right goals but fail to finish them.
Critical strengths
Matched task variants keep the initial state, goals, and budget fixed while varying event activity or stated importance. This supports controlled comparisons (an important trait for benchmark paper i think). Full execution traces and metrics beyond binary success, make the dominant failure patterns easier to inspect.
Critical weaknesses
Execution bottleneck and planning trade-offs The main failure mode is reaching the cart without submitting an order, including on attribute-based selection with no NPCs. Importance edits mainly change how long agents interact, not whether orders complete. A large share of the measured gap may reflect CUA workflow competence instead of concurrency-aware multi-goal planning.
Limited scale Several key contrasts rest on small paired sets (e.g., two-goal dynamic with only a few pairs), and four-goal dynamic success falls to zero. Floor effects and sparse runs limit how well the benchmark evalaute planning strategies or models once competition is added.
Questions for authors
How do you think we can solve the problem?