acceptodds
Withdrawn from ICLR 2027

ParallelWorld: Test-Time Scaling for Agentic Embodied Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Embodied AI has emerged as a key paradigm for autonomous systems operating in complex physical environments. While end-to-end models have established strong local capabilities for perception and control, scaling these capabilities to extended, multi-stage tasks remains difficult. Agentic embodied systems address this challenge by introducing high-level planners that organize perception, reasoning, and executable skills over longer interaction horizons. However, existing agentic embodied frameworks typically generate actions for direct execution without explicitly simulating how alternative action sequences shape future observations and task progress, leading agents to commit to suboptimal interactions before considering their long-term consequences. To address this limitation, we propose ParallelWorld, an adaptive-horizon test-time scaling framework that grounds agentic embodied reasoning in the prospective simulation of action outcomes. Specifically, we design an agent system to simulate and evaluate alternative trajectories in parallel before committing to an action. A verifier assesses intermediate state transitions with respect to the task objective, dynamically pruning unpromising branches and guiding subsequent expansion. Based on the evaluated long-horizon outcomes, the agent selects a trajectory, executes only its first action, and replans as new observations become available. Experiments on ESI-Bench and LIBERO-Pro demonstrate improvements in active spatial reasoning and language-conditioned manipulation, respectively, highlighting the effectiveness of prospective planning for both understanding and manipulation tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
test time scaling, embodied reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tiDzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces ParallelWorld, a framework for enhancing the decision making performance of embodied agents through test-time search in simulation / world-model. Evaluated on two simulated benchmarks, the method shows improved performance compared to baseline methods that do not perform test-time search.

Critical strengths
  • The empirical results show that the method leads to increased performance on simulated domains.
  • The use of agents to prune down search branch seems interesting.
Critical weaknesses
  • The idea itself is pretty trivial - if you have access to a model of the environment then you can do search. This is essentially a simple form of model-predictive control, and the community has known for a very long time that with a good enough model we can get performance boosts. It is not clear what novel insight this paper contributes other than showing that this works with an agent-based policy.

  • It seems that for this method to work, an accurate model is required. However it is unclear how we can get an accurate world model / simulator for real-world applications such as on real robots, limiting the applicability of the method. There is also no real-world evaluation of the method.

  • In the ESI-Bench results, the agent can directly access the ground truth state and the ground truth environment. This is an unrealistic assumption for most real-world domains and should be explicitly discussed.

  • The inference time cost is a big concern in my opinion, especially since the agent is also involved in pruning the search branch, which will add more inference time. With such a high inference cost, it is not clear whether the framework can be extended to real systems.

  • The experiments lack comparison against alternative planning methods given the same simulator or world-model access. It is not clear whether the introduced search mechanism is needed at all.

  • No statistical significance or variance estimation is provided

Questions for authors

See weakness.

Reviewer_zvFEOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

ParallelWorld adds verifier-guided, multi-step lookahead over semantic action primitives, using simulator rollouts for ESI-Bench and image-model predictions for LIBERO-Pro. It adaptively selects branches and search depth, executes the first selected action, and replans. Its potential contribution is this embodied search configuration and its empirical utility.

Critical strengths
  • Results exceed Active Exploration in 26/28 ESI-Bench subcategories and HarnessVLA in all eight LIBERO-Pro settings; Spatial-S improves from 56% to 83% (Tables 1 and 2).
  • On the small ablation subset, adaptive width achieves 73.3% success versus 36.7% for width four, with lower runtime (Table 3). The application spans evidence acquisition and manipulation.
Critical weaknesses
  • The novelty of the proposed method is insufficiently distinguished from existing work (see below). ParallelWorld's semantic primitives and adaptive search configuration may provide useful integration, but the "test-time scaling" label does not establish a new planning principle. It's questionable what specific search or allocation mechanism is novel.

  • The scaling and adaptive-horizon claims lack decisive controls.

    • Table 3 varies width on only three tasks/30 episodes; it reports neither realized depths nor performance across controlled compute budgets.
    • Adaptive search takes 2402.2 seconds versus 417.4 for direct execution. Does it outperform one-step best-of-N and fixed-depth search with the same models, primitives, information access, and total budget? Such comparisons would establish whether the proposed adaptivity adds value beyond conventional lookahead.
  • The results are encouraging, but the paper doesn’t clearly show what’s new about its planning method, or whether the gains come from the method itself rather than from existing search techniques or extra evidence.

    • ESI-Bench mixes additional computation with additional evidence. Equation (10) restores true simulator states and obtains observations from alternative actions, potentially revealing hidden information.
    • Branch-query accounting and retention of prospective evidence are unclear. Do gains persist when baselines receive equivalent simulator/reset access and query budgets, and which imagined observations reach the answer model?
Questions for authors
  • Does adaptive search outperform one-step best-of-N and fixed-depth search with the same models, primitives, information access, and total inference budget, isolating the benefit of horizon adaptation?
  • What imagination depths are actually used, and how does the exploration-depth cap of 15 in Section 4.1.1 relate to the inner imagination horizon?
  • On ESI-Bench, do simulated branch actions count toward the exploration-depth cap of 15, and what total query budget is enforced?
  • Do gains persist when baselines receive equivalent simulator-reset access and query budgets?
Reviewer_8DsDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ParallelWorld, an adaptive-horizon test-time scaling framework for agentic embodied agents. Instead of executing the planner's next primitive directly, the agent expands a tree of candidate action sequences in a "prospective world", uses a verifier to score and prune branches, executes only the first action of the selected trajectory, and replans (receding horizon). It is evaluated on ESI-Bench (active spatial reasoning, where prospective simulation restores simulator state and applies the true transition) and LIBERO-Pro (manipulation on top of HarnessVLA, where prospective simulation is a single image generated by an image model per primitive). Reported gains include +18.9/+20.0 points on two ESI-Bench subcategories and up to +27 points on LIBERO-Pro Spatial-S over HarnessVLA.

Critical strengths

(1) The receding-horizon design — committing only to the first action of the selected trajectory — is principled, and the largest gains occur exactly where lookahead should matter (occlusion / unobserved-change tasks). (2) A single formulation covers both evidence-acquisition and manipulation, and the failure analysis (Sec. 4.3) honestly identifies regimes where the method does not help.

Critical weaknesses

(1) On ESI-Bench, "simulation" is oracle lookahead with the ground-truth simulator transition and restorable state (Eq. 10). This is privileged information unavailable in any real deployment and unavailable to the Active Exploration baseline, so the result shows that search with a perfect model helps in simulation, not that the proposed method is deployable; simulator calls per real step are never reported or controlled. (2) The three core components — candidate generation, the verifier, and the stop/continue controller, including how K_h, M_h and the horizon are chosen — are specified only as symbols, with no model identities, prompts, or scoring rules, making the work irreproducible and "adaptive" undefined. (3) There is no compute-matched baseline (e.g., best-of-N with the same verifier at equal wall-clock/API budget) and no ablation of the verifier, horizon, or imagination source, so gains cannot be attributed to prospective simulation rather than to more compute; the single ablation uses 30 episodes on an unexplained, much-harder subset. Cost is also a concern: ~2,402s per episode vs 417s for direct execution (Table 3), with no performance-vs-compute curve despite the test-time-scaling framing. Imagination fidelity and verifier accuracy on LIBERO-Pro are never measured, and no confidence intervals are reported anywhere (several headline cells differ by 1–3 points at n=100, and one ESI subcategory regresses by 10 points).

Questions for authors

(1) How many simulator rollouts are used per real step on ESI-Bench, and how does the method perform when imagination must come from a learned model rather than the ground-truth transition? (2) Which exact models/prompts implement the planner, verifier and controller in each benchmark, and what are the empirical distributions of K_h, M_h and horizon? (3) At equal wall-clock/API budget, how does ParallelWorld compare to best-of-N HarnessVLA / Active Exploration with the same verifier? (4) What is the measured agreement between imagined images and true outcomes, and the verifier's ranking accuracy, on LIBERO-Pro? (5) Can you report per-subcategory n, aggregate scores and confidence intervals for Tables 1–3?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.