acceptodds
Under review as a conference paper at ICLR 2027

ForceWorld: A Force World Simulator with Visual and Force Feedback

20 views

Abstract

Many existing world models focus on predicting future visual observations conditioned on robot actions. In contact-rich manipulation, however, distinct physical outcomes can appear visually similar. In such cases, force signals can provide key information about how the robot interacts with its environment. Therefore, we present ForceWorld, an action-conditioned world simulator that provides visual and force feedback to support closed-loop rollouts and evaluation of force-aware policies. Built on a pretrained video diffusion transformer, ForceWorld jointly denoises video and force representations to generate multi-view RGB observations and external joint-torque. To capture evolving contact conditions and their dependence on robot actions, we therefore augment framewise torque embeddings with a temporal encoder and directly modulate torque denoising with action inputs. Across four real-world contact-rich manipulation tasks, ForceWorld reduces high-load torque prediction RMSE by 26.6% compared with TACO, while also outperforming both evaluated video-prediction baselines. The predicted feedback enables closed-loop policy rollouts over minute-scale task horizons. This supports downstream policy evaluation, with success rates strongly correlated with those in real-world execution (Pearson r=0.956).

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotics World Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_u2FWOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ForceWorld, an action-conditioned world model that jointly predicts multi-view RGB observations and external joint-torque proxies estimated from motor currents. Built on a pretrained video diffusion transformer, it introduces temporal torque encoding, direct action modulation, and load-weighted training. The generated visual and torque observations are returned to fixed force-aware policies for closed-loop imagined execution. Experiments evaluate prediction quality, apparent contact behavior, and the agreement between real and imagined policy performance across four manipulation tasks, three policy families, and several policy checkpoints.

Critical strengths
  1. The paper addresses a useful evaluation problem and tests the predicted feedback in an actual interaction loop.Force-aware policies require an evaluation environment that reproduces their torque observations as well as their visual observations. ForceWorld supplies both modalities and allows the policy’s subsequent actions to depend on generated feedback. This gives the prediction model a concrete downstream purpose.

  2. The real-versus-imagined comparisons provide encouraging initial evidence. The main evaluation includes 240 real and 240 imagined trials across three policy families and four tasks, with additional comparisons across checkpoints. The reported success-rate correlation and within-task ranking metric support the feasibility of the approach. Including failed demonstrations and excluding evaluation-policy rollouts from world-model training are also sensible choices that improve the credibility of this comparison.

Critical weaknesses
  1. The incremental contribution needs a stronger demonstration of its value for policy evaluation. World-model-based policy evaluation has already been studied in WorldEval, and joint video–force generation is present in TACO. The temporal encoder, action modulation, and load-weighted objective are reasonable extensions, but the current experiments mainly establish prediction improvements. They do not show whether these additions produce substantially better policy evaluation than a visual world model combined with a simpler torque predictor. The backbone and conditioning configurations used for the retrained baselines also need to be clearer. Consequently, I am not yet convinced that the demonstrated evaluation benefit establishes a sufficiently strong advance over existing approaches.

  2. The evidence for a reliable, reusable evaluator is limited in both coverage and statistical resolution. The main policy-evaluation results cover the same four tasks represented in training and a single robot platform. Excluding evaluation-policy trajectories provides useful separation, but there is no systematic evaluation on unseen tasks, objects, or contact conditions. Furthermore, the reported correlation of 0.956 is calculated over only twelve policy–task success rates, each estimated from twenty trials. The within-task MMRV is a useful complement, but uncertainty in the rankings is not quantified. These results leave open whether ForceWorld can reliably distinguish closely performing policies or support evaluation beyond the demonstrated task settings.

Questions for authors
  1. What improvement in policy-evaluation accuracy comes specifically from joint visual–torque modeling? Could the authors compare against a matched visual world model with a simple independent torque predictor? Please also clarify the backbone, pretraining, and action-conditioning configurations used for the retrained TACO baseline.

  2. With the world-model checkpoint held fixed, how well are policy rankings preserved under unseen objects or contact conditions? Could the authors quantify uncertainty in the within-task rankings and MMRV, given the twenty trials per policy–task pair?

  3. How robust are the current-derived torque labels across recording sessions, payload changes, and temperature variation? Is there independent calibration or drift analysis beyond the free-motion and elastic-band checks, particularly for the high-load labels used in training and evaluation?

  4. What is the wall-clock cost of an imagined rollout? Please clarify how execution-time force feedback is implemented within the approximately 4.8-second action chunks, including whether the evaluated FAWAM implementation retains its online residual correction.

3 comments

We thank the reviewer for a detailed and expert review. We acknowledge the limitations in statistical resolution and coverage and restrict our claims to what the evidence supports. On the size of the increment, we respectfully differ, for the reasons below.

W1 — Contribution beyond prior work.

We argue that the core contribution of ForceWorld is to enable closed-loop evaluation of force-aware policies by jointly predicting video and torque conditioned on the policy's own actions and returning both as observations during rollout. TACO takes no action input and therefore cannot directly serve as a closed-loop policy evaluator. It generates joint video–force correction segments for post-training, without returning force observations to a policy during rollout. WorldEval returns generated images but cannot supply the torque observations required by force-aware policies; Table 6 shows the impact of withholding this feedback. The temporal encoder, action modulation, and load-weighted objective improve torque prediction for this closed-loop use (Table 2).

Q1 — Joint modeling benefits and baseline configurations.

We compare ForceWorld with a matched visual-only world model equipped with a separately trained torque prediction head. Using the same training data and closed-loop evaluation protocol, the decoupled baseline achieves a correlation of 0.926 and an MMRV of 0.058, compared with 0.956 and 0.0125 for ForceWorld. Without force information to constrain visual generation, a visual-only world model can produce penetration artifacts, as shown in our Push-T experiments. A separate torque prediction head then operates on these physically inconsistent frames, which can compromise torque accuracy during contact.

All retrained baselines use the same training trajectories as ForceWorld and are evaluated on the same held-out windows, with the same start frames and, where applicable, recorded actions; their backbones, pretraining and action conditioning are listed below. TACO and Ctrl-World are retrained from their released code and recipes.

ModelBackbone and initializationAction input
TACOWan2.2-TI2V-5B, from the robot-pretrained Wan2.2-5B-Robot DiTNone, by design
Ctrl-WorldStable Video Diffusion, from the released SVD image-to-video weightsSame 14-D commanded end pose
ForceWorldWan2.1 image-to-video, 14B14-D commanded end pose, via an additive timestep embedding and AdaLN modulation

W2 and Q2 — Coverage and ranking uncertainty.

Coverage. ForceWorld reproduces the observed real-world ranking of all three policies under this object change. We additionally evaluated the Wipe board task using a different blackboard eraser to test object generalization. The table below compares real and imagined success rates and policy rankings with the new object.

PolicyReal success rate (%)Imagined success rate (%)Real rankImagined rank
ForceVLA255022
π0.5+τ507511
FAWAM153533

Uncertainty. We have quantified it from the per-cell success counts in Appendix D. The within-task rankings are significantly better than chance (exact permutation test, p = 0.005 for MMRV), and resampling the 20 trials of each cell gives an MMRV of 0.037 (95% interval [0.008, 0.092]), against 0.033 for a repeated real evaluation.

Q3 — Robustness of current derived labels.

Our weight tests indicate that the torque labels are consistent across recording sessions and operating time, and respond approximately proportionally to payload.

  • Session and warm-up consistency. We placed a 300 g calibration weight on a plate and commanded the robot arm to lift the plate. Across recording sessions and operating durations, the mean torque magnitude during the same lifting phase varied by at most 6% relative to the reference trial. Motor temperature was not logged, so operating time is only a proxy for warm-up.
  • Payload response. Across 300–600 g, labels increased approximately proportionally with load.
  • External validation. FACTR2(Corl2026)reports a contact L1 error of 0.547 N·m for NEXT versus 1.471 N·m for a disturbance observer, measured against Franka's built-in external torque estimate. This supports the estimator but does not independently calibrate high-load labels on our platform.

Q4 — Wall clock cost and feedback timing.

Including policy inference, an imagined rollout takes 1.7–12.1 minutes for ForceVLA and π0.5+τ and 2.4–19 minutes for FAWAM, depending on the task.

TaskRobot time simulatedForceVLA, π0.5+τFAWAM
Wipe board34 s1.9 min2.7 min
Unlock30 s1.7 min2.4 min
Screw bulb75 s4.1 min6.8 min
Peel cucumber220 s12.1 min19 min

In the re-run trials, the policy receives new RGB and torque observations after every 10 executed actions (1 s), matching real-robot execution. The 4.8 s refers to the maximum world-model generation window, not the feedback interval.

FAWAM's online residual correction was disabled in both the real and the imagined evaluation.

Thank you for the additional experiments and clarifications. The comparison with a matched visual model plus an independent torque predictor directly addresses my main concern. The lower MMRV provides useful evidence that joint modeling benefits policy evaluation beyond improving prediction metrics alone.

The new-object experiment, ranking uncertainty analysis, and clarification of feedback timing also strengthen the evaluation. I appreciate the explicit reporting of rollout costs and the fact that FAWAM's residual correction was disabled in both real and imagined execution.

Some limitations remain. The object-generalization test is narrow, and the imagined success rates overestimate real performance despite preserving the observed ranking. The uncertainty analysis does not directly establish a significant advantage over the decoupled baseline, and high-load torque calibration on this platform remains limited. These points should qualify claims about generalization and evaluator reliability.

Overall, the response provides enough additional evidence to change my recommendation to 3 (weak acceptance). Please incorporate these results and qualifications into the manuscript.

Reviewer_3rfEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an action-conditioned visual-torque world model that jointly generates future multi-view RGB observations and external joint-torque estimates. This method uses TARE to capture temporal torque information, AMP to strengthen action conditioning, and PACE to emphasize prediction during high-load phases. It achieves strong visual prediction performance on the evaluated tasks and supports the effectiveness of torque modeling through ablation studies.

Critical strengths
  1. This paper proposes an action-conditioned world simulator that jointly models joint torques and visual observations.
  2. It combines temporally aligned torque prediction with action conditioning and high-load supervision, effectively integrating torque information into future observation prediction.
Critical weaknesses
  1. Although the data sources and general split strategy are described, the total training data volume and its distribution across tasks remain insufficiently specified.
  2. The torque representation and model configuration used for the TACO baseline are not described in sufficient detail, making it difficult to assess the fairness and comparability of the evaluation.
  3. This paper states that the policy retains joint-space state inputs, while the world model primarily generates RGB observations and torque feedback. However, it does not clearly explain how subsequent joint states are obtained or how their consistency with the generated feedback is maintained.
Questions for authors
  1. Could the authors provide a complete dataset details, including the number and duration of trajectories for each task, the training / validation / test splits, and the criteria for collecting successful and failed trajectories?
  2. Given that TACO uses end-effector force-torque measurements, how were its torque representation and model configuration adapted to ensure a comparable evaluation with ForceWorld?
  3. During imagined execution, how are the joint-space states supplied to the policy updated? Are they derived from action commands, an additional state predictor, or another source? When contact causes motion to deviate from the commands, how is consistency maintained between the joint states, generated RGB observations, and torque feedback?
3 comments

We thank the reviewer for the careful reading and constructive feedback. Below, we address each point and provide the relevant details and clarifications in our responses.

Q1 — dataset details.

We provide the dataset breakdown below.

TaskTrajectoriesDuration (h)Mean length (s)
Wipe board2,87426.733
Unlock2,99926.832
Peel cucumber59133.5204
Screw bulb85026.2111
Total7,314113.256
  • Splits. Each task is split by trajectory into training, validation and test sets at 98 : 1 : 1, so no validation or test trajectory contributes any window to training.
  • Failed trajectories. A trajectory is labeled successful when the operator completes the task. Failed trajectories come from two sources. Some are operator errors that make the task impossible to finish, e.g., an object dropped to the floor or out of the arms' reach. Others are recorded on purpose to imitate typical policy failures: a bulb misaligned with the socket, a jittering arm, a key that does not line up with the keyhole, or writing left on the board after wiping.
  • Policy data. The evaluated policies are trained on a subset of this dataset that contains only the successful trajectories. No rollouts of the evaluated policies enter world-model training.

Q2 — TACO configuration.

We ran TACO's released code and recipe and changed only what our data requires.

  • Backbone. TACO keeps its own backbone: Wan2.2-TI2V-5B, initialized from the robot-pretrained Wan2.2-5B-Robot DiT.
  • TACO adaptation and fairness. We take explicit steps to align the prediction task while preserving TACO's original force-modeling design. We adapt its original fingertip force–torque representation to the same 12-D external joint-torque signal used by ForceWorld, comprising six joints from each arm, for both initial force conditioning and future prediction. Since TACO's force channel already supports 12-D sequences, we retain its released force-channel architecture and first-frame anchoring scheme, while recomputing normalization statistics on our data using its original script. TACO is retrained from its released code on the same trajectories and evaluated using the same windows and sampling seeds.
  • Data and evaluation. Training trajectories, 49-frame 10 Hz windows and the three-view canvas are the same as ours. TACO is evaluated on the same windows, from the same real start frame and first-frame torque.
  • Action input. TACO takes no action input by design, which may contribute to its higher torque error.

Q3 — joint state during imagined execution.

The joint states are derived from action commands; there is no additional state predictor. At each replanning step, the joint state is set to the last commanded joint target of the executed segment (for FAWAM, which acts in end-effector space, the last commanded end pose). This follows the practice of policy-in-the-loop world models whose policies read proprioception. Ctrl-World (Guo et al., 2025) maps the policy's joint-velocity outputs to future joint configurations with a learned adapter and feeds them back as the next joint state. RoboWorld (Jeon et al., 2026) updates the joint state by integrating the policy's joint-velocity commands.

When contact makes the motion deviate from the commands, consistency is maintained by the world model, which generates RGB and torque jointly from the same commanded actions. Its training data are also conditioned on commands, and there a command that presses into an object leaves the arm blocked and raises the external torque. ForceWorld therefore renders the arm stopped at the contact and predicts the corresponding torque rise. On Push-T, the gripper passes through the rigid block in 5% of rollouts, versus 65% for the visual-only model (Fig. 5). The joint state supplied to the policy remains the commanded value, as in the works above, and the blocked motion reaches the policy through the generated frames and torque.

We believe these answers resolve the points raised. If so, we would be grateful if you would consider updating your assessment.

Thank you for the additional clarifications. I have a clear understanding of the dataset composition, the criteria for comparison with the TACO baseline, and how the joint-space states derived during imagined execution. It has addressed most of my concerns, so I have decided to update my score to 3 (weak accept).

We thank the reviewer for the thoughtful feedback and for updating their assessment. We are glad that our clarifications addressed the main concerns, and we sincerely appreciate the time and care the reviewer devoted to evaluating our work.

Reviewer_bgShOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

this work introduces an action-conditioned world model designed for contact-rich robotic manipulation that jointly predicts (multi-view) video and joint torques. This unlocks closed-loop evaluation of force-aware robot policies entirely in imagination. There are results for reduced sim2real gap when evaluating models - which is important for robotics tasks.

Critical strengths
  1. The conceptual contribution of ForceWorld is expanding learned world simulation from visual-only frame prediction to joint video and force co-denoising.

  2. Specialized Components for Torque Denoising (e.g. TARE for predicting contact phases).

  3. Reduced visual penetration due to vision-torque denoising (but see weaknesses section about video quality metrics).

Critical weaknesses
  1. Table1 claims better visual quality over baselines Ctrl-World and TACO. However, it is not clear whether that improvement is due to the base model (Wan 2.1 DiT ckpt) or due to their changes. However, from table 4 (appendix B.3), we see that the video-only and video-torque models have similar video quality metrics (seen in all 3 metrics: PSNR SSIM LPIPS). This suggests that major improvements to video quality come from the choice of base model rather than than from joint force-video denoising. This seems to contradict the stated contribution that joint denoising improves visual prediction quality.

  2. sim2real transfer metric: Pearson r=0.956 This high correlation between sim and real seems good at first, but using the feed-zero policy reaches r=0.888 already (table 6). So I wonder if this high Pearson is because the author's are taking task success numbers across different tasks with distinct clusters (easy, medium, hard) and squashing them together in one metric. An evaluator that reflects that "Wipe board is easy and Peel cucumber is hard" will work equally easily in sim and real. This will produce a high linear correlation across those 12 pooled points.

Most robot arms carry no force sensor, so we take the force signal from what every actuated arm already logs, following NEXT (Oh et al., 2026)

I think it would be good to have some ground truth information to compare how good your force prediction is to the ground truth (e.g. adding a Force Torque sensor or using at least one robot with force-torque sensing). Without that, it’s hard to claim that this is a ‘force’ predictor; rather it is a proxy for force. This proxy might still be useful for force-based VLAs, but that is orthogonal to the point of labelling what ForceWorld predicts as ‘force’. Regardless of the label, this sensorless supervision pipeline allows world models to apply across cheaper manipulators.

Questions for authors

please address weaknesses section

2 comments

We thank the reviewer for the positive assessment and constructive feedback. Below, we address the concerns raised in W1 and W2. Regarding W3, we provide a more precise explanation of what the signal represents.

W1 — Can the video quality gains be attributed to joint force–video denoising?

ForceWorld achieves better overall visual prediction metrics than the baselines in Table 1. To examine the contribution of joint modeling, we compare it with a matched visual-only counterpart using the same backbone. On Push-T, both models use the same training data, recorded action sequences, pre-contact observations, prediction horizon and denoising budget. Joint video–torque prediction reduces apparent gripper–block penetration from 13/20 rollouts (65%) to 1/20 (5%), alongside improvements in PSNR, SSIM and LPIPS (Table 3, Fig. 5; Appendix C).

The reduction in penetration matters beyond image appearance. A gripper passing through a rigid block may affect only a small image region, yet changes whether the policy observes blocked motion or continued movement. In closed-loop evaluation, such an error can mislead the next action and propagate through subsequent predictions, distorting the estimated policy success rate. Frame-level metrics do not directly measure this contact behavior. Push-T provides direct evidence of improved visual contact consistency.

W2 — pooled correlation and task difficulty.

Thank you for examining Table 6 closely. To assess agreement beyond differences in task difficulty, we compare policy rankings within each task. Predicted torque feedback yields a lower task-averaged MMRV than zero feedback (0.0125 vs. 0.0708), with lower ranking error on Unlock, Peel cucumber and Screw bulb, as shown below.

Task-level MMRV computed from the success counts in Appendix D (Tables 5–6), with 20 trials per policy–task pair in each evaluation condition. Lower values indicate smaller ranking errors, weighted by the real-success-rate gaps (Eq. 7).

TaskPredicted torque: MMRV ↓Zero feedback: MMRV ↓
Wipe board0.01670.0000
Unlock0.00000.0333
Peel cucumber0.03330.0667
Screw bulb0.00000.1833
Task average0.01250.0708

On Wipe board, the only ranking disagreement under predicted feedback is a tie between two policies whose observed real success rates differ by one trial out of 20. On Peel cucumber, the reversed pair likewise differs by only one real success. These distinctions matter because MMRV measures the real-performance gap associated with a ranking error.

After subtracting each task’s mean from the real and imagined success rates, the correlation remains 0.82 with predicted torque feedback, versus 0.62 with zero feedback. These results support within-task agreement beyond differences in task difficulty, while the 20-trial evaluation does not reliably resolve small differences between policies.

W3 — ground truth for the force signal.

Thank you for this thoughtful comment and for recognizing the value of sensorless supervision. The estimator we use, NEXT (CoRL 2026), has been validated against sensor measurements by its authors. On a Franka arm, compared with the arm's built-in joint-torque sensors, its mean error during contact is 0.55 N·m, versus 1.47 N·m for a disturbance observer. In free space, its error (0.41 N·m) is even lower than that of the sensor itself (0.45 N·m). On our own arms, we further checked the estimate in two settings where the answer is known in advance.

We agree that the label is a proxy, and Section 3.1 introduces it as an external joint-torque proxy. More precisely, it is an external joint-torque proxy. Its main difference from a calibrated torque is a per-joint scale set by the motor constants. ForceWorld is trained and evaluated on per-joint normalized torque, which removes this scale, so what the model learns, namely when each joint is loaded and how strongly relative to its range, does not depend on the absolute calibration. We also fully agree that the proxy is useful for force-aware policies.

This is also why the sensorless pipeline you highlight matters to us. Our aim is to let world models obtain force information on arms without force sensors, and we plan to extend the pipeline to more embodiments so that force-aware world models can be trained across platforms.

Thank you for the response and additional analyses.

  1. Visual prediction vs. contact consistency (W1): The Push-T ablation shows that joint video–torque denoising improves physical contact consistency by reducing gripper–block penetration, which is good. However, this does not fully resolve the concern about the broader claim in Section 1 and Table 1 that joint modeling "improves visual prediction quality" over Ctrl-World and TACO. In Table 4, adding the torque stream yields virtually identical video metrics, suggesting the large Table 1 visual gains over baselines stem primarily from the 14B Wan 2.1 backbone, while joint torque prediction mainly benefits collision-heavy cases like Push-T. For camera-ready, please update the Section 1 claim on general visual prediction quality and contact consistency.

  2. Task-difficulty pooling for Pearson correlation (W2): Thanks for the per-task MMRV comparison. Even after removing task-difficulty baselines, predicted torque feedback provides signal over feed-zero — raising the correlation from 0.62 to 0.82 and reducing ranking error across the harder, contact-rich task . Please update the text/appendix to show the per-task statistics as well.

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your reviews of this submission. The authors have now posted their responses, and the author–reviewer discussion period ends on November 18, 2026 (AoE).

Please read the authors' rebuttal and let them know whether it addresses your concerns. A short reply is enough, for example:

  • which of your concerns are now resolved;
  • which remain open, and what evidence or clarification would resolve them;
  • whether you plan to update your rating.

If the rebuttal changes your assessment, please update your review and rating before the discussion period ends. If you have already engaged, thank you, and no further action is needed unless the authors reply again.

Your participation is important for a fair and well-informed decision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.