PhysTrace: A Standardized Benchmark for Evaluating Sim-to-Real Physical Fidelity

Physical simulation provides scalable interaction environments for robot learning, yet understanding how physical fidelity affects the transfer of robot policies to the real world requires systematic investigation through comparable physical experiments, real-world observations, and task evaluations. We establish a benchmark for evaluating physical simulation quality with a focus on real-world transfer. Using six simulation platforms as experimental testbeds, the benchmark integrates fundamental physics experiments, everyday interaction tasks, and real-world experimental comparisons into a hierarchical evaluation framework. It establishes consistent experimental procedures for asset representation, physical properties, initial conditions, loading protocols, and observation specifications. Platform adaptation, native configuration verification, and temporal alignment of data ensure reproducible experimental conditions and traceable data pipelines, while explicitly documenting differences in physical implementations and parameter semantics across platforms. Through multifactor experimental matrices, controlled single-variable comparisons, and paired simulation–real-world comparisons, we evaluate the accuracy, stability, and parameter sensitivity of physical responses. We further use process data from control, contact, motion, and sensing to locate the stages at which discrepancies arise and analyze their propagation mechanisms. Under the tested conditions, agreement between simulation and real-world observations depends on the task, interaction stage, and configuration, with platforms exhibiting distinct responses during frictional transitions, transient collisions, and support transitions. Process observations and controlled comparisons provide evidence of how initial states, native numerical configurations, and contact responses contribute to some of these discrepancies. Building on these findings, supplementary short-horizon manipulation experiments with vision-language-action policies examine the relationships among physical discrepancies, task performance, and real-world transfer. The benchmark reveals that simulation fidelity is task- and stage-dependent: agreement in overall responses can coexist with discrepancies in critical interaction processes. This finding offers an important perspective on real-world transfer performance and supports a shift in simulation-based training for embodied models, from evaluating overall realism toward targeted calibration of task-critical physical processes.