CrystalBench Dynamics: A Multimodal Benchmark for dynamic reasoning in crystal systems

Scientific reasoning over crystals requires connecting structural perturbations to changes in symmetry, relaxation response, and finite-temperature evolution. Existing materials-science benchmarks evaluate domain knowledge, property prediction, multimodal interpretation, or structure manipulation, but rarely connect controlled perturbations with their simulated physical consequences. We introduce CrystalBench Dynamics, a simulation-grounded benchmark containing 4,016 questions across 18 text and multimodal tasks derived from 1,500 real crystal structures. Our generation pipeline applies five structural perturbations, including lattice strain, vacancy creation, atomic substitution, interstitial insertion, and atomic displacement, followed by geometry relaxation and finite-temperature molecular dynamics. The resulting tasks are organized into four capability families: Perception, Symmetry Reasoning, Symmetry–Response Reasoning, and Dynamics Prediction. All reference answers are computed programmatically from crystal structures, optimization records, and simulation trajectories rather than generated by language models. Across contemporary language and vision-language models, the strongest system achieves only 51.96% task-macro accuracy on the Full Track, remaining more than 20 percentage points below the expert performance. Enabling thinking raises Gemma-4-31B-IT from 31.26% to 43.04%, demonstrating the value of inference-time reasoning while leaving substantial headroom. These results show that recognizing crystal structures does not imply the ability to predict how their symmetry and physical state evolve under perturbation.