MAGAR: Multimodal Agents for GUI-Based 3D Mechanical Assembly and Minimal Rework

A GUI command can be correct at the interface level yet still produce an incorrect assembly. An agent must identify the relevant components, respect their struc- tural relationships, and determine which parts should move during recovery. We introduce MAGAR-Bench, a benchmark of 4,000 multimodal samples from 26 assembly families that separates Assembly Decision, Atomic Action, and Min- imal Rework. For rework, the objective counts distinct moved parts within the validated recovery candidate set for each state, rather than the number of actions. We also introduce MAGAR, a selective correction framework built on a frozen policy obtained by supervised fine-tuning. Its key design matches correction gran- ularity to output dependencies. Structured candidates revise assembly fields and rework scope, while complete action candidates keep native GUI primitives cou- pled with their parameters. A learned selector retains the original prediction un- less a candidate passes a task-specific intervention threshold. On recorded states from held-out assembly families, MAGAR improves Overall, the task-balanced aggregate, from 37.50 to 41.39, with gains on all three tasks. Across 209 interven- tions, 70 correct an error and 22 introduce one. These results support task-specific correction while highlighting the remaining tradeoff between fixing errors and preserving reliable predictions.