LensDreamer: Exploring Optical Design Basins via World-Model Imagination

Automatic lens design seeks optical system configurations that satisfy target imaging specifications. This task is fundamentally a search over a highly non-convex merit landscape. Existing automatic lens-design methods generally explore this landscape by repeatedly evaluating candidate systems with an optical simulator, making broad search computationally demanding. We take a different perspective: rather than evaluating every candidate trajectory with an optical simulator, we use a learned world model to imagine the consequences of design actions, thereby exploring a broader solution space and identifying promising paths out of inferior local minima. Building on this idea, we introduce LensDreamer, an action-conditioned optical world model that encodes layouts, lens prescriptions, design requirements, and other optical observations into a multimodal state and predicts its evolution under candidate actions. Its Basin-Map Planner organizes imagined endpoints into distinct candidate basins, selects representative designs for physical verification, and passes them to a local optimizer for refinement. We further construct LensDreamer-Traj, a trajectory dataset for world-model training, and LensDreamer-2K, a benchmark for automatic lens design. Experiments demonstrate the potential of world-model imagination for navigating non-convex optical design landscapes. More broadly, we hope this work sheds light on applying world models to automatic lens design and other non-convex scientific design problems. Codes and datasets will be released upon acceptance.