acceptodds
Under review as a conference paper at ICLR 2027

LensDreamer: Exploring Optical Design Basins via World-Model Imagination

Abstract

Automatic lens design seeks optical system configurations that satisfy target imaging specifications. This task is fundamentally a search over a highly non-convex merit landscape. Existing automatic lens-design methods generally explore this landscape by repeatedly evaluating candidate systems with an optical simulator, making broad search computationally demanding. We take a different perspective: rather than evaluating every candidate trajectory with an optical simulator, we use a learned world model to imagine the consequences of design actions, thereby exploring a broader solution space and identifying promising paths out of inferior local minima. Building on this idea, we introduce LensDreamer, an action-conditioned optical world model that encodes layouts, lens prescriptions, design requirements, and other optical observations into a multimodal state and predicts its evolution under candidate actions. Its Basin-Map Planner organizes imagined endpoints into distinct candidate basins, selects representative designs for physical verification, and passes them to a local optimizer for refinement. We further construct LensDreamer-Traj, a trajectory dataset for world-model training, and LensDreamer-2K, a benchmark for automatic lens design. Experiments demonstrate the potential of world-model imagination for navigating non-convex optical design landscapes. More broadly, we hope this work sheds light on applying world models to automatic lens design and other non-convex scientific design problems. Codes and datasets will be released upon acceptance.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Optical Design, World Model

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_k4EnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LensDreamer, a world-model-assisted optimization framework for optical lens design. By learning the multi-step changes in optical response during lens parameter editing, the method predicts in latent space the endpoints that different design action sequences may reach, and uses a Basin-Map planning strategy to allocate a limited budget for high-fidelity physical evaluations across different candidate design regions. The paper also constructs the LensDreamer-Traj trajectory dataset, which is used to train and evaluate the model's ability to predict lens parameter changes and optical responses.

Critical strengths
  1. The paper tackles a practically meaningful system design problem and backs it with fairly thorough experiments. It treats optical lens design as a complex non-convex optimization task and combines multi-step optical response prediction, exploration of candidate design regions, and allocation of physical optimization budget into a complete framework. Its effectiveness is demonstrated not only by final optimization results but also by multi-step prediction, dynamics-component ablations, and candidate action selection from a shared pool. These results suggest that the proposed model improves prediction and action selection under the tested settings, rather than only achieving strong final system-level metrics. This supports acceptance, since the method is empirically justified at multiple levels and addresses a problem with clear practical value.

  2. The dataset and physical optimization evaluation framework also have research value. The authors construct the LensDreamer-Traj trajectory dataset and the LensDreamer-2K benchmark, and combine predictive evaluation with final physical optimization evaluation. Under controlled comparisons with matched physical evaluation budgets, LensDreamer outperforms Optiland multi-start, with paired confidence intervals in the supplementary material supporting this improvement. The paper also reports final design error and attempts to assess coverage of different stable optimization endpoints, offering a complementary view of search behavior in non-convex optical design spaces. This supports acceptance because the dataset, evaluation pipeline, and physical optimization experiments have potential research value beyond the specific model architecture.

Critical weaknesses
  1. The lens parameter updates are deterministic and known (Eq. 3), making exact parameter updates combined with a nonlinear optical response surrogate a natural baseline. Such a model could also evaluate multi-step action sequences. Without this comparison, it is difficult to assess the added value of recurrent world-model dynamics. Moreover, the comparison between greedy one-step planning and the full Basin-Map planner changes both the planning horizon and basin allocation strategy, making their individual contributions difficult to isolate.

  2. The reported basin coverage depends on how locally optimized endpoints are grouped, but the numerical tolerances used to identify distinct basins are not specified. The DLS stopping criteria and sensitivity to these tolerances are also unclear. Since discovering distinct basins is a central claim of the paper, these details are important for reproducing the results and assessing the robustness of the reported improvements.

Questions for authors
  1. Have the authors considered a baseline that uses exact parameter updates with a nonlinear optical response surrogate? How would it compare with the proposed world model under the same training data and planning budget?

  2. What numerical tolerances are used to determine whether two DLS-optimized endpoints belong to the same basin, and what are the DLS stopping criteria? How sensitive are the reported basin coverage results to these settings?

8 comments
AuthorsOpenReview ↗

Response to Reviewer k4En (R4)

Thank you for your comments. We respond point by point below and have revised the manuscript and supplementary material. Please download the latest main-paper and supplementary PDFs to view the revisions.

W1/Q1: Added value of learned dynamics and multi-step planning.

We trained a Direct Optical Surrogate combining exact prescription updates with nonlinear response prediction:

pk+1=A(pk,ak),Y^k+1=fθ(pk+1,S).p_{k+1}=\mathcal A(p_k,a_k),\qquad \hat Y_{k+1}=f_\theta(p_{k+1},S).

Direct Optical Surrogate: A shared three-layer Transformer block repeated four times predicts responses from exact action-updated prescriptions and specifications using bidirectional attention.

Action-Conditioned World Model: Transformer–RSSM uses causal action-history attention, recurrent stochastic rollouts, and a shared three-layer block repeated four times per latent step.

Both share training data/schedule, 2,000 tasks, Basin-Map Planner (H=8), ≤180 online Optiland calls, and local refinement. Selection uses a shared exact-prescription/predicted-response representation, excluding private latent features and uncertainty scoring. LensDreamer (Ours) retains its original planner configuration.

MethodFeasible ↑SR ↑SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} ↓Feasible SpecGAP\mathrm{SpecGAP} ↓RMS (μ\mum) ↓Axis MTF ↑Corner MTF ↑∣Distortion∣\lvert\mathrm{Distortion}\rvert ↓
Direct Optical Surrogate83.45%46.85%2.150.59109.8311.86%2.06%0.21
Action-Conditioned World Model84.75%48.60%1.960.51102.4112.49%2.23%0.17
LensDreamer (Ours)85.20%50.35%1.890.4895.6713.27%2.37%0.14

Under the common planner, the world model reduces SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} by 8.8% and RMS by 6.8%, and improves SR by 1.75 percentage points, with gains on all metrics. The full LensDreamer system reaches 43.60% OOD SR versus 39.20% for the surrogate (+4.40 percentage points; 11.2% relative), supporting success on unseen specifications.

We ran H=1, 2, 4, and 8 with the same full Basin-Map Planner, checkpoint, candidate population, verification slots, other settings, DLS, tasks, and seeds, with ≤180 Optiland calls per task. H=1 retains basin-aware selection, unlike the original greedy baseline.

HorizonSpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} ↓SR ↑Feasible ↑Certified stable basins ↑Anchor escape ↑Beat physics-only ↑
H=12.4240.85%81.15%1.2743.15%58.65%
H=22.1845.10%82.90%1.4350.70%65.20%
H=42.0348.25%84.05%1.5458.35%70.65%
H=81.8950.35%85.20%1.6362.5%73.25%

From H=1 to H=8, SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} decreases by 21.9% and SR increases by 9.50 percentage points, while stable basins rise from 1.27 to 1.63. All horizons retain basin-aware selection and execute only the first action before replanning.

Thank you for these suggestions. Section 3.4, Tables 4–5, and Appendix F.2 now isolate the contributions of dynamics and planning.


W2/Q2: Endpoint grouping, DLS stopping, and tolerance sensitivity.

We already described endpoint grouping in Section 2.4/Appendix F, coverage metrics in Appendix G, and tolerance-based grouping in Appendix H, Figure 2.

Following your suggestion, we have added numerical criteria in Appendix F.1 and sensitivity results in Appendix H.1. Certification is a separate post-hoc audit, outside the online 180-call budget, allowing ≤1,024 residual evaluations per audit refinement plus certification calls. Escape uses all 2,000 tasks. Grouping uses a complete-link rule with structural checks and tolerance-normalized prescription/response distances. Prescription tolerance is 10−310^{-3}. Online and audit TRF use separately specified stopping rules. Certification additionally requires physical validity and projected-gradient infinity norm ≤10−410^{-4}, passing checks at both 10−610^{-6} and 10−710^{-7} difference steps; a budget stop alone does not certify convergence.

We scale grouping tolerances on fixed audited endpoints. Mean basin counts are:

MultiplierFull Basin-MapScalar top-kk/CEMRandom diverse
0.50×1.841.321.23
0.75×1.721.231.14
1.00×1.631.141.07
1.50×1.541.081.01
2.00×1.411.020.96

The ranking remains unchanged across the full range, also for anchor escape. With matched endpoint counts, Full-minus-CEM remains positive (0.16–0.25). Thank you for this suggestion. Appendices F.1/G give the protocol and statistics; Appendix H.1, Tables 10–11, adds sensitivity results for reproducible coverage analysis.

AuthorsOpenReview ↗

Kind Request to Review Our Rebuttal and Updated Manuscript

Dear Reviewer k4En,

Thank you for your time and thoughtful feedback. We have posted our point-by-point responses, including additional experiments and clarifications, and uploaded the revised manuscript and supplementary material. We would greatly appreciate it if you could review these updates when you have a chance. If any questions or concerns remain, we would be happy to provide further explanations or discuss them with you. Thank you again for your time and consideration.

Best regards, Author of submission 1572

Reviewer_k4EnOpenReview ↗

Thank you for the author's reply. I'll keep my score.

Thank you for the author's reply. I'll keep my score.

Reviewer_gs9vOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LensDreamer, a world-model-based framework for automated lens design under a limited budget of high-fidelity physical evaluations. By integrating a multimodal optical state encoder, Transformer–RSSM dynamics, and a Basin-Map Planner, the framework imagines multi-step design trajectories and selects diverse candidates for physical verification. Experiments on the LensDreamer-2K benchmark demonstrate significant improvements in design feasibility, specification compliance, and the exploration of distinct DLS-stable endpoints.

Critical strengths

S1. Strong empirical performance under controlled physical-evaluation budgets.

LensDreamer demonstrates consistent gains across 2,000 lens-design tasks, boosting the feasibility rate from 79.35% to 85.20% and lowering SpecGAPFA from 2.83 to 1.89 over the strongest budget-matched baseline. Crucially, all final designs are re-evaluated within the same physical simulator, lending strong credibility to the reported improvements.

S2. Effective basin exploration backed by solid experimental validation.

The Basin-Map Planner integrates imagined endpoint clustering with cross-round exploration mechanisms to enhance search diversity. Empirical results highlight clear gains in anchor escape rates and the discovery of distinct DLS-stable endpoints, confirming that the proposed strategy actively uncovers superior local optima rather than merely refining initial solutions.

Critical weaknesses

W1. The independent contribution of learned multi-step dynamics remains insufficiently isolated. While the full planner outperforms greedy one-step planning, the comparison also changes the basin-aware selection strategy. Moreover, since prescription updates are explicitly defined by the action operator, it remains unclear whether learning multi-step prescription dynamics offers a substantial advantage over directly applying these transitions and using a simpler surrogate to predict optical properties. A controlled comparison would better establish the necessity of the proposed world model.

Questions for authors

Q1. Contribution of Multi-Step Planning.
Table 3 shows that the full planner outperforms greedy one-step planning, but the comparison also changes the basin-aware selection strategy. Could the authors evaluate different planning horizons (e.g., H=1, 2, 4, 8) while keeping the Basin-Map Planner and physical-evaluation budget fixed? This would help clarify whether longer-horizon imagination actually improves action selection and final design quality.

Q2. Necessity of Learned Dynamics.
Since prescription updates are explicitly defined by the action operator, could the authors compare against a simpler approach that directly computes future prescriptions from action sequences and uses a surrogate model to predict their optical properties? Such a comparison under the same Basin-Map Planner would help establish whether learning the full multi-step dynamics provides a substantial advantage.

Q3. Computational Cost and Practical Efficiency.
Could the authors report the end-to-end optimization time per task, including multimodal encoding, imagined rollouts, and physical verification? Given the additional model computation and offline training cost, under what conditions does LensDreamer offer a practical advantage over simpler surrogate-assisted optimization methods?

5 comments
AuthorsOpenReview ↗

Response to Reviewer gs9v (R3)

Thank you for your comments. We respond point by point below and have revised the manuscript and supplementary material. Please download the latest main-paper and supplementary PDFs to view the revisions.

W1/Q2: Exact prescription updates followed by a direct optical surrogate.

We trained a Direct Optical Surrogate using the known update pk+1=A(pk,ak)p_{k+1}=\mathcal A(p_k,a_k) and response prediction Y^k+1=fθ(pk+1,S)\hat Y_{k+1}=f_\theta(p_{k+1},S).

Direct Optical Surrogate: A shared three-layer Transformer block repeated four times independently predicts responses from exact prescriptions using bidirectional surface attention.

Action-Conditioned World Model: Transformer–RSSM combines causal action-history attention with recurrent stochastic rollouts.

Both use the same data, training schedule, Basin-Map Planner (H=8), ≤180 online Optiland calls, and local refinement on all 2,000 tasks. Selection uses exact prescriptions and predicted responses in the same representation, excluding private latent features and uncertainty scoring. LensDreamer (Ours) reports the full system with its complete planner configuration.

MethodFeasible ↑SR ↑SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} ↓Feasible SpecGAP\mathrm{SpecGAP} ↓RMS (μ\mum) ↓Axis MTF ↑Corner MTF ↑∣Distortion∣\lvert\mathrm{Distortion}\rvert ↓
Direct Optical Surrogate83.45%46.85%2.150.59109.8311.86%2.06%0.21
Action-Conditioned World Model84.75%48.60%1.960.51102.4112.49%2.23%0.17
LensDreamer (Ours)85.20%50.35%1.890.4895.6713.27%2.37%0.14

Under the common planner, the world model reduces SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} by 8.8% and RMS by 6.8%, increasing SR by 1.75 percentage points and feasibility by 1.30 points. This supports learned optical-response dynamics even when exact prescription updates are available. The full LensDreamer system achieves 43.60% OOD SR versus 39.20% for the surrogate (+4.40 percentage points; 11.2% relative), supporting better success on unseen specifications. Thank you for this suggestion. The comparison is in Section 3.4, Table 4, with settings in Appendix F.2, strengthening the controlled evaluation.


W1/Q1: Contribution of the planning horizon.

We ran separate closed-loop optimizations at H=1/2/4/8 with the full Basin-Map Planner fixed, alongside the checkpoint, candidate population, verification slots, other planner settings, tasks, local refinement, and 180-call cap. H=1 retains basin-aware selection. Each setting executes only the first selected action before replanning.

HorizonSpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} ↓SR ↑Feasible ↑Certified stable basins ↑Anchor escape ↑Beat physics-only ↑
H=12.4240.85%81.15%1.2743.15%58.65%
H=22.1845.10%82.90%1.4350.70%65.20%
H=42.0348.25%84.05%1.5458.35%70.65%
H=81.8950.35%85.20%1.6362.5%73.25%

From H=1 to H=8, SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} falls 21.9% and SR rises 9.50 percentage points, while stable basins increase from 1.27 to 1.63. This supports longer lookahead with basin allocation fixed. Thank you for this suggestion. We added this ablation to Section 3.4, Table 5, with settings in Appendix F.2.


Q3: Computational cost and practical efficiency.

Warm latency includes rendering, encoding, imagination/planning, physical verification, local refinement, and finalization. Loading, training, and the separate basin audit are excluded. Each task uses one B200 and 16 CPU cores; reported times average per-task medians of three timing repetitions.

MethodWarm time (s) ↓SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} ↓SR ↑
Direct Optical Surrogate242.772.1546.85%
Action-Conditioned World Model258.351.9648.60%
LensDreamer (Ours)216.831.8950.35%

The full system is 10.7% faster than the direct surrogate in this comparison. Encoding takes 12.64 s, imagination/planning 34.19 s, and verification plus refinement 164.73 s versus 226.41 s for the direct surrogate. Reduced physical work offsets the model overhead. The shared-planner world model is slower overall (258.35 s), so learned dynamics alone do not establish a speed advantage.

Our primary goal is better lens-design quality under a limited physical budget. Offline training takes 50.48 hours on four B200 GPUs once; the same checkpoint serves all 2,000 tasks without retraining, sharing its cost across design queries. This reuse carries the learned quality gains across evaluated requirements. Thank you for this suggestion. Appendix H.3, Table 14 details timing and cost.

AuthorsOpenReview ↗

Kind Request to Review Our Rebuttal and Updated Manuscript

Dear Reviewer gs9v,

Thank you for your time and thoughtful feedback. We have posted our point-by-point responses, including additional experiments and clarifications, and uploaded the revised manuscript and supplementary material. We would greatly appreciate it if you could review these updates when you have a chance. If any questions or concerns remain, we would be happy to provide further explanations or discuss them with you. Thank you again for your time and consideration.

Best regards, Author of submission 1572

AuthorsOpenReview ↗

Discussion Period About to Finish

Dear Reviewer gs9v,

Thank you for your time and thoughtful feedback. We have posted our point-by-point responses, including additional experiments and clarifications, and uploaded the revised manuscript and supplementary material. We would greatly appreciate it if you could review these updates when you have a chance. If any questions or concerns remain, we would be happy to provide further explanations or discuss them with you. Thank you again for your time and consideration.

Best regards, Author of submission 1572

Reviewer_yubQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper formulates automatic lens design as action-conditioned world modeling. Given a target effective focal length, F-number, detector size, spectral range, MTF requirements, and RMS target, the system encodes 2D and 3D lens layouts, prescriptions, specifications, physical observations, and the legal action domain into a multimodal state. LensDreamer uses a frozen DINOv3 ViT-L/16 for visual features and a LoRA-adapted Qwen3-VL-32B-Instruct backbone for multimodal fusion, followed by a Transformer-RSSM that imagines length-eight action sequences without invoking the high-fidelity simulator at every step.

The Basin-Map Planner clusters imagined endpoints into candidate regions, selects diverse representatives, verifies only the first action of each selected sequence with the physical evaluator, re-encodes the verified outcomes in an external MPC loop, and sends retained designs to local DLS refinement. The paper introduces LensDreamer-Traj, with 13,696 training trajectories and 109,568 high-fidelity transitions, and LensDreamer-2K, with 2,000 evaluation tasks consisting of 500 in-distribution and 1,500 specification-OOD cases.

On the reported benchmark, LensDreamer obtains an 85.20% feasible rate, SpecGAP_FA of 1.89, feasible-sample SpecGAP of 0.48, RMS of 95.67 micrometers, axis MTF of 13.27%, corner MTF of 2.37%, and absolute distortion of 0.14. The authors also report broader verified coverage and more distinct DLS-stable endpoint groups for the full Basin-Map Planner. The manuscript explicitly limits its claims to specification-OOD evaluation with fixed topology and does not claim global optimality.

Critical strengths
  1. The problem formulation is clear and potentially novel. The paper unifies local-minimum escape, multi-step decision making, and a limited physical-evaluation budget in a world-model loop. The design of imagining candidate sequences, verifying first actions, feeding back physical observations, and then refining retained designs is a natural match for non-convex lens optimization.
  2. The empirical study has a useful breadth. The authors evaluate 2,000 tasks with a common Optiland re-evaluation protocol, compare matched-budget physics-only baselines, include a planner ablation, and report open-loop H1-H8 prediction results. The reported improvements over Optiland multi-start are sizeable: feasible rate increases from 79.35% to 85.20%, SpecGAP_FA decreases from 2.83 to 1.89, and RMS decreases from 104.61 to 95.67 micrometers.
Critical weaknesses
  1. The evaluation scope is too narrow to support the broader generalization claims. The benchmark changes target specifications but does not test unseen topologies or unseen lens families, and the action space preserves topology. The paper itself acknowledges this limitation. Therefore, the current results primarily establish effectiveness on the authors' Optiland-based benchmark; they do not yet show that the method transfers to new optical structures or to other non-convex scientific-design problems.

  2. The key comparisons are not yet reproducible or fully cost-normalized. The manuscript states that controlled comparisons match high-fidelity physical calls, while LensDreamer uses additional model-side computation; DeepLens and QGSO retain their native procedures. The supplied PDF also defers the definitions of SpecGAP and SpecGAP_FA, invalid-prescription penalties, action constraints, clustering thresholds, novelty and annealing rules, DLS settings, and loss weights to an unavailable appendix. Without those details, independent verification of the ranking and attribution of gains is difficult.

Questions for authors
  1. Please provide the complete appendix or supplementary material. In particular, please specify the exact formulas and normalization for SpecGAP and SpecGAP_FA, the fixed penalty for invalid prescriptions, the feasibility threshold, the legal action space and its discrete/continuous encoding, Basin-Map clustering distance and threshold, novelty archive, annealed acceptance, triggered jump, DLS settings, loss weights, and the complete accounting of the 180 physical calls per task.

  2. Can you compare LensDreamer, Optiland multi-start, physical CEM, DeepLens, and QGSO under the same total physical calls, wall-clock time, or total compute, while reporting training cost, inference cost, actual Optiland calls, independent random seeds, and 95% confidence intervals? If the advantage persists after cost normalization, the main soundness concern would be substantially reduced.

  3. How are LensDreamer-Traj and LensDreamer-2K separated by lens family, topology, initial prescription, and specification generator? Please report stratified ID and specification-OOD results and add a held-out test on unseen lens families or topologies. If OOD tasks share the same families and topology, the current OOD evidence should be interpreted more narrowly.

9 comments
AuthorsOpenReview ↗

Response to Reviewer yubQ (R2), Part 1

Thank you for your comments. We respond point by point below and have revised the manuscript and supplementary material. Please download the latest main-paper and supplementary PDFs to view the revisions.

W1/Q3: Generalization scope and dataset separation.

We already state in Discussion and Limitations (Section 4) that evaluation covers specification-OOD tasks and the action space preserves topology. Topology is fixed within each task, while the benchmark spans different lens structures. Training covers five families and 14 topologies; ID covers five families and 12 topologies; specification-OOD covers five families and 14 topologies. All use the same Optiland prototype library. ID targets lie within training-supported ranges; specification-OOD targets exceed at least one range. Trajectories are split disjointly, but this is not a family-, topology-, or initial-prototype-held-out test.

Lens elementsTraining trajectoriesIDSpec-OODAll test
3367121931
4691234265
51,1834187128
61,75467146213
72,931116291407
82,63794352446
91,81962247309
101,22343173216
117272792119
12364155166
Total13,6965001,5002,000

The table shows structural diversity, not unseen-topology transfer. Our supported generalization result concerns unseen target specifications within the evaluated families and topologies:

SplitTasksSpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} ↓SR ↑Feasible ↑
ID5000.8570.60%94.40%
Specification-OOD1,5002.2343.60%82.13%

On specification-OOD tasks, SR exceeds physical CEM's 24.27% by 19.33 percentage points. Unseen-family/topology and cross-domain transfer remain future evaluation directions. Thank you for this suggestion. We narrowed the Abstract, clarified scope in Sections 3.1/4, and added dataset composition and stratified results in Appendices D/H.2, Tables 12–13, making the evaluation scope explicit.

AuthorsOpenReview ↗

Response to Reviewer yubQ (R2), Part 2

Thank you for your suggestions on reproducibility and cost. We clarify the submitted material and revised settings below.

W2/Q1: Reproducibility and supplementary details.

Already in the submitted supplement: The original supplement contains action encoding (D), architecture and loss formulas/weights (E), endpoint representations, novelty/annealing equations, jump rules and budgets (F), and normalized SpecGAP, failure handling, validity and paired intervals (G).

Expanded details and consolidated settings (including original values):

  • Metrics (G). SpecGAP adds normalized MTF shortfalls, relative EFL error (weight 0.75), F-number violation and RMS excess (weight 0.25). MTF weights sum to one; required conditions cover 50/100/200/300 lp/mm, task-defined fields, wavelengths and T/S directions. Reported axis/corner MTF is at 100 lp/mm. SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} assigns 10 to invalid outputs. Feasible means successful tracing/analysis with complete, finite required responses, not specification satisfaction. SR additionally requires geometric checks, 1% EFL tolerance, F-number/RMS limits and every MTF requirement; failed tasks remain in the denominator.
  • Actions (D). Each action encodes operation, surface and u∈[−1,1]u\in[-1,1], with Δ=clip⁡(γsau,−ba,ba)\Delta=\operatorname{clip}(\gamma s_a u,-b_a,b_a). Scales for radius, thickness, air gap, conic and refocus are 0.10/0.08/0.10/0.15/0.10; clipping limits are 0.75, except 2.00 for conic. Radius/spacing updates are multiplicative; conic updates are additive. Shared masks enforce legal operations; materials, stop and topology stay fixed.
  • Planning and exploration (F). Each round uses 4,096 sequences, H=8, six proposal updates, four restarts, eight clusters and eight verification slots. Endpoint vectors combine latent state, masked prescription and predicted responses, then undergo L2 normalization. Clustering uses cosine distance, deterministic farthest-first initialization, at most eight iterations and center tolerances atol=rtol=10−410^{-4}. The archive stores predicted endpoints after first-action verification; H8 states remain imagined. Center/novelty/uncertainty weights are 0.08/0.15/0.03; ensemble risk is zero. Annealing uses physical cost changes, with temperature 0.10→0.01. After three stagnant rounds, action scale increases fivefold for one round. The best verified feasible design is stored separately.
  • Refinement and budget (F.1). Bounded TRF minimizes weighted EFL/RMS residuals, not the full SpecGAP. It uses shared initial-prescription bounds, two-point Jacobians, automatic difference steps, Jacobian scaling and ftol=2×10−52\times10^{-5}, xtol=gtol=10−810^{-8}. The online cap is 180 actual calls: initial 1 + first-action verification 64 + serialization reevaluation 1 + DLS residual evaluations 102 + endpoint observations 12. Four DLS quotas are 26/26/25/25. Independent basin certification is accounted for separately and cannot improve the online returned design.
  • Learning/statistics (E/G). MTF/first-order/feasibility/prescription/auxiliary/KL loss weights are 1/0.5/0.25/0.25/0.25/0.02; the two consistency weights are 0.25 each. Paired 95% CIs use 4,000 task-level percentile bootstrap replicates.

Thank you for this suggestion. Appendices D–G now make the implementation and accounting more explicit.


W2/Q2: Computational cost and comparison scope.

Warm latency includes rendering, encoding, planning, verification, refinement and finalization. Loading, training and the separate basin audit are excluded. Each task uses one B200/16 CPU cores; reported times average per-task medians of three timing repetitions.

MethodWarm time (s) ↓SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} ↓SR ↑
Direct Optical Surrogate242.772.1546.85%
Action-Conditioned World Model258.351.9648.60%
LensDreamer (Ours)216.831.8950.35%

The full system is 10.7% faster than the surrogate. Verification plus DLS takes 164.73 s versus 226.41 s, offsetting model overhead. The common-planner world model improves quality but takes 258.35 s.

Our focus is better design quality under a limited physical budget. One 50.48-hour training run on four B200 GPUs produces the same checkpoint used on all 2,000 tasks without retraining, amortizing its cost. Mean calls are 179.84 for Optiland multi-start, 176.91 for physical CEM and 169.37 for LensDreamer, capped at 180. Physics-only comparisons match call budgets; DeepLens/QGSO use their own optimization procedures with common final reevaluation, not equal-time comparisons.

Thank you. Appendix H.3, Table 14 adds the cost breakdown and timing protocol.

AuthorsOpenReview ↗

Kind Request to Review Our Rebuttal and Updated Manuscript

Dear Reviewer yubQ,

Thank you for your time and thoughtful feedback. We have posted our point-by-point responses, including additional experiments and clarifications, and uploaded the revised manuscript and supplementary material. We would greatly appreciate it if you could review these updates when you have a chance. If any questions or concerns remain, we would be happy to provide further explanations or discuss them with you. Thank you again for your time and consideration.

Best regards, Author of submission 1572

Reviewer_w7KNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an approach for automatic lens design under a limited optical simulation budget. The method uses an action-conditioned world model to predict how a sequence of edits affects a lens prescription and its optical response. The planner groups predicted endpoints and selects sequences from different regions of the design space. It verifies the first edit of each selected sequence in Optiland and replans from the verified designs. Selected designs are then refined using damped least squares. The paper introduces a dataset of simulated design trajectories and evaluates the method on 2000 lens design tasks.

Critical strengths
  1. The paper addresses a relevant problem in automatic lens design. The result of local optimization depends on the starting prescription. The proposed planner uses learned predictions to allocate simulator calls across different regions of the design space.
  2. The approach shows significant improvements over the physics-only baselines under the same simulator-call budget.
Critical weaknesses
  1. The contribution of the world model is unclear.The method finds better lenses than the physics-only methods, but the paper does not compare with a simpler model that takes a lens prescription and predicts its optical performance. For example, Sheng et al. (2022), cited by the paper, combines a Kriging surrogate with evolutionary optimization. Tseng et al. (TOG 2021) also learns a proxy for compound-lens simulation. Comparing LensDreamer with a direct optical-performance predictor trained on the same data and used with the same search procedure, simulation budget, and local refinement would show whether the proposed world model adds value.
  2. It is unclear how often the lenses meet the required specification. Table 1 shows that LensDreamer produces more valid designs and gets closer to the targets on average. A valid design can still miss a required MTF or spot size. Reporting how many designs meet all requirements will show how often the method successfully solves the design task.
Questions for authors
  1. How does LensDreamer compare with a direct optical surrogate, following approaches such as Sheng et al. (2022) or Tseng et al. (2021), when trained on the same data and used with the same planner, simulation budget, and local refinement?
  2. What fraction of designs meet all the specifications within the stated tolerances?
5 comments
AuthorsOpenReview ↗

Response to Reviewer w7KN (R1)

Thank you for your comments. We respond point by point below and have revised the manuscript and supplementary material. Please download the latest main-paper and supplementary PDFs to view the revisions.

W1/Q1: Comparison with "a simpler model that takes a lens prescription and predicts its optical performance".

The code for Sheng et al. (2022) and Tseng et al. (2021) is not publicly available, so we could not directly compare with these methods.

Our experiments include DeepLens (Yang et al., 2024) and QGSO (Gao et al., 2025) ("a simpler model that takes a lens prescription and predicts its optical performance"), representative open-source methods that evaluate prescriptions through optical simulation. LensDreamer outperforms both on all reported final-design metrics after common Optiland reevaluation (Table 1).

Following your suggestion, we trained a Direct Optical Transformer Surrogate as an alternative.

Direct Optical Surrogate: A three-layer Transformer block repeated four times with shared weights independently predicts responses from exact action-updated prescriptions and task specifications, using bidirectional surface attention.

Action-Conditioned World Model: The LensDreamer Transformer–RSSM uses a three-layer Transformer block repeated four times per latent step, with shared weights, causal action-history attention, and recurrent stochastic rollouts.

We match the training data and schedule, all 2,000 tasks, Basin-Map Planner (H=8), ≤180 online Optiland calls, and local refinement. Selection uses the same exact-prescription/predicted-response representation, without private latent features or uncertainty scoring. LensDreamer (Ours) reports our full system with its complete planner configuration.

MethodFeasible ↑SR ↑SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} ↓Feasible SpecGAP\mathrm{SpecGAP} ↓RMS (μ\mum) ↓Axis MTF ↑Corner MTF ↑∣Distortion∣\lvert\mathrm{Distortion}\rvert ↓
Direct Optical Surrogate83.45%46.85%2.150.59109.8311.86%2.06%0.21
Action-Conditioned World Model84.75%48.60%1.960.51102.4112.49%2.23%0.17
LensDreamer (Ours)85.20%50.35%1.890.4895.6713.27%2.37%0.14

Under the common planner, the Action-Conditioned World Model reduces SpecGAPFA\mathrm{SpecGAP}_{\mathrm{FA}} by 8.8% and RMS by 6.8%, while increasing SR by 1.75 percentage points and feasibility by 1.30 percentage points over the Direct Optical Surrogate. This supports world-model response prediction under the same planner and physical budget.

Thank you for this suggestion. Section 3.4, Table 4, and Appendix F.2 now present the controlled comparison and settings.


W2/Q2: What fraction of designs meet all specifications within the stated tolerances?

We computed Success Rate (SR) from saved physical evaluations. A design is successful only if it is physically valid, passes the geometric checks, and meets all task requirements within the evaluation tolerances, including every required MTF condition, the EFL tolerance, and the F-number and RMS limits. SR includes all tasks, counting invalid designs as failures. SpecGAP\mathrm{SpecGAP} measures continuous specification deviation.

ID SR and OOD SR report success on 500 in-distribution and 1,500 specification-OOD tasks. OOD tasks have at least one target outside the training range. Successes counts successful tasks; All SR divides this count by 2,000. Feasible measures valid ray tracing and optical analysis, regardless of specification satisfaction.

MethodID SR (%) ↑OOD SR (%) ↑Successes / 2,000All SR (%) ↑Feasible (%) ↑
Optiland single-start38.4014.7341320.6572.20
Optiland multi-start43.4021.2053526.7579.35
Physical random41.0018.8048724.3577.35
Physical CEM45.8024.2759329.6576.75
DeepLens50.4021.4057328.6568.35
QGSO27.8015.0736518.2546.45
Direct Optical Surrogate69.8039.2093746.8583.45
Action-Conditioned World Model68.8041.8797248.6084.75
LensDreamer (Ours)70.6043.601,00750.3585.20

LensDreamer achieves the highest ID and specification-OOD SR, succeeding on 1,007/2,000 tasks (50.35%). Its OOD SR reaches 43.60% versus 39.20% for the Direct Optical Surrogate, a gain of 4.40 percentage points (11.2% relative). Overall SR also exceeds physical CEM by 20.70 percentage points. Thank you for this suggestion. SR is now in Table 1, with its definition and tolerances in Appendix G.

AuthorsOpenReview ↗

Kind Request to Review Our Rebuttal and Updated Manuscript

Dear Reviewer w7KN,

Thank you for your time and thoughtful feedback. We have posted our point-by-point responses, including additional experiments and clarifications, and uploaded the revised manuscript and supplementary material. We would greatly appreciate it if you could review these updates when you have a chance. If any questions or concerns remain, we would be happy to provide further explanations or discuss them with you. Thank you again for your time and consideration.

Best regards, Author of submission 1572

AuthorsOpenReview ↗

Kind Request to Review Our Rebuttal and Updated Manuscript

Dear Reviewer w7KN,

Thank you for your time and thoughtful feedback. We have posted our point-by-point responses, including additional experiments and clarifications, and uploaded the revised manuscript and supplementary material. We would greatly appreciate it if you could review these updates when you have a chance. If any questions or concerns remain, we would be happy to provide further explanations or discuss them with you. Thank you again for your time and consideration.

Best regards, Author of submission 1572

Comments on the paper

3 comments
AuthorsOpenReview ↗

Comments by Authors

Dear Area Chair,

Thank you for overseeing the review process. We have carefully addressed the reviewers' concerns through extensive additional experiments, statistical analyses, and a revised manuscript. However, we have not yet received further feedback from the reviewers.

As the discussion period will soon come to an end, we would greatly appreciate it if you could kindly encourage the reviewers to revisit our responses and share any remaining concerns.

We are fully committed to addressing unresolved questions and conducting further experiments if necessary. Early feedback would allow us to make the most of the remaining discussion period and provide any additional evidence that may help the reviewers reach a well-informed assessment.

Thank you very much for your time and consideration.

Best, Authors

AuthorsOpenReview ↗

Clarification of Responses to Reviewer k4En’s Concerns

Dear Area Chair,

Following our updated summary, we would like to clarify the status of the only remaining rating of 2, from Reviewer k4En. We believe the review's concerns have been addressed through the added controlled experiments and detailed reproducibility material.

1. The major methodological questions by Reviewer k4En overlap with those of reviewers who raised their ratings.

  • Exact prescription updates plus a direct optical surrogate. k4En, w7KN, and gs9v all requested this comparison. Our response to each explains the same experiment under matched data, planner, refinement, and physical budgets, using a common selection representation. The world model reduces SpecGAPFA 2.15→1.96 and increases SR 46.85%→48.60%. Section 3.4, Table 4; Appendix F.2.
  • Separating horizon length from basin allocation. k4En and gs9v raised this confound. We added H=1/2/4/8 with the full Basin-Map Planner fixed, including basin-aware selection at H=1. SR increases 40.85%→50.35% and SpecGAPFA decreases 2.42→1.89. Section 3.4, Table 5; Appendix F.2.

These answers were supplied directly to k4En and in our replies to the other reviewers. w7KN confirmed that the surrogate comparison and success-rate results address their main concerns. gs9v explicitly confirmed resolution of both the surrogate and horizon questions. Both raised their ratings to 3. yubQ has now also raised the rating to 3, stating that most concerns were resolved. Thus, all three other reviewers have positively reassessed the paper after the rebuttal.

2. The numerical and reproducibility questions also received specific answers.

For k4En's endpoint-grouping and refinement questions, Appendices F.1/G specify grouping rules, normalized distances, numerical tolerances, and online/audit stopping criteria. The prescription tolerance is 10^-3; stability checks additionally require a projected-gradient infinity norm of at most 10^-4, checked with difference steps of 10^-6 and 10^-7. Audit calls are recorded separately and do not improve the online returned design.

Appendix H.1, Tables 10–11, regroups fixed audited endpoints over 0.5×–2.0× tolerances while keeping audit refinement and certification fixed. Full Basin-Map retains higher endpoint-class counts and anchor escape throughout; its gain remains 0.16–0.25 classes/task after matching endpoint counts. These are explicit numerical criteria and sensitivity results, beyond a verbal assurance of reproducibility.

3. The unchanged rating has not been accompanied by a remaining technical objection.

k4En's follow-up on 17 November states only:

“Thank you for the author's reply. I'll keep my score.”

We subsequently asked which original concerns remained unresolved and offered to provide further clarification. At the time of writing, no specific outstanding issue has been identified in response. We therefore cannot determine what further evidence the reviewer considers necessary. We respectfully ask the AC to evaluate the concern-to-evidence mapping above when interpreting this unchanged rating.

We have worked carefully to substantiate this initial application of action-conditioned world-model planning to lens design. The supplementary material provides extensive numerical results and implementation settings, and we will release code, configurations, checkpoints, datasets, and reproduction scripts upon acceptance, enabling others to inspect, reproduce, and extend the work. This release complements the concrete reproducibility details already supplied in the paper.

We sincerely hope the final assessment will recognize the strengthened evidence, the positive reassessments by three reviewers, and the contribution of this work within its stated scope.

Finally, we sincerely thank the AC for the time, effort, and careful consideration devoted to our submission. The research community advances through the generous service of people like you, and we deeply appreciate your contribution.

Best regards,

Authors

AuthorsOpenReview ↗

Summary of Author-Reviewer Discussion Period for the Area Chair

Dear Area Chair,

Thank you for overseeing the review of LensDreamer. Reviewers w7KN, gs9v, and yubQ raised their ratings from 2 to 3 after the rebuttal. We summarize the revisions and remaining unchanged rating.

1. Shared concerns and our responses

  • Exact updates plus a direct surrogate (w7KN, gs9v, k4En). We added a nonlinear optical surrogate using exact prescription updates. Both predictors share training data/schedule, planner, local refinement, the 180-call cap, and all 2,000 tasks. Selection uses a common representation without private latent features or uncertainty scoring. The controlled world model reduces SpecGAPFA 2.15→1.96 (8.8%) and increases SR 46.85%→48.60%. Section 3.4, Table 4; Appendix F.2.

  • Horizon versus basin allocation (gs9v, k4En). We added H=1/2/4/8 runs with the full Basin-Map Planner and other settings fixed. H=1 retains basin-aware selection. From H=1 to H=8, SpecGAPFA decreases 2.42→1.89, SR increases 40.85%→50.35%, and certified endpoint-class counts increase 1.27→1.63. Section 3.4, Table 5; Appendix F.2.

  • All-specification success (w7KN). We added SR requiring physical validity, geometric checks, and every task requirement to pass, counting invalid outputs as failures. LensDreamer succeeds on 1,007/2,000 tasks (50.35%), with 70.60% ID SR and 43.60% specification-OOD SR. Table 1; Appendix G; H.2, Table 13.

  • Grouping criteria and reproducibility (k4En, yubQ). We specify numerical grouping tolerances, refinement stopping rules, stability checks, model/action/planner settings, and actual-call accounting. The independent certification audit is separate from online optimization. Across 0.5×–2.0× grouping tolerances, the coverage ordering is preserved; matching endpoint counts leaves a gain of 0.16–0.25 classes/task over scalar top-k/CEM. Appendices D–G; H.1, Tables 10–11.

  • Scope and cost (yubQ, gs9v). The submitted Section 4 already limits evaluation to specification-OOD within the evaluated families/topologies and distinguishes physical-budget allocation from reducing total compute. The revised first-page Abstract and Sections 3.1/4, with H.2, Tables 12–13, make this explicit. Our results support generalization across target specifications within those structures, not transfer to unseen lens families or different, unseen topologies. H.3, Table 14, reports learned-system online timings (216.83 s for LensDreamer versus 242.77 s for the direct surrogate) and separately discloses training of 50.48 hours on four B200 GPUs, with one checkpoint reused across all tasks. We claim design-quality gains under a physical-call cap, not lower total computation or a demonstrated runtime advantage over physics-only baselines.

2. Updated reviewer feedback

  • w7KN confirmed that the direct-surrogate comparison and success-rate results “address my main concerns” and raised the rating to weak accept.
  • gs9v confirmed that the surrogate and horizon experiments address the main methodological concerns, and raised the rating from 2 to 3, while noting structural transfer and offline cost as limitations.
  • yubQ, after further clarification, stated: “The author has solved most of my concerns, so I raise the score to 3.”
  • k4En remains at 2. The only follow-up explanation is “Thank you for the author's reply. I'll keep my score.” We asked which concerns remain unresolved, but have not received a further explanation at the time of writing.
  • Overall, three of the four reviewers have raised their ratings from 2 to 3 following the rebuttal, bringing the overall scores from 2/2/2/2 to 3/3/3/2.

We believe the added evidence addresses k4En's methodological and reproducibility concerns. The principal methodological questions overlap with those explicitly resolved for w7KN and gs9v; the numerical and reproducibility questions received detailed answers as well. We provide a separate note mapping this remaining review to the revisions, so that the unchanged rating can be assessed alongside the new evidence.

To our knowledge, LensDreamer introduces the first optical world model for this lens-design task. We believe its learned planning approach, trajectory dataset, and 2,000-task benchmark offer a useful foundation for further research. Extensive numerical results and implementation details are provided in the supplementary material. Upon acceptance, we will release code, configurations, checkpoints, datasets, and evaluation scripts to enable independent reproduction and further development. Thank you for considering the revised evidence and the discussion as a whole.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.