acceptodds
Under review as a conference paper at ICLR 2027

LensRSI: Towards Recursive Self-Improving Optical Design Agents

Abstract

Automatic lens design seeks to construct and optimize optical systems with minimal human intervention. Although numerical optimization and physics-based simulation have automated important parts of this process, agentic systems that can autonomously manage complete optical design workflows remain underexplored, and enabling such systems to improve their own design policies poses a further challenge. We study recursive policy improvement for optical design agents and introduce LensRSI, a verifier-gated framework that converts simulator-measured state–action–outcome trajectories into validated, inheritable search policies. LensRSI couples a task-solving loop with a policy-improvement loop: the agent interacts with a physics-based optical simulator, learns which actions work in different optical states, and proposes an updated design policy. To keep this process reliable, every candidate policy is evaluated independently and inherited only when it improves design quality while preserving held-out performance and physical validity. All policy comparisons restart from the same fixed initial prescriptions, so only verified search knowledge, rather than optimized lenses, is carried across rounds. Experiments on LensArenaEx, our benchmark of extreme lens-design tasks, show that LensRSI achieves smaller root-mean-square spot radii and better generalization to challenging specifications than non-evolving and limited-update alternatives under matched simulation budgets, while maintaining optical validity. Results demonstrate the potential of recursive self-improvement (RSI) to build optical design agents and we hope to inspire further research into RSI for scientific discovery. Codes and datasets will be released on acceptance.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
optical lens design, agentic systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MpHqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LensRSI, which builds on LensDesigner and couples a task-level design loop (prescription retrieval, typed lens edits, simulator feedback) with a cross-task loop that distills simulator-measured trajectories into a candidate policy, consisting of a state-conditioned action-score table and allowed search controls. A separate verifier, LensHarness, re-runs the parent and candidate from identical initial prescriptions under matched simulator budgets and promotes the candidate only if it improves without degrading validation performance or physical validity. Experiments on the LensArena and LensArenaEx benchmarks demonstrate the effectiveness of the proposed method.

Critical strengths
  • This paper studies an interesting method for optical design via a self-improving agent

  • The paper introduces LensArenaEx, a benchmark source to support this domain.

  • Experiments demonstrate consistent gains across LLM backbones.

Critical weaknesses
  • The most confusing point for me is that the recursive self-improvement mechanism is not formally defined in the setting of this paper. Even Eq.3 writes ak=πt(sk,x)a_k = \pi_t(s_k, x), but πt\pi_t is only a score table with control values. So the authors never seem to specify how the scores become actions or what the LLM does inside G.

  • The promotion rule can accept overfit candidates. δdev\delta_{dev} is measured on the same development tasks used to build the candidate, the same LensArena tasks are reused every round without multiple-testing correction, and the ϵval\epsilon_{val} tolerance lets small regressions accumulate across rounds. It would be better to provide clarification on that.

  • The policy class is too coarse for lens design. Edits cover only radius, spacing, and conic; surfaces are pooled into front/middle/rear, and the function Q ignores the specification x, so one score is shared across wide-angle and narrow-field tasks.

  • Some figures are too small to read. And Fig. 2 contains typos (e.g., 'Postive' should be 'Positive').

Questions for authors

Please see my Weaknesses section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer MpHq - Part 1

Thank you for your comments and for recognizing the value of optical policy improvement and the consistent results across foundation models. We respond point by point below.

W1: How scores become actions and what the LLM does inside G.

The notation πt=(Mt,Ct)\pi_t=(M_t,C_t) identifies the inherited policy state. A fixed executor reads this state to generate concrete actions. Eq. 3 abbreviates this execution process, rather than treating a score table as a standalone action generator. An explicit way to write it is

ak=F(sk,x;Mt,Ct),a_k = F(s_k,x;M_t,C_t),

where FF is the fixed task-level executor. It constructs candidate edits with specific surface indices, parameter values, directions, and step sizes. The memory resolves action preferences using the current optical state and available task context. Higher-scoring edits receive priority, while the execution schedule retains baseline search coverage. These are priorities over measured candidates, not predictions that replace the simulator.

The LLM has two distinct roles. During task execution, it proposes allowed optimizer settings from the specification and measured state, with inherited controls and routes affecting the executable program. During policy improvement, it proposes allow-listed program revisions from Development evidence. The memory update and proposed program controls form an isolated candidate π~t+1\widetilde{\pi}_{t+1}. LensHarness decides whether that candidate becomes active.

The supplementary section on State-Conditioned Cross-Task Policy Update already explains action reordering and the use of scores as planner context. Its execution examples show a proposed program followed by an inherited state-route override. These details specify the execution path behind πt\pi_t and GG.

W2: Development overfitting, repeated promotion, and tolerance accumulation.

Development improvement is a learning-side requirement, not the claim of generalization. Candidate generation uses Development trajectories, and promotion also requires comparison on a fixed 60-task stratified subset of LensArena. These tasks do not overlap with Development. The final policy is then tested on LensArenaEx, whose results never influence learning or promotion. Full LensArena results include the promotion subset and are not presented as held-out test evidence.

Reusing promotion tasks can lead to adaptive selection effects. The gate uses fixed empirical thresholds rather than a sequence of statistical significance tests, so it should not be interpreted as a hypothesis-testing guarantee. The evidence for transfer comes from the separate LensArenaEx evaluation.

The tolerance is a local acceptance margin, not a guarantee of monotonic validation improvement from the initial policy. Small permitted regressions could accumulate in principle. The empirical effect of verification is tested in Table 2: on LensArenaEx, removing the gate changes Avg. RMS from 184.4 to 252.3 μm and Success from 58.7% to 46.0%. These results support independent verification before inheritance, without treating the gate as a universal guarantee against overfitting.

AuthorsOpenReview ↗

Response to Reviewer MpHq - Part 2

W3: Coarse action categories and dependence on specification x.

The front/middle/rear categories are memory-sharing keys, not the resolution of physical edits. Every executed edit still targets a concrete surface and parameter. Relative regions allow experience to transfer between prescriptions with different surface counts, rather than requiring every surface index to have an independent history.

The full decision process also depends on the task specification. The memory resolver receives both the targets and current measurements. Its task-context preferences distinguish lens-element counts and FoV regimes, while the compact state includes the RMS-to-target ratio, MTF margin, search phase, and validity. The planner receives the specification as well. The execution therefore does not apply a single undifferentiated preference to every wide-angle and narrow-field task.

The local table Qt(ξ,z)Q_t(\xi,z) is a compact component of this process, not a representation of every physical variable in xx. Task-context preferences and state-conditioned entries are combined with fallback rules, so we do not claim a separate full table for every specification. The representation’s utility is tested by the State-blind control: on LensArenaEx, Avg. RMS improves from 336.2 to 184.4 μm and Success from 38.0% to 58.7% with the full method. These results support useful state-dependent transfer despite the compact representation.

W4: Figure readability and spelling.

Thank you for pointing this out. We have replaced Fig. 2 in the rebuttal manuscript with the corrected framework PDF. We will also improve figure readability and correct the remaining typographical issues.

Questions for Authors: “Please see my Weaknesses section.”

The four responses above address the questions in the same order: policy execution, promotion and overfitting, policy granularity and specification dependence, and figure presentation.

AuthorsOpenReview ↗

Kind Request to Review Our Rebuttal and Updated Manuscript

Dear Reviewer MpHq,

Thank you for your time and thoughtful feedback. We have posted our point-by-point responses, including additional experiments and clarifications, and uploaded the revised manuscript and supplementary material. We would greatly appreciate it if you could review these updates when you have a chance. If any questions or concerns remain, we would be happy to provide further explanations or discuss them with you. Thank you again for your time and consideration.

Best regards, Author of Submission #10190

Reviewer_rxqEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores how an optical lens design agent can autonomously manage the entire optical design workflow using a recursive strategy. The manuscript presents LensRIS, a lens design agent framework that implements a recursive strategy. Built upon LensDesigner, the approach transforms objects inherited across rounds from the prescriptive solution of a single task into cross-task, validated, and inheritable search strategies. During each recursive iteration, the parent strategy generates a set of simulator-grounded trajectories for the development task. The system aggregates these trajectories into search preferences under specific state conditions, thereby constructing a candidate strategy. Each candidate strategy is independently evaluated by LensHarness and is inherited as the parent strategy for the next round only if it improves design quality while maintaining generalization performance and physical validity. The final strategy is evaluated on LensArenaEx, which has never participated in the recursion. The experiment reports that LensRSI outperforms QGSO, DeepLens, and LensDesigner in terms of average RMS and success rates on both LensArena and LensArenaEx.

Critical strengths
  1. This paper solves automated lens design problem through large language model agents and recursive policy improvement. The manuscript reframes the improvement of lens design through agents from “prescription optimization within a task” to explicit recursive policy, and introduces LensHarness as an independent execution and verification environment. The attempt to ground the verifiability of the effectiveness of recursive self-improving in a concrete scientific task and to introduce objective, reproducible simulator feedback signals is itself valuable.
  2. The manuscript proposes the LensArenaEx benchmark building upon LensArena. The specifications of the proposed LensArenaEx benchmark are indeed more extreme than those of the conventional LensArena benchmark, and the distribution in Figure 3 and the layout comparison in Figure 4 also indicate the difficulty of the tasks. If the code and data are released as promised in the reproducibility statement, this benchmark can serve as a comparison platform for subsequent work on automated lens design.
Critical weaknesses
  1. The core method does not demonstrate innovation in representation learning. The effectiveness of the recursive strategy will largely depend on running more rounds or selecting from multiple candidate solutions; however, this paper does not distinguish the recursive strategy from running multiple times with the same strategy. Equation (9) is an empirically weighted average of tabular Q-values, Equation (5) is an accept/reject style recursion, and Equation (8) is a standard discounted return. These mechanisms are routine operations in bandits and evolutionary strategies. What is noteworthy is that the paper does not provide an ablation experiment comparing a fixed strategy run repeatedly for N rounds without recursion. The "Non-evolving" and "One-step" settings in Table 2 are not multiple executions of the same strategy but controls without updating the Q-table. Figure 5(a) shows that RMS decreases as tasks used increases, but this only indicates that using more tasks is better; it does not show that the recursive update itself brings more gain than "running more". If the update in Equation (9) were removed and only the same initial strategy were executed repeatedly, how much would performance drop? The absence of this experimental data leaves the core claim that "RSI is effective" in doubt.
  2. The experiments lack an important ablation study on the influence of the initialization strategy. The manuscript uses RAG to retrieve starting forms from a mixed pool of classical designs, curated patents, online searches, and ab initio optimization (Section 3.2), but Table 2 does not include a comparison of different retrieval pools or different initialization quality. If the performance improvement mainly comes from better seeds, then the contribution of policy improvement is overestimated.
  3. The experimental data for LensDesigner differ from the original paper. In Table 1, LensDesigner + Gemini-3.1-pro-preview performs worse than the original LensDesigner + Gemini-3.0-flash. The paper does not explain this discrepancy, nor does it state whether the baseline was reproduced under a unified implementation, a unified budget, and a unified evaluation script. If the baseline was not fairly reproduced, the comparison in Table 1 cannot support the claim that LensRSI outperforms LensDesigner.
  4. Several important experimental implementation details are missing from the manuscript. The horizontal axes in Figure 5 lack numerical values, making it impossible to judge the number of recursive rounds, the number of tasks per round, and the order of magnitude of the total budget. In addition, the prompt templates are not provided, yet this is an LLM-agent work in which prompts may have a considerable impact on the results.
Questions for authors
  1. Necessity of recursive self‑improvement vs. repeated execution The core mechanisms (e.g., Equations 5, 8, 9) appear to be standard bandit/evolutionary operations, and it remains unclear whether the performance gains stem from the recursive update itself or merely from running more trials/selecting from multiple candidates. Could you provide an ablation study where Equation (9) is removed and the same initial strategy is executed repeatedly for N rounds without recursion?

  2. Influence of initialization strategy The manuscript uses RAG to retrieve starting forms from a mixed pool (classical designs, patents, online searches, ab initio optimization), yet Table 2 lacks comparisons of different retrieval pools or initialization quality. If performance improvements mainly come from better seeds rather than policy improvement, the contribution of recursive updating is overestimated. Could you add an ablation study that varies the initialization source to disentangle the gains from initialization versus recursive policy updates?

  3. Fair baseline reproduction and discrepancy with original LensDesigner results Table 1 shows LensDesigner + Gemini‑3.1‑pro‑preview performing worse than the original LensDesigner + Gemini‑3.0‑flash, and the paper does not explain this discrepancy. Could you clarify whether all baselines were reproduced under a unified implementation, unified budget, and unified evaluation script?

  4. Missing experimental details The horizontal axes in Figure 5 lack numerical values, making it impossible to judge the number of recursive rounds, tasks per round, and the order of magnitude of the total budget. Prompt templates are not provided.

3 comments
AuthorsOpenReview ↗

Response to Reviewer rxqE - Part 1

Thank you for your comments and for recognizing the value of verifiable policy inheritance and the LensArenaEx benchmark. We respond point by point below.

W1/Q1: Recursive updates versus repeated execution.

We do not claim that accept/reject selection, discounted returns, or tabular averaging are individually new algorithms. The contribution is their use in an executable optical policy that learns from measured design trajectories and is independently verified before inheritance. The recursive dependency is that an accepted policy generates the experience used to construct the next policy.

The existing evaluation separates this from simply continuing lens optimization. Policy comparisons restart from fixed initial prescriptions. No optimized prescription is carried into the next round. Final evaluation uses a frozen policy with a fixed per-task budget, so its results do not accumulate optimization steps from previous RSI rounds.

Table 2 provides two related controls that retain the task-solving setting while restricting policy improvement:

Policy variantLensArenaEx Avg. RMS (μm) ↓LensArenaEx Success (%) ↑
Non-evolving521.429.3
One-step439.744.0
LensRSI (Full)184.458.7

Full outperforms both controls under the reported protocol. The Random update control also performs worse than Full, at 364.9 μm Avg. RMS and 33.3% Success. These comparisons support constructing policies from measured experience rather than relying on an unchanged policy or feedback-blind changes.

These controls address the value of policy updating at the reported evaluation budget. We do not present them as an exhaustive comparison against every best-of-NN allocation of additional computation.

W2/Q2: Initialization quality versus policy improvement.

The initialization does not improve across policy rounds. For each task, the starting prescriptions are fixed before parent–candidate comparisons. Both policies receive the same starting prescriptions, so a candidate cannot obtain an advantage by retrieving a better seed. The matched-initialization ablations test policy changes within the same task-solving setting.

Table 2’s Fixed LensDesigner Skill control further retains our task-solving pipeline but uses fixed LensDesigner skills without recursive policy improvement. On LensArenaEx, it obtains 285.8 μm Avg. RMS and 40.7% Success, compared with 184.4 μm and 58.7% for Full. This result supports a contribution beyond supplying the pipeline with existing skills.

Initialization can affect absolute performance, but it is held fixed in these policy comparisons. The gains therefore cannot be explained solely by changing seed quality during improvement. Our claim concerns policy improvement under the stated initialization protocol, rather than independence from every possible initialization source.

W3/Q3: LensDesigner reproduction and differences from its original results.

The LensDesigner numbers in Table 1 come from new runs, not copied results from its original paper. The original LensDesigner used Zemax for local optimization and optical evaluation. We reran LensDesigner using the open-source Optiland environment and computed its results with the evaluation procedure used in this paper.

The numerical results therefore come from different optimization and evaluation environments and are not expected to reproduce the original table entry by entry. The comparison in our Table 1 uses the reproduced Optiland results rather than mixing original Zemax numbers with newly measured LensRSI numbers. Our stated protocol also matches the per-task physical-evaluation budget.

The Fixed LensDesigner Skill ablation provides a separate comparison within our task-solving pipeline, keeping the execution setting while replacing the learned policy knowledge with fixed prior skills. It supports the value of the policy-improvement component alongside the full-system baseline comparison.

AuthorsOpenReview ↗

Response to Reviewer rxqE - Part 2

W4/Q4: Round counts, total budget, and prompts.

The supplementary material describes the task-level optimizer schedule, the 64-call accounting rule, and the data-access restrictions. It also includes concrete system/user prompt excerpts, allowed program changes, and an execution example showing how an inherited route modifies the proposed optimizer program. Prompt information is therefore present, although the manuscript could point to it more clearly.

The actual run used three RSI rounds, with one successful promotion. Each round evaluated the same 120 Development tasks and the same 60 Promotion tasks, restarting from fixed initial prescriptions. Promotion tasks were stratified from LensArena's 120 tasks, rather than using the full benchmark for each gate comparison.

RSI roundDevelopment tasksPromotion tasksDecision
112060Accept candidate policy
212060Retain current policy
312060Retain current policy

The table reports attempted policy-improvement rounds, while the 64-call allowance is the budget of an individual task run. Saved policies are evaluated on LensArenaEx retrospectively, and these results do not determine the stopping round.

AuthorsOpenReview ↗

Kind Request to Review Our Rebuttal and Updated Manuscript

Dear Reviewer rxqE,

Thank you for your time and thoughtful feedback. We have posted our point-by-point responses, including additional experiments and clarifications, and uploaded the revised manuscript and supplementary material. We would greatly appreciate it if you could review these updates when you have a chance. If any questions or concerns remain, we would be happy to provide further explanations or discuss them with you. Thank you again for your time and consideration.

Best regards, Author of Submission #10190

Reviewer_SZFKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

This paper introduces LensRSI, a framework that involves LLM's in the lens design workflow using a new LensHarness. During training, there is an inner and outer loop. In the inner loop, the LensHarness tries to complete a single lens design task by setting optimizer schedules and making active changes to candidates. In the outer loop, a policy is learnt that includes memory (a collection of (state, action, outcome) tuples) and a program ("named routes" which can override local optimizer settings). Finally, the policy is frozen and evaluated on a new LensArenaEx dataset with 150 lens design tasks, where it outperforms three competitors while being robust against swapping out the LLM.

Critical strengths

Agentic design with a good feedback mechanism is a relevant direction for current research, including optical design workflows.

Critical weaknesses
  1. This paper builds on another paper called "LensDesigner. A Self-Improving Agent for Optical Lens Design" by Sun et al.. Starting from the introduction, this paper assumes a deep understanding of LensDesigner (i.e., how an agent is able to steer a lens design task). I think it would be helpful for the paper to be more self-contained.
  2. The evaluation only targets LensDesigner's LensArena (a private dataset) and the new LensArenaEx dataset. While the author's do compare against two competing methods, a more traditional global+local approach which they mention in the related work section is missing. Additionally, it is unclear what simulation-budgets were used for the competitors. I would expect simulations calls to be cheap, so only the per-task budget of 64 would be quite restrictive.

There is no split between validation and test set, meaning the method might be over-fitted to LensArenaEx (or, more generally, the dataset-generation pipeline used for both).

The "recursive self improvement" (RSI in LensRSI) term seems misleading, as no model parameters are changing, only LLM context. The authors themselves refer to it mostly as "recursive policy improvement" which appears more accurate to me.

Questions for authors
  1. What exact parameter choices separate LensArenaEx from LensArena, and where do the development tasks come from? (i.e., how similar are they) In general, some statistics on the datasets would be appreciated.
  2. What are the time and cost budgets? How long do 64 OptiLand simulations (i.e., your budget) take, and how much time/tokens does the LLM involvement take?
  3. In the abstract, you mention "numerical optimization and physics-based simulation have automated important parts of this process". How strong are these with larger budgets compared to your agentic approach? As your approach seems to rely on particle swarm + damped least squares internally, how far is it from an expert-designed optimized baseline with a much larger budget? (1000s of calls, this question is only relevant when simulations are sufficiently cheap, which they should be)
  4. Since LLM's are probabilistic, what is the variance between evaluations on the same design task?
5 comments
AuthorsOpenReview ↗

Response to Reviewer SZFK - Part 1

Thank you for your comments and for recognizing the relevance of agentic optical design with physical feedback. We respond point by point below.

W1: LensDesigner’s publication status and the relationship between the papers.

We would like to correct a factual misunderstanding: LensDesigner had been accepted to NeurIPS 2026 before our full-paper submission and is a peer-reviewed work. The project website linked in our reference already states this acceptance: LensDesigner. The cited work is LensDesigner: A Self-Improving Agent for Optical Lens Design, Sun et al., 2026. It also appears in the NeurIPS 2026 official listing.

Our paper explicitly identifies LensDesigner as prior work and builds on its task-solving setting. The contribution concerns constructing, independently validating, and inheriting a search policy across tasks. A public research system can serve as the basis for subsequent work regardless of which group conducts that work. We respectfully ask that the submission be assessed on its technical content rather than inferred author identities.

W1(1): Whether the paper requires prior understanding of LensDesigner.

The method description is self-contained and does not require reading LensDesigner first. Sec. 3.1 defines the task, prescription, policy, and recursive update. Sec. 3.2 describes the complete workflow from natural-language specifications and initialization through lens edits, simulation feedback, and policy proposals. Secs. 3.3–3.4 define the action scores, candidate construction, and verification rules. The supplementary material provides execution details and worked examples. The LensDesigner citation establishes the relationship to prior work rather than supplying missing definitions needed to understand LensRSI.

W1(2): Benchmark coverage, optimization baselines, and simulation budgets.

Our experiments include QGSO, an optimization-based quasi-global lens search method, alongside DeepLens and the agentic LensDesigner baseline. These cover optimization-based, learning-based, and agentic approaches to automatic lens design.

Table 2 also controls for the task-solving machinery. The Non-evolving variant retains the task-level optimization workflow but removes policy updates across tasks. On LensArenaEx, it obtains 521.4 μm Avg. RMS and 29.3% Success, compared with 184.4 μm and 58.7% for the full method. This comparison supports a contribution from policy improvement beyond the numerical optimization components themselves.

Promotion uses a fixed, stratified subset of 60 tasks from the 120-task LensArena benchmark. This subset covers different optical design conditions while reducing evaluation cost in each RSI round. LensArenaEx is a separate set of 150 extreme design tasks used after the policy is frozen.

The reported comparison uses a per-task physical-evaluation budget of 64 calls, rather than 64 unrestricted optimizer invocations. Candidate validation and internal optimizer objective/residual evaluations consume this budget. Parent–candidate comparisons use matched budgets and fixed initial prescriptions, controlling the optical feedback available to each policy.

Matching physical evaluations controls the amount of optical feedback available to each method. It does not assume that every simulation is expensive or that equal simulation counts imply equal wall-clock or API costs. We address time and token costs separately in Question 2.

W2: Validation/test separation and possible overfitting to LensArenaEx.

The protocol separates policy development, promotion validation, and final testing. Each RSI round uses the same 120 Development tasks and the same 60 Promotion tasks. Development tasks are independently constructed from the retrieval library and do not overlap with Promotion tasks. Only active-parent Development trajectories update policy memory. The Promotion set is a stratified subset of LensArena, selected to cover different optical design conditions while reducing the cost of each promotion comparison. Its outcomes determine acceptance or rejection, while its trajectories never enter policy memory. LensArenaEx does not participate in candidate generation, promotion, or the choice of the number of rounds.

Saved policies are evaluated on LensArenaEx retrospectively, after the improvement procedure has finished. These measurements are not fed back into the procedure. LensArena results include policy-selection tasks, while LensArenaEx provides the held-out evaluation.

Repeated use of a validation benchmark and direct access to a test benchmark are different issues. The full 120-task LensArena results include the 60 tasks used for promotion, so we do not present that aggregate as an untouched test result. The independent LensArenaEx results are the evidence for transfer beyond the tasks used in learning and policy selection.

AuthorsOpenReview ↗

Response to Reviewer SZFK - Part 2

W3: “Recursive self-improvement” versus changes to model weights.

Thank you for suggesting a more precise description. The foundation-model weights remain fixed. The inherited objects are persistent state-conditioned action preferences and permitted optimizer-program controls. These affect execution: the memory reorders candidate edits, and the program controls alter allowed search settings.

We will use recursive policy improvement to make the scope explicit. The recursive step is that an accepted policy generates the experience used to propose its successor. This is a system-policy update, rather than foundation-model training or merely appending reflection text to a prompt.

Q1: Dataset specifications, Development sources, and similarity.

The two benchmarks contain 120 LensArena tasks and 150 LensArenaEx tasks. The 120 Development tasks are constructed independently from feasible designs in the frozen retrieval library, with small perturbations to FoV, f-number, and EFL within its parameter coverage. They are distinct from the 60 stratified LensArena tasks used for promotion. These Development and Promotion sets remain fixed across RSI rounds.

LensArenaEx contains parameter-extrapolation tasks. Each task goes beyond the observed range of the frozen retrieval library in at least one of FoV, f-number, or EFL. We checked task IDs and FoV–f-number–EFL combinations and found no identical tasks across Development, LensArena, and LensArenaEx. Development and LensArenaEx do reference the same frozen library during construction, so this is task-level separation rather than a claim of unrelated data provenance.

The tables below report task counts for the full benchmarks, not just the Promotion subset. Each interval includes its lower boundary and excludes its upper boundary, except the final interval, which includes both boundaries.

FoV (°)LensArenaLensArenaEx
[0, 20)429
[20, 40)1417
[40, 60)2018
[60, 80)1414
[80, 100)1412
[100, 120)1220
[120, 140)740
[140, 160)270
[160, 180]80
Total120150
f-number (f/#)LensArenaLensArenaEx
[0, 1)80
[1, 2)4025
[2, 3)2117
[3, 4)1522
[4, 5)1522
[5, 6)1814
[6, 7)340
[7, 8]010
Total120150

These distributions show different coverage rather than a uniform shift toward larger FoV or faster apertures. For example, LensArenaEx contains 40 tasks in the 120°–140° FoV bin, compared with 7 in LensArena, while LensArena also includes tasks above 140°. The difficulty of a lens-design task depends on the combination of specifications, not an individual FoV or f-number bin alone. The tables describe benchmark composition, while the separate learning, promotion, and testing roles are explained in W2 above.

Q2: Wall-clock time, tokens, and cost.

Timing was measured over five evaluations of all 150 LensArenaEx tasks, using Gemini 3.1 Pro Preview and an Intel Xeon Platinum 8568Y+ CPU for optical computation. For a task run with 64 Optiland evaluations, the measured averages are:

QuantityMean per task
Simulation and optimization time5.23 s
LLM request time41.16 s
End-to-end time46.46 s
LLM requests2
Total API token usage14,828.6 tokens

Across the repeated evaluations, end-to-end time was 46.5 ± 10.8 seconds per task (mean ± standard deviation).

LLM requests dominate the measured task time, while simulation and optimization together take 5.23 seconds. We therefore do not motivate the fixed physical-evaluation budget by claiming that optical simulation alone dominates runtime. The budget controls the amount of physical feedback available to each method. These per-task measurements are distinct from the total offline cost of policy improvement.

Q3: Comparison with expert numerical optimization using thousands of calls.

PSO and DLS are important components of our task-level execution. LensRSI studies how experience improves the policy that organizes these numerical tools. Table 2’s Non-evolving control retains the task-solving workflow while removing cross-task policy updates. Under the same budget, its LensArenaEx Avg. RMS and Success are 521.4 μm and 29.3%, compared with 184.4 μm and 58.7% for the full method. These results support a benefit from policy improvement on top of numerical optimization.

The matched simulation budget controls computation and does not rely on simulations being expensive. Our results do not quantify the gap to an expert optimizer given thousands of calls. The supported conclusion is that the learned policy obtains better designs under the reported equal-budget comparison, rather than that it dominates numerical optimization at every possible budget.

AuthorsOpenReview ↗

Response to Reviewer SZFK - Part 3

Q4: Variability on repeated evaluations of the same task.

We evaluated each of the 150 LensArenaEx tasks five times using Gemini 3.1 Pro, for 750 task evaluations. The initial prescription, frozen policy, search seed, and per-task evaluation budget were held fixed. Only the LLM responses were independently resampled, isolating their effect under these execution conditions.

Foundation modelTasksRepetitions per taskFinal Avg. RMS (μm) ↓Success (%) ↑
Gemini 3.1 Pro1505363.70 ± 3.2746.00 ± 1.05

Values are the mean and sample standard deviation across five complete benchmark evaluations. The Success standard deviation is expressed in percentage points. Final Avg. RMS measures the optimized endpoint, not the reduction from the initial prescription. These measurements quantify variation due to resampling LLM responses while the policy and optical-search conditions remain fixed.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Comments by Authors

Dear Area Chair,

Thank you for overseeing the review process. We have carefully addressed the reviewers' concerns through extensive additional experiments, statistical analyses, and a revised manuscript. However, we have not yet received further feedback from the reviewers.

As the discussion period will soon come to an end, we would greatly appreciate it if you could kindly encourage the reviewers to revisit our responses and share any remaining concerns.

We are fully committed to addressing unresolved questions and conducting further experiments if necessary. Early feedback would allow us to make the most of the remaining discussion period and provide any additional evidence that may help the reviewers reach a well-informed assessment.

Thank you very much for your time and consideration.

Best, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.