acceptodds
Under review as a conference paper at ICLR 2027

Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization

Abstract

Large language models (LLMs) are increasingly used to formulate optimization models from natural-language problem descriptions, yet realistic operations research (OR) requests are often incomplete: missing objectives, constraints, or business rules can change the resulting mathematical program. Existing evaluations largely assume a complete specification and therefore overlook whether an agent knows when clarification is needed before modeling. We introduce OR-Clarify, a benchmark for pre-formulation clarification. Each task presents a partial public problem description, withholds structured hidden slots, and evaluates agents through bounded interaction with a simulated user. The benchmark supports both open-ended and choice-based clarification, and measures slot recovery, stopping behavior, silent assumptions, and interaction cost. We further propose Interactive Optimization (InterOPT), a two-stage framework that identifies unresolved formulation-critical gaps and uses them to guide whether to ask the next question or to stop. In our choice-based experiments, InterOPT substantially outperforms all baselines in exact slot recovery; in the open-ended setting, it remains competitive with strong prior methods. Together, OR-Clarify and InterOPT reframe OR assistance as a selective completeness decision: clarify when needed, stop when ready, and quantify what remains missing.

Primary area:
optimization
Keywords:
optimization modeling, large language models, interactive optimization, clarification, benchmarks

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_TM6ROpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies pre-formulation clarification in LLM-based optimization modeling. It introduces OR-Clarify, a benchmark containing 100 incomplete business specifications with hidden requirements, and InterOPT, a two-stage framework that tracks formulation gaps and guides clarification questions and stopping decisions. Experiments show substantial improvements in choice-based requirement recovery, although the method is less competitive in open-ended interactions.

Critical strengths

S1. A meaningful and practically relevant problem.

Recognizing missing requirements before constructing an optimization model is an important but underexplored capability. The paper usefully distinguishes formulation-critical ambiguities from ordinary missing details.

S2. Structured evaluation and informative diagnostics.

OR-Clarify provides hidden-slot annotations, controlled user simulation, and separate Open and Choice protocols. The Choice-setting improvements are substantial, and the stopping analyses, question-budget curves, and supplementary implementation provide useful evidence for evaluating clarification behavior.

Critical weaknesses

W1. The benchmark does not fully validate the central claim of formulation readiness. [Q1]

Section 3 defines completeness in terms of whether plausible business completions induce the same mathematical formulation. However, the evaluation primarily measures whether agents explicitly ask about predefined hidden slots, without verifying that the resulting specification supports a correct formulation.

Furthermore, all benchmark cases contain missing requirements, so the ability to recognize an already-complete specification and stop appropriately remains untested. This creates a gap between the proposed completeness criterion and the actual evaluation.

W2. The gap-tracking and stopping mechanisms are not sufficiently justified. [Q2]

The Choice implementation marks gaps as ASKED immediately after question selection, without explicitly tracking whether the user's response resolves them. The supplementary materials also reveal differences between the Open and Choice implementations that are not adequately explained in the unified method description.

These concerns are reflected in the Open results: InterOPT underperforms ORPilot in Core Exact (0.538 vs. 0.583), while producing more silent assumptions (0.560 vs. 0.268 per run). Removing Stage 2 further reduces silent assumptions to 0.254. The evidence therefore provides limited support for the reliability of the proposed stopping mechanism.

W3. The stage ablations do not adequately isolate the proposed contributions. [Q3]

The supplementary results reveal substantial differences in question budgets. In Choice, full InterOPT asks 10.04 atomic questions on average, compared with 3.46 and 4.86 for the two stage ablations.

Although the paper reports question-budget curves for the main baselines, equivalent controls are missing for these ablations. Thus, it remains unclear how much of the observed improvement comes from the proposed mechanisms rather than additional interaction.

The absence of complete benchmark cases and case-level evaluation artifacts also limits independent verification of the reported results.

Questions for authors

[Q1] Formulation completeness and benchmark validity.

Could the authors include already-complete specifications to evaluate unnecessary questioning and premature stopping? More importantly, could they verify whether recovered requirements lead to structurally correct mathematical formulations, rather than only successful hidden-slot matching?

[Q2] Gap resolution and stopping reliability.

Why does the explicit ledger track ASKED rather than answer-conditioned resolution? How are ambiguous or incomplete answers handled? Could the authors also clarify the differences between the Open and Choice implementations and explain why removing Stage 2 substantially reduces silent assumptions in Open?

[Q3] Fair isolation of the proposed stages.

Could the authors provide question-budget-matched comparisons between full InterOPT and its stage ablations? A corresponding LLM-call or token-budget comparison would also help establish whether the gains arise from the proposed mechanisms rather than additional deliberation.

6 comments
AuthorsOpenReview ↗

Response to Reviewer TM6R - Part1

Dear Reviewer TM6R,

Thank you for recognizing the practical relevance of our problem and the value of our structured evaluation, particularly the improvements under Choice. Below, we provide additional explanations and experimental evidence addressing your concerns regarding formulation readiness, stopping decisions, and stage ablations.

W1.1 & Q1.2: Downstream Mathematical Modeling Accuracy

Following your insightful suggestions, we compare four clarification methods end-to-end on our proposed OR-Clarify, using OptiMUS [1] as the downstream automated modeling framework. The public problem statements together with the information elicited during the clarification dialogue are combined as inputs for OptiMUS. We report both the clarification metrics (i.e., All-Slot Exact and Core Exact) and the final modeling accuracy as follows:

SettingClarification MethodOptimization MethodAll-Slot Exact ↑Core Exact ↑Final Modeling Accuracy ↑
OpenORPilotOptiMUS54.60%58.30%28%
OpenGATEOptiMUS52.80%56.00%23%
OpenInterOPTOptiMUS49.20%53.80%23%
ChoiceInterOPTOptiMUS63.80%67.50%32%

We can observe that our InterOPT-Choice achieves the highest final modeling accuracy. This demonstrates that a better elicitation strategy can benefit the downstream optimization modeling performance.

W1.2 & Q1.1: Stopping on Initially Complete Specifications

Thank you for your constructive suggestions! We further evaluate InterOPT and the strongest baseline under both Open & Choice settings, using the complete specifications corresponding to the 100 OR-Clarify cases, without informing the methods that the specifications are already complete. We impose a 20-turn interaction limit per problem and report the following three metrics:

  • Zero-question stops: The percentage of problems for which a method immediately issues READY TO MODEL without asking any clarification questions.
  • Mean interactive turns per problem: The average number of clarification turns (i.e., question–answer exchanges) per problem before the method issues READY TO MODEL or reaches the turn limit. Immediate stops count as zero.
  • Turn Limit Reached Without Stopping: The number of problems (out of 100) for which a method reaches the 20-turn limit without issuing READY TO MODEL.
SettingMethodZero-question stopsMean interactive turns per problemTurn Limit Reached Without Stopping
OpenORPilot (Best Baseline)0%7.227/100
OpenInterOPT29%4.395/100
ChoiceReadyGate (Best Baseline)32%5.7721/100
ChoiceInterOPT26%3.060/100

Overall, InterOPT requires fewer clarification turns and reaches the turn limit less frequently than the strongest baseline under both settings. This suggests its effectiveness in avoiding unnecessary questions and recognizing when to stop.

Your suggestion highlights another important and underexplored challenge in pre-formulation clarification for OR: determining when to stop asking questions and proceed to optimization modeling, particularly when the initial specification is already complete. We appreciate this insight and include this additional evaluation to examine whether clarification agents can recognize complete specifications and avoid unnecessary questioning.

W2.1 & Q2.3: Open/Choice Implementation

In the Choice setting, the agent presents structured alternatives, and the user selects among predefined options or selects Choice D instead to provide supplementary information. In the Open setting, the agent asks free-form questions, and the user responds in natural language. Both protocols share the same two-stage clarification logic, using formulation gaps to guide question selection. We clarify these differences in Section 5 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer TM6R - Part2

W2.2 & Q2.1 & Q2.2: Answer-Conditioned Resolution & Ambiguous User Answers

First, we thank you for this insightful suggestion, which motivates us to introduce an answer-conditioned variant (InterOPT-Resolution) that explicitly checks whether a gap is resolved and follows up on the same gap when the user's response is insufficient. Specifically, at the next planning step, InterOPT-Resolution classifies each queried gap as open, partially resolved, or resolved based on the user's response. More details are provided in our revised paper.

Second, your suggestion also motivates us to introduce an additional evaluation protocol, where the simulated user provides an incomplete or underspecified answer to a clarification question with a predefined probability. This protocol better captures a practical challenge in real-world interactions: users, particularly non-experts, may not fully understand what information a question seeks and may therefore provide only partial answers. More details are provided in our revised paper.

We evaluate this protocol under both Open and Choice settings and report the results below:

SettingMethodAll-Slot Exact ↑Core Exact ↑Silent/run ↓
OpenORPilot (Best Baseline)0.5000.5360.467
OpenInterOPT0.5330.6070.167
OpenInterOPT-Resolution0.5670.6070.133
ChoiceReadyGate (Best Baseline)0.4670.5000.533
ChoiceInterOPT0.6330.6430.167
ChoiceInterOPT-Resolution0.6330.6790.333

From the results, we draw two observations:

  • Answer-conditioned resolution improves requirement recovery under incomplete user answers. It improves All-Slot Exact in Open and Core Exact in Choice, suggesting that explicitly tracking gap resolution and following up on insufficient answers helps recover missing requirements, although its effect on silent assumptions varies across protocols.
  • ORPilot is less robust to incomplete user answers. Despite its strong performance under Open settings with complete user answer in the original paper, its recovery performance drops and silent assumptions increase under incomplete answers. We attribute this partly to its tendency to ask broad invitation questions (e.g., "Is there anything else we should know?"), which may elicit uncertain or incorrect information in this setting. Further analysis is provided in our response to Reviewer 7gVL (Point 2).

W2.3 & Q2.4: Stage 2 ablation under Open settings w.r.t. Silent Assumptions

Your observation highlights an important trade-off in Stage 2. Stage 2 uses gap information from Stage 1 and candidate questions to select the most informative question at each turn, rather than asking about every possible detail. As a result, some lower-priority details with natural default values may remain unconfirmed. If the agent later treats these defaults as confirmed in its final summary, they count as silent assumptions.

We illustrate this with Case 59 from InterOPT-Open, which involves monthly production planning for three candy brands using three raw materials:

  • Turn 2: After the user confirms profit maximization in Turn 1, the agent considers asking whether profit per kg is calculated as "selling price minus raw material cost minus processing fee."
  • Question prioritization: Instead, it asks about the missing composition constraints for brand B, reasoning that "The profit calculation detail is important but has a natural default interpretation." It then asks about production bounds in Turn 3 and stops.
  • Turn 4: In its READY TO MODEL response, the agent applies the profit formula without asking the user to confirm it. The judge therefore records one silent assumption (H1), since the agent treats the formula as confirmed without checking whether both raw material costs and processing fees are subtracted.

Based on this case and the ablation experiment, we can see that Stage 2 enables fewer, more targeted questions, although some assumptions with natural default interpretations may remain unconfirmed. By prioritizing the most informative questions, Stage 2 reduces atomic questions (4.82 vs. 9.50) and improves Core Exact (53.8% vs. 50.6%). However, some details, such as the profit calculation formula in Case 59, may remain unconfirmed and become silent assumptions in the final summary, leading to a higher Silent/run (0.560 vs. 0.254). Nevertheless, this does not come at the cost of overall requirement recovery and downstream modeling accuracy, as InterOPT still achieves higher All-Slot Exact and Core Exact.

AuthorsOpenReview ↗

Response to Reviewer TM6R - Part3

W3.1: Fine-grained ablation w.r.t. turn budgets

To address your concerns about the ablation under the Choice setting, we conduct a fine-grained evaluation of full InterOPT, its two ablation variants, and the strongest baselines under controlled interaction budgets of up to 3, 4, and 5 turns per OR-Clarify case. The results are reported below:

Interaction BudgetMetricFull InterOPT (Choice)w/o Stage 1 (Choice)w/o Stage 2 (Choice)ReadyGate (Choice)ORPilot (Open)
Up to 3 turnsCore Exact ↑54.47%53.19%49.36%47.45%49.79%
All-Slot Exact ↑51.00%48.80%46.20%44.00%46.80%
Up to 4 turnsCore Exact ↑58.09%55.11%53.83%49.57%52.77%
All-Slot Exact ↑54.00%50.80%49.80%46.00%49.60%
Up to 5 turnsCore Exact ↑60.64%55.96%56.60%50.85%55.96%
All-Slot Exact ↑56.00%51.60%52.40%47.00%52.20%
Full trajectory (20-turn limit)Core Exact ↑67.45%56.38%61.49%51.70%58.30%
All-Slot Exact ↑63.80%52.00%57.20%47.60%54.60%

From the table above, we draw three observations:

  • Larger interaction budgets benefit all methods. Both All-Slot Exact and Core Exact improve as the interaction budget increases, as additional turns allow agents to recover more hidden requirements from user responses.

  • InterOPT consistently outperforms the ablations and baselines under the same budget. This suggests that its improvements do not simply come from having more interaction turns, but from our core clarification strategy.

  • InterOPT makes more effective use of additional interaction turns. Its performance advantage becomes more pronounced as the budget increases. This suggests that InterOPT better exploits additional interaction opportunities, while other methods would experience diminishing returns in later turns due to less targeted or repetitive questions.

W3.2: LLM API Calls & Token Costs

Following your suggestion regarding the computational cost of InterOPT and its variants, we further report the average number of LLM API calls and input/output tokens per question turn, together with the corresponding requirement-recovery performance:

SettingMethodAll-Slot Exact ↑Core Exact ↑API Calls per Question Turn ↓Input Tokens per Question Turn ↓Output Tokens per Question Turn ↓
Choicew/o Stage 152.0%56.4%4.9811,7028,921
Choicew/o Stage 257.2%61.5%3.667,8245,527
ChoiceInterOPT (ours)63.8%67.5%5.3818,81910,169

As expected, using both Stage 1 and Stage 2 incurs more LLM calls and token usage than either ablation variant. However, this additional computational cost comes with substantially better requirement recovery: InterOPT achieves the highest All-Slot Exact and Core Exact. Together with our downstream modeling evaluation, these results suggest that the additional computation is worthwhile, as it helps recover more formulation-critical requirements and supports more accurate downstream modeling.

References

[1] OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models.

Reviewer_7gVLOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies premature formulation of an optimization problem in natural language when interacting with an LLM agent. It studies whether an LLM agent recognizes that the user request is missing facts that change the final solution, and how it asks them before writing the mathematical equations. The authors introduce OR-Clarify, a benchmark of 100 optimization problems in which roughly half of the critical facts for formulation are withheld as 178 hidden sots, each labeled by severity. An agent questions an LLM-simulated user, and an LLM judge scores the transcript on exact slot recovery and silent assumptions. The authors also propose InterOPT, a two-stage method that first keeps a persistent record of open formulation gaps and then generates candidate questions that fill these gaps.

Critical strengths

I think identifying that natural language description of an optimization problem leaves out several details that change the final solution (for example, what should be done if demand is not met) is a very interesting observation by the authors. And then, building a benchmark for this purpose helps future works to evaluate their approaches.

Critical weaknesses

As someone outside this research area, my main concern is that the evidence for InterOPT is somewhat narrow. The method clearly outperforms other approaches only under the multiple-choice protocol, where the baselines are authors's own variants, while under the free-form priot methods such as ORPilot and GATE achieve higher recovery.

Questions for authors

None

2 comments

Dear Reviewer 7gVL,

Thank you for your thoughtful review. We appreciate your recognition of the importance of identifying missing formulation-critical requirements and the value of OR-Clarify as a benchmark for future research. We would like to address your concerns from the following three aspects:

  • Adapt the SOTA baselines (i.e., ORPilot and GATE) from open setting to choice setting.
  • Conduct in-depth analysis of why ORPilot and GATE can succeed in open setting.
  • Explanation of why choice setting is better than open setting from the perspective of user effort.

1. Generalizability of Choice Results with External Baselines

To evaluate whether the Choice results extend to additional baseline frameworks, we adapt ORPilot and GATE to the same A–D interaction format and evaluate them on all 100 problems. InterOPT-Choice achieves 63.8% All-Slot Exact, exceeding both adapted baselines by 16.8 percentage points. Its Core Exact is 67.5%, compared with 48.9% for ORPilot-Choice and 52.1% for GATE-Choice.

SettingMethodAll-Slot Exact ↑Core Exact ↑Silent Assumptions per Run ↓
ChoiceORPilot47.0%48.9%0.74
ChoiceGATE47.0%52.1%0.31
ChoiceInterOPT (ours)63.8%67.5%0.366

2. Understanding Elicitation Behavior under Open setting

In the original Open evaluation, ORPilot and GATE achieve higher slot recovery than InterOPT-Open with similar numbers of atomic questions. To examine this difference, we study the behavior of broad invitations in conversation traces and find that broad invitation behavior is the key factor.

Broad invitations, such as "Is there anything else we should know?", trigger the simulated user to identify and supply omissions that the agent has not explicitly targeted. A single invitation can therefore elicit additional requirements without a separate targeted question for each one. ORPilot uses broad invitations much more frequently (35.2% of all turns), which may help explain its higher recovery without a proportional increase in question count.

In the table below, All-Slot Exact, Core Exact, and Silent/run are taken from the main evaluation (100 problems, five runs per problem), and the final column reports the number of audited dialogue turns containing a broad invitation:

SettingMethodAll-Slot Exact ↑Core Exact ↑Silent/run ↓Number of Invitation Turns
OpenORPilot0.5460.5830.268174
OpenGATE0.5280.5600.41240
OpenInterOPT0.4920.5380.56028
ChoiceMC-D0.4740.5060.69215
ChoiceInterOPT0.6380.6750.36616

3. User Effort: Choice versus Open settings

User effort refers to effort required to answer clarification questions, measured by user words per turn and per problem. InterOPT-Choice uses only 23.27 words per problem (vs. 131.2–175.3 in Open) while achieving the highest recovery (63.8% All-Slot, 67.5% Core Exact).

Offering targeted options lets users confirm exact modeling conditions without drafting explanations from scratch. Reaching higher accuracy with less input makes InterOPT-Choice more practical for real-world deployment, where domain users often lack optimization expertise or time for long responses:

SettingMethodAll-Slot Exact ↑Core Exact ↑User words/turn ↓User words/problem ↓
OpenORPilot54.6%58.3%35.5175.3
OpenGATE52.8%56.0%30.2135.0
OpenInterOPT (ours)49.2%53.8%29.2131.2
ChoiceMC-D47.4%50.6%4.9311.3
ChoiceInterOPT (ours)63.8%67.5%2.5223.27

Dear Reviewer 7gVL,

Thank you for your time and thoughtful efforts in evaluating our work. As the discussion period is drawing to a close, we would like to kindly check if our responses and revisions have adequately addressed your concerns. We would greatly appreciate any further feedback you may have, as your insights are very valuable for improving our paper.

Best regards, Authors

Reviewer_zx7HOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the premature formulation problem in optimization modeling. Most evaluations of LLM-based optimization agents assume a sufficient specification, but the information is not always available. The authors introduce ORClarify, a benchmark for pre-formulation clarification. They propose Interactive Optimization (InterOPT), a two-stage framework that identifies unresolved formulation-critical gaps and uses them to guide whether to ask the next question or to stop. Experiments demonstrate the improvements in this setting.

Critical strengths

Strengths:

  1. The view of pre-formulation clarification is interesting. The task includes deciding whether a request contains enough information. This setting is critical.
  2. The designed search framework is a practical method to address this problem.
Critical weaknesses

Weaknesses:

  1. Maybe I missed some important details of the paper. The statistics and detailed information about the constructed benchmark seem to be missing.
  2. Some baselines for the harness of optimization modeling are missing, such as OptiMUS, SAC-Opt, and OptiTree. The authors should evaluate them on the constructed benchmarks.
  3. The authors have not verified whether recovering more slots actually translates into a higher quality of the final generated optimization model. I recommend adding a small-scale end-to-end comparative experiment to close the loop between clarification and final modeling.
Questions for authors

Please see the weaknesses above.

2 comments
AuthorsOpenReview ↗

Official Response from Authors

Dear reviewer zx7H,

Thank you for your thoughtful review. We appreciate your recognition of the importance of pre-formulation clarification and the practicality of our proposed search framework. Below, we provide additional explanations and experimental evidence addressing your concerns regarding benchmark statistics, downstream modeling frameworks, and end-to-end modeling accuracy.

Weakness 1 — How large is OR-Clarify, and what does it contain?

OR-Clarify contains 100 problems and 178 hidden requirements (1–5 per problem, a mean of 1.78):

  • 75 blocking conditions that change the problem definition (P0),
  • 83 substantive modeling conditions (P1),
  • and 20 secondary boundary or interpretive conditions (P2).

Each hidden requirement is grounded in a fact from the original problem and paired with a user answer and a recovery rubric. The benchmark construction section details masking and annotation, and full statistics have been added to Section 4.1 (Table 1) and Appendix A (Figure 5).

Weakness 2 — How does InterOPT relate to modeling frameworks such as OptiMUS, SAC-Opt, and OptiTree?

InterOPT handles the initial clarification step—eliciting business requirements that affect the objective, variables, or constraints—whereas frameworks such as OptiMUS [1], SAC-Opt [2], and OptiTree [3] handle subsequent formulation, making the two stages complementary. To isolate how clarification affects formulation, we adopt OptiMUS as the downstream modeler and pass each method's elicited information to it (see Weakness 3 below).

Weakness 3 — Are elicited requirements correctly represented in the final mathematical model?

We compare four clarification methods end-to-end on our proposed OR-Clarify, using OptiMUS as the downstream automated modeling framework. The public problem statements together with the information elicited during the clarification dialogue are combined as inputs for OptiMUS. We report both the clarification metrics (i.e., All-Slot Exact and Core Exact) and the final modeling accuracy as follows:

SettingClarification MethodOptimization MethodAll-Slot Exact ↑Core Exact ↑Final Modeling Accuracy ↑
OpenORPilotOptiMUS54.60%58.30%28%
OpenGATEOptiMUS52.80%56.00%23%
OpenInterOPTOptiMUS49.20%53.80%23%
ChoiceInterOPTOptiMUS63.80%67.50%32%

We can observe that our InterOPT under the Choice setting achieves the highest final modeling accuracy. This demonstrates that a better elicitation strategy can benefit the downstream optimization modeling performance.

References

[1] OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models.

[2] SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling.

[3] OptiTree: Hierarchical Thoughts Generation with Tree Search for LLM Optimization Modeling.

Dear Reviewer zx7H,

Thank you for your time and thoughtful efforts in evaluating our work. As the discussion period is drawing to a close, we would like to kindly check if our responses and revisions have adequately addressed your concerns. We would greatly appreciate any further feedback you may have, as your insights are very valuable for improving our paper.

Best regards, Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

Dear Chairs,

We sincerely thank the reviewers for their thoughtful feedback. We are encouraged by their recognition of pre-formulation clarification, the value of OR-Clarify, and the effectiveness of InterOPT.

We would like to highlight the current status of our rebuttal:

  • Our paper now receives ratings of 3, 3, 3 across all three reviewers.
  • Reviewers zx7H and TM6R confirmed that their main concerns are addressed. We have also provided comprehensive new experiments and clarifications addressing Reviewer 7gVL's concerns.

Below we provide a summary of the reviewer consensus and our revisions to support your assessment.


Consensus Among Reviewers

  • Importance & Practical Relevance (All reviewers): All reviewers (TM6R, 7gVL, zx7H) recognize pre-formulation clarification as an important and practical problem.
  • Benchmark & Evaluation Value (TM6R, 7gVL): Reviewer 7gVL highlights the value of OR-Clarify for future research, and Reviewer TM6R recognizes the value of our structured evaluation.
  • Framework Practicality & Choice Gains (TM6R, zx7H): Reviewer zx7H acknowledges the practicality of our search framework, and Reviewer TM6R specifically stresses the improvements achieved under the Choice setting.

Summary of Revisions and Clarifications

We address every concern raised by the reviewers with new experiments, analyses, and revisions:

  • Downstream Modeling (TM6R, zx7H):
    • Complementary Roles: We clarify that InterOPT handles pre-formulation clarification, whereas OptiMUS, SAC-Opt, and OptiTree handle subsequent modeling; the two stages are complementary.
    • End-to-End Evaluation with OptiMUS: With clarification outputs passed to OptiMUS, InterOPT-Choice yields the highest accuracy (32% vs. 28% for ORPilot-Open; 23% for GATE-Open/InterOPT-Open).
  • Choice vs. Open Generalizability & Practicality (7gVL, TM6R):
    • External Baselines in Choice: We adapt ORPilot and GATE to the same A–D Choice format on all 100 problems; InterOPT-Choice achieves 67.5% Core Exact, outperforming ORPilot-Choice (48.9%) and GATE-Choice (52.1%) by +15.4 to +18.6 percentage points.
    • User Effort Reduction: We quantify user effort and show that InterOPT-Choice requires only 23.27 words/problem (vs. 131.2–175.3 in Open), validating its practicality for domain users with limited optimization expertise.
    • Protocol Clarification: We clarify in Section 5 that Open and Choice share the same two-stage gap-guided logic, differing only in interaction format (free-form Q&A vs. structured A–D choices).
  • Open-Setting Clarification Behavior (7gVL): We analyze dialogue traces and show that ORPilot relies on broad invitations (e.g., "Is there anything else?") in 35.2% of turns (174 vs. 28 for InterOPT-Open), which may help recover requirements through user-volunteered information.
  • Stopping on Complete Specifications (TM6R): We analyze stopping behavior on 100 complete specifications. InterOPT stops in fewer turns than the baselines in both Open (4.39 vs. 7.22 for ORPilot) and Choice (3.06 vs. 5.77 for ReadyGate), reaching the turn limit 0/100 times in Choice (vs. 21/100 for ReadyGate).
  • Incomplete Answer Resolution (TM6R): We introduce InterOPT-Resolution (with gap states: open, partially resolved, and resolved). Under incomplete user answers, it improves over InterOPT on Open All-Slot Exact (53.3% → 56.7%) and Choice Core Exact (64.3% → 67.9%).
  • Detailed Stage Ablations & Trade-offs (TM6R):
    • Budget-Controlled Ablations: We conduct matched-budget ablations (3, 4, and 5 turns) and confirm that full InterOPT-Choice consistently outperforms both stage ablations, ReadyGate, and ORPilot at every budget level.
    • Silent Assumptions and Stage 2 Prioritization (Open): We analyze Case 59 to show how selective prioritization may contribute to Stage 2's higher Silent/run: choosing the most informative question leaves lower-priority natural defaults unconfirmed, which are treated as silent assumptions if used in the summary. Importantly, this does not compromise overall requirement recovery (53.8% vs. 50.6% Core Exact).
  • Computational Cost Analysis (TM6R): We report per-turn API calls and token costs in Choice, showing that InterOPT's higher cost (5.38 calls/turn vs. 3.66–4.98 for ablations) brings substantial gains (67.5% vs. 56.4%–61.5% Core Exact).
  • OR-Clarify Benchmark Statistics (zx7H): We add complete benchmark statistics to Section 4.1 (Table 1) and Appendix A (Figure 5), clarifying that OR-Clarify comprises 100 problems and 178 hidden requirements (75 P0, 83 P1, and 20 P2 conditions).

We have highlighted all modifications in the revised manuscript in blue. We hope the Area Chair will consider our rebuttal and believe these additions address the reviewers' concerns.

Thank you again for your time and effort!

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.