acceptodds
Under review as a conference paper at ICLR 2027

Interactive Clarification for Cloud Infrastructure-as-Code Synthesis

Abstract

The scale and complexity of modern cloud infrastructure have made "Infrastructure-as-Code" (IaC) essential for managing deployments through declarative configurations. While large language models (LLMs) are increasingly used to generate IaC configurations from natural language, user requests are often ambiguous and underspecified. Unlike traditional code generation, it is costly and time-consuming to test IaC configurations during synthesis, forcing the LLMs into an almost one-shot regime. We observe that ambiguity in IaC synthesis exhibits a compositional structure: configurations decompose into three axes (resources, topology, attributes) where higher-level decisions constrain lower-level ones. We propose a training-free, multi-level disambiguation framework that generates diverse candidate specifications, identifies structural disagreements across these axes, ranks them by informativeness, and produces targeted clarification questions that progressively narrow the configuration space. We further introduce Ambig-IaC, an expert-verified benchmark of 300 validated IaC tasks with ambiguous requests, and define evaluation metrics based on graph edit distance and exact attribute matching. Comprehensive experiments show that our method achieves the best structural fidelity at every interaction budget and the best attribute fidelity at larger budgets, at a fraction of the cost of information-gain alternatives, with gains that scale with the interaction budget and generalize across models. Extensive ablation studies and analyses further demonstrate its robustness for interactive IaC synthesis.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Applications, Cloud computing, Infrastructure as code, Ambiguity resolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jz7COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper clarifies ambiguous infrastructure requests through binary questions about candidate resources, topologies and attributes, followed by pruning and regeneration. AMBIG-IAC contains 300300 tasks with plan-checked reference configurations; experts audit 100100 ambiguous requests. Experiments evaluate configuration recovery and cost across models and question budgets.

Critical strengths
  1. Clarifying infrastructure intent is a useful problem, and the resource/topology/attribute decomposition gives question selection an explicit, interpretable basis.
  2. Structural gains and lower token use and latency than ATD at K=5K = 5 support the framework’s usefulness. Multiple models and budgets, output-generation controls, and simulated-user audits strengthen the evaluation.
Critical weaknesses
  1. The metrics can miss incorrect attribute-to-resource assignments. Resource+Attribute F1\text{Resource+Attribute }F_1 pools (type,key,value)(\text{type}, \text{key}, \text{value}) items across instances, while Structure Score ignores attributes (§5; Appendix C). Swapping port/CIDR settings between same-type security groups attached to different roles can therefore yield perfect scores for an incorrect configuration. This is a constructed blind spot; its frequency in actual outputs is unknown. An instance-aligned assessment is needed to substantiate recovery of the target’s attributes.
  2. Structural consensus can end clarification before attributes are resolved. Section 4 deduplicates candidates by resource types and topology and stops when structurally distinct candidates cannot be generated. This can remove the attribute-only alternatives needed for further questions. The tradeoff is acknowledged, but its impact on attribute recovery has not been isolated, leaving uncertainty about whether the design supports clarification across all three intended axes.
Questions for authors
  1. How do attribute comparisons change under a topology-consistent resource-instance alignment, particularly for repeated resource types?
  2. How are unresolved attribute-only alternatives retained once candidates agree structurally? What changes if deduplication preserves meaningful attribute differences?
3 comments
AuthorsOpenReview ↗

Response to Reviewer jz7C

Thank you for the thoughtful review. We investigated both concerns through implementation changes and additional evaluations. For Q1, we corrected the metric and re-scored all submitted outputs and baselines. The revised metric detects the reported attribute-matching errors, while our structural and attribute gains remain. For Q2, we implemented and tested an attribute-preserving variant across question budgets to quantify the impact of structural deduplication. The variant enables more attribute-level clarification, but does not significantly improve overall R+A F1.

Q1. Topology-consistent attribute comparisons

We revised the metric and re-scored all results. R+A F1 now credits an attribute only on the generated resource instance aligned to the corresponding reference instance. We retain the item definitions of Appendix C.4, but replace pooled matching with instance-aligned matching and account for every repeated nested-block element.

The alignment starts from the node correspondence used by the Structure Score's graph edit distance (GED), ensuring topology consistency. Since multiple correspondences may have the same edit cost, we break ties in favor of attribute agreement. Specifically, we iteratively swap the matched node-pairs of same-type resource instances whenever doing so increases the number of matched attributes without increasing the edit cost, until no further improvement is possible. Within each aligned resource pair, repeated nested-block elements, such as ingress rules, are matched one-to-one to maximize matched attribute leaves, recursively. The submitted implementation retained only one element per repeated block; the revised implementation counts every element. Re-scoring actual outputs lowers each method's R+A F1 by 0.6–2.8 points, primarily by removing credit previously assigned across resource instances or nested-block elements.

The structural advantage and attribute gains over the three lightweight baselines remain. Table R1 reports the submitted GPT-4o-mini runs under the revised metric. Structure Score is unchanged. Table R1. Submitted runs under the revised metric. The difference between ATD and Ours is not significant under paired Wilcoxon tests with Holm correction within each setting.

MethodStructure K=5K=10K=15Aligned R+A F1 K=5K=10K=15
Direct Q Generation43.0844.3846.3116.7317.5217.21
Best-of-N41.2843.5243.1217.5417.7318.71
Self-Consistency38.5938.4138.1914.8815.4415.52
ATD42.2341.89—21.1522.68—
Ours (submitted)48.9550.9854.8420.8122.6424.78

Our method significantly exceeds Direct Question Generation, Best-of-N, and Self-Consistency on both metrics at every budget shown. The differences from ATD on aligned R+A F1 are not statistically significant, but at a much higher cost. This is consistent with the original findings in the paper.

AuthorsOpenReview ↗

Response to Reviewer jz7C (Part 2)

Q2. Impact of Attribute-Aware Candidate Preservation

We tested the impact of preserving attribute-only alternatives. To investigate Q2, we implemented a variant that preserves candidates with meaningful per-instance attribute differences, while retaining the existing exclusions for names and identifiers. Disagreement extraction considers attribute values across all instances of each resource type and ranks them by partition entropy. The remaining framework is unchanged. On GPT-4o-mini, the mean pool sizes increase from 5.8/5.2/5.0 to 6.9/6.8/6.8, containing more candidates with attribute differences.

Table R2. GPT-4o-mini results under the instance-aligned metric. Bold marks the highest mean across all methods, including Submitted.

MethodStructure K=5K=10K=15Aligned R+A F1 K=5K=10K=15
Direct Q Generation43.0844.3846.3116.7317.5217.21
Best-of-N41.2843.5243.1217.5417.7318.71
Self-Consistency38.5938.4138.1914.8815.4415.52
ATD42.2341.89—21.1522.68—
Ours (attribute-preserving)49.0852.6853.6921.4622.3624.46
Ours (submitted)48.9550.9854.8420.8122.6424.78

The main comparative findings remain, but overall attribute gains are limited. The variant still exceeds all baselines on Structure and the three lightweight baselines on aligned R+A F1 at every budget. Differences from ATD on R+A F1 are not significant. Compared with Submitted, among reference attribute items missed by the variant, 62–68% belong to resource types absent from the final output, suggesting that resource recovery remains an important limitation.

Takeaway. Attribute-aware candidate preservation restores attribute-level clarification without significantly improving overall R+A F1. We suspect that better attribute question selection alone may be insufficient: attribute recovery also depends on generating the intended resources and attribute values. Moreover, even with attribute-aware candidate preservation, attribute questions still compete with resource and topology questions under a fixed interaction budget. A more adaptive strategy could prioritize structural clarification until a convergence threshold is reached, then allocate more questions to unresolved attributes. We see this as a promising direction for future work.

AuthorsOpenReview ↗

Friendly Reminder

Dear Reviewer,

Thank you for recognizing the importance of the problem and our contributions to both the method and the benchmark. We have conducted additional experiments to address your concerns and hope you will have a chance to review our rebuttal. We would greatly appreciate any further feedback!

Reviewer_RS6rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies interactive clarification for infrastructure-as-code synthesis from underspecified natural-language requests. It represents candidate configurations through resources, dependency topology, and attributes, extracts symbolic disagreements among sampled candidates, and uses entropy ranking with cross-axis balancing to guide binary clarification questions. User answers prune the candidate pool, which is regenerated when necessary, and a surviving candidate is returned as the final specification. The paper also introduces AMBIG-IAC, comprising 300 tasks adapted from IaC-Eval, and evaluates recovery of a hidden reference configuration using a reference-informed simulated user, graph similarity, and resource–attribute F1. Experiments across five model backbones show consistent improvements in structural reference fidelity, with substantially lower cost than ATD. Additional analyses examine output preservation, computational cost, simulated-user reliability, and candidate-coverage failures.

Critical strengths
  1. A useful and efficient implementation of structured clarification. The paper identifies a practical way to generate informative questions directly from differences between structured infrastructure candidates, avoiding much of the question–candidate cross-evaluation required by ATD. The structural improvements are consistent across the evaluated backbones and interaction budgets. At K=5 with GPT-4o-mini, the method uses 31.8k tokens and 28 seconds per task, compared with 129.1k tokens and 303 seconds for ATD, while obtaining higher structural fidelity. This provides a consequential contribution toward making interactive disambiguation practical for structured artifacts.

  2. An informative benchmark and analysis of what drives performance. AMBIG-IAC establishes a controlled setting for studying clarification in cloud configuration, with a three-expert audit of 100 rewritten requests. The output-protocol ablation is especially useful: it distinguishes improvements from the clarification procedure from those obtained by preserving a surviving candidate instead of generating a fresh final specification. The failure analysis also identifies missing architectural options and shared incorrect assumptions as prominent bottlenecks. Together, these results provide useful guidance about where to improve interactive generation systems beyond simply asking more questions.

Critical weaknesses
  1. The fidelity evaluation can give perfect scores to configurations with consequential behavioral differences. Appendix C.4 pools attribute items by resource type, attribute path, and scalar value, without retaining their association with individual resource instances; nested blocks also flatten under a shared prefix. Consider a security group allowing port 22 only from a private CIDR and port 443 from the public internet. Exchanging the two CIDRs preserves both the resource/dependency graph and the flattened attribute bag, so both reported metrics can remain perfect even though SSH is now publicly accessible. This is a false-positive match, distinct from the acknowledged limitation of penalizing valid alternative configurations. Consequently, the current results leave uncertainty about whether the improvements persist when attributes are evaluated together with the resources and nested blocks to which they belong. The Terraform validation experiment does not close this gap: the emission prompt prioritizes plan success over completeness, and fidelity after conversion and repair is not reported.

  2. The specified disagreement representation can miss architectural differences already present in the candidate pool. Appendix A.3 defines resource disagreements through type presence and topology disagreements through type-level edge presence. These features do not generally distinguish resource multiplicities or instance-level wiring. For example, candidates containing two versus three otherwise identically configured instances can share the same resource types, type-level dependencies, and per-type attribute values. Under the written definitions, the disagreement set can therefore be empty and Algorithm 1 can terminate despite unresolved architectural differences. This creates a mismatch with the instance-sensitive structural evaluation and limits the central structure-aware clarification mechanism even when a relevant alternative has already been sampled. Additional handling in the implementation could resolve this concern, but it is not specified clearly enough to assess.

Questions for authors
  1. Attribute associations and evaluation. Does the implemented resource–attribute metric retain any association between attributes and individual resource instances or nested blocks beyond the definition in Appendix C.4? If it does not, do the main comparisons persist under an evaluation that preserves these associations, including cases such as exchanging the CIDRs of two ingress rules?

  2. Resource multiplicity and instance-level topology. How does the implementation distinguish candidates with the same resource types and type-level dependencies but different resource counts or instance-level wiring? Please clarify the handling of multiple instances of the same type and whether the algorithm can terminate with such differences unresolved; their frequency in the existing runs would materially affect my assessment.

  3. Fidelity after Terraform emission and repair. Since the emission prompt prioritizes a plan-clean configuration over completeness, how often do conversion or repair change the selected specification or violate previously clarified requirements? Do the relative improvements remain when fidelity is evaluated on the final Terraform configurations?

4 comments
AuthorsOpenReview ↗

Response to Reviewer RS6r

Thank you for the thoughtful review. We addressed all three concerns through implementation changes and additional evaluations. For Q1, the implemented metric retained no association between attributes and instances; we corrected the metric and re-scored all submitted outputs and baselines. The revised metric detects the CIDR exchange, while our structural and comparative attribute gains remain. For Q2, the submitted implementation did not distinguish resource counts; we implemented and tested a variant which extended the disagreement representation to cover counts, and re-ran the method at every budget. Count differences reach the final output in at most 2% of tasks, and the comparative results are unchanged. For Q3, we scored the emitted Terraform with the same two metrics and checked every clarified answer against it. The relative improvements remain on the final Terraform.

Q1. Attribute associations and evaluation

The implemented metric retained no association, and we have revised it. The submitted implementation followed Appendix C.4 exactly, so exchanging the CIDRs of two ingress rules would have scored perfectly. R+A F1 now credits an attribute only on the generated resource instance aligned to the corresponding reference instance. We retain the item definitions of Appendix C.4, but replace pooled matching with instance-aligned matching and account for every repeated nested-block element.

The alignment starts from the node correspondence used by the Structure Score's graph edit distance (GED), ensuring topology consistency. Since multiple correspondences may have the same edit cost, we break ties in favor of attribute agreement. Specifically, we iteratively swap the matched node-pairs of same-type resource instances whenever doing so increases the number of matched attributes without increasing the edit cost, until no further improvement is possible. Within each aligned resource pair, repeated nested-block elements, such as ingress rules, are matched one-to-one to maximize matched attribute leaves, recursively. In the reviewer's example, the SSH rule's CIDR is therefore compared only with the SSH rule's CIDR. The submitted implementation retained only one element per repeated block; the revised implementation counts every element. On perturbed copies of the references, the revised metric detects every exchange of settings between same-type instances with different roles and between nested-block elements, all of which the pooled metric scored as perfect, while still awarding perfect scores when interchangeable instances are exchanged. Re-scoring actual outputs lowers each method's R+A F1 by 0.6–2.8 points, primarily by removing credit previously assigned across resource instances or nested-block elements.

The main comparisons persist. Table R1 reports the submitted GPT-4o-mini runs under the revised metric. Structure Score is unchanged.

Table R1. Submitted runs under the revised metric. The difference between ATD and Ours is not significant under paired Wilcoxon tests with Holm correction within each setting.

MethodStructure K=5K=10K=15Aligned R+A F1 K=5K=10K=15
Direct Q Generation43.0844.3846.3116.7317.5217.21
Best-of-N41.2843.5243.1217.5417.7318.71
Self-Consistency38.5938.4138.1914.8815.4415.52
ATD42.2341.89—21.1522.68—
Ours (submitted)48.9550.9854.8420.8122.6424.78

Our method significantly exceeds Direct Question Generation, Best-of-N, and Self-Consistency on both metrics at every budget shown. The differences from ATD on aligned R+A F1 are not statistically significant, but at a much higher cost. This is consistent with the original findings in the paper.

AuthorsOpenReview ↗

Response to Reviewer RS6r (Part 2)

Q2. Resource multiplicity and instance-level topology

We address resource multiplicity by implementing and testing a multiplicity-aware variant. In the revised method, candidates are duplicates only if they agree on the count of every resource type, the count of every type-level edge and the per-instance attribute values. A count disagreement partitions the pool by count (e.g., two versus three aws_subnet), is ranked by partition entropy alongside the presence disagreements on the same axis, and each answer becomes a constraint that prunes regenerated candidates. The loop therefore cannot terminate due to apparent consensus while count differences remain in the current candidate pool. The same per-instance comparison also retains candidates that differ only in meaningful attribute values. All other components of Algorithm 1 are unchanged.

Table R2. GPT-4o-mini results under the revised metric. Bold marks the highest mean across all methods, including Submitted.

MethodStructure K=5K=10K=15Aligned R+A F1 K=5K=10K=15
Direct Q Generation43.0844.3846.3116.7317.5217.21
Best-of-N41.2843.5243.1217.5417.7318.71
Self-Consistency38.5938.4138.1914.8815.4415.52
ATD42.2341.89—21.1522.68—
Ours (variant)49.0852.6853.6921.4622.3624.46
Ours (submitted)48.9550.9854.8420.8122.6424.78

The main comparative findings remain robust. The revised method still exceeds all baselines on Structure and the three lightweight baselines on aligned R+A F1 at every budget; differences from ATD on R+A F1 are not statistically significant, at a fraction of its cost. Revised and Submitted are statistically indistinguishable, at a similar cost (29.5k versus 31.8k tokens per task at K=5). This is consistent with our analysis of the submitted runs: count differences appeared in about 10% of candidate pools, but only up to 2% of final outputs had correct resource types with incorrect counts. Together with the statistically indistinguishable results of the revised variant, this suggests that the omission of multiplicity had limited impact on the main findings.

Takeaway. The revised method addresses resource multiplicity without changing the main comparative findings. Extending the Ambig-IaC benchmark with more multiplicity-sensitive and instance-level topology cases would enable a more targeted evaluation of these distinctions.

AuthorsOpenReview ↗

Response to Reviewer RS6r (Part 3)

Q3. Fidelity after Terraform Emission and Repair

We address all three questions using the deployability study (Claude Opus 4.6, K=5; ATD was evaluated only on GPT-4o-mini). We evaluate the final Terraform using the same two metrics, extracting resources, dependencies, and attributes consistently with the reference evaluation.

Conversion largely preserves the selected specification; repair has limited impact. Averaged over the four methods, the emitted Terraform preserves the exact resource-type multiset in 66% of tasks and retains 99% of the selected specification's resources overall; the gap between the two reflects added rather than removed resources. The main change is adding supporting resources (IAM roles, log groups, data sources), averaging 0.9 per task. The Terraform retains 85% of the specification's attribute items; among the remaining items, more than half differ only in serialization (e.g., nested blocks and policy documents), while approximately 5% of all items are omitted and 1% have changed values. It also adds approximately seven attribute items per task from the clarified intent, contributing to the higher R+A F1 observed after emission. Repair affects 119 of 1,200 method-task pairs, improving the Structure scores of those tasks by 0.8 points on average.

Emission preserves most clarified answers. An LLM judge (GPT-5.1) checked each clarified answer against both the selected specification and emitted Terraform. Across all four methods, emission introduces contradictions in 1.5% of clarified answers (3% for ours); repair newly violates 18 answers across the 119 repaired method-task pairs. Most contradictions involve supporting resources introduced during emission to produce plan-valid configurations, despite their absence from the selected specification and reference.

The relative improvements persist on the final Terraform. Table R3 compares the selected specification with the emitted Terraform.

Table R3. Structure / aligned R+A F1, Claude Opus 4.6, K=5.

MethodSelected specificationEmitted Terraform
Direct Q Generation53.2 / 24.052.1 / 27.9
Best-of-N57.7 / 27.656.6 / 31.6
Self-Consistency57.0 / 26.456.3 / 30.2
Ours64.7 / 29.565.2 / 34.2

On the final Terraform, our method remains significantly ahead of all baselines on Structure and of Direct Question Generation and Self-Consistency on aligned R+A F1. The difference from Best-of-N on aligned R+A F1 is not significant, consistent with Table 4. Scoring configurations that fail terraform plan as zero changes any mean by at most 1.3 points.

Takeaway. Conversion and repair largely preserve the selected specifications and clarified requirements across all methods. Crucially, our structural and attribute-fidelity advantages persist on the final Terraform, showing that the main findings are not artifacts of evaluating intermediate specifications.

AuthorsOpenReview ↗

Friendly Reminder

Dear Reviewer,

Thank you for recognizing the importance of the problem and our contributions to both the method and the benchmark. We have conducted additional experiments to address your concerns and hope you will have a chance to review our rebuttal. We would greatly appreciate any further feedback!

Reviewer_uScpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes an interactive approach to generating Infrastructure-as-Code from ambiguous requests. It samples candidate configurations, identifies disagreements about resources, connections, and attributes, and uses those disagreements to generate clarification questions. The authors introduce Ambig-IaC, a benchmark of 300 Terraform tasks, and show improvements over several question-generation strategies across multiple models and interaction budgets.

Critical strengths

The paper addresses a meaningful problem. Infrastructure requests often omit decisions that affect connectivity, security, and resource selection. Making those decisions explicit before deployment is useful, and the resource, topology, and attribute decomposition provides an understandable structure for doing so.

I also appreciate the experimental coverage. The authors evaluate several backbones, report uncertainty estimates, and include ablations that separate question selection from candidate preservation. The benchmark and failure analysis could be useful resources for studying infrastructure assistants.

Critical weaknesses

The absence of an environment-aware baseline using read-only operations is a major concern. Inspecting existing resources, network configuration, deployment state, and relevant policies can provide substantial context without provisioning or modifying infrastructure. These operations are often relatively inexpensive and could resolve some of the uncertainty that the proposed system asks the user. The paper checks Terraform validity after generation, but it does not compare against an assistant that gathers environment information before deciding what to ask.

This omission matters because the central comparison is between different ways of asking questions. It is unsurprising that targeted, structured follow-up questions improve performance on underspecified requests. The more consequential question is which uncertainties require user input after the assistant has gathered the information already available to it.

Questions for authors

Is there a comparison against an assistant that uses read-only cloud API or CLI operations to inspect relevant environment state before asking clarification questions? A hybrid baseline that inspects first and asks only about unresolved decisions would be particularly informative.

Under a comparison that accounts for both tool calls and user interactions, how much of the reported improvement remains attributable to the proposed question-selection strategy vs tool-calls.

2 comments
AuthorsOpenReview ↗

Response to Reviewer uScp

Thank you for the thoughtful review and constructive suggestion. We would like to clarify the stage of infrastructure management that our work addresses. The lifecycle is commonly divided into planning and design (Day 0), provisioning and deployment (Day 1), and ongoing operation and maintenance (Day 2). Our work focuses on the first stage: translating an underspecified request into an infrastructure configuration aligned with user intent.

Specifically, we target pre-deployment design (Day 0), where little or no task-relevant infrastructure has yet been deployed. The main uncertainties concern what the user wants to build—its resources, topology, and attributes. Inspection can reveal existing constraints, but does not necessarily resolve these intended design choices. Ambig-IaC captures this setting by evaluating configuration design from a request, without an associated initial cloud deployment.

We agree that inspection becomes more informative when planning within or modifying an established environment. Our method is compatible with the proposed inspect-first workflow: the agent could inspect relevant resources and policies, incorporate that information into candidate generation, and then apply our question-selection strategy to the remaining design disagreements. Environment inspection and structured clarification are therefore complementary.

Our current experiments evaluate whether structured question selection improves intent recovery given the same available task information and interaction budget. The comparisons directly address this question, and the ablation in Appendix shows that the benefit persists independently of candidate preservation. Together, these results establish the contribution of question selection to the Day 0 infrastructure-design task studied in this work.

Beyond the clarification method, our work also contributes an expert-verified benchmark for studying the design decisions that guide subsequent deployment and operation. Ambig-IaC contains 300 Terraform tasks spanning 167 resource types and 44 service families, pairing ambiguous requests with validated reference configurations representing the intended designs. This makes implicit design choices evaluable and enables controlled comparisons of how effectively agents recover user intent under limited interaction.

Your suggestion is a natural extension of this foundation. Adding initial cloud environments would support evaluation of the combined inspection-and-clarification workflow under a joint tool-call and user-interaction budget. It would also support infrastructure-update tasks—for example, changes in response to incidents or evolving application requirements—where inspecting the current environment is especially relevant.

We will clarify the pre-deployment design scope in the revision and explain how our method and benchmark can support this environment-aware extension. Thank you for highlighting this opportunity to build on our work.

AuthorsOpenReview ↗

Friendly Reminder

Dear Reviewer,

Thank you for recognizing the importance of the problem and our contributions to both the method and the benchmark. We have clarified the scope and motivation of our work. We hope you will have a chance to review our response and would greatly appreciate any further feedback!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response to the Area Chair and Reviewers

We thank the reviewers for their thoughtful feedback and are encouraged that all three recognize the importance of the problem: "a meaningful problem" (uScp), "a useful problem" (jz7C), and a method that is "a consequential contribution toward making interactive disambiguation practical for structured artifacts" (RS6r). The reviewers also appreciated the clear presentation (uScp, jz7C), the interpretable resource/topology/attribute decomposition (uScp, jz7C), the structural gains at much lower cost than ATD (RS6r, jz7C), the Ambig-IaC benchmark (uScp, RS6r), and the thorough evaluation (uScp, RS6r, jz7C).

Nearly all modern software and AI services run on cloud infrastructure, and AIOps, the use of LLMs to automate its design and operation, is a high-stakes application area where configuration errors translate directly into outages, security exposure, and wasted spend. This work matters because LLMs are increasingly asked to generate cloud infra from underspecified requests, and a cloud configuration cannot be cheaply tested during synthesis. An assistant that silently guesses produces infrastructure that is valid but misaligned with user intent on connectivity, security, and resource choices.

We have taken substantial actions in response to the reviews. The concerns centre on evaluation rigor and scope, and two of the three reviewers rated their soundness concerns as "minor or fixable" (RS6r, jz7C). We addressed each one with implementation changes and new experiments, and the paper's conclusions held. Below we summarize the key changes.

1. Instance-aligned attribute metric (RS6r, jz7C). We replaced pooled attribute matching with topology-consistent, instance-aligned matching and re-scored all methods. Our method still has the best Structure Score at every budget, significantly exceeds the three lightweight baselines on both metrics, and is statistically on par with ATD on attribute fidelity at roughly a quarter of the tokens.

2. Multiplicity- and attribute-aware disagreements (RS6r, jz7C). We implemented a variant that distinguishes resource counts and preserves attribute-only alternatives, and re-ran it at every budget. The comparative findings are unchanged, and the variant is statistically indistinguishable from the submitted method.

3. Fidelity of the final Terraform (RS6r). We scored the emitted Terraform with the same metrics. Our method remains significantly ahead of all baselines on Structure (65.2 vs. 56.6 for the best baseline) and keeps the highest attribute fidelity (34.2 vs. 31.6).

4. Scope (uScp). Our work targets pre-deployment design (Day 0), where Ambig-IaC tasks have no initial cloud deployment to inspect. Environment inspection is complementary to our method, and we will state this scope explicitly in the revision.

All new results will be incorporated into the revision. We have not yet received follow-up from the reviewers, so we hope this summary helps the Area Chair assess how the concerns have been resolved.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.