- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies computational resource exhaustion attacks against finite element method (FEM) solvers and proposes CloakFEM, a framework that uses LLM agents to automatically generate FEM programs with high computational costs. The method uses a unified intermediate representation (IR) to describe modifiable FEM parameters and combines three agents—Strategy, Generation, and Selection—to iteratively search for inputs that significantly increase computational costs based on solver feedback. It also uses validity checks to ensure that the generated programs can run and complete the solution process.
The authors conduct experiments across three FEM environments, 17 problem categories, and 1,068 initial samples, reporting a maximum amplification of 5,752.72× in solver iteration counts. The method is compared with five conventional black-box search methods. In addition, the paper includes comparisons under a restricted search space, component ablations, cross-solver transfer experiments, and defense experiments.
Critical strengths
S1. The research problem has some exploratory value.
This paper focuses on the phenomenon that valid inputs can significantly increase the computational costs of FEM solvers and studies this problem from the perspective of computational resource exhaustion attacks. It provides a direction worth exploring for evaluating the security of traditional numerical computing software.
S2. An automated search framework across different FEM solvers is implemented.
The authors design a unified FEM intermediate representation and a validity-checking mechanism, allowing the framework to handle inputs from different FEM environments and automatically search for computationally expensive programs based on feedback from actual solver executions. This has some engineering value.
Critical weaknesses
W1. Limited methodological novelty, and the proposed multi-agent workflow lacks substantial technical contributions.
The core workflow of CloakFEM uses three agents to propose modification directions, generate candidates, and select candidates, followed by iterative optimization based on solver feedback. This basically follows the common paradigm of existing LLM-agent-based search and optimization methods. The FEM-specific IR and validity checks are more like domain-specific engineering adaptations rather than new search algorithms or optimization mechanisms. Therefore, I think the methodological novelty of this paper is relatively limited.
W2. The practical necessity and application value of the method are unclear.
The paper constructs inputs with high computational costs through parameter search, but the main mechanisms it uses, such as near-incompressibility, material coefficient contrast, and mesh distortion, are all known causes of numerical difficulties. Section 4.3 also attributes some results to the classical phenomenon of volumetric locking. If attackers can directly use these known mechanisms to construct computationally expensive inputs, what additional value does introducing a complex multi-agent search framework provide? The paper does not sufficiently demonstrate that CloakFEM has clear advantages over methods that directly use existing numerical knowledge. Therefore, its practical contribution and application value remain unclear.
W3. The experimental comparisons are not sufficiently fair, making it difficult to demonstrate the practical advantages of the method.
In the main experiments, CloakFEM uses FEM domain knowledge and structured modification operations, while the conventional black-box baselines mainly search over encoded parameters. Therefore, their search capabilities are not entirely consistent. Although Section 4.2.2 uses a unified restricted search space, Table 2 reports median amplification factors of only 1.038× and 1.017× for CloakFEM and Evolutionary Search, respectively, indicating a limited difference in practice. In addition, the experiments only equalize the number of FEM solver calls without considering the additional costs of extensive LLM inference. Therefore, the current results are not sufficient to demonstrate that CloakFEM has significant practical advantages over conventional search methods.
W4. The paper has obvious formatting and figure issues, and the experimental organization and result presentation are also confusing, seriously affecting readability.
The paper contains several obvious formatting and labeling problems. For example, references on pages 2 and 3 are displayed incorrectly, some text in Figure 1 is incorrectly truncated, and names such as pdeagent_cd_3d_asymmetric_beta appear without explanation. The plots and legends in Figure 4 are also unclear. In addition, the experimental settings are inconsistent, with different experiments using different data subsets, search spaces, and numbers of generated candidates, making the results difficult to compare. The experimental section also reports many secondary statistical metrics without clearly highlighting the most important results or their relationship to the core contributions. These problems significantly affect the readability of the paper and the understandability of its experimental conclusions.
Questions for authors
See W2.
4 comments
Reply to Reviewer DE4w (continued)
W3. Fair comparison and end-to-end search efficiency
Matched action space. As specified in Section 4.2.2, all three methods operate within the same bounded edit space and share the execution interface, validity checks, and query budget.
CloakFEM matters. Table 2(a) show a pronounced upper tail that is relevant to resource-exhaustion risk. In the comparison on 439 common finite seeds, CloakFEM achieves P90 amplification of 3.052× versus 1.763× for Evolutionary Search, and exceeds 3× on 10.5% of seeds versus 2.1%—approximately five times as frequently. Its maximum amplification is 12.811×, increasing linear iterations from 95 to 1,217. Thus, bounded, valid-input edits can still produce substantial computational amplification, and CloakFEM discovers these high-cost cases more reliably.Appendix A.1 provides paired statistical analysis and absolute iteration counts. WEDGE demonstrates the utility of performance-stressing inputs for exposing bottlenecks and improving code efficiency, PathoGen generates pathological inputs causing practical slowdowns. These studies motivate our emphasis on high-cost input discovery alongside median behavior.
Including LLM overhead. The equal-time comparison in Table 2(c) includes model calls and retries within the same total search deadline. CloakFEM retains higher mean amplification and a higher frequency of high-amplification inputs than evolutionary search after this overhead is included. Appendix A.1 details the protocol and outcome accounting.
W4. Case identifiers and experimental organization
We have checked the manuscript for the reported reference-display issue on pages 2-3 and would appreciate the specific citation or cross-reference location so that we can verify the exact item.
The identifier pdeagent_cd_3d_asymmetric_beta, which is a task ID, denotes a three-dimensional convection–diffusion case from PDE-Agent-Bench, with an asymmetric advection vector. Its source is YusanX/pde-agent-bench, data/benchmark_v2.jsonl, case cd_3d_asymmetric_beta; the corresponding repository IR file is docs/seed_corpus/ir_out/pdeagent/pdeagent_cd_3d_asymmetric_beta.json
We have updated Fig.2 and Fig.4 .
The experiments use cohorts selected for distinct evaluation questions and execution requirements. Under the experimental budget, this organization enables broad main evaluation and targeted controlled comparisons:
| Experiment | Cohort | Purpose |
|---|---|---|
| Primary search (Table1, Fig 4, Table 4, Fig.4) | 1,027 seeds: 577 Elmer, 444 PDE-Agent, 6 NAFEMS | Amplification prevalence and complete-system search effectiveness |
| Bounded-edit control (Table 2(a)) | 444 PDE-Agent seeds | Comparison under matched actions |
| Component ablation (Fig. 6) | 444 PDE-Agent seeds | Contribution of the model components |
| Non-LLM domain-rule comparison (Table 2(a)) | 444 PDE-Agent seeds | Added value over domain rules with incumbent memory and two-field edits |
| Cross-solver replay (Fig.5, Table 10, Fig.9-10) | 99 selected saved cases supported by the replay adapters | Reproducibility of selected inputs across implementations |
The 667-seed count-metric cohort is the finite intersection across all seven methods. Count and wall-time metrics are reported separately. Secondary distribution summaries are provided in the appendix.
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns.
Regarding W4, the PDF reference issue is clearly observable. For example, the left border of page 3 and the header "Under review as a conference paper at ICLR 2027" are incorrectly linked to the reference by Gilbert Strang and George J. Fix, An Analysis of the Finite Element Method. Similar abnormal hyperlink regions also appear on page 2.
Overall, while the additional experiments have addressed some of my concerns, I do not think they sufficiently resolve my main concerns. Therefore, I will maintain my original score.
Thank you for pointing out the abnormal hyperlink regions on pages 2–3. We have fixed this in the revised PDF and verified that the page border and running header are no longer linked. We believe this issue comes from the PDF rendering instead of our reference mistake.
We also note that you find the main concerns insufficiently resolved. Could you clarify which concerns remain insufficiently addressed so that we can improve the revised manuscript?
Reply to Reviewer DE4w
Thank you for the detailed review and for acknowledging the exploratory and engineering value of this work. We address your concerns about clarity and soundness below.
W1. Methodological contribution
CloakFEM provides a semantic search framework for heterogeneous FEM inputs under explicit validity and execution constraints. Its shared IR exposes physical inputs and supported node edits, source-specific writers reconstruct executable native inputs, and deterministic pre-execution checks and post-solve validation determine candidate acceptance. Solver methods, tolerances, execution limits, and resolution controls remain protected. The Strategy, Generation, and Selection components respectively identify edit targets, instantiate concrete magnitudes, and allocate solver queries using execution history. This design connects semantic reasoning to an auditable, solver-independent execution interface.
RQ3.1 directly quantifies the search gains from semantic generation, solver feedback, and learned selection (Figure 6(a, c); Appendix A.3). Replacing semantic Generation with a zero-knowledge generator reduces mean amplification from 127.38× to 10.25× and P90 from 188.13× to 1.46×. Removing solver feedback lowers these metrics to 46.00× and 42.09×, while removing learned Selection lowers them to 51.74× and 17.66×. These ablations hold the IR, validity policy, and query cap fixed, demonstrating substantial contributions from CloakFEM’s generation, feedback, and selection mechanisms on the evaluated cohort. We report both arithmetic mean and P90 to distinguish aggregate amplification from upper-tail behavior.
W2. Added value over known numerical mechanisms
Near-incompressibility, coefficient contrast, and mesh distortion provide useful numerical knowledge. The task addressed by CloakFEM is to turn such knowledge into effective, executable FEM program for red-teaming testing: identifying supported fields, coordinating modifications, choosing their magnitudes, and adapting to solver feedback while satisfying the acceptance constraints. A known difficult regime supplies a search direction; CloakFEM automates the seed-specific search needed to realize computational amplification.
As in computer-system red teaming, agents can turn established domain knowledge, such as buffer-overflow vulnerabilities, into concrete test inputs for effective vulnerability discovery. Similarly, CloakFEM can help FEM solver developers identify limitations in their implementations and use these findings to develop mitigation strategies.
Table 2(a) directly compares CloakFEM with a non-LLM domain-rule baseline equipped with search memory and coordinated parameter edits, serving as an explicit numerical-knowledge control. Built within the CLOAKFEM framework, the baseline follows a fixed schedule of domain-informed rules, modifying one or two parameters at predefined seed-relative magnitudes, and supports small sampled node movements. Its memory retains the best accepted input as the parent and records proposed candidates to avoid duplicates. Appendix A.1 details the baseline and bounded-edit protocol.
All three methods are evaluated on 444 PDE-Agent seeds. Table 2(a) reports results on the same 439 seeds with finite outcomes for every method, reproduced below for convenience.
Against this specific Non-LLM domain-rule baseline, CloakFEM’s its paired rate above 3× is 4.78 percentage points higher, almost douled than the baseline(95% CI [2.73, 7.06]).