acceptodds
Withdrawn from ICLR 2027

Simulate, Reason, Decide: Scientific Reasoning with LLMs for Simulation-Driven Decision Making

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Scientific simulators are increasingly being integrated into LLM-driven systems for high-stakes simulation-driven decision-making. However, existing frameworks primarily use LLMs to generate, calibrate, or execute simulators, treating them as black-box interfaces rather than as structured mechanistic systems that can be reasoned about. As a result, current approaches lack the ability to identify, represent, and reason about the assumptions and mechanisms underlying simulator behavior, limiting traceability, systematic verification, and decision justification. We introduce MechSim, a mechanism-grounded neuro-symbolic reasoning framework for executable scientific simulators. Unlike prior neuro-symbolic approaches that primarily reason over static symbolic structures, MechSim enables LLM agents to reason about the mechanisms, assumptions, and execution behavior of scientific simulators. Our framework represents simulators using a shared structured schema that captures assumptions, variables, mechanism dependencies, and execution traces. On top of this representation, LLM agents operate as constrained reasoning engines that generate structured, evidence-grounded explanations linking simulator outcomes to their underlying mechanisms. We evaluate our approach across multiple high-stakes domains and show that it improves mechanism-level explanation quality, simulator analysis, and downstream decision-making reliability.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
scientific simulation, mechanism-grounded reasoning, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_4pHdOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MechSim, a multi-agent framework that constructs Simulation Reasoning Graphs (SRGs) to represent simulator mechanisms, assumptions, and execution traces. LLM agents reason over these graphs to generate explanations and decision recommendations. Experiments across three domains report improvements in explanation quality and decision-making.

Critical strengths

The problem of mechanism-grounded reasoning over scientific simulators is important, and explicitly representing simulator assumptions and mechanisms is potentially useful.

Critical weaknesses
  • Limited novelty and weak baselines. The framework largely combines existing techniques (RAG, graph extraction, sensitivity analysis, and LLM verification). The main baselines are not designed for reasoning over executable simulators. A strong coding-agent (even just codex or opencode harness) baseline with equivalent simulator access, tools, and test-time compute is essential. The current coding-agent ablation is insufficiently characterized.

  • Potentially biased evaluation. The LLM judge explicitly evaluates explanations against the proposed SRGs, favoring MechSim's representation. The ablations also fail to isolate the contribution of SRGs from additional information, tool use, and verification.

  • Limited benchmark validity. The experiments focus on relatively simple simulators and synthetic oracle-based evaluations. Notably, Appendix F shows that ARIMA substantially outperforms all candidate mechanistic simulators on COVID-19 forecasting, raising concerns about the practical relevance of selecting among these models.

Questions for authors

How does MechSim compare with a strong coding agent or generic agentic harness?

2 comments

We thank the reviewer for the helpful comments.

Novelty. We would like to clarify that our contribution lies less in the individual components than in how they constrain LLM reasoning. MechSim represents each simulator as a Simulation Reasoning Graph and requires the LLM to reason along its paths, so that every claim cites the edges, assumptions, and evidence it rests on and can be tested by re-executing the simulator. This makes the reasoning traceable and auditable rather than free-form. The problem itself is also new: existing work uses LLMs to build, calibrate, or run simulators, and to the best of our knowledge, reasoning about the mechanisms and assumptions of executable simulators has not been studied. The SRG is a new representation as well. Unlike knowledge graphs or causal graphs extracted from text or data, it is tied to an executable simulator and brings its variables, transitions, policy bindings, assumptions, and execution traces into one schema shared across heterogeneous simulators.

Baselines. We have a similar coding-agent baseline in the paper: the variant without mechanism reasoning (Appendix C.1, Figure C.1) uses a coding agent that generates query-specific arguments, and it performs clearly worse than MechSim (0.53 vs. 0.70 in P/R@k, 4.40 vs. 2.66 in Top-k regret). We agree that a stronger coding-agent baseline is valuable, and we will add one in our revised version of the paper. However, we expect it to perform worse than MechSim for the same reason: such an agent can run simulators and cite code, but nothing checks its claims against the simulator's mechanisms.

Evaluation criteria. We respectfully disagree that our evaluation favors our method. Using LLM judges together with a smaller-scale human evaluation is the standard way to evaluate generated explanations, and our four criteria follow prior work [1, 2], with the full rubric in Appendix E.6. These criteria evaluate the explanation rather than the method: they ask whether an explanation is complete, consistent with the simulator's mechanisms, scientifically sound, and faithful to the simulation outputs.

[1] DeYoung et al. ERASER: A Benchmark to Evaluate Rationalized NLP Models. ACL 2020. [2] Liu et al. G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment. EMNLP 2023.

Thanks again for your review.

Thanks for authors' response. I still think the baselines are pooly justified and the evals are not solid enough for claiming MechSim is helpful. My score remains the same.

Reviewer_2gJqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes the MechSim framework. In the framework, one agent turns the assumptions, variables, dependencies, etc. into a graph (SRG), and another agent grounds its reasoning about the simulator outcomes in the constraints set by the graph. The aim is to improve interpretability by providing evidence-backed explanations of policy outcomes and linking those outcomes to the underlying mechanisms. The authors tested the framework on 3 evaluation tasks, and MechSim performs better than the baseline reasoning methods in most cases.

Critical strengths

I think the goal of the framework is sound, and a more interpretable and grounded LLM-agent framework for reasoning over scientific simulators would definitely contribute a lot to the community.

Critical weaknesses
  • My biggest concern is that multiple verification and validation steps within the framework are performed by LLM agents without proper evaluation. In effect, the framework relies on LLMs evaluating other LLMs, and neither the verifier nor the evaluation judge is itself validated. I think this leads to a kind of “pseudo-explainability” where the explanations look grounded in the SRG, but the link between the reasoning agent’s claims and the graph’s variables and assumptions is never independently checked.
  • It is unclear how to evaluate the quality of the constructed graph. While 23 experts rate the constructed graphs on 3 metrics (Section 5.4), there is no definition of these metrics or of the ground-truth “corrected graph.” There is also no mention of how the scores are calculated, and no report of interrater reliability among the experts, which should be standard in evaluations like this. Also, only 3 of the 15 simulators were evaluated by humans, with one graph each. As a result, the scores in Table 2 seem like arbitrary numbers, and are hard to interpret. I would like to see a more thorough human evaluation of the constructed graphs, since they are the backbone of the whole framework.
  • After the graph is constructed, I also cannot see how the framework ensures that the SRG-constrained agent strictly follows the SRG paths in its reasoning. The reasoning agent’s verifier is itself another agent. I think checking whether the reasoning path follows the constructed graph could be done with a deterministic script, so I don’t see why this needs to be done by another LLM agent. I also could not find information about the verifier agent, such as which backbone LLM it uses. There is no evidence that the verifier agent’s verdicts are themselves checked, and no interrater reliability is reported between different agents (which, in my opinion, should be very close to 1 to justify the claim that the SRG-constrained agent is indeed doing what is intended).
  • The final human expert ratings in Table 1 have the same issue as the Table 2 ratings described above, i.e., we don’t know how the numbers are calculated, what the rubric is, etc. Since this is the headline finding, a thorough evaluation with a detailed description of the metrics is necessary. The LLM judges here are also not properly evaluated. No interrater reliability is reported between the LLM judges, and the LLM judges’ scores in Table C4 disagree quite a lot with the human experts’ scores on the baseline. The LLM judges’ prompts in E.6 also leak method names, which may lead to bias.
  • Another minor issue is that Section 3.2 states that ABC is used to estimate the posterior. However, as I understand it, only the measles forecasting tasks used ABC in the experiments. One of the references for ABC (Temfack and Wyse 2025) is incorrect. This paper is about SMC (particle filtering), not ABC. The COVID-19 forecast settings returns a best-fit value instead of a distribution. I think Section 3.2 should reflect how the experiments were actually conducted.
  • Other small stuff:
    • And I think Table C2 has typos for Claude-supply chain entries, since the values are 10 times larger than the other 2 LLMs.
    • I also think there is a lot of room for improvement in the writing, especially in the abstract, Section 1, and Section 2. The text is a bit too wordy and generally quite hard to read.
Questions for authors
  • How are the LLM judges validated, and what is the interrater reliability between them? How do the authors explain the disagreement between the LLM judges’ scores in Table C4 and the human experts’ scores on the baseline? How is potential bias from including method names in the prompts in E.6 addressed?
  • How are the 3 metrics in Section 5.4 defined, what constitutes the ground-truth “corrected graph,” and how are the scores in Table 2 calculated? What is the interrater reliability among the 23 experts?
1 comment

We thank the reviewer for the detailed comments.

SRG evaluation. The criteria for the three metrics are given in Section 5.4: overall correctness measures the fidelity of the graph to the simulator's mechanisms, variable coverage measures the coverage of variables and policy levers, and relationship correctness measures the correctness of graph edges. Experts rated each metric on a 1–5 scale, and Table 2 reports the mean score over experts.

Deterministic path checking. We thank the reviewer for this suggestion and agree that it is worth exploring, but it is not trivial. The reasoning is written in natural language, so checking whether it follows the edges of the SRG first requires extracting the entities it mentions and their order, for example with named entity recognition. This step itself needs a rule-based system or another language model, so the check would not be fully deterministic. We will explore this direction in our revised version of the paper.

Rubric for Table 1. The human experts scored the explanations on the same four criteria and 1–5 scale used by the LLM judges; the full rubric is given in Appendix E.6. Each entry in Table 1 is the mean score over experts. We have computed inter-rater agreement for the LLM judges in Table C.3; for the human experts, the sample (23 experts) is too small to compute it reliably.

Thanks again for your review.

Reviewer_qyEgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MechSim converts each scientific simulator into a Simulation Reasoning Graph (SRG: variables, mechanism transitions, assumptions, execution traces, sensitivity artifacts). LLM agents calibrate simulators (ABC), run parametric and structural sensitivity analyses, retrieve literature, and produce structured explanations (interpretation, mechanism paths, evidence, claims) that a verification agent checks and refines. On three domains (COVID-19, supply chain, measles; an oracle simulator plus candidate ensembles) and three LLM backbones, MechSim is compared with Causal-Copilot, Logic-LM and Graph-of-Thoughts on simulator selection (regret), policy selection (F1@K) and explanation quality (LLM judges and 23 domain experts). It is generally best, and an ablation replacing it with a coding agent that writes query-specific arguments performs worse.

Critical strengths
  1. Well-motivated problem with a realistic, multi-domain benchmark. Heterogeneous simulators (compartmental, agent-based, inventory), held-out real forecast data (COVID forecast hub, M5), three backbones, and candid reporting (some comparisons not significant; a symbolic solver matches MechSim when rankings follow output magnitudes). This is a more realistic testbed than most LLM-for-science work.
  2. Concrete, reusable artifact. The SRG schema and released code make mechanism claims traceable, and the paired significance testing (14/18 comparisons significant) indicates the improvement over the chosen baselines is not noise.
Critical weaknesses
  1. The evaluation does not isolate the claimed cause. The paper attributes gains to SRG-constrained reasoning and verification, but the baselines only see simulator outputs (Sec. 4.2) while MechSim additionally runs its own calibration, perturbation experiments and retrieval, so gains may come from extra information and compute. The relevant control is a strong tool-using coding agent with the same information and budget; the "coding agent" ablation (0.70 vs. 0.53 P/R@k) is reported in aggregate without call/token/simulator-run budgets. Simple non-LLM references are also missing as far as I can tell (best pre-cutoff RMSE simulator, ranking by calibrated outputs, random F1@K, which is already high with 6-9 policies and K in {3,5}). Without these, the value added by the SRG/verification design cannot be judged.
  2. Explanation quality is judged mainly by subjective ratings although ground truth is available, and the claimed verification is weaker than presented. The oracle's true mechanisms are known but claims are not scored against them. Several expert-score margins are small (Measles 4.13 vs. 4.11 for GoT), inter-rater agreement is not reported in the main text, the human study uses one backbone, and the SRG study rates only MechSim's own graphs. The SRG is LLM-extracted and the verifier is an LLM; only the sensitivity analysis is executable evidence, "when available". Ground truth being another simulator also limits the "high-stakes decision" framing.
Questions for authors
  1. What are the inter-rater agreement and blinding procedure in the 23-expert study, and how many items did each expert rate?
  2. Under matched information (same calibrated posteriors, sensitivity runs, retrieved documents) and matched LLM budget, how do the baselines and the coding-agent variant perform?
  3. What are the results of best-pre-cutoff-RMSE selection, output-based policy ranking and random F1@K?
  4. Can mechanism claims be scored automatically against the oracle's true mechanisms, and which module (SRG, verifier, retrieval, sensitivity) drives the gain?
1 comment

We thank you for your suggestions.

For the first weakness, all methods share the same pipeline up to the reasoning stage: every baseline uses the same calibrated simulators and receives the same simulation outputs and simulator structure as MechSim (Section 4.2; Appendix B.1). Only the reasoning module differs (causal discovery in Causal-Copilot, symbolic solving in Logic-LM, graph-structured deliberation in GoT, and SRG-constrained reasoning in MechSim).

We agree that retrieved evidence and perturbation results are currently available only to MechSim. In the future, we will (1) provide the same retrieved evidence to all baselines, and (2) add a tool-using coding-agent baseline with the same simulators, information, and budget as MechSim but without the SRG. But we think this baseline will perform worse than MechSim: our ablation without mechanism reasoning (Appendix C.1, Figure C.1) is essentially a coding agent that generates query-specific arguments, and it already performs clearly worse (0.53 vs. 0.70 in P/R@k).

For the second weakness, our evaluation is not based on subjective ratings alone: policy rankings are compared with the oracle ranking (Table C.1), simulator selection is evaluated by regret against real post-cutoff observations (Table C.2). Both decision tasks include inter-rater agreement in Table C.3. For the expert evaluation, the sample is too small to compute a reliable inter-rater agreement.

Thanks again for your review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.