acceptodds
Under review as a conference paper at ICLR 2027

OntoAct: Hypothesis-Based Neuro-Symbolic Agents Through Ontological Abstraction

Abstract

Learning from experience requires recovering reusable knowledge and determining when it applies to a new situation. We introduce OntoAct, a hypothesis-based neuro-symbolic framework built on ontological abstraction. OntoAct turns individual experiences into axioms over entity types, relations, and conditions, making their underlying patterns reusable across situations. A large language model (LLM) uses this world knowledge to propose hypotheses and actions, while a symbolic verifier assesses their supporting conditions against current observations. Verification identifies missing evidence, and new observations and intervention outcomes update hypothesis assessments and guide subsequent actions. We also introduce Tacit-FLE, a benchmark built on the Factorio Learning Environment (FLE) for diagnosing and repairing hidden faults in existing factories using inherited operational records. Its synthetic maintenance corpus distributes evidence across records, and an executed repair may leave the fault unresolved, requiring agents to reassess their diagnoses through interaction. Across Tacit-FLE, Mars, and ALFWorld, OntoAct achieves leading task performance with efficient interaction and inference.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Neuro-Symbolic AI, Ontology, agentic ai

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SVnQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces OntoAct, a neuro-symbolic agent framework that extracts reusable knowledge from historical experience through ontological abstraction. The framework converts operational records into typed assertions and conditional axioms. During interaction, an LLM proposes hypotheses and actions, while a symbolic verifier checks their consistency with the learned knowledge and current observations. Hypotheses are updated as new evidence and action outcomes become available.

The authors also introduce Tacit-FLE, a benchmark built on the Factorio Learning Environment for diagnosing and repairing hidden faults using historical maintenance records. Experiments are conducted on Tacit-FLE, Mars, and ALFWorld, with comparisons against ReAct, Reflexion, IfR, NeSyC, and WALL-E. The reported results show improvements in task success, reduced unsuccessful interventions, and lower inference costs in several settings.

Critical strengths
  1. Meaningful problem formulation and execution-based evaluation.

The paper addresses the important problem of transferring knowledge from historical experience to new situations while verifying whether previously learned conditions remain applicable. Tacit-FLE provides an interesting evaluation setting in which an executed repair does not necessarily resolve the underlying fault. Measuring both successful repairs and unsuccessful interventions offers more meaningful evidence than relying solely on textual assessments.

  1. Detailed diagnostic analyses.

The paper provides extensive supplementary analyses, including hypothesis tracking, candidate fault coverage, resource usage, and action-level verification diagnostics. These analyses are useful for understanding the behavior of the proposed system and provide some evidence that explicit knowledge verification can reduce unnecessary interventions.

Critical weaknesses
  1. Serious concerns about the validity and fidelity of a key neuro-symbolic baseline.

My primary concern is the evaluation of NeSyC, a closely related neuro-symbolic learning framework.

In Table 1, NeSyC achieves 0% repair success across all 104 Tacit-FLE evaluation episodes. More importantly, Appendix D.1 reveals that this result is not simply due to poor fault diagnosis or unsuccessful repair attempts. Across the 3,000 evaluation turns on the small and medium suites, every executed action is a waiting action. Among the 64 initial planner outputs, 47 contain no plan and the remaining 17 contain only a waiting action.

This is a substantial concern. The reported behavior suggests that the evaluated NeSyC implementation was effectively unable to perform the intended repair task. Such a result does not provide meaningful evidence that OntoAct outperforms NeSyC in knowledge abstraction, hypothesis evaluation, or fault diagnosis.

Appendix C.2 states that the authors reconstructed NeSyC's induction and rule-application procedures using the released implementation as a reference and introduced additional adaptations to connect its symbolic planning mechanism to the target environments. However, the paper does not establish that the reconstructed baseline was independently validated as a functioning implementation before comparing its performance with OntoAct.

A similar issue appears in ALFWorld. Appendix D.5 reports that the reconstructed NeSyC completes 0/134 tasks under standard observations, with all 6,566 executed steps being idle actions. Providing additional initial-state information raises its performance to 35/134, but this does not resolve the fundamental concern about the validity of the baseline adaptation.

I appreciate that the authors disclose these failures rather than concealing them. Nevertheless, including a baseline that consistently fails to produce meaningful actions substantially weakens the interpretation of the comparison, particularly because NeSyC is one of the most directly relevant prior neuro-symbolic approaches.

The other baseline comparisons are not automatically invalidated by this issue. However, the evidence presented is insufficient to establish a reliable advantage over this important line of prior work. A functional and validated baseline implementation is necessary before the reported comparative conclusions can be considered convincing.

  1. Insufficient isolation of the proposed methodological contributions.

The main conceptual components of OntoAct—LLM-based knowledge extraction, symbolic rule representations, hypothesis evaluation, and action verification—are closely related to existing neuro-symbolic agent frameworks, including NeSyC and WALL-E.

The proposed distinction is the use of ontological abstraction to generalize historical experiences into reusable relational and conditional knowledge, together with an explicit hypothesis-management mechanism.

However, the paper does not provide sufficiently direct component-level ablations to establish the independent contributions of these mechanisms.

In particular, it remains unclear how much of the improvement comes from ontological abstraction itself, rather than the predefined domain-specific schemas, the quality of extracted rules, the hypothesis ledger, or the action verifier. Appendix B further shows that the supported knowledge representations and verification procedures differ substantially across Tacit-FLE, Mars, and ALFWorld.

Without controlled experiments that remove or replace these components while preserving the remaining system, the results primarily demonstrate the effectiveness of the complete engineered pipeline. They do not convincingly establish the necessity or generalizable value of the proposed ontological abstraction.

This concern is particularly important given the substantial overlap between OntoAct and prior neuro-symbolic approaches. The absence of direct mechanism-isolation experiments, combined with the questionable validity of a key baseline, makes the claimed methodological advancement difficult to assess.

Overall, although the proposed system achieves promising results in the reported environments, I do not find the current experimental evidence sufficiently reliable or the independent methodological contribution sufficiently established to support acceptance.

Questions for authors

Q1. Can the authors demonstrate that their NeSyC reconstruction is a valid and functional implementation of the original approach?

Appendix D.1 reports that NeSyC executes only waiting actions across 3,000 evaluation turns on the small and medium Tacit-FLE suites. What specifically causes its planning mechanism to return no executable plans or only waiting actions?

Can the authors provide implementation validation results, such as successful execution on the original benchmark or controlled diagnostic cases demonstrating that the reconstructed planner can generate meaningful actions under its intended assumptions?

Q2. How do the authors distinguish a baseline implementation or environment-adaptation failure from a genuine methodological limitation of NeSyC?

The reconstructed NeSyC also fails on all ALFWorld tasks under standard observations and improves only after additional initial-state information is provided. Does this indicate that the baseline requires different observation or planning interfaces?

Can the authors clarify whether the evaluated configuration preserves the essential operating assumptions of the original method and provide evidence from a functional adaptation under comparable conditions?

Q3. What is the independent contribution of ontological abstraction and explicit hypothesis management?

Can the authors provide controlled ablations that independently remove ontological abstraction, the hypothesis ledger, and action verification while maintaining the same backbone, environment access, and relevant knowledge?

In particular, would the framework retain similar performance if its learned relational knowledge were represented as ordinary structured rules without the proposed ontology?

These results are necessary to determine whether the improvements arise from the claimed methodological contributions or from the overall system configuration.

4 comments
Reviewer_SVnQOpenReview ↗

Follow-up on the ASP Representation Control and Potential Rating Revision

Thank you for conducting the additional ASP-based experiment. I appreciate the authors' continued efforts to address my concerns through targeted experiments.

This comparison directly addresses my previous request for an alternative structured representation retaining symbolic verification and hypothesis management. The three-run results and task-level breakdowns are valuable.

The reported overall improvements are substantial: +23.8, +39.9, and +52.5 percentage points across Small, Medium, and Large settings. If the comparison is fair and the ASP baseline functions correctly, these results would provide stronger evidence for OntoAct's methodological contribution.

However, I would appreciate clarification of two remaining issues.

1. Equivalence of structural priors and available information

OntoAct uses a predefined ontology schema specifying entity types, relations, observable attributes, and admissible axiom templates.

Does the ASP control receive equivalent information, including entity types, relation constraints, diagnostic templates, and action semantics?

Does its induction prompt explicitly encourage variable-based generalization over entity roles and types?

This matters because ASP supports reusable variable-containing rules. If the ASP baseline receives weaker structural guidance, the performance gap may partly reflect differences in domain knowledge or inductive bias rather than representation alone.

I do not require identical induction algorithms, since structured induction is part of the proposed contribution. However, the comparison should clarify which advantages arise from the representation and induction mechanisms versus differences in supplied prior knowledge.

2. Functional validation and failure analysis of ASP

The task-level results reveal an interesting pattern: ASP outperforms OntoAct on Small Chain (69.2% versus 56.4%) but performs substantially worse on Conjunction, declining from 33.3% to 15.2% across scales, while OntoAct achieves 100%.

The explanation that induced ASP rules predominantly reference specific units is plausible, but requires supporting evidence.

Could the authors provide representative rules from successful Chain and failed Conjunction cases and clarify:

  • Whether reusable variable-containing rules are successfully induced;
  • Whether clingo correctly grounds and validates these rules;
  • Whether hypothesis management applies them to new entities;
  • Whether failures arise during induction, verification, hypothesis selection, or execution?

Representative execution traces or brief stage-level diagnostics would help establish that the ASP baseline functions as intended.

Additionally, ASP reports fewer overall spurious interventions. Could the authors clarify whether this reflects fewer repair attempts or more conservative behavior? Reporting the proportion of episodes without repair attempts would help interpret this result.

Updated assessment and potential rating revision

The rebuttal has substantially strengthened the paper. My earlier concerns about missing component ablations and unexplained NeSyC behavior have been meaningfully addressed. The ASP comparison also directly targets my remaining concern about the independent contribution of ontological abstraction.

I am now seriously considering increasing my rating from 2 (Weak Rejection) to 3 (Weak Acceptance).

My decision depends primarily on whether the ASP baseline receives comparable domain information and whether its induction, verification, and hypothesis-management components function correctly.

If these points are satisfactorily clarified, I would regard the new results as substantial evidence for OntoAct's contribution and would be inclined to update my recommendation.

Thank you again for the constructive discussion and additional experimental effort.

Reviewer_SVnQOpenReview ↗

Clarifications on the ASP Control and Remaining Interpretation

Thank you for these concrete examples and the repair-activity analysis. The successful reuse of a variable-containing Chain rule substantially addresses my concern that the ASP condition might be restricted to instance-specific knowledge. The Conjunction traces also make the distinction between proposing a correct target and providing admissible diagnostic support much clearer.

I appreciate the explicit clarification that this comparison evaluates representation together with structured induction, rather than representation alone. This is a reasonable scope for the experiment. Please retain this qualification in the revised claims: the observed target-specific rule heads characterize the evaluated induction pipeline, not an inherent limitation of ASP.

The repair counts also show that ASP is not simply inactive in most episodes. However, fewer repairs per episode can contribute to lower SI, and the current counts do not establish how much of the SI difference reflects repair volume versus intervention accuracy. Please present success, SI, and executed repairs together and avoid attributing the SI difference to a single explanation. A descriptive spurious-repair fraction from the existing logs would further help, provided its event definition is consistent.

Please incorporate the induction prompts, representative rules, and execution traces into the revised manuscript or supplement, as proposed. These additions substantially reduce my concerns about the ASP control. Together with the previously reported results, they make me inclined toward 3 (weak accept), with the remaining reservation concerning the attribution of gains to representation versus structured induction.

Reviewer_SVnQOpenReview ↗

Final Assessment of the Revised ASP Comparison

Thank you for incorporating the additional analyses into the revised manuscript.

I have reviewed the updated Section 5.3, Table 8, and Appendix E.5. The revision satisfactorily addresses my main outstanding concerns about the ASP comparison.

In particular, reporting success, SI, executed repairs, and the spurious-repair fraction together makes the interpretation substantially clearer. The results appropriately distinguish reduced intervention volume from intervention accuracy. The representative rules and execution traces also demonstrate that the ASP condition can generalize across targets, while identifying specific limitations of the evaluated induction pipeline.

I also appreciate the explicit qualification that the comparison evaluates representation together with structured induction, rather than establishing an inherent limitation of ASP.

The remaining distinction between representation effects and induction-procedure effects is a methodological limitation, but it is now appropriately acknowledged and does not invalidate the overall system-level contribution.

Given these substantive additions and clarifications, I am updating my rating to 3 (Weak Accept).

Thank you for the careful and constructive discussion throughout the review process. Your questions helped us strengthen the experimental evidence and clarify the scope of our contribution. We appreciate the time you devoted to reviewing the additional analyses and revised manuscript.

Reviewer_dM11OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces OntoAct, a neurosymbolic agent framework that uses ontology abstraction to translate historical experience into structured knowledge for decision-making. The framework extracts entity relations and conditional rules, together with their provenance, from maintenance logs or interaction trajectories. A language model then proposes hypotheses and actions based on the current observation, while a symbolic module checks their structural support and applicability conditions. For fault diagnosis, the system maintains explicit hypotheses and updates their status using subsequent observations and intervention outcomes.

The paper also introduces Tacit-FLE, a benchmark that evaluates agents' ability to use historical records to diagnose and repair hidden faults through dependency-chain, conditional-conjunction, and troubleshooting tasks. The reported experiments show competitive performance and fewer ineffective interventions, with additional evaluations on MARS and ALFWorld examining the framework's applicability beyond Tacit-FLE.

Critical strengths
  1. An explicit and inspectable approach to using historical experience. OntoAct represents structural relations, applicability conditions, and hypothesis states explicitly, allowing the agent to identify missing evidence and revise its judgments in response to action outcomes. This provides a concrete mechanism for determining when past experience is applicable to the current situation, which is a meaningful contribution to experience-driven agents.

  2. Evaluation that examines behavior and cost beyond task success. The paper analyzes candidate coverage, hypothesis retention, repeated and incorrect repairs, action reselection, and executed commands. It also separates knowledge construction costs from evaluation costs and breaks down model calls by role in ALFWorld, attributing much of the additional call count to outcome prediction. These analyses make the behavioral and efficiency claims easier to assess.

Critical weaknesses
  1. The contributions of the core mechanisms are not sufficiently isolated. OntoAct combines typed knowledge, diagnostic schemas, a hypothesis ledger, pre-execution condition checks, post-failure hypothesis updates, and feedback on repeated actions. The analysis of different language models does not establish which of these mechanisms account for the gains or whether they are complementary. More targeted ablations are needed to distinguish the benefits of ontology abstraction, language-model hypothesis generation, and symbolic checking. This limits the conclusions that can be drawn about why the framework works.

  2. The diagnostic analyses do not fully separate pre-intervention evidence from post-intervention refinement. Figure 3 and Appendix D.7 report target precision among retained hypotheses and explicitly include updates after unsuccessful repairs. These are appropriate measurements of the complete interactive agent. However, the analyses do not isolate how much of the improvement arises from observation-based diagnosis before repair versus the explicit outcome-based hypothesis-refutation rule. Although Table A26 provides candidate coverage at different ranks, it is not aligned to the first executed repair. An intervention-aligned analysis, together with a controlled ablation of post-failure hypothesis refutation, would strengthen the attribution of the gains in repair success and reduced ineffective interventions.

Questions for authors
  1. Can you provide controlled ablations that isolate the main contributions of ontology abstraction, hypothesis tracking, pre-execution condition checking, and post-failure updates? In particular, which mechanisms account for the reduction in ineffective interventions, and what evidence supports their complementarity?

  2. Given the update and measurement protocols in Appendices B.3 and D.7, could you report target precision or true-fault coverage immediately before the first executed repair? A controlled comparison that disables only the explicit post-failure hypothesis-refutation rule, while preserving access to the same environment feedback, would also help distinguish the contribution of pre-repair diagnosis from that of structured use of intervention outcomes.

Reviewer_UqJYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces OntoAct, an LLM agent that learns from an inherited corpus of logs or transitions in two steps. First, an LLM extracts typed ground assertions and conditional axioms under a fixed, hand-specified schema per environment, and a deterministic admission step checks them for typing, source citation and corpus support or counterexamples. Then, during interaction, the LLM proposes structural claims, diagnostic hypotheses with observable condition atoms, and an action. A symbolic layer evaluates the atoms in three-valued logic, marks each hypothesis confirmed, open or refuted, and backs a repair action only when a confirmed hypothesis supports it. Unsuccessful repairs refute the hypothesis that motivated them. The authors also contribute Tacit-FLE, a fault diagnosis and repair benchmark built on the Factorio Learning Environment. It has three task types, chain, conjunction and elimination, at three plant scales, with 104 evaluation episodes and synthetic maintenance corpora. With GPT-5.6 Luna, OntoAct reaches 78.6, 83.3 and 90.0 percent repair success against at most 66.7 for the best baseline, with far fewer spurious interventions. It also has the best seven-world average Score on Mars and solves 132 of 134 ALFWorld tasks with about half the LLM calls of WALL-E. The appendices add a lot of supporting analysis: candidate coverage, hypothesis retention and refutation, cost accounting, and per-seed Mars rewards.

Critical strengths
  1. The intervention-efficiency result on Tacit-FLE is large, consistent and well instrumented. OntoAct cuts spurious interventions by 70 to 91 percent against IfR at every scale, for example 1.6 versus 17.8 per episode on large plants. Its executed replacement precision is 25 to 36 percent, against at most 9.9 percent for every baseline. The diagnostics in Appendix D.7 go past outcome metrics: a ledger-level analysis shows 95.6 to 99.6 percent of off-target hypotheses being refuted, and the coverage-at-three gains over IfR have world-cluster bootstrap intervals that exclude zero. In a maintenance setting where each wrong intervention has a real cost, this is the right thing to measure. It is the clearest evidence I have seen that hypothesis-level verification changes agent behavior rather than only success rates.

  2. Tacit-FLE is a thoughtfully designed benchmark. Its three task types map to distinct inferential demands drawn from model-based diagnosis: composing local dependencies, combining a precedent with a plan and a reading, and elimination. Evaluation repairs are held out of the corpus, identifiers are anonymized, and an unsuccessful repair does not end the episode. That last choice makes repeated wrong interventions visible, which success-only benchmarks such as standard ALFWorld cannot show. The construction checks and the exact regeneration of all 45 corpora in Table A7 make it a credible community resource, assuming the promised release happens.

Critical weaknesses
  1. There are no ablations, so the main causal claim, that ontological abstraction plus symbolic verification produces the gains, is not established. I searched appendix. The paper never reports OntoAct without the verifier gate, without admission checks, with free-form rather than typed hypotheses, or without the corpus. The only controls are the ALFWorld initial-state disclosure and the model sweeps. Page 24, line 1269 mentions that "The no-corpus control omits the records", but it never defines this control. It most likely refers to the ReAct baseline, and no OntoAct no-corpus variant is reported. This matters because OntoAct carries several hand-built components that baselines do not get. The paper discloses these as given representation in Table A8. The observation map turns stock trends into inflow readings by a "fixed interpretation of public observations, rather than an induced axiom", per page 31, lines 1671 to 1672. The reasoning context lists "the stopped units that no recorded pattern explains", and an out-of-pattern explanation "is marked as supported when its cause is one of these unexplained units", per page 27, lines 1431 to 1433. That second piece is essentially an elimination heuristic written into the scaffold, and it is how OntoAct handles elimination faults that the corpus deliberately omits. Repairs also need a confirmed hypothesis to pass the gate. Since the gate is soft, the paper should report how often unbacked repairs still executed. Appendix B.3, lines 1442 to 1443, says that at the proposal limit, which is two on Tacit-FLE, the final proposal is passed to the executor. Without a baseline that gives ReAct the same structured context, and without a variant of OntoAct with the verifier switched off, I cannot separate the gains from ontology induction from those due to the engineered context and the gate. For a framework paper whose title names ontological abstraction as the mechanism, that gap is decisive.

  2. Benchmark, schema and method all come from the same authors, and the statistical support is thin for a single-run protocol. The Tacit-FLE schema closes the entity types and relation names, and the authors design both the diagnostic pattern forms and the fault families: supply cuts and off-plan configurations, per page 16, lines 831 to 832, and page 26, lines 1367 to 1374. So the setting favors a method that is handed the right vocabulary. Tacit-FLE results come from one run per episode with the provider's default temperature, with no repeated seeds reported, and some cells are tiny: 7 small conjunction episodes and 8 small and 8 medium elimination episodes, per Table A6. Episodes within a world share a corpus, so they are not independent. On small plants the gap over IfR is 22 versus 18 of 28. On Mars, the claim that OntoAct "retains the highest mean Score" across model conditions comes with no dispersion for Score and rests on five seeds per world. The Reward margins are not separated: under LunaR the Reward Average is 8.35±1.26 for OntoAct against 8.31±0.68 for IfR. The seven-world average follows the convention of Zhou et al. and including Default would not change the ranking. Still, on Default, ReAct* reaches a Score of 68.27 against 30.26 for OntoAct under LunaR. Finally, the Mars verification story in Table 5 is narrower than the main text suggests. Zero violations hold only for the shared conditions, which come from 22 pair-specific matches made by hand per Appendix D.4, lines 2103 to 2105. Page 40, lines 2156 to 2158 reports that across OntoAct's 781 issued crafting or placement commands, there are 363 violations of real resource requirements, all within the 622 commands whose action has no typed cost rule. The main text gives this a single clause, "broader resource-rule coverage remains incomplete" on page 8, lines 429 to 430. It should be stated as a coverage limitation of the induced ontology.

Questions for authors
  1. What are OntoAct's results on medium Tacit-FLE with the verifier gate disabled, so that proposals execute directly while the hypothesis ledger is still shown? What exactly is the no-corpus control on line 1269, and what does OntoAct score without the corpus? These numbers would largely settle Weakness 1.
  2. How many of OntoAct's executed repairs on Tacit-FLE were passed to the executor at the proposal limit without a backing confirmed hypothesis?
  3. How does ReAct or IfR perform when given the same structured context OntoAct sees: the inflow-mapped readings, the list of unexplained stopped units, and the unresolved-condition-to-probe mapping? If most of the gap closes, the contribution shifts from ontology induction to context engineering.
  4. How much of the Tacit-FLE schema and template design was tuned on the five development worlds per cell, and was any of it revised after looking at evaluation worlds? Would the same schema transfer to a new fault family, such as power faults injected at test time, without edits?
  5. Can you report Tacit-FLE success and SI over at least three independent runs, with paired significance against IfR per scale?
  6. Can the schema itself be induced? The conclusion acknowledges this, but some evidence of robustness to a coarser or misspecified schema would make the "ontological abstraction" framing much more convincing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.