acceptodds
Under review as a conference paper at ICLR 2027

One Hypothesis Is Not Enough: Abductive Reasoning with Agentic Hypothesis Refinement over Knowledge Graphs

Abstract

Abductive reasoning over knowledge graphs (KGs) seeks a first-order logic hypothesis whose answer set explains a given set of observed entities. Since many hypotheses can explain the same observations, controllable hypothesis generators condition generation on entities, relations, or logical patterns, but they treat generation as a single step. A generated hypothesis may be well-formed and satisfy the given conditions yet still fail to explain the observations, and the model has no mechanism to detect or correct this mismatch. Closing the abductive cycle requires revising a discrete, structured hypothesis, which large language models cannot do reliably, even though they iterate readily in natural language. We propose HypoAgent, an agentic hypothesis refinement framework that treats condition signals not only as expressions of user intent but also as operators that steer generation. A Hypothesis Proposal Agent calls a small trained generator to propose a hypothesis. Root-cause analysis then uses fragment-level coverage to locate branches worth inspecting and gathers neighborhood evidence from the training graph. A Hypothesis Refiner Agent combines this diagnosis with previously generated hypotheses to produce updated conditions and directly revised hypotheses. HypoAgent outperforms one-shot generation in single-turn and multi-turn settings on BioKG, PharmKG8k, and DBpedia50, and in the unconditional setting on DBpedia50.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Knowledge Graph, Abductive Reasoning, LLM Agent

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PRtLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HypoAgent, an iterative framework for controllable abductive reasoning over knowledge graphs. Starting from observed entities and user-specified conditions, a lightweight conditional hypothesis generator proposes a structured logical hypothesis. If execution on the training graph does not sufficiently recover the observations, the framework performs root-cause analysis using two signals: fragment-level execution coverage of the current hypothesis and neighborhood probing around the observed entities. An LLM-based refiner then uses these signals and generation history to either update the conditions passed back to the structured generator or directly revise the hypothesis. Candidate hypotheses are evaluated on the training graph, prioritized by adherence to the original user conditions and then by semantic similarity to the observations. The paper evaluates the framework on BioKG, PharmKG8k, and DBpedia50 in single-turn and multi-turn controllable settings, as well as unconditional refinement on DBpedia50. HypoAgent substantially improves semantic similarity over one-shot CtrlHGen, repeated CtrlHGen sampling, and ReAct-style baselines across the reported settings. The paper also studies the contributions of fragment diagnosis and neighborhood probing, and reports lower remote-LLM usage and cost than ReAct-style agents on DBpedia50.

Critical strengths
  1. A well-motivated repair interface for structured hypothesis generation. The most interesting aspect of the paper is the use of conditions not only as user-control signals, as in prior controllable hypothesis generation, but also as an interface for iterative repair. This is a sensible division of labor: execution and graph evidence identify useful directions, an LLM decides how the proposal should change, and the specialized generator remains responsible for constructing a valid structured hypothesis. I find this specific system design novel and technically well motivated.

  2. Strong and reasonably broad full-system empirical results. HypoAgent substantially improves semantic similarity over one-shot and agentic baselines across three knowledge graphs, and the gains persist across three different LLM backends. The component ablations also show that both fragment diagnosis and neighborhood probing provide useful information. These results provide convincing evidence that the complete system is practically effective, even though the attribution of the gains to individual mechanisms is less clear.

Critical weaknesses
  1. The central claim that the gains come from diagnosis-guided refinement rather than repeated sampling is not yet cleanly established. The paper explicitly interprets the improvement over CtrlHGen best-of-k as evidence that diagnosis-guided refinement, rather than repeated sampling, is responsible for the gains. However, the appendix states that kk is matched only to the number of refinement rounds, not to the total number of candidates or generator invocations used by HypoAgent. HypoAgent additionally benefits from multiple proposal routes and adaptive execution-based selection. Therefore, the current comparison does not isolate the effect of diagnosis-guided refinement from a larger or more adaptive search budget. A matched control using the same generator, the same number of proposal calls, and the same execution-based selection, but without RCA-driven condition updates, would directly resolve this concern.

  2. The paper does not yet provide an intervention-level ablation for its most distinctive mechanism, condition-mediated refinement. The candidate-removal analysis in Tables 7–8 is performed post hoc on candidate pools produced by the full system. As the authors acknowledge, later retained candidates can depend on histories that were themselves created using condition-update candidates. Thus, this analysis shows that condition-update candidates are often useful within full-system trajectories, but not what would happen if the mechanism were absent. Since condition-mediated repair is arguably the paper's most distinctive contribution, I would like to see the system rerun with the updated-condition route disabled. Ideally, the removed proposal should be replaced by an additional unchanged-condition sample so that the number of generator calls is matched.

  3. The naming "root case analysis" is a little stronger than it is actually. The paper does not define what is a "root cause" for a "bad" hyperthesis and it may not be "bad" at all. A more proper naming should be "improvement direction diagnosis".

Questions for authors
  1. Matched repeated-sampling control. Could the authors compare HypoAgent against a CtrlHGen-style repeated-sampling baseline with the same hypothesis generator, the same number of generator invocations/candidates, and the same training-graph execution-based selection? This would materially affect my assessment because the current best-of-k comparison does not match the effective proposal/search budget.

  2. True ablation of condition-mediated refinement. Could the authors rerun the system with the updated-condition route disabled, rather than removing its candidates post hoc? A particularly clean control would replace the updated-condition proposal with an additional generation under the unchanged condition, keeping the proposal budget fixed. I consider this important because condition-mediated repair appears to be the most distinctive methodological contribution.

  3. Fairness of the underlying hypothesis generator. Please clarify whether the generator pθp_\theta used inside HypoAgent and the CtrlHGen baseline are trained with the same architecture, training data, optimization recipe, and model initialization/checkpoint. If they differ, could the authors report the differences or provide a controlled comparison using the same generator? This clarification is important for attributing improvements to the refinement framework rather than to the underlying proposal model.

  4. Clarification of the refinement implementation. Please clarify (i) how invalid or unsupported LLM-generated updated conditions are detected/handled, and (ii) the exact construction and pruning strategy for two-hop neighborhood candidates and intersection_candidates. These details are currently insufficient to determine the robustness and computational scope of the refinement procedure.

Solving my questions here can raise the final score. If none of them is addressed, the score will be lean weak reject.

Reviewer_Fg9iOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper "ONE HYPOTHESIS IS NOT ENOUGH: ABDUCTIVE REASONING WITH AGENTIC HYPOTHESIS REFINEMENT OVER KNOWLEDGE GRAPHS" addresses the problem of abductive reasoning over knowledge graphs. It proposes an agentic framework for hypothesis refinement to discover first-order logic hypotheses.

Critical strengths
  1. Principled Hybrid Architecture Coupling Specialized Generators with Agentic Steering: By treating control conditions as both user-intent constraints and iterative steering operators, HypoAgent combines the inductive bias and syntax guarantees of a trained 6-layer Transformer generator with the high-level semantic reasoning and error diagnosis of an LLM. Table 1 shows this hybrid design substantially outperforms both pure LLM ReAct agents (Jaccard 0.224 on BioKG) and one-shot neural generators (Jaccard 0.718).
  2. Comprehensive Diagnostic Feedback via Dual-Perspective RCA: The combination of internal fragment coverage diagnosis (which isolates faulty conjuncts) and external KG neighborhood probing (which discovers missing relational links) provides actionable diagnostic signals. The ablation study (Table 4) shows that removing RCA causes severe performance drops across all three benchmarks (-14.6 Jaccard points on BioKG, -18.3 on PharmKG8k, -11.8 on DBpedia50), proving that structured execution diagnosis is the central driver of refinement gains.
Critical weaknesses
  1. Scalability in Large-Scale KGs: In Section 5.1, the evaluation protocol filters the test benchmarks to retain only queries whose complete answer sets contain at most 32 entities on GtestG_{test}, following prior benchmark protocols. The author-reported BioKG scaling study evaluates neighborhood probing for observation sets up to 1,024, including high-degree nodes. Intersection-based probing and top-10 output limits keep probe outputs small, bu probing latency remains 42.5–53.1 seconds despite median latency below 0.6 seconds. Bounded prompt size therefore does not establish bounded intermediate computation, and the study does not establish end-to-end hypothesis-generation accuracy or runtime for larger observation sets. This limits the evidence for scalability beyond the main benchmark regime.

  2. Limited Evidence for Reactive Multi-Turn Recovery: The main aggregate multi-turn evaluation in Section 5.4 uses scripted target-reaching sequences with fixed subsequent requests. The supplied reactive dialogue demonstrates adaptation to feedback about preceding hypotheses, but one trajectory does not establish robustness across users, erroneous hypotheses, or ambiguous corrections. The evidence supports scripted sequence tracking and a qualitative example of reactive interaction, while broader conversational error-recovery claims remain insufficiently evaluated.

Questions for authors
  1. Generalization Beyond Scripted Target-Reaching: Does the reactive-dialogue evidence extend beyond the illustrated trajectory? If additional evaluations exist, please specify their sample size, feedback source, and recovery or condition-adherence results. This would help assess robustness to open-ended feedback that directly criticizes the preceding hypothesis.

  2. Behavior on High-Degree Nodes and Dense Subgraphs: What operations dominate the reported 42.5–53.1-second probing latency, and are expansion caps or anchor pre-filtering implemented and evaluated, or proposed for future work? Any available end-to-end runtime and accuracy measurements on the larger observation sets would help establish the practical significance of this tail cost.

Reviewer_r9XROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem in knowledge graph abductive reasoning that “controllable hypothesis generation is performed only once, and a hypothesis that satisfies the conditions may not necessarily explain the observation.” It proposes HypoAgent: a hypothesis proposal agent invokes an in-domain trained 6-layer Transformer to generate structured hypotheses; root cause analysis locates errors from two perspectives—fragment-level coverage and knowledge graph neighborhood; then an LLM-driven hypothesis revision agent directly rewrites the hypothesis or generates update conditions, feeds them back to the generator for iterative revision, and selects candidates according to user condition compliance and training graph Jaccard. In the single-round, multi-round, and unconditional settings on BioKG, PharmKG8k, and DBpedia50, its answer-set similarity outperforms single-generation and repeated-sampling baselines.

Critical strengths

This paper proposes that in abductive reasoning, a single hypothesis in hypothetical reasoning is insufficient, and conditional hypothesis revision is required. This is common knowledge in the commonsense domain. The author introduces this common sense into the hypothetical reasoning of abductive reasoning, and the motivation is reasonable.

Critical weaknesses
  1. The method in this paper relies on an in-domain trained generator for conditional hypothesis generation, but the training details and training results of this generator are not disclosed, making it difficult for readers to reproduce. It is recommended that more training details and results be disclosed.

  2. This paper mentions that LLMs have difficulty generating strongly structured hypotheses and that the ReAct baseline performs poorly. However, it lacks necessary comparative experiments as conditional hypothesis generators, such as fine-tuning LLMs with a small number of parameters, constrained decoding with LLMs, and LLMs combined with answer templates.

  3. This paper does indeed perform edge-level train/test isolation within the same graph, and the test graph is not accessed during inference; this point is clear. However, this isolation only supports transductive edge-holdout prediction and does not support cross-domain generalization; moreover, the specific uses of the sampled graph for training samples and the validation graph are not fully explained.

Questions for authors

1.The performance of this paper depends on a 6-layer Transformer conditional generator trained on the target graph, but the paper only discloses the number of layers, the optimizer, the learning rate, weight decay, and the number of epochs for each dataset; it does not report the training quality of the standalone generator, making it difficult for readers to judge whether it has been sufficiently trained or to reproduce the results. Q1.1: Please supplement the complete training configuration of the generator: hidden dimension, number of attention heads, FFN dimension, batch size, loss function, vocabulary size, dropout, positional encoding, maximum sequence length, and training sample size. Q1.2: Please report the training/validation curves of the generator as a standalone component, including training loss, validation loss, convergence behavior, and the generation success rate or answer-set Jaccard on the validation set, rather than only the test metrics of the final system.

  1. This paper mentions that LLMs have difficulty generating strongly structured hypotheses and that the ReAct baseline performs poorly. However, it lacks necessary comparative experiments as conditional hypothesis generators, such as fine-tuning LLMs with a small number of parameters, constrained decoding with LLMs, and LLMs combined with answer templates. Q2.1: Why is a fine-tuned small-parameter LLM (e.g., at the 1B–7B scale) not used as the conditional hypothesis generator and compared with the 6-layer Transformer in this paper under the same training data and the same inference budget? Q2.2: Have you tried grammar-constrained decoding or constrained decoding to make a general-purpose LLM output only valid first-order logic queries? If so, what were the results; if not, why not try it? Q2.3: Have you tried the approach in which the LLM only generates conditions or drafts, which are then converted into logical formulas by templates/rules? How does this compare with the division of labor in this paper, where the LLM performs planning and revision while a dedicated generator writes the logical formulas?

  2. This paper does indeed perform edge-level train/test isolation within the same graph, and the test graph is not accessed during inference; this point is clear. However, this isolation only supports transductive edge-holdout prediction and does not support cross-domain generalization; moreover, the specific uses of the sampled graph for training samples and the validation graph are not fully explained. Q3.1: The paper only performs a single 8:1:1 edge split. Could the authors supplement at least 3–5 random splits or multi-seed experiments and report the mean, standard deviation, or confidence intervals to show that the conclusions do not depend on a particular split? Q3.2: Could the authors supplement cross-dataset experiments: train the generator on one dataset, freeze it, and then directly test it on another dataset? This would directly test whether the method relies on in-domain training.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.