acceptodds
Under review as a conference paper at ICLR 2027

NeSyFS: A Neuro-symbolic Fast-Slow Thinking Framework for LLM Agent under Partial Observability

Abstract

Recently Large Language Models (LLMs) have been increasingly deployed as autonomous agents in applications such as self-reflection, retrieval-augmented generation, and scientific discovery. In these settings, agents must act based on limited observations rather than full environmental states, leading to partial observability. This introduces several key challenges: belief state inference, task objective misalignment, and planning under uncertainty. Prior approaches typically condition actions on full or summarized action-observation histories whose redundant and irrelevant information can mislead the decision making of LLM agent. Inspired by human cognition, we propose a novel Neuro-Symbolic Fast-Slow thinking (NeSyFS) framework for LLM agent, addressing the challenges introduced by partial observability in a unified approach. We use a knowledge graph (KG) to represent the belief state, providing triplets as context for every module of NeSyFS. The fast-thinking module performs reactive action, while slow-thinking conducts a new uncertainty-aware planning by following the high-level structure of twisted sequential Monte Carlo (TSMC) algorithm. To mitigate the misalignment of task objective, a reflection module is used to reflect fast-thinking actions, and also switches to the slow-thinking module whenever reactive actions repeatedly fail. Experiments on three representative benchmarks, i.e. ALFWorld, Webshop, and ScienceWorld, demonstrate significant advantages over previous methods.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
neuro-symbolic learning, llm agent, reflection, planning, twisted sequential Monte Carlo

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_maXtOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Neuro-Symbolic Fast-Slow thinking framework (NeSyFS) for LLM Agent to address the problems of partial observability, such as belief state inference, task objective misalignment, and planning under uncertainty. It contains a knowledge graph to represent the belief state, the fast-thinking module to perform reactive action, and the slow-thinking module to conduct new uncertainty-aware planning. The experimental results show that the proposed method has advantages on the ALFWorld, Webshop, and ScienceWorld over baseline methods.

Critical strengths
  1. The fast-slow thinking framework to address partial observability is interesting.
  2. The use of different neuro-symbolic representations for distinct states helps separate information for LLM agents.
  3. The results of three different benchmarks show that the proposed method performs better than baselines.
Critical weaknesses
  1. The LLMs only use GPT-5 and Llama families. It lacks generalization.
  2. The experiments need to add some ablation studies, such as the hyperparameter of K and every component of NeSyFS.
  3. The proposed method maybe adds inference time.
Questions for authors
  1. What are the differences between History, Belief State, and KG?
  2. How are the weights of particles trained in the LLM agent?
  3. How long is the inference time and how many tokens does the proposed method?
  4. If the mismatch between the LLM proposal distribution qq and the prior transition distribution p0p_0 in the task, what does happen for the method?
2 comments

Thanks for your time on our work and helpful comments. Our response to the weakness and questions is as below.

Response to weakness 1: Due to the limit of research budget, we cannot evaluate our method on other models. We think GPT and Llama are representative enough.

Response to weakness 2: This paper includes ablation studies designed to investigate the contributions of individual components. Specifically, Figure 5 compares the performance of various context representations—knowledge graphs, interaction histories, and aggregated belief states—both with and without a "reflection" mechanism, illustrating the distinct benefits provided by knowledge graph-based memory and the reflection process itself. Furthermore, Appendix E.3 is dedicated to evaluating the "slow thinking" module.

The ablation study of different K values is added as Appendix D.6 in the revised version of this paper.

Response to weakness 3: We acknowledge that maintaining a knowledge graph, performing reflection, and executing planning processes entail additional computational overhead. However, NeSyFS is designed to strike a balance between computational efficiency and decision quality by adaptively activating the "slow thinking" mechanism. In particular, the system initiates the more resource-intensive planning process only when the actions taken via "fast thinking" repeatedly fail during the reflection phase, thereby avoiding unnecessary planning at every decision step. Additionally, knowledge graph-based retrieval provides a concise context that is highly relevant to the task; compared to processing lengthy interaction histories, this approach helps reduce token consumption. Analyses regarding computational overhead and token costs are provided in Appendices E.4 and E.5.

Answer to Q1: History is the sequence of past observations and actions within currrent episode. Belief state is estimated hidden state in partially observable environment. KG is the representation of hidden state in our approach.

Answer to Q2: In NeSyFS, particle weights are not learned through model training; instead, they are computed dynamically during the planning process. Specifically, for each particle, a "progress detector"—based on a large language model (LLM)—evaluates whether its predicted state transition advances the completion of the task, outputting a binary signal (δ∈{0,1}\delta \in \{0, 1\}) accordingly. The particle weights are then calculated using the formula ω=exp⁡(λδ)\omega = \exp(\lambda\delta), where the parameter λ\lambda serves to adjust the degree of preference for "promising" trajectories. These weights determine the resampling probabilities, ensuring that particles demonstrating positive progress toward task completion are selected more frequently, while still maintaining the possibility of exploring other trajectories. This mechanism eliminates the need to train a separate value function or reward model, and it also reduces reliance on fine-grained evaluations by the LLM—a type of assessment that is often unreliable.

Answer to Q3: Please see the last section of Appendix.

Answer to Q4: As discussed in Section 3.3, our simplified weighting rule follows the base-model-as-proposal setting of TSMC (Zhao et al. 2024), where the proposal distribution (q) equals the base distribution (p_0), allowing their likelihood ratio to cancel.

The NeSyFS paper can assume q=p0q=p_0 only if p0p_0 is defined as the transition distribution induced by the same pretrained LLM used as the proposal model. It cannot generally assume that the LLM proposal distribution equals the true environment transition distribution. However, since the transitions (next observations) in the slow-module planning are generated by the same LLM as the proposal model, we can assume q=p0q=p_0, and we don't need to consider the case q!=p0q!=p_0

Thanks for your responses. My concerns are resolved. I will keep my positive score.

Reviewer_ZSw6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes NeSyFS, a fast-slow thinking framework for LLM agent under partial observability. The ReAct style fast system maintains a knowledge graph of the state of the environment and directly interacts with the environment. The graph consists of triplets about relationships and attributes of entities and events. NeSyFS monitors and appends newly acquired facts, and prunes outdated triplets. At each fast round, the fast system proposes reactive action. If the action passes the reflection module, then the agent acts reactively in the environment. If the reactive actions keep failing the reflection judge for K times, the slow thinking module takes the charge and conducts a search for the optimal action with TSMC. NeSyFS shows advantages on LFWorld, Webshop, and ScienceWorld.

Critical strengths
  1. The paper rises to a well-motivated problem in applications and present an intuitive, reasonable framework.
  2. The paper presents results across three benchmarks and demonstrates significant improvements.
Critical weaknesses
  1. Weak evidence on the effectiveness of the slow module. When the slow module is on, GPT-5 and GPT-5 mini both receive very marginal improvements comparing to slow-module-off. Furthermore, slow module almost contributes nothing to GPT-5 comparing to mini, implying that the slow module does not help for stronger models.
  2. The description and implementation of TSMC algorithm is unclear. In line 260, the paper writes "particles are guided toward high-probability regions of a target distribution which corresponds to trajectories that successfully accomplish the task". However, the paper does not present the use of successful trajectories here or in Appendix D.
  3. Missing compute matched experiments. NeSyFS could introduce extra computation and maintaining an extra knowledge graph. The paper would benefit by comparing with scaled multi-round ReAct, reflexion.
Questions for authors
  1. What's the average fire rate of slow modules in the experiments?
  2. Could you provide additional computation-matched experiments (as described in weakness)?
2 comments

We thank the reviewer for these constructive comments. Our response to weakness is as below.

  1. "Weak evidence on the effectiveness ......"

Response: The effectiveness of the "slow thinking" module: Although the overall improvement brought about by "slow thinking" is relatively limited, this does not necessarily imply that the planner itself has limited value. NeSyFS employs an adaptive "fast-slow" architecture: "slow thinking" is activated only when actions taken via "fast thinking" repeatedly fail during the reflection phase, rather than being triggered at every decision-making step. Consequently, since a large proportion of decisions are successfully handled through "fast thinking," the contribution of the "slow thinking" module to overall performance may appear "diluted." The smaller gain observed for GPT-5 also aligns with the following logic: the more capable a model is, the less additional deliberation it requires. More importantly, this planner addresses a unique challenge that graph memory and reflection mechanisms cannot handle on their own—namely, reasoning about uncertain future observations and their potential consequences.

  1. "The description and implementation ......"

Response: Our approach does not require the observation of successful trajectories. Instead, it employs sequential importance weighting and resampling to construct an approximation distribution—guided by a notion of "progress"—that favors successful trajectories. Specifically, each particle represents a hypothetical future trajectory, and its weight is updated via an exponential function based on the task progress predicted by a large language model (LLM). This process progressively biases the sampling toward trajectories with higher predicted progress. We note that, due to potential biases in progress estimation, this surrogate distribution may not perfectly align with the true distribution of successful trajectories.

  1. "Missing compute matched experiments ......"

Response: First, our approach is already built upon multi-turn ReAct, since the reflection is multi-turn and slow module is only initiated after multiple turns of action generations and reflections. Second, we agree that additional computation could contribute to performance improvements. Nevertheless, NeSyFS is designed to allocate deliberative computation selectively, invoking expensive planning only when reactive decisions are unreliable. Moreover, KG-based context provides concise task-relevant information, potentially reducing redundant reasoning over lengthy interaction histories.

In summary, the contribution of NeSyFS lies in combining structured future-state modeling with selectively activated, progress-guided probabilistic planning under partial observability.

Answer to Q1: please see the last section of Appendix. The firing rate of slow module is evaluated there.

AuthorsOpenReview ↗

Kind Request for Your Feedback on Our Rebuttal

Thank you again for your thoughtful review and constructive feedback. We hope that our rebuttal and revised paper have addressed your questions and concerns. With the author–reviewer discussion period ending in about a day, we would be grateful to hear whether any points remain unresolved or would benefit from further clarification. Please let us know, and we will respond promptly. Thank you for your time and consideration.

Reviewer_GM8EOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This work proposes a framework NESYFS, which addresses LLM agents that must act with incomplete information about their environment: it stores observed facts in a knowledge graph, using an LLM to propose actions and checks whether each action supports the task objective. After repeated rejection, a planner generates several imagined action sequences and probabilistically retains those judged to make progress. The graph supplies context to all these components. The paper reports substantial improvements on three interactive benchmarks.

Critical strengths
  • The approach design is coherent on the explicit planning and knowledge graph. The same dynamically updated graph supports action generation, action checking, and planning, giving these processes a common representation of the environment. Using repeated action-checking failures to trigger additional planning is also a sensible way to allocate deliberation to difficult decisions.
  • The reported empirical improvements are substantial. Results span three different interactive environments and multiple model backbones, suggesting that the framework’s usefulness is not confined to one task or model. These gains make the approach worth investigating even if its individual ingredients are largely established.
Critical weaknesses
  • The design choice of the proposed approach is not unique to this work. AriGraph already equips agents with graph-based memory and planning, while SwiftSage combines fast action generation with slower deliberation. NeSyFS brings these ideas together with an action checker that decides when to invoke planning. This integration may be useful, but the paper should clarify what new capability or insight follows from this particular combination.

  • The evaluation presentation is short in the main pages, and especially lacks insightful analysis on how the proposed approach surpasses baselines significantly, e.g. when its planner adds meaningful value beyond graph memory plus reflection, and whether that advantage specifically comes from handling incomplete information. In particular, the planner itself needs a clearer methodological contribution. Its basic procedure is to generate several possible futures, score their progress using an LLM, and resample promising candidates. Sequential Monte Carlo methods already provide this general search pattern. NeSyFS’s adaptation uses graph-based imagined states and binary progress judgments. The paper does not establish why these choices provide a substantive advance over simpler sampled search. Moreover, weighting trajectories by an LLM’s judgment of progress does not by itself justify the connection to sampling successful trajectories.

  • There could be more recent baselines added for comparison which adopts the trial-and-error/planning methodologies. 4 out of 6 baselines are pre-2023. There are more SOTA methods in 2025/2026.

Questions for authors

At comparable inference cost, how much does the proposed planner improve over graph memory plus reflection and simpler search methods?

3 comments

We thank the reviewer for this insightful comment. While KG-based memory and fast–slow thinking have been explored in AriGraph and SwiftSage, respectively, NeSyFS goes beyond their integration by introducing two key methodological contributions that address the challenges of partial observability. Our response to the weakness is as below.

  1. "The design choice of the proposed approach is not unique to this work......"

Response: First, our work verifies the effectiveness of structured KG-based context in improving reflection and world modeling. Unlike prior approaches that primarily use KGs for memory and decision-making, NeSyFS leverages task-relevant KG triplets to support both action verification and future-state prediction. In reflection, structured context reduces irrelevant information while preserving critical state relationships, enabling more reliable verification of action alignment with task objectives. Table 2 demonstrates that KG-based reflection achieves fewer detection errors and higher effective reliability than history-based or summarized-belief-based alternatives. Furthermore, in world modeling, KG-retrieved triplets ground LLM predictions of future observations, which are subsequently incorporated into the KG to construct structured representations of hypothetical future states. Thus, the KG serves not merely as memory, but as a shared symbolic representation supporting both reflection and planning.

Second, NeSyFS introduces a new planning method, i.e. neuro-symbolic uncertainty-aware planning algorithm based on twisted sequential Monte Carlo (TSMC), fundamentally different from the subgoal-oriented deliberation, such as SwiftSage. Our algorithm maintains multiple particles representing possible future actions, observations, and KG states. Through LLM-based prediction, task-progress evaluation, and probabilistic resampling, it explores promising trajectories while mitigating errors in LLM-based evaluations. This provides a distinct mechanism for planning under partial observability rather than simply incorporating an existing planning component.

  1. "The evaluation presentation is short in the main pages......"

Response: First, by explicitly reasoning about uncertain future states, our planner offers capabilities that extend beyond those provided by "knowledge graph (KG) memory" and "reflection" mechanisms. While KG memory can represent observed information and reflection mechanisms can identify deviant behaviors, neither approach can explore potential future observations or their consequences. Our planner addresses this limitation through multi-trajectory simulation. Table 1 demonstrates that our method achieves significant improvements over AriGraph and SwiftSage, while Figure 5 further illustrates the individual contributions of KG memory and reflection mechanisms when planning is not employed.

Second, our methodological contribution lies in the application of Sequential Monte Carlo (SMC) methods to neuro-symbolic planning in partially observable environments. In this framework, each particle maintains a dynamically updated knowledge graph that represents a hypothesized future state. By retrieving triples from these knowledge graphs, we can guide observation predictions generated by large language models (LLMs). Simultaneously, we employ binary progress assessments to reduce reliance on potentially unreliable, fine-grained value estimates. Furthermore, our probabilistic resampling mechanism prioritizes promising trajectories while still allowing for the exploration necessary when occasional deviations are detected during evaluation. These mechanisms distinguish our approach from simple sampling-based search.

Third, we establish a connection to "successful trajectory sampling" through a technique we term "progress-guided distribution tilting," rather than simply assuming that an LLM’s assessment of progress is equivalent to the probability of success. Specifically, we employ an exponentially weighted mechanism to tilt the particle distribution toward trajectories that exhibit greater predicted progress. We acknowledge that, unless the progress signal perfectly reflects the likelihood of ultimate task completion, this method does not guarantee the sampling of successful trajectories. We will further clarify this distinction—and the underlying assumptions—in our theoretical discussion, so as to avoid making an overly absolute claim that successful trajectories can necessarily be sampled.

  1. "There could be more recent baselines......"

Response: First, we would like to clarify that our baselines are not predominantly pre-2023: AriGraph and ABBEL were introduced in 2025, while SwiftSage, RAFA, and Reflexion are from 2023.

Other methods proposed within 2 years have their own domain-specific assumptions, e.g. robotis and web navigation, or added extra components, e.g. task decomposition or subgoal graph. There is no need to compare against these methods.

AuthorsOpenReview ↗

Kind Request for Your Feedback on Our Rebuttal

Thank you very much for your thoughtful review, as well as the valuable questions and constructive suggestions you provided. In our rebuttal and revised paper, we have done our best to address all of your concerns and clarifications. As the author-reviewer discussion period will end in about 1 day, we would like to kindly confirm whether we have sufficiently addressed all of your concerns. If you have any remaining questions or would like further clarification on any point, please feel free to let us know and we will respond promptly. Thank you very much for your time and consideration!

Thanks for the response and I appreciate the evidence for KG-based reflection and the qualification of the theoretical claim. However, as prior works like AriGraph and SwiftSage already incorporate planning, the presence of deliberative search alone does not feature a novel technique, and the overall agent design feels engineering-oriented. It is also uncertain that why the proposed planner improves upon simpler sampled search. I will raise score to 2 but still maintain my overall rejection assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.