- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a Neuro-Symbolic Fast-Slow thinking framework (NeSyFS) for LLM Agent to address the problems of partial observability, such as belief state inference, task objective misalignment, and planning under uncertainty. It contains a knowledge graph to represent the belief state, the fast-thinking module to perform reactive action, and the slow-thinking module to conduct new uncertainty-aware planning. The experimental results show that the proposed method has advantages on the ALFWorld, Webshop, and ScienceWorld over baseline methods.
Critical strengths
- The fast-slow thinking framework to address partial observability is interesting.
- The use of different neuro-symbolic representations for distinct states helps separate information for LLM agents.
- The results of three different benchmarks show that the proposed method performs better than baselines.
Critical weaknesses
- The LLMs only use GPT-5 and Llama families. It lacks generalization.
- The experiments need to add some ablation studies, such as the hyperparameter of K and every component of NeSyFS.
- The proposed method maybe adds inference time.
Questions for authors
- What are the differences between History, Belief State, and KG?
- How are the weights of particles trained in the LLM agent?
- How long is the inference time and how many tokens does the proposed method?
- If the mismatch between the LLM proposal distribution and the prior transition distribution in the task, what does happen for the method?
2 comments
Thanks for your responses. My concerns are resolved. I will keep my positive score.
Thanks for your time on our work and helpful comments. Our response to the weakness and questions is as below.
Response to weakness 1: Due to the limit of research budget, we cannot evaluate our method on other models. We think GPT and Llama are representative enough.
Response to weakness 2: This paper includes ablation studies designed to investigate the contributions of individual components. Specifically, Figure 5 compares the performance of various context representations—knowledge graphs, interaction histories, and aggregated belief states—both with and without a "reflection" mechanism, illustrating the distinct benefits provided by knowledge graph-based memory and the reflection process itself. Furthermore, Appendix E.3 is dedicated to evaluating the "slow thinking" module.
The ablation study of different K values is added as Appendix D.6 in the revised version of this paper.
Response to weakness 3: We acknowledge that maintaining a knowledge graph, performing reflection, and executing planning processes entail additional computational overhead. However, NeSyFS is designed to strike a balance between computational efficiency and decision quality by adaptively activating the "slow thinking" mechanism. In particular, the system initiates the more resource-intensive planning process only when the actions taken via "fast thinking" repeatedly fail during the reflection phase, thereby avoiding unnecessary planning at every decision step. Additionally, knowledge graph-based retrieval provides a concise context that is highly relevant to the task; compared to processing lengthy interaction histories, this approach helps reduce token consumption. Analyses regarding computational overhead and token costs are provided in Appendices E.4 and E.5.
Answer to Q1: History is the sequence of past observations and actions within currrent episode. Belief state is estimated hidden state in partially observable environment. KG is the representation of hidden state in our approach.
Answer to Q2: In NeSyFS, particle weights are not learned through model training; instead, they are computed dynamically during the planning process. Specifically, for each particle, a "progress detector"—based on a large language model (LLM)—evaluates whether its predicted state transition advances the completion of the task, outputting a binary signal (δ∈{0,1}) accordingly. The particle weights are then calculated using the formula ω=exp(λδ), where the parameter λ serves to adjust the degree of preference for "promising" trajectories. These weights determine the resampling probabilities, ensuring that particles demonstrating positive progress toward task completion are selected more frequently, while still maintaining the possibility of exploring other trajectories. This mechanism eliminates the need to train a separate value function or reward model, and it also reduces reliance on fine-grained evaluations by the LLM—a type of assessment that is often unreliable.
Answer to Q3: Please see the last section of Appendix.
Answer to Q4: As discussed in Section 3.3, our simplified weighting rule follows the base-model-as-proposal setting of TSMC (Zhao et al. 2024), where the proposal distribution (q) equals the base distribution (p_0), allowing their likelihood ratio to cancel.
The NeSyFS paper can assume q=p0 only if p0 is defined as the transition distribution induced by the same pretrained LLM used as the proposal model. It cannot generally assume that the LLM proposal distribution equals the true environment transition distribution. However, since the transitions (next observations) in the slow-module planning are generated by the same LLM as the proposal model, we can assume q=p0, and we don't need to consider the case q!=p0