- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
LLM-based brain-signal agents may generate workflows that are “semantically relevant but still computationally invalid” when required preprocessing steps or intermediate artifacts are missing. BrainAgent-CGP separates semantic goal interpretation from executable workflow construction, where a Constraint-Guided Planner builds dependency-valid workflows from the required and produced artifacts declared by each tool. The method is evaluated on a 30-task benchmark against three LLM-based planners with ablations, together with language-robustness, failure-recovery, and real-data experiments on ISRUC-Sleep and Sleep-EDF Expanded.
Critical strengths
- This paper provides a clear separation with a deterministic planner.
- A reasonable failure handling process is provided.
Critical weaknesses
-
The main results are largely guaranteed by construction. CGP only expands tools whose required artifacts are available, so PVR = 1.00 and ITR = 0.00 follow directly from the algorithm, on a 30-task benchmark built from the same registry. The ablation shows that the cost and reliability terms do not change any workflow, so Eq. 9 is not validated. The failure-recovery experiment relies on three experiment-only fallback tools, and the real-data study checks only that a single four-tool workflow runs without errors.
-
The planner comparison is hard to interpret. Prompts are not provided, and it is unclear whether the LLM baselines receive the same artifact declarations as CGP. Only one backbone (gpt-5.6-luna) is used, and there is no comparison with EEG Agent or BrainAgent. Figure 3 is also inconsistent with Table 11.
-
Multiple citation errors. Several cited works do not match how they are described in the text (Ref. N denotes the N-th entry in the paper’s reference list):
- Jiang et al. (2025) [Ref. 6] studies how LLM interactions affect users’ cognition using EEG, not tool-using agents.
- Liu et al. (2026) [Ref. 11] is a survey and did not propose Plan-over-Graph, which is Zhang et al. (2025b) [Ref. 17].
- Ming et al. (2025) [Ref. 12] is a brain-controlled robotic-arm paper, not GraSP [1].
- Wu et al. (2025) [Ref. 14] is a seizure-prediction review; the described dependency-aware parallel planning matches GAP [2].
- The GraphPlan entry [Ref. 9] also contains placeholder text.
- Limited novelty and terminology disconnected from prior work. The paper introduces new terms for established concepts without relating them to prior work. Semantic goal interpretation is NL-to-goal parsing, required/produced artifacts are preconditions/effects, and CGP is standard forward state-space search. This makes the paper harder to follow and obscures the close overlap with STRIPS/PDDL-style planning[3] and LLM+P[4], neither of which is discussed.
[1] GraSP: Graph-Structured Skill Compositions for LLM Agents, Xia et al., 2026.
[2] GAP: Graph-based Agent Planning with Parallel Tool Use and Reinforcement Learning, Wu et al., NeurIPS 2025.
[3] STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving, Fikes et al., 1971.
[4] LLM+P: Empowering Large Language Models with Optimal Planning Proficiency, Liu et al., 2023.
Questions for authors
Clarifications:
- Do the LLM baselines receive the same required/produced artifact declarations as CGP, and is any validation or repair allowed? Please provide the prompts and justify using a single backbone.
- Does the 0.038 s planning latency of BrainAgent-CGP include semantic goal interpretation? Which interpreter (controlled parser or hybrid semantic interpreter) is used in Table 1?
- How is “w/o Artifact Constraints” implemented? Why does routing accuracy drop to 0.767 when routing is deterministic?
- Why do Figure 3 and Table 11 report different tool counts and redundant selections?
- In the real-data study, is output correctness checked beyond the absence of execution errors? Which preprocessing settings produce the abrupt drop in the PSD curves of Figure 2?