- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an automatic LLM-based framework for synthesizing local control barrier functions (CBFs) for hybrid systems. The LLM guides the candidate generation and refinement. The proposed approach is evaluated in two case studies, demonstrating its ability to synthesize local CBFs.
Critical strengths
- The main contribution (a framework for synthesizing CBFs automatically) sounds relevant and indeed seems novel.
- 6 successful runs out of 9, compared to 0 by the naive baseline, which shows the effectiveness of the method.
Critical weaknesses
- This is a paper of type "use generative AI as an oracle for problem X." I am generally reserved regarding such papers. By default, I expect that this works quite well in many cases. I was not convinced that any challenge was faced or any clever tool was needed, nor that there are any deeper insights from the general approach.
- The paper only compares to a "naive baseline" (378). I do not think this is sufficient to argue that the proposed approach is the right one. I accept that paper addresses an unsolved problem, so no established baseline may exist. Still, I would expect that the authors could also make comparative experiments in simplified settings that other tools can deal with.
- The paper lacks a systematic analysis or discussion of the synthesized CBFs, limiting the possible conclusions drawn about the proposed method.
Questions for authors
- Could the authors compare their approach with at least one other established baseline to investigate its positioning?
- Could the authors provide a systematic evaluation of the quality and properties of the certificates generated by the LLM, e.g., the control performance?
3 comments
Thank you for adding the new results.
Regarding the new baseline, I understand that Reviewer qfYB suggested to look at fixed templates. In my opinion, using an established tool is preferred to demonstrate the advances. Reviewer 6pTw suggested an SOS baseline for instance. (I saw your reply that you expect that SOS will work better for these systems.)
Regarding my weakness 1: I understand that simply letting the LLM solve the end-to-end problem is not going to work. My point was that there are many well-known algorithmic schemes that consist of a refinement loop with an oracle. I am not saying that this type of work should not be done, but I consider it relatively low-hanging in the technical contribution.
Response to the Follow-up Comment of Reviewer Er4j
Thank you very much for the follow-up comment.
In my opinion, using an established tool is preferred to demonstrate the advances.
We agree that a comparison with an established tool is important. Currently our comparison with existing methods is limited to single-mode settings as most of them are for single-mode systems.
I consider it relatively low-hanging in the technical contribution.
We understand the reviewer's take. That said, we believe that using LLMs as synthesis oracles is still worth studying as LLMs are so flexible, and how to use them as oracles in search is not necessarily trivial. As a promising future direction, we can also consider giving LLMs not only formal specifications but also domain knowledge from users.
Thank you again for the fruitful discussion!
Response to Reviewer Er4j
Thank you very much for reviewing our paper and for your constructive comments. Our General Response reports the experiments we added.
First, we answer the two questions.
Q1.
For the two hybrid tasks, we ran a baseline with the LLM replaced by a fixed polynomial template; please see Point (1) of our General Response. For non-hybrid settings, Appendix C compares the proposed method with that of BarrierBench (Taheri et al., 2026) on its 86 continuous-time problems.
Q2.
Appendix D.3 shows, for the reported design of every certified run in the Rover task, the safe-set fraction, goal reaching, arrival time, and clearance over 32 trajectories. Appendix D.4 shows the trajectories in the ACC task at three gains. In the revised version, we have added a table of all certified designs, with their degrees and numbers of monomials; please see Point (4) of our General Response and Appendix E.4 for details.
Let us also comment on the point raised as a weakness.
Although we understand the reviewer's view, we respectfully disagree. We do not think that the task of finding CBFs can be solved merely by using an LLM as an oracle. In fact, as one of the baselines, we repeatedly queried the same LLM with the prompt used in the first iteration of our method, with the same fitter and solver but without the search mechanism; this produced no certificates in 540 proposals for the Rover task. LLMs indeed play an important role in writing function forms of barriers, but the other components of the framework are also indispensable: the fitter tunes the coefficients, a design is accepted only when the solver proves every condition (including safety at each mode switch), and the search mechanism is key to trying different possibilities beyond what LLMs alone can suggest.