acceptodds
Under review as a conference paper at ICLR 2027

Beyond Feasibility: Enforcing Safety Rules on Language-Model Planners with World-Aware Look-Ahead

Abstract

Language models can turn natural-language instructions into robot plans, but their plans carry no safety guarantee: a plan can be executable and reach its goal yet still violate safety rules. When the rules and the action model are formally specified, one would like a planner that never returns an unsafe plan, preserves the language model’s preferences among safe plans, and needs only a single decoding pass. We introduce GUARDCTRL, which weights each candidate token by the probability that the model completes it into a safe plan. This look-ahead tracks the world state and the progress of each rule along every hypothetical continuation, and is computed exactly with a hidden Markov model distilled over grounded action and object symbols. Its support guarantees that one decoding pass returns a safe plan whenever one exists within the token horizon, and its magnitude preserves the model’s preferences among safe plans. We evaluate on the DESPITE benchmark and on a new, harder set with rules over the action history. With Llama-3.1-8B-Instruct, GUARDCTRL is safe on 98.4% of DESPITE hard tasks and 98.2% of tasks with temporal rules, against 13.8% and 6.4% for rejection sampling with up to ten attempts, and close to GPT-5 given the same rules. Its extra time, mostly a one-time compilation, is about twice that of one unconstrained generation. It also follows preferences stated only in the instruction.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Constrained Decoding, Formal Guarantees, Robot Planning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_N17COpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks how to make an LLM robot planner obey safety rules beyond reaching the goal, such as not grabbing a hot cup bare-handed. Given a PDDL model and formal rules, the method reweights each next token by the probability that the plan can still be finished safely, following each hypothetical action into the world it produces. This probability is computed exactly on a small HMM distilled from the LLM over action and object symbols, with the product of parser, rules and world explored lazily. Since the weight is positive exactly when a safe completion exists, one pass returns a safe plan whenever one exists within the horizon, and since it follows the LLM, the plan should stay close to the model's own choice. On DESPITE and a new set with temporal rules, an 8B model becomes safe on about 98% of tasks, against well under 30% for rejection sampling.

Critical strengths

The framing is clean and useful. Splitting the look-ahead into its support, which gives safety and completeness, and its size, which keeps the LLM's preferences, makes clear what each part of the method is for, and the link to shields and nonblocking supervisors is stated openly. The move to a symbol-level surrogate is also well motivated by a concrete case where a subword HMM badly misjudges two equally likely routes.

The experimental bookkeeping is careful. Held-out test tasks, an audit of the rule translation, paired tests, attribution of every failure, and the honest admission that a classical planner already solves almost every task when everything is formalized all make the results easy to trust.

Critical weaknesses

My main concern is that the part that is really new, using the size of the look-ahead to stay faithful to the LLM, is not isolated. The safety guarantee alone is what a shield gives, and a classical search over the same world and rules solves nearly every task in about a second. What would show the idea works is an ablation that samples from the LLM restricted to the exact safe support, with no HMM weighting, compared on preference following. Another natural baseline is to let the planner enumerate safe plans and have the LLM pick among them by likelihood.

The case for needing the LLM at all is also thin. It rests on a small preference study built from easy tasks with LLM-written phrasings, and on a text-only rule study where the guarantee no longer holds; the comparison to the target distribution uses only a few dozen selected instances. The local mask baseline fails because it lets the model start spelling an action that cannot be taken, which a standard mask over admissible action prefixes would avoid. Finally, the "about twice one generation" overhead comes from a sample whose mean compile time is about half the full-set mean, and scaling to larger worlds is untested.

Questions for authors

What preference-following rate do you get by sampling the LLM restricted to the exact safe support, without HMM weighting? This is the cleanest test of what the look-ahead magnitude buys.

How does the method compare with having the planner enumerate safe plans and the LLM rerank or sample among them, and with weighted rejection sampling or SMC at matched compute?

How does a mask over currently admissible action prefixes perform, and how many local-mask failures are real action-level dead ends rather than spelling dead ends?

What is the overhead over the full test sets, and how does compile time grow with the number of objects, plan length and rules?

Is the backward recursion computed in log space, and could numerical underflow turn a positive look-ahead into zero and break the completeness guarantee?

4 comments
AuthorsOpenReview ↗

Response to Reviewer N17C (1/3)

Dear Reviewer N17C,

We apologize for the delay in posting our detailed response and appreciate your patience. Thank you for your careful review and for recognizing the clear framing of our approach and the motivation for our symbol-level HMM, as well as the care and transparency of our experimental evaluation and reporting. Following your comments, we compare GUARDCTRL with an exact safe-support mask to assess preference following, plan length, and runtime (A1). Then we evaluate planner enumeration followed by LLM ranking or sampling, weighted rejection sampling, and SMC (A2), and test a mask over admissible action prefixes while analyzing the original local mask’s failures (A3). We also report runtime across all 1,000 test tasks and how compilation time varies with the number of reachable states (A4), and clarify the numerical implementation and how underflow can affect the completeness guarantee (A5). We provide detailed point-by-point responses below.

Q1: Preference following with exact safe-support masking

A1: We agree that comparing GUARDCTRL with an exact safe-support mask helps isolate the contribution of our probabilistic weighting. The clearest benefit is shorter plans: GUARDCTRL averages 6.6 actions, compared with 22.2 for the exact mask (Table 1). Both methods achieve comparable high safe success and preference following.

Both methods use the same parser, PDDL world, rule monitors, 512-token horizon, and temperature 1. Under the assumptions of Proposition 1, they allow the same tokens (see A5 for finite precision). Only GUARDCTRL weights them by estimated acceptance probabilities. Section 4.4 tests two routes for each of 100 easy tasks using LLM-written requests (200 explicit, 200 implicit). Both methods follow preferences more often than the instruction-blind planner.

The mask produced longer plans on 291 instances and shorter ones on 29, averaging 15.7 extra actions. GUARDCTRL's mean length is close to the 5.9 actions of accepted LLM plans. Five mask outputs of 86–98 actions timed out during checking and count as unverified failures in S. GUARDCTRL had the lower mean runtime and the mask the lower median.

Table 1. Safe success, preference following, length, and runtime on 400 preference instances (Llama, Section 4.4). Percentage pairs are explicit / implicit requests, with all 200 instances per phrasing in the denominator. S counts safe completions and P additionally requires the preferred action.

MethodSafe success S (%) ↑Preference following P (%) ↑Length (actions), mean (median)Runtime (s), mean (median) ↓
Classical planner (blind to instruction)100.0 / 100.049.5 / 49.54.7 (4)1.0 (0.9)
Exact safe-support mask (new)98.0 / 99.595.0 / 86.522.2 (9)12.6 (5.7)
GUARDCTRL100.0 / 100.093.0 / 81.56.6 (6)8.5 (7.1)

On the main benchmarks in Section 4.2, the mask averages 24.1 versus 7.1 actions for GUARDCTRL on D-Short, and 24.6 versus 7.4 on D-Long, with similar safe success (98.0% versus 98.4% and 97.0% versus 98.2%, respectively).

A package-delivery task from the preference set shows how extra steps can accumulate in an accepted plan. Both plans carry a package downstairs and pass the acceptance checks.

  • GUARDCTRL (6 actions) (check_stairs) → (grab_package) → (clear_human) → (start_descending_stairs) → (complete_descent) → (place_package bottom_floor)
  • Exact mask (14 actions) (check_stairs) → (clear_human) → (grab_package) → (place_package top_floor) → (grab_package) → (place_package top_floor) → (grab_package) → (start_descending_stairs) → (place_package top_floor) → (grab_package) → (place_package top_floor) → (grab_package) → (complete_descent) → (place_package bottom_floor)

The mask puts the package down upstairs and picks it up again four times, adding eight actions. The binary mask leaves the LLM's relative probabilities unchanged among the allowed tokens, including the tokens for an extra put-down whenever a safe finish is still reachable. Thus, when termination is blocked, the model can take another allowed action, including a repeat that leaves enough tokens to finish.

GUARDCTRL weights those choices by their estimated probability of leading to an accepted plan. If the HMM estimates that having to pick up the package again reduces that probability, weighting makes the extra put-down less likely. The two trajectories illustrate how weighting can shorten plans even when the mask follows preferences well overall. We did not trace step-by-step probabilities for this instance. We will include these exact-mask comparisons in the revised main tables.

AuthorsOpenReview ↗

Response to Reviewer N17C (2/3)

Q2: Planner enumeration, weighted rejection sampling, and SMC

A2: Following your suggestions, we compare GUARDCTRL with planner enumeration followed by LLM ranking or sampling, weighted rejection sampling, and SMC. GUARDCTRL solved 48 of the 61 main-set tasks where enumeration returned no candidate within the search budget (Table 2). On the 50 ablation tasks, GUARDCTRL solved all 50, compared with 33–40 for the tested weighted rejection sampling and SMC configurations. GUARDCTRL also had the lowest mean runtime per task (Table 3).

The planner enumerates safe plans up to two actions longer than the shortest plan. On the preference instances, the resulting lists contain both routes by construction and average 228 plans per instance. Selecting the most likely plan gives the preference-following rates in Table 2. Sampling from the same list in proportion to LLM likelihood yields P of 99.0% for explicit requests and 82.0% for implicit requests. On 33 D-Short and 28 D-Long tasks, the planner returned an empty list, so neither selection strategy could return a plan.

Table 2. Planner enumeration with LLM selection (Llama). P, plans listed, and runtime are measured on the 400 preference instances in Section 4.4. P is reported for explicit / implicit requests. The final two columns give results on D-Short / D-Long (500 tasks each).

MethodPreference following P (%) ↑Plans listed per instance, mean (max) ↓Runtime (s), mean (median) ↓Safe success S (%) ↑Tasks with no listed plan ↓
Enumeration + most likely plan (new)98.5 / 82.5228 (3,604)22.1 (2.8)93.2 / 94.033 / 28
GUARDCTRL93.0 / 81.5—8.5 (7.1)98.4 / 98.2—

For the particle comparison, each method ran alone on one A40. Weighted rejection sampling draws independent paths using the action-prefix mask in A3. Increasing the particle count from three to ten solved seven more tasks and increased both runtime and sampled tokens. SMC resamples paths by weight. With five particles in this run, it solved 37 tasks to weighted rejection sampling’s 38. The weights reflect the LLM probability mass retained along a prefix. GUARDCTRL uses QtQ_t to estimate the chance of safe completion within the remaining token budget.

Table 3. Timed runs on the 50 Llama ablation tasks from Table 15 in our paper. Time covers setup through generation and excludes judging. Sampled tokens are summed over particles and averaged over tasks. Length is measured in actions over returned plans. All returned plans were safe.

Method (particles)Tasks solved ↑Time (s), mean (median) ↓Sampled tokens, mean ↓Length (actions), mean (median)
Weighted rejection sampling (3)3311.7 (4.5)6329.6 (8)
Weighted rejection sampling (5)3814.1 (14.8)1,07010.9 (8)
Weighted rejection sampling (10)4018.1 (26.6)2,1018.1 (7)
SMC (5)3719.6 (5.2)81610.3 (8)
GUARDCTRL (1 path)506.7 (4.7)417.2 (7)

With ten particles, SMC also solved 37 of 50 tasks and returned no plan on the other 13. Its returned plans averaged 8.4 actions (median 8). We used one random seed and did not tune the SMC resampling parameters.

AuthorsOpenReview ↗

Response to Reviewer N17C (3/3)

Q3: Masking admissible action prefixes and causes of local-mask failures

A3: We agree that masking admissible action prefixes can provide a stronger local baseline. We therefore reran Llama on the 50 ablation tasks from Section 4.5 (25 D-Short and 25 D-Long) with a mask over currently admissible action prefixes, which increases safe completions from 0/50 to 27/50 but still underperforms GUARDCTRL's 50/50 (Table 4). The original mask’s “No plan” outcomes in Table 4 and Appendix Table 15 of our paper were all traced to blocked action prefixes in Appendix L.3, where all 50 failures were identified as spelling dead ends and none as action-level dead ends.

After (navigate_to_spill), the original mask lets Llama start (get_mop and blocks the closing parenthesis because the mop must be fetched in the storage area. The suggested mask blocks the first token that no longer matches any action allowed at the spill. This eliminated spelling dead ends. Of its 23 remaining failures, five had no applicable, permitted action with the goal or an obligation still unmet, and 18 reached the token limit. A1’s exact mask looks ahead to check whether an accepted completion remains reachable within the budget.

Table 4. Mask comparison on the 50 ablation tasks in Section 4.5 (Llama). The four outcome counts sum to 50 per row. A dash indicates an unreported length.

MethodSafe completion (tasks)Spelling dead end (tasks)Action-level dead end (tasks)Token limit reached (tasks)Length (actions), mean (median)
Local mask05000-
Action-prefix mask27051810.6 (8)
GUARDCTRL500007.2 (7)

Q4: Full-set runtime and scaling with task size

A4: We agree that the timing results should cover the full test sets, and we have now added runtime measurements for all 1,000 tasks (Table 5). Across all 1,000 tasks, GUARDCTRL averaged 13.6 s per task for Llama and 18.9 s for Qwen, including 10.7 s and 12.8 s for compilation, respectively (Table 5). The 50-task sample underestimated mean compilation time, but its median was within 0.1 s of the full-set median for both models. Mean runtime outside compilation was also similar across the two sets.

A few large reachable state graphs raise the full-set mean. Compilation accounts for the entire 5.6 s increase in mean end-to-end time for Llama and 6.0 s of the 6.6 s increase for Qwen. Median compilation time rises from 1.8 s in the smallest quarter of graphs to 34.7 s in the largest 5%. Larger graphs also have longer plans, so we cannot isolate the effect of plan length. Object counts vary too little to show a trend. Comparing rule counts across different tasks leaves the cost of adding a rule to a fixed task unmeasured.

Table 3 in our paper times one unconstrained generation on the 50-task sample (Llama 2.4 s, Qwen 2.6 s). We will clarify that “about twice one generation” describes Llama's extra cost on that sample. A full-set ratio requires the same baseline on all 1,000 tasks.

Table 5. GUARDCTRL time per task on one A40. Runtime is end-to-end time minus compilation time.

Model and task setCompilation (s), meanCompilation (s), medianRuntime (s), meanEnd-to-end (s), mean
Llama, 50-task sample5.13.52.98.0
Llama, all 1,000 tasks10.73.52.913.6
Qwen, 50-task sample6.84.45.512.3
Qwen, all 1,000 tasks12.84.56.118.9

Q5: Numerical underflow in the look-ahead probabilities

A5: Numerical underflow is a valid concern for our implementation. The submitted backward recursion computes QQ in probability space using double precision. If the only feasible next token has a positive QQ that rounds to zero, the decoder can fail even when a valid plan exists.

However, returning no plan can sometimes be a good outcome. Suppose the only route to the goal requires action B immediately after action A. If the HMM assigns a vanishingly small probability to completing the task along this sole route, that low weight may reflect how rarely B follows A in the LLM-generated plans used for distillation. The learned weights can therefore act as a soft constraint on an action sequence that the explicit rules permit. If the consequences of executing B after A are poorly understood, it may be good to stop rather than recommend a plan that depends entirely on that sequence.

Thank you again for your insightful review. We will incorporate these new results and clarifications into the revised paper within 8 hours, with changes highlighted in blue.

We apologize again for the delayed response and for leaving you limited time to review these additions. We hope these results and analyses address your concerns, and we are happy to provide further clarification.

Best regards,

Authors of Submission 59705

AuthorsOpenReview ↗

Revision Uploaded: Follow-Up to Reviewer N17C

Dear Reviewer N17C,

Thank you again for your careful reading and detailed questions, and for your patience with our delayed response. We have revised the paper to incorporate the additional baseline comparisons, distributional analysis, full-set timing results, and clarification of the numerical limitations discussed in our responses. We have also added a table of contents for the appendices to make the relevant experiments, analyses, and clarifications easier to find. The table below lists where each of your concerns is addressed.

Your commentRevisionLocation
W1 / Q1: What probabilistic weighting adds beyond exact safe supportAdded an exact safe-support mask using the same parser, PDDL model, rule monitors, and horizon. The comparison reports safe success, preference following, plan length, and runtime.Section 4.5 and Appendix M.1
W1 / Q2: Planner enumeration followed by LLM ranking or samplingAdded both selection strategies and evaluated preference following and main-set performance.Section 4.4 and Appendix M.4
Q2: Weighted rejection sampling and SMC at matched computeAdded comparisons with weighted rejection sampling and SMC on the same 50 ablation tasks, reporting safe completions, plan lengths, and end-to-end runtimes under the same hardware and timing protocol.Section 4.5 and Appendix M.4
W2: Limited evidence for fidelity to the LLM’s distributionAdded complete-plan distribution comparisons on 16 tasks, using accepted LLM plans as the reference and reporting TV and KL divergence. We also measure variation between two reference samples and clarify that fidelity with the distilled HMM is empirical rather than guaranteed.Section 3.4, Section 4.5, Section 6 and Appendix M.2
W2: Scope of the evidence for needing the LLMExpanded the comparison with planner-based alternatives and clarified when enumeration with LLM selection is a strong alternative.Section 4.4, Section 6 and Appendix M.4
W2 / Q3: Action-prefix masking and the causes of local-mask failuresAdded a mask over currently admissible action prefixes.Section 4.5, Appendix L.3 and Appendix M.4
W2 / Q4: Whether the timing sample represents the full test setsAdded compilation and end-to-end timings over all 1,000 tasks.Section 4.3 and Appendix M.8
W2 / Q4: Scaling with objects, plan length, and rulesAdded an analysis of compilation cost across reachable graph sizes. We explain why the available tasks do not isolate the effects of plan length, object count, or added rules, and explicitly acknowledge that scaling to larger worlds remains untested.Section 6 and Appendix M.8
Q5: Numerical underflow and the completeness guaranteeClarified that the backward recursion uses probability-space double precision. Underflow can cause the decoder to return no plan on a solvable task, so the implementation’s completeness is limited by numerical precision.Section 6 and Appendix C

Thank you again for helping us strengthen the comparisons and clarify the scope of our claims. We would be happy to address any remaining questions.

Best regards,
The Authors of Submission 59705

Reviewer_ExDnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method for generating plans with an LLM but ensuring these plans satisfy some "safety constraints". The approach -- called GuardCtrl -- requires having a PDDL description of the environment and actions, and method to check if the safety constraints are violated. GuardCtrl works by using the LLM to generate a distribution over actions, and using a limited lookahead to filter out actions that lead to dead ends. This way we get safe plans and also a planning process that is biased towards the preference of the LLM (which are supposedly obtained from natural language description of what a human user wants).

There is an interesting evaluation against two strawmen baselines, showing improved results in terms of the amount of safe plans generated.

Critical strengths
  • The proposed method is interesting and timely - related to the big issue of governance of AI agents and safety.
  • I like the integration of a symbolic model with an LLM planner, and the one proposed here is somewhat novel.
  • An interesting experimental design.
  • I appreciate the honesty in the "AI Use Statement".
Critical weaknesses
  • Writing quality should be improved. There are quite a few phrases I could not make sense of, such as
    • Page 2: "... its support and its size do different jobs. ..." - what is the support of the lookahead? what are these jobs?
    • Page 2: "The constraint is no longer fixed" - which constraint? what is this referring to?
    • Page 2: Lines 99-102 do not make sense to me. What are "subword tokens" or "spelling" in this context?
    • In Fig. 2: "mitt and pick still needed: too late" - what is this?
  • The use of HMM is not well explains. I expected a definition of an HMM, and what exactly are you doing with it in a more formal and accurate manner.

See also my rebuttal questions, which mainly revolve on explains what you did.

Questions for authors

Q1. Line 470 in page 9: "As concurrent work (August 2026), it is discussed, not reproduced" - please explain this sentence. Q2. Line 137 in page 3: "... and leaves no obligation pending" - what are these obligations? Q3. Please clarify the notation used in Equation 2. e.g., what is X_{<t} Q4. Line 171 in page 4: "When the constraint..." - what is "the constraint"? also what is a "A linear context head"? Q5. Line 186 in page 4 mentions a "budget". What is this "budget"? Q6. Please explain Equation 3. The notation there has not been defined before (A_{zz'}, B_{z'},...) Q7. Page 6. Please explain the following sentence: "This checker equals the one that scores the output"

7 comments
Reviewer_ExDnOpenReview ↗

No authors response?

I am surprised the authors chose not to respond to any of our concerns.

AuthorsOpenReview ↗

Thank you for the reminder

Dear Reviewer ExDn,

Thank you for the reminder, and we sincerely apologize for the delayed response. The reviewers' questions touch on several components of our framework, and we wanted to address each of them with a dedicated experiment that is simple and easy to interpret. Designing these controlled experiments took considerably more time than we anticipated.

We are also grateful to all reviewers for their insightful suggestions, which have helped us make the work considerably more convincing. We have completed all the additional experiments addressing the concerns raised across the reviews and are organizing our point-by-point responses currently. We will post them within the next 12 hours and update the revised paper within the next 20 hours.

Best regards,

The Authors of Paper 59705

AuthorsOpenReview ↗

Response to Reviewer ExDn (1/4)

Dear Reviewer ExDn,

We apologize again for the delay in posting our detailed response and appreciate your patience. Thank you for your insightful review and for recognizing on the timeliness of our work on AI safety, the integration of a symbolic model with LLM planning, and our experimental design. Following your comments, we compare GUARDCTRL with an exact safe-support mask (A1-1) and conduct a controlled comparison of uniform and trained HMM emissions (A1-2). We clarify the terminology and the Figure 2 label (A2), and explain what the HMM models and how it guides generation (A3). We then answer Q1–Q7 in order, explaining the relationship to Meta-Ctrl (A4), pending obligations (A5), the notation in Equation 2 (A6), the acceptance condition and context head (A7), the token budget (A8), the recursion in Equation 3 (A9), and how the same checker is used for retries and final scoring (A10). We provide detailed point-by-point responses below.

Baseline comparison 1: Exact safe-support masking

AB-1: We agree that stronger baselines are needed to assess what probabilistic weighting adds beyond symbolic reachability. We therefore compare GUARDCTRL with an exact safe-support mask and find that both methods achieve comparable high safe success and preference following, while GUARDCTRL produces substantially shorter plans (Table 1).

Exact safe-support mask. GUARDCTRL weights each candidate token by QQ, the estimated probability of an accepted completion. The mask instead gives weight 1 whenever such a completion is reachable. We used the same parser, PDDL world, rule monitors, 512-token horizon, and temperature 1 on the 400 preference instances in Section 4.4. Under Proposition 1's assumptions, the allowed tokens are identical.

The mask averaged 22.2 actions to GUARDCTRL's 6.6, compared with 5.9 for accepted LLM plans. Its plans were longer on 291 instances and shorter on 29. The mask had a higher mean runtime (12.6 vs. 8.5 s) and a lower median (5.7 vs. 7.1 s). Five mask outputs of 86 to 98 actions exceeded the checker's time limit and count as unverified failures in S.

Table 1. Safe success, preference following, length, and runtime on 400 preference instances (Llama, Section 4.4). Percentage pairs are explicit / implicit requests, with 200 instances each. S counts safe completions. P also requires the preferred action.

MethodSafe success S (%) ↑Preference following P (%) ↑Length (actions), mean (median)Runtime (s), mean (median) ↓
Classical planner (blind to instruction)100.0 / 100.049.5 / 49.54.7 (4)1.0 (0.9)
Exact safe-support mask (new)98.0 / 99.595.0 / 86.522.2 (9)12.6 (5.7)
GUARDCTRL100.0 / 100.093.0 / 81.56.6 (6)8.5 (7.1)

On D-Short and D-Long (Section 4.2), mask/GUARDCTRL mean lengths were 24.1/7.1 and 24.6/7.4 actions, respectively, with similar safe success (98.0%/98.4% and 97.0%/98.2%).

Both plans below safely deliver a package downstairs.

  • GUARDCTRL (6 actions) (check_stairs) → (grab_package) → (clear_human) → (start_descending_stairs) → (complete_descent) → (place_package bottom_floor)
  • Exact mask (14 actions) (check_stairs) → (clear_human) → (grab_package) → (place_package top_floor) → (grab_package) → (place_package top_floor) → (grab_package) → (start_descending_stairs) → (place_package top_floor) → (grab_package) → (place_package top_floor) → (grab_package) → (complete_descent) → (place_package bottom_floor)

Putting the package down and picking it up four times adds eight actions. The mask allows each detour because delivery remains possible and leaves the choice to the LLM. GUARDCTRL can downweight a put-down if the HMM estimates that the extra pickup lowers the chance of an accepted finish. We did not trace the token probabilities in this example.

Reviewer_svZiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GUARDCTRL is a frozenLLM planner guided by supplied PDDL dynamics plus formal safety rules. It reweights the next token probabilities applying the probability of the distilled hidden markov reaching an accepted completion, tracking the parser, the hypothetical world states and the temporal rule monitors. It explores this product lazily and uses action/object symbols with declaration-conditioned emissions to allow for actions that were not seen during the distillation phase. Under exact computation and support assumptions the paper proves both soundness and bounded horizon completeness. Experiments were performed on DESPITE with supplied safety rules and an extension which includes temporal rules.

Critical strengths
  1. A integrated framework for enforcing feasibility and safety together. Tracking hypothetical PDDL transitions in conjunction with rule monitor states deals with the interaction between action applicability, goal reachability and temporal obligations. The distinction between the support of the look-ahead and the magnitude of that look-ahead in a probabilistic sense provides a useful way of conceptualising the approach. This is a very good thing as it provides a principled way of constraining language model planning within an explicit specification.

  2. Substantial empirical gains with a useful structured surrogate. The reported safe-success gains are seen across two language models and both state dependent and temporal rules. The spelling bias example provides a concrete motivation for the use of action/object symbols and the preference experiments look at information that a purely symbolic planner would not be able to make use of. Taken together this suggests benefit beyond just unconstrained generation and checking.

Critical weaknesses
  1. The experiments dont go far enough to isolate the contribution of probabilistic weighting beyond symbolic reachability. The key baseline would be to use the same parser, world model, rule monitors and horizon but substitute the look-ahead magnitude with a simple binary flag indicating whether or not an acceptable completion exists. The current local mask is considerably weaker and can get stuck while spelling out an inapplicable action. So its failure doesnt show the benefit of the learned completion probabilities. This is important as the probabilistic guidance and preference preservation are the key contributions of the approach, the classical planner already being able to solve the vast majority of the tasks. The lack of any demonstrable benefit from the learned emissions in the reported ablation still leaves the evidence for the learned components very weak.

  2. Several claims and reported figures require some resolution. The true language model look-ahead is required to achieve exact preservation of the conditioned language model distribution and exact inference under a distilled HMM does not achieve this. Section 3.4 is aware of this, but the abstract and conclusion are promising much stronger, so they kind of don't support each other. There's also an possible issues in Table 2. With 13.8% of the tasks being successful and the unsuccessful tasks using up 10 attempts each the mean number of attempts has to be at least (0.138 + 10*0.862) = 8.758 which is greater than the 8.56 reported. Also, the description of D-Long as "500 further" tasks is a bit ambiguous in the context of the 854 held-out tasks available.THese issues reduce the confidence in the main claims and the experimental reporting.

3.Llama’s feasibility under rejection sampling is 14%. This paper says that scores are computed from raw output without DESPITE’s output cleaner, and GuardCtrl’s parser enforces the format by construction. Some of the gap could therefore be formatting rather than planning.

(Minor issues:)

  1. LLM+P: It uses the 8B param model as a translator, and most of its failures are JSON schema violations. That makes the baseline weak for reasons unrelated to the paper’s argument about formalization limits.
  2. Text mentions 95 LLM+P failures but 94 invalid translations. SO presumably there is 1 other failure type; please explain.
Questions for authors
  1. Do the existing experiments incorporate a binary bounded-horizon reachability mask using precisely the same symbolic machinery as GUARDCTRL? If yes, how does it perform in terms of safe success, preference following and runtime? That would go some way towards establishing what contribution the look-ahead magnitude makes beyond its very basic support.
  2. Can you reconcile the reported 13.8% success rate with the 8.56 mean number of attempts under the stated 10 attempt rejection sampling protocol? Please can you clarify points with regard to early termination, runs which were excluded or any other differences in the denominator?
  3. What's the overlap between D-Short and D-Long and do both of those exclude all development work? Are the claims releated the effect of temporal rules actually backed up by a paired comparison on the same underlying tasks?
  4. Clarification: Is preference preservation meant to be an approximate empirical property rather than a guarantee for the distilled surrogate? What evidence actually measures fidelity to the conditioned language model distribution other than preference-following accuracy?

Meta-Ctrl is closely related concurrent work but a more precise positioning statement would have helped readers distinguish GUARDCTRL's lazy PDDL-based construction and declaration-conditioned emissions.

(Note: I haven't checked the long appendix; please refer to the relevant section if it's there)

6 comments
AuthorsOpenReview ↗

Response to Reviewer svZi (1/5)

Dear Reviewer svZi,

We apologize for the delay and appreciate your patience. Thank you for recognizing the value of enforcing feasibility and safety together and the empirical gains in our experiments. We compare probabilistic weighting with exact masking (A1-1) and trained with uniform HMM emissions (A1-2), then assess agreement with accepted LLM plans (A2-1). We explain the rejection-sampling attempt counts and full rerun (A2-2), report task overlap, development use, and the paired temporal-rule study (A2-3), and measure how much format cleaning improves safe success (A3). We also explain the LLM+P failures (A4) and how GUARDCTRL differs from Meta-Ctrl (A5).

W1-1 & Q1: Comparing probabilistic guidance with exact safe-support masking

A1-1: We agree that determining what probabilistic weighting adds beyond symbolic reachability is important. We therefore compare GUARDCTRL with an exact safe-support mask and find that both methods achieve comparable high safe success and preference-following rates, while GUARDCTRL produces substantially shorter plans (Table 1).

We tested the mask with GUARDCTRL's parser, PDDL world, safety-rule monitors, 512-token horizon, and temperature 1 on the 400 preference instances of Section 4.4. Under Proposition 1's assumptions, both methods allow the same tokens.

Long mask plans often repeat actions. The mask averages 22.2 actions versus GUARDCTRL's 6.6, and produces longer plans on 291 instances versus shorter plans on 29. Among the 105 mask plans with at least 15 actions, repeated actions account for 79% of actions per plan on average. GUARDCTRL is also closer in mean length to accepted LLM reference plans, which average 5.9 actions. A2-1 examines distributional agreement directly. The mask has a higher mean runtime (12.6 versus 8.5 s) and a lower median (5.7 versus 7.1 s). Five mask outputs contain 86 to 98 actions and exceed the official checker's time limit. They count as unverified failures in S.

Table 1. Safe success, preference following, length, and runtime on 400 preference instances (Llama, Section 4.4). Percentage pairs are explicit / implicit requests, with all 200 instances per phrasing in the denominator. S counts safe completions and P additionally requires the preferred action.

MethodSafe success S (%) ↑Preference following P (%) ↑Length (actions), mean (median)Runtime (s), mean (median) ↓
Classical planner (blind to instruction)100.0 / 100.049.5 / 49.54.7 (4)1.0 (0.9)
Exact safe-support mask (new)98.0 / 99.595.0 / 86.522.2 (9)12.6 (5.7)
GUARDCTRL100.0 / 100.093.0 / 81.56.6 (6)8.5 (7.1)

The same pattern holds on the main benchmarks in Section 4.2. The mask averages 24.1 versus 7.1 actions for GUARDCTRL on D-Short, and 24.6 versus 7.4 on D-Long, with similar safe success (98.0% versus 98.4% and 97.0% versus 98.2%, respectively). Some mask outputs of 58 to 98 actions exceed the official checker's time limit. The following is an example showing why masking leads to much longer plans.

One preference task shows where the extra actions come from. Both plans carry a package downstairs and pass the acceptance checks.

  • GUARDCTRL (6 actions) (check_stairs) → (grab_package) → (clear_human) → (start_descending_stairs) → (complete_descent) → (place_package bottom_floor)
  • Exact mask (14 actions) (check_stairs) → (clear_human) → (grab_package) → (place_package top_floor) → (grab_package) → (place_package top_floor) → (grab_package) → (start_descending_stairs) → (place_package top_floor) → (grab_package) → (place_package top_floor) → (grab_package) → (complete_descent) → (place_package bottom_floor)

The four rounds of putting the package back upstairs and picking it up again account for the eight extra actions. Putting the package down and continuing downstairs both leave an accepted completion reachable, so the mask gives the allowed tokens for either choice weight 1 and leaves their relative probabilities to the LLM. If the plan cannot yet end in acceptance, the mask blocks the end token but still lets the LLM repeat an earlier action, provided enough tokens remain to finish afterward.

GUARDCTRL weights those same tokens by the estimated probability of an accepted completion. Because putting the package down requires another pickup before delivery, the put-down gets less weight if the HMM estimates a higher chance of acceptance from proceeding downstairs. The trajectories illustrate how this difference in weighting can lead to shorter plans, although we did not trace the step-by-step probabilities for this example. We will include these comparisons in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer svZi (2/5)

W1-2: Effect of learned emissions on route selection

A1-2: In our controlled study, trained emissions better preserve the route choices of accepted LLM plans without a route request and improve following of explicit route requests (Tables 2 and 3). Uniform emissions can overweight unlikely failures on a feasible route.

Why uniform emissions hurt: an example. A filled pot is on an unlit gas burner and an electric kettle is also available. Stove rules require turning off the burner and removing the pot after use, and forbid four actions while the burner is on, including draping a towel over the rail. The user asks: “Please heat the water for tea on the gas stove.” Trajectories and numbers below are illustrative, not measurements.

Two normal, safe routes:

  • Stove (requested): (light_gas_burner) → (heat_water pot) → (turn_off_burner) → (remove_pot pot burner) → (pour_water pot cup) → EOS
  • Kettle (misses the requested method): (fill_kettle) → (switch_on_kettle) → (boil_water kettle) → (switch_off_kettle) → (pour_water kettle cup) → EOS

Low-probability failure trajectories under the LLM:

  • Unsafe action: (light_gas_burner) → (drape_towel stove_rail) → …
  • Unfulfilled obligations: (light_gas_burner) → (heat_water pot) → EOS

At unit guidance strength, GUARDCTRL reweights next-token probabilities by Q, the estimated probability of an accepted completion. In this two-choice, action-level illustration, suppose the LLM initially favors the stove 9:1:

  • Uniform emissions overestimate the unlikely failure trajectories above. Accepted mass of 2 versus 104 per 1,000 continuations gives stove/kettle Q = 0.002/0.104, a 52-fold advantage for the kettle. Reweighting then favors the kettle with probability 0.104 / (9 × 0.002 + 0.104) ≈ 85%.
  • Trained emissions better approximate the LLM's action correlations. Accepted mass of about 500 versus 360 per 1,000 gives Q = 0.50/0.36, preserving the requested stove route with probability 9 × 0.50 / (9 × 0.50 + 0.36) ≈ 93%.

Both safe routes remain available and the difference is the estimated probability of completing them safely.

Controlled experiment. In 24 scenarios, we fix actions and goals and add rules only to feasible route A. We compare No added rules, Preparation rules, and Preparation + completion rules. Preparation adds one follow-up obligation and two prohibited actions. The combined setting adds a second obligation and two more prohibited actions at completion. For the stove, the obligations are to turn off the burner after lighting it and remove the pot after heating. For each scenario, setting, and HMM, we sample 24 plans with no route request and 24 with an explicit request for A. All pass the checks.

Without a route request, uniform guidance increasingly underselects A as rules are added, while trained guidance starts with A about as often as the accepted LLM plans do (Table 2). Both routes remain feasible and all plans are accepted. The shift is consistent with uniform emissions assigning too much weight to unlikely failures on A.

Table 2. Relative frequency of starting with A without a route request (Llama, 24 scenarios). Entries are ratios to the frequency in accepted unconstrained LLM plans: 1 means equal frequency, below 1 means less frequent. In Tables 2 and 3, Trained HMM denotes the controlled-study configuration, not the submitted model.

MethodNo added rulesPreparation rulesPreparation + completion rules
Uniform HMM0.960.790.57
Trained HMM1.001.001.01

With an explicit request, that route bias becomes a loss of instruction following, and trained guidance largely avoids this loss (Table 3). Uniform guidance's A/B look-ahead ratio falls sharply as rules are added, weakening the LLM's preference for the requested route. At the strongest setting, request following is 64.9% with uniform emissions versus 96.5% with the trained HMM. The contribution of learning is therefore to preserve useful LLM preferences within the safe support: additional obligations need not cause a feasible, requested route to be excessively downweighted.

Table 3. Look-ahead weighting and instruction following with an explicit request for A (same 24 scenarios). Values follow the order: No added rules / Preparation rules / Preparation + completion rules. The A/B ratio divides estimated acceptance probabilities after each route's first step, summarized by the geometric mean across scenarios. Request following is the percentage of generated plans that carry out A and are accepted.

MethodA/B look-ahead ratioPlans following the A request (%) ↑
Uniform HMM1.00 / 0.095 / 0.01997.7 / 85.2 / 64.9
Trained HMM2.73 / 2.37 / 2.1698.4 / 93.9 / 96.5
AuthorsOpenReview ↗

Response to Reviewer svZi (3/5)

W2-1 & Q4: How closely generated plans follow the LLM's probabilities

A2-1: We agree that it is important to directly evaluate how closely GUARDCTRL preserves the LLM’s distribution over accepted plans. With a distilled HMM, exact preservation of this distribution is not guaranteed. We therefore compare complete-plan distributions on 16 tasks using KL divergence and total variation, and discover that GUARDCTRL is closer to the acceptance-conditioned LLM reference than exact masking (Table 4).

Table 6 in our paper (Appendix G) examines the relative look-ahead weights of two routes in a development example. We extend this local diagnostic to KL/TV comparisons of complete-plan distributions. The exact-mask control retains the same safe support, isolating the contribution of probabilistic weighting to plan selection. For each of 16 tasks selected by a criterion fixed in advance, we collect two batches of 100 accepted LLM plans through rejection sampling and 100 plans per method. The first LLM batch is the reference, and the second measures variation between samples from the same model.

GUARDCTRL reduces median estimated KL from the mask's 4.1 to 1.1, has lower KL on 14 of 16 tasks, and matches the reference's mean length. Reported TV intervals are [0.25, 0.49] for GUARDCTRL and [0.39, 0.59] for the mask. Compared with exact masking, probabilistic weighting better preserves which safe plans the LLM prefers and how often it generates them. Exact recovery of p⋆p^\star requires matching the true LLM acceptance-probability ratios (Section 3.4). We will revise the abstract and conclusion to distinguish the safe-support guarantee from the empirical evidence for approximate distributional fidelity.

Table 4. Complete-plan distribution comparison (Llama, 16 tasks). Total variation (TV) and KL divergence use the first batch of accepted LLM plans as the reference. TV ranges from 0 to 1, lower TV and KL indicate closer agreement.

MethodEstimated TV ↓Estimated KL (nats), median ↓Mean length (actions)
Rejection sampling (second batch)0.1404.9
Exact safe-support mask0.494.112.3
GUARDCTRL0.371.14.9

W2-2 & Q2: Explaining the mismatch between success rate and attempt count

A2-2: Thank you for pointing out this discrepancy. It came from interrupted runs: unfinished attempts were included in the mean, while these affected tasks remained failures in the full denominator. We therefore conducted an independent full rerun following the stated stopping protocol, which satisfies the attempt-count bound and yields a similar overall safe-success rate (Rerun 1). The other rejection-sampling rows also satisfy the bound, and GUARDCTRL’s advantage remains.

You are correct: at 13.8% safe success, the mean must be at least 0.138+10×0.862=8.7580.138 + 10 \times 0.862 = 8.758 if every unsuccessful task uses ten attempts. In the submitted run, however, 54 tasks stopped during their fourth attempt after three completed, rejected attempts, all remained in the 500-task denominator as failures. The reported 8.56 counted 4,282 attempt slots, including the 54 unfinished attempts, whereas only 4,228 produced outputs. We therefore reran all 500 tasks independently in Rerun 1, using batches of 24 and a different sampling random stream. Every task ran to acceptance or ten attempts, with 436 using all ten. At the rerun's 14.0% safe success, the same calculation gives 0.14+10×0.86=8.740.14 + 10 \times 0.86 = 8.74, consistent with the observed 9.21. We will replace the affected row of Table 2 in our paper and update dependent statistics.

The other three rejection-sampling rows contain no interrupted tasks and satisfy the bound: Qwen D-Short, 8.25≥7.508.25 \geq 7.50, Llama D-Long, 9.62≥9.429.62 \geq 9.42, Qwen D-Long, 8.81≥8.318.81 \geq 8.31. GUARDCTRL has 98.4% safe success, paired outcomes across 500 tasks are 423 solved only by GUARDCTRL, 1 only by rejection sampling, 69 by both, and 7 by neither (exact McNemar p<10−100p < 10^{-100}).

Rerun 1. Submitted run and full rerun of rejection sampling (Llama, D-Short, 500 tasks, at most 10 attempts). Protocol completion means reaching acceptance or ten attempts. F and S use all 500 tasks, parentheses in S give successful / total tasks. The submitted mean includes unfinished attempt slots.

RunProtocol completed (tasks / total)Feasibility F (%)Safe success S (%)Mean attempts per task
Submitted run446 / 50014.213.8 (69 / 500)8.56 (attempt slots)
Full rerun500 / 50014.414.0 (70 / 500)9.21

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviewers and Revision Submitted

Dear Area Chair and Reviewers,

We sincerely thank you for your careful evaluation and constructive feedback, and apologize for our delayed responses. We have posted point-by-point responses to all three reviewers and uploaded the revised manuscript. Below, we summarize the initial assessments and the main additions for convenient reference.

Summary of Reviews

The initial reviews included two weak acceptances (ExDn and N17C) and one weak rejection (svZi). The reviewers recognized the value of integrating symbolic world models with LLM planning: svZi highlighted the joint treatment of feasibility and safety and the empirical gains, ExDn appreciated the timely motivation and symbolic–LLM integration, and N17C valued the clear framing and motivation for the symbol-level surrogate.

The central concern was whether probabilistic weighting contributes beyond symbolic reachability, alongside requests for stronger baselines, clearer explanations, and more complete experimental reporting.

Summary of Rebuttals and Revisions

Main concernNew evidence and revisionsLocation
What does weighting add beyond exact safe support? (svZi, ExDn, N17C)Added an exact mask using the same symbolic machinery and horizon. Both methods achieve similarly high safe success, while **GUARDCTRL produces much shorter plans: a mean of 6.6 versus 22.2 actions across 400 preference instances with Llama, with similar reductions on both main sets. The mask follows preferences slightly more often, we therefore identify shorter plans, rather than higher preference accuracy, as the clearest benefit.§4.5, App. M.1
Does weighting preserve the LLM’s distribution, and when does learning help? (svZi, N17C)Added complete-plan comparisons on 16 tasks: GUARDCTRL reduces median estimated KL from 4.1 to 1.1 relative to exact masking. A separate 24-scenario study shows that trained emissions reduce route distortion as rules accumulate. We distinguish this study’s HMM from the main surrogate and clarify that distributional fidelity is empirical, not guaranteed.§3.4, §4.5, Apps. M.2–M.3
How competitive are stronger alternatives? (svZi, N17C)Added planner enumeration with LLM selection, action-prefix masking, weighted rejection sampling, and SMC. Enumeration is a strong alternative when candidates are available, GUARDCTRL solves 48 of the 61 main-set tasks where enumeration finds none within its budget. On the 50-task ablation, GUARDCTRL solves all tasks, versus 27 for action-prefix masking and 38/37 for five-particle weighted rejection sampling/SMC.§4.4–§4.5, App. M.4
Are the evaluation and reporting reliable? (svZi)Corrected the interrupted rejection-sampling run through a full 500-task rerun, clarified test-set overlap and development use, and audited formatting and translation failures. A new same-task temporal-rule comparison shows rejection sampling falling from 13.2% to 6.4%, while GUARDCTRL remains near 98%.§4.1–§4.2, §4.2, Apps. H, K, M.5–M.7
Are the method, cost, and limitations clear? (svZi, ExDn, N17C)Expanded the HMM definition, notation, examples, and positioning relative to Meta-Ctrl. Added timings across all 1,000 tasks, qualified the sample-based overhead estimate, and explicitly acknowledged compilation costs, untested larger-world scaling, and possible loss of completeness through numerical underflow.§2–§3, §4.3, §5–§6, Apps. C, M.8, N

These additions clarify the contribution: safe reachability provides the foundation, while probabilistic guidance yields substantially shorter plans and closer empirical agreement with the LLM’s distribution over accepted plans. We appreciate the reviewers’ suggestions for making this distinction testable and explicit, and respectfully ask that the new evidence and revised manuscript be considered in the assessment.

Thank you again for your time. We welcome any remaining questions.

Best regards,
The Authors of Submission 59705

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.