acceptodds
Under review as a conference paper at ICLR 2027

MetaAct: Self-Evolving Executable Action Constraints for Embodied Language Agents

Abstract

Self-evolving language agents have emerged as a promising paradigm for continuously improving agent behavior through experience. However, most existing approaches evolve textual artifacts, which can influence action preferences but cannot reliably enforce action admissibility in embodied environments with discrete and often irreversible actions. We propose **MetaAct**, which shifts self-evolution from textual guidance to executable action constraints by synthesizing **GEAR** (Grounded Executable Action Rules) from execution feedback. Each rule is represented as a typed specification, compiled into a deterministic guard, and validated offline through trajectory replay without additional model calls or environment interactions. Across four benchmarks, **MetaAct** improves task performance; on ScienceWorld and WebShop, violations drop from 70% to 0% and 40% to 1%, with score gains of +10.1 and +8.0. Our code is available at https://anonymous.4open.science/r/MetaAct-D567.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Self-Evolving Agents, Embodied Language Agents, Executable Action Constraints

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_hQHYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses a clear and practical research problem: converting lessons from experience into constraints at the action execution interface can prevent language agents from repeatedly taking actions they already know should be avoided. The paired experiments and the decomposition of action masking and rule messages provide valuable evidence. However, the results most strongly support the narrower conclusion that execution-time blocking and feedback help under specific rules and task subsets. The evidence and experimental protocol descriptions remain insufficient to establish reliable offline verification, general-purpose automatic rule discovery, or performance gains attributable to self-evolution across all four environments.

Critical strengths
  1. The method is well designed. MetaAct uses failed and successful trajectories as contrastive evidence, prompting an LLM to construct typed GEAR specifications from six manually designed primitives and predefined goal extractors. A fixed compiler converts these specifications into deterministic guards, which reject rule-violating actions before execution and return trigger messages to the planner.

  2. The framework is comprehensive, and the evaluation covers a diverse set of benchmarks.

Critical weaknesses

W1. Replay verification does not establish effectiveness after intervention. It measures blocking on historical actions, but cannot determine whether the resulting alternative trajectory improves task completion or causes harmful restrictions.

W2. The main results do not sufficiently isolate GEAR’s contribution. Same-scaffold guard-on/off comparisons and simpler rule-selection baselines are needed to attribute gains to the proposed constraints and synthesis procedure.

W3. Online self-evolution is demonstrated only in a limited setting. The system quickly acquires one specification and then remains unchanged, providing little evidence of sustained refinement, accumulation, or retirement.

W4. Repetitive and formulaic writing reduces readability. The experimental discussion often restates table entries without highlighting the core findings, relevant comparisons, and limitations.

Questions for authors
  1. How are harmful and catastrophic constraints labeled in Figure 5, and how is the candidate pool constructed? Is the held-out split defined by trajectory or task type? Are the labels independent of the eff/ok proxies, and do these metrics predict actual task-performance degradation after deployment?

  2. Can the authors provide guard-on/off comparisons using the same MetaAct scaffold, particularly on ALFWorld and TextCraft, distinguishing existing fixed guards from newly synthesized ones? How do single-round synthesis or simple rule selection perform with the same evidence?

  3. Please clarify the configurations, deployed rule specifications, seeds, and evaluation subsets used in Tables 1/2/3/8. Why does the SciWorld 70%→0% violation result use 20 episodes, whereas Figure 4 uses 16? How do the 17 use-thermometer constraints relate to the four shown in Figure 9?

  4. What is the full-stream paired score difference between online MetaAct and the no-learning/textual baselines? Do the gains persist under different episode orders? Please clarify whether the current evidence is limited to one specification accepted early in the stream.

Reviewer_8uFYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MetaAct, a framework that synthesizes structured rules from successful and failed agent trajectories and compiles them into executable constraints that block violating actions. Candidate rules are validated offline through trajectory replay before deployment. The authors report performance improvements across four interactive benchmarks, with target-rule violation rates dropping from 70% to 0% on ScienceWorld and from 40% to 1% on WebShop.

Critical strengths

The paper studies a practically relevant gap between knowing a rule and following it, exploring how experience can be converted into enforceable action constraints.

Critical weaknesses
  1. Offline replay cannot establish deployment benefits. Blocking an action changes subsequent decisions and state distributions, which historical trajectories cannot capture. Replay therefore validates blocking behavior, not downstream task improvement.

  2. Failure coverage lacks causal grounding. Blocking any action in a failed trajectory can earn coverage credit, even if that action did not cause the failure. This objective may favor constraints with high coverage but little practical value.

  3. Risk of adaptive overfitting to sparse evidence. Repeatedly generating, revising, and selecting rules on very few trajectories may turn incidental correlations into hard constraints. Sparse successful trajectories also limit reliable estimation of false-blocking risk.

  4. The definition of effective actions is unreliable. Using observation changes as a proxy for action effectiveness overlooks hidden state changes and information-gathering value, potentially causing necessary actions to be classified as ineffective and blocked.

  5. The violation metric is partly self-confirming. Using the same rules to enforce constraints and measure violations makes lower violation rates partly a consequence of the mechanism itself, rather than independent evidence of rule correctness. The zero-violation result on ScienceWorld also applies only to a specific rule family.

  6. Self-evolution is limited in scope. The method relies on manually predefined rule primitives and extractors, primarily selecting rules and instantiating parameters. No rules are generated for some tasks, and evidence for long-term adaptation and correction of erroneous rules remains limited.

  7. The experiments do not sufficiently isolate the core contribution. Comparisons omit strong optimizers such as GEPA and alternative search methods operating within the same rule space, while token consumption varies substantially across methods. Evaluation on unseen task types, distribution shifts, and extended interaction streams is also insufficient to support broad claims about generalization and efficiency.

Questions for authors
  1. Offline replay cannot observe the alternative decisions made after an action is blocked. Could the authors further validate whether rules that pass replay consistently improve actual task performance?

  2. Failure coverage counts blocking any action in a failed trajectory as successful coverage. How does the method distinguish actions that caused the failure from otherwise reasonable actions in the same trajectory?

  3. Could the authors compare against optimizers such as GEPA under the same rule space, replay scoring, and computational budget to separate the contribution of the search method from those of the rule representation and hard enforcement?

Reviewer_KHRVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MetaAct, a framework that synthesizes executable action constraints (GEAR) from agent execution trajectories. The constraints are compiled into deterministic guards and validated through offline trajectory replay before deployment. Experiments across four benchmarks demonstrate improvements in task performance and reduced action violations.

Critical strengths
  1. Practical motivation. Converting execution experience into enforceable constraints can help agents avoid repeating known mistakes.
  2. Empirical evaluation. The paper evaluates multiple benchmarks and includes ablations of the constraint synthesis and verification mechanisms.
Critical weaknesses

1. Limited methodological novelty.

The core idea closely resembles existing action masking, shielding, and runtime constraint methods, including AgentSpec (Wang et al., 2026) and OntoGuard (Liu et al., 2026). MetaAct mainly uses an LLM to select and instantiate predefined rules, followed by trajectory-based filtering, without introducing a substantially new mechanism for learning or optimizing action constraints.

2. Limited constraint expressiveness and generalization.

GEAR is restricted to six manually predefined rule primitives and cannot discover new constraint structures. Appendix B.2 shows that constraints are synthesized for only 10 of 30 ScienceWorld task types, while no constraints are produced on ALFWorld (Appendix C.3), raising concerns about the method's general applicability.

3. Limited reliability of offline verification and unclear violation reporting.

Offline replay only checks whether constraints would block actions in recorded trajectories, without evaluating the outcomes of subsequent replanning. Figure 5 shows that accepted constraints still exhibit a 6.5% harmful blocking rate, while Appendix D.3 clarifies that the reported 0% violation rate on ScienceWorld applies only to focus-related rules, with 78 additional disagreements under the unrestricted evaluator. These limitations weaken the claims about reliable constraint enforcement.

4. Insufficient evidence for continuous self-evolution.

The paper emphasizes continuous evolution of executable constraints, but Section 3.3 states that only 2 of 23 ScienceWorld constraints reach the four-use threshold, leaving the retirement mechanism inactive in the reported results. Although Section 4.6 demonstrates online constraint acquisition on WebShop, the evaluation is limited to a single 100-episode stream and does not convincingly establish sustained self-improvement.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.