acceptodds
Under review as a conference paper at ICLR 2027

Safe, Persistent, and Evolving Agent Harness for Understanding Partially Observable Worlds

Abstract

Large language model agents can invoke tools fluently, but enterprise workflows demand more than selecting the right tools: actions must strictly comply with organizational policies, tool feedback often conceals hidden side effects under partial observability, and long-horizon tasks require persistent state tracking across multiple records. To address these challenges, we introduce E-Ledger, a multi-agent harness for safe and persistent execution. E-Ledger employs a code approval layer that checks every proposed action against policy before execution, and maintains a world ledger of verified hidden rules alongside evidence-backed dynamic state. Because hidden rules are typically unknown a priori, we further propose WorldAbduct, an abductive, world-model-driven harness evolution framework. WorldAbduct diagnoses execution trajectories across four complementary views (state consistency, world-observation gap, policy-gate correctness, and goal judgment) to hypothesize latent rules, and verifies them through targeted abductive interactions before integrating them into the ledger. On the enterprise benchmark World of Workflows, E-Ledger with WorldAbduct improves safe task completion across four LLM backbones, outperforming the strongest evolution baseline by 5–15 percentage points. Experiments in ScienceWorld and DiscoveryWorld further show that abductive harness evolution carries over to scientific environments.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
llm agent, agent harness

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_umU3OpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper aims to solve the key problem: how agents perform safely and consistently in a partial-observed and complex enterprise-level tasks. This paper introduce E-Ledger, a multi-agent harness for safe and persistent execution. E-Ledger employs a code approval layer that checks every proposed action against policy before execution, and maintains a world ledger of verified hidden rules alongside evidence-backed dynamic state.

Critical strengths
  1. The research question is interesting and of great significance. We do need enterprise agent to be deployed in a partial-observed environments. By combining "non-bypassable" and "unobserved side effect", E-Ledger is more promising than naive prompt enhancement or other methods.

  2. The framework architecture is well-designed. Agents are clearly separated, with well-defined boundaries for greatly taking each responsibility.

Critical weaknesses
  1. It seems only comparing the test-time cost (as shown in Table 1). How about training-time cost? Compared to other methods, does E-Ledger have an advantage in terms of training costs?
  2. The data scale of the benchmark test appears to be small. It may not effectively demonstrate the advantages of E-Ledger.
Questions for authors
  1. It seems only comparing the test-time cost (as shown in Table 1). How about training-time cost? Compared to other methods, does E-Ledger have an advantage in terms of training costs?
  2. I think single-run is acceptable limitation in the field. But you list it as key limitation (as shown in Appendix G). What other limitation does E-Ledger may have?
1 comment
Reviewer_umU3OpenReview ↗

Comments by Reviewer umU3

Thanks for the author's response. Although I have raised my rating to 3, I have lowered my confidence level, and I consider this a borderline paper. I hope AC may take the feedback from other reviewers into careful consideration.

Reviewer_Z56xOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces E-Ledger, a multi-agent harness designed for safe task execution in partially observable enterprise environments. E-Ledger combines a code-based policy approval layer with a persistent world ledger that maintains verified hidden transition rules (static knowledge) and task-relevant dynamic state. The authors further propose WorldAbduct, a harness evolution framework that analyzes execution trajectories through four diagnostic perspectives: state consistency, world-observation gap, policy-gate correctness, and goal judgment. Candidate hidden-rule hypotheses are evaluated through targeted interactions in fresh environment instances before being incorporated into the static world ledger.

The approach is evaluated primarily on World of Workflows (WOW) using four LLM backbones, with additional experiments on ScienceWorld and DiscoveryWorld. On WOW, WorldAbduct achieves safe task completion rates of 65–80%, outperforming evolution baseline by 5–15 percentage points. Component ablations and a manual audit of discovered rules provide evidence for the importance of persistent world knowledge and hypothesis verification. The scientific-environment experiments suggest that the approach can also be applied outside enterprise workflows.

Critical strengths

The paper addresses an important limitation of tool-using agents, observable tool feedback may not reflect the complete consequences of an action. The proposed combination of persistent state tracking, executable policy enforcement, and targeted verification of hidden-rule hypotheses seems technically coherent and useful. In particular, the distinction between learning from observed trajectories and actively testing alternative explanations is a great design choice. The cost-revaluation case study illustrates how verification can correct an initially plausible but incorrect hypothesis about an environment's transition dynamics.

Authors evaluate multiple LLM backbones on WOW, provide component and evolution-view ablations, and extend the approach to two different domains. While these experiments have important limitations, the results provide preliminary evidence that persistent world knowledge and active verification can improve execution reliability and policy safety.

Critical weaknesses

The primary WOW evaluation contains 20 test tasks, and the authors explicitly report single-run results without confidence intervals, standard deviations, or repeated trials. Consequently, each 5-percentage-point improvement corresponds to just one additional successful task. The reported 5–15 percentage-point improvement over MemoHarness therefore represents only one to three additional successful tasks, depending on the backbone. For example, the improvement on GLM-5.3-Flash is from 65% to 75%, corresponding to 13 versus 15 successful tasks. Without paired per-task outcomes or repeated evaluations, it is difficult to determine whether these differences provide robust evidence of superiority. Am I missing something here?

More fundamentally, the training, validation, and test splits share the same task families, with different variants assigned to each split. Thus, the evaluation primarily measures whether discovered rules transfer across variants of already encountered workflows, rather than whether the method generalizes to previously unseen hidden dynamics. A similar concern applies to ScienceWorld and DiscoveryWorld, where task types or discovery themes are shared across splits. I think the improvements over WorldEvolver on these scientific benchmarks are also small, approximately one additional successful test episode in each environment. I consider the existing evaluation does not sufficiently establish the statistical reliability of the gains or their generalization beyond previously encountered mechanisms.

On the other hand, WorldAbduct differs from the compared evolution methods not only in its use of abductive reasoning but also in its ability to perform additional targeted environment interactions. Appendix E reports approximately 146–380 additional verification interactions per backbone during evolution. Although Appendix F provides MemoHarness and WorldEvolver with one additional evolution round, the paper does not establish that these methods receive comparable environment-interaction, token, or computational budgets. An additional evolution round is not necessarily equivalent to the information obtained through targeted experimentation.

The ablations also leave the mechanism of improvement partly unresolved. On GLM-5.3-Flash, removing abduction leaves task completion unchanged at 80%, while policy safety decreases from 95% to 85%. This suggests that the measured benefit of verification is primarily associated with safety rather than improved task completion, but the experiment does not distinguish the effects of abductive hypothesis revision, additional observations, and filtering of uncertain rules.

Furthermore, the formalization describes the diagnostic objectives and abduction process largely through LLM-based proposal and decision interfaces, without a clearly specified criterion for hypothesis verification or identification. It remains unclear how systematically the method distinguishes competing causal explanations or determines whether a rule's applicability conditions have been sufficiently established.I think the current results do not clearly show that abductive world-model learning is what drives the improvements, even though this is a central contribution of the paper.

Questions for authors
  1. Given that WOW has only 20 test tasks and performance differences often correspond to one or two episodes, can the authors provide paired per-task outcomes, uncertainty estimates, or results across independent runs? Are the improvements over MemoHarness and WorldEvolver consistent across task families, or concentrated in a small number of tasks?

  2. Since all WOW task families appear in both training and testing, to what extent do test tasks rely on hidden transition mechanisms already discovered during training? Can the authors distinguish performance on previously learned mechanisms from performance on genuinely unseen hidden rules, or provide a held-out workflow-family evaluation?

  3. What are the total evolution-time environment interactions, LLM calls, and computational costs for WorldAbduct and each baseline? Would WorldAbduct retain its advantage if the competing methods were given equivalent active-interaction budgets? In particular interested in understanding how much improvement comes from abductive hypothesis revision and verification rather than simply obtaining additional observations through targeted exploration?

  4. What operational criteria determine that a hidden-rule hypothesis has been sufficiently verified? How does the abduction agent distinguish between competing explanations that agree with the observed validation trajectory, and how are overly broad or overly narrow applicability conditions handled?

  5. Could the authors report results separately for feasible and infeasible WOW tasks? Appendix A.2 indicates that a correct infeasibility label is sufficient for task-completion credit, without evaluating the correctness of the refusal explanation. It would therefore be useful to determine how much of the STCR improvement comes from successfully executing feasible workflows versus correctly refusing infeasible tasks. Additionally, how frequently do policy violations arise from incorrect or stale policy-context bindings despite the deterministic code approval layer?

Reviewer_iEszOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes E-Ledger, a multi-agent framework for improving the safety and reliability of LLM agents in partially observable environments. It further introduces WorldAbduct, which identifies and verifies hidden environmental rules through trajectory analysis and targeted interactions. Experiments on WOW, ScienceWorld, and DiscoveryWorld demonstrate improvements in task performance and safety.

Critical strengths

The paper addresses important challenges in reliable LLM agents, including policy compliance, persistent state tracking, and hidden environmental dynamics. The proposed framework provides a reasonable architectural solution to these problems. The paper evaluates four LLM backbones on WOW and extends the experiments to ScienceWorld and DiscoveryWorld. The ablation studies and hidden-rule verification examples provide useful insights into the framework's behavior.

Critical weaknesses

The WOW evaluation contains only 20 test tasks, meaning that the reported 5–15 percentage-point improvements correspond to just 1–3 additional successful tasks. Moreover, all configurations are evaluated using single runs, without confidence intervals or statistical significance analysis. The train/test split also uses different variants of the same task types, limiting the evidence for generalization to unseen workflows. The improvements on ScienceWorld and DiscoveryWorld are similarly small, making the claimed generalization benefits insufficiently convincing. The framework primarily combines established techniques, including persistent memory, policy checking, trajectory-based reflection, and environment verification. Its methodological novelty beyond this integration is not clearly demonstrated. Furthermore, WorldAbduct introduces substantial additional environment interactions during training, while the baselines are not evaluated under fully matched exploration and computational budgets. Although ablation experiments are provided, they do not sufficiently distinguish the contribution of the proposed four-view diagnosis from the general benefits of active hypothesis verification. The figures contain several noticeable errors that reflect insufficient proofreading and quality control. For example, “priority” is misspelled as “priorty” in Figure 1, while Figure 2 contains “predition” instead of “prediction” and “ommited” instead of “omitted.” Such avoidable errors in the main framework illustrations undermine the professionalism and clarity of the presentation. The authors should carefully review and correct the figures.These issues further weaken the overall presentation quality.

Questions for authors

Q1. Statistical reliability and task-level results Could the authors provide repeated evaluations with different random seeds and report confidence intervals or paired task-level results? Given that WOW contains only 20 test tasks, how robust are the reported improvements? Could the authors also separately report successful completion of feasible tasks and correct refusals of infeasible tasks? Q2. Experimental fairness and contribution attribution Could the authors provide a more comprehensive comparison of the total evolution budgets, including LLM calls, token consumption, and environment interactions? How would WorldAbduct compare against baselines equipped with a similar active-verification budget? In particular, what evidence demonstrates that the four-view diagnosis contributes beyond generic hypothesis testing? Q3. Generalization to unseen environmental dynamics Since the WOW evaluation uses held-out variants of previously encountered task types, to what extent can the learned hidden rules generalize to genuinely unseen workflow categories or new environmental transition mechanisms?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.