acceptodds
Withdrawn from ICLR 2027

Org-Agent: Beyond Personal Assistants Towards Organizational Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language model agents serving organizations must coordinate requests from multiple users while using knowledge distributed across their interactions. We identify two complementary capabilities for this setting, namely cross-user interaction and decision-making, as well as cross-user memory and knowledge use. Both capabilities are governed by organizational constraints across three aspects: user identity, authority, and access permissions; the attribution and temporal validity of information; and rules for resolving conflicting requirements across users and completion requirements for joint decisions. These constraints shape what information or decisions must be obtained before an action can proceed and what conditions must be satisfied during its execution. Motivated by this, we introduce Org-Agent, a unified constraint-centric reasoning framework that organizes task execution in three stages. Specifically, Org-Agent decomposes a task into atomic subtasks and constructs a task dependency graph whose edges encode the dependencies among them. Building on this graph, it schedules the subtasks in dependency order through topological sorting. It then executes each subtask while accounting for the task's constraints, supported by evidence-acquisition and memory-management tools. Experiments on MUSES-Bench and GroupMemBench demonstrate the effectiveness of Org-Agent on both capabilities, and ablations further support the contributions of dependency modeling and tool use. Our code is available at https://anonymous.4open.science/r/ORG-AGENT-80C3.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Organizational Agents, Organizational Constraints, DAG

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_o4xMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates an agent architecture, termed ORG-AGENT, designed to coordinate multi-user requests and leverage knowledge distributed across multiple team members in organizational contexts. The authors identify two primary capabilities required for this setting: cross-user interaction and decision-making, and cross-user memory and knowledge use. To address these problems, the system decomposes tasks into atomic subtasks, models their dependencies via a dynamic Task Dependency Graph (TDG), schedules node execution using topological sorting, and executes nodes while referencing organizational constraints with the aid of evidence-acquisition and memory-management tools. The approach is evaluated across two recent benchmarks, MUSES-Bench and GroupMemBench, where the authors demonstrate improvements over naive baseline prompts and existing agent memory frameworks.

Critical strengths

Dynamic formulation of multi-user dependencies into an executable graph: Translating unstructured, multi-party requests and organizational authority constraints into an on-the-fly task dependency graph (TDG) that updates as new conversational inputs arrive. In a traditional systems, database, or workflow conference, the work would likely be critiqued for reinventing known wheels. In the current LLM agent literature, it is accepted primarily as a useful empirical baseline that applies 30-year-old systems principles to compensate for the reasoning limits of language models.

Unified constraint-guided execution across coordination and memory: Integrating topological subtask scheduling with metadata-filtered evidence retrieval and memory management to jointly handle cross-user decision-making and cross-user information retrieval within a single framework.

Critical weaknesses

1. Insufficient related work survey Workflow research and development has a long-lasting history. Unfortunately, this paper does not cite the foundational literature on workflow management, business process execution, or distributed coordination. Its citations are confined almost entirely to the recent natural language processing and LLM agent bubble from 2023 to 2026 (such as MemGPT, Mem0, Zep, HippoRAG, GroupMemBench, and MUSES-Bench). When the authors introduce directed acyclic graphs, they cite a 2021 graph neural network paper: Directed Acyclic Graph Neural Networks (Thost & Chen, ICLR 2021). This citation refers to representation learning on DAGs with GNNs, rather than the decades of established systems literature on task dependency modeling, workflow scheduling, and organizational access control.

Example references:

  1. The Application of Petri Nets to Workflow Management (Wil M.P. van der Aalst, Journal of Circuits, Systems and Computers, 1998).

  2. Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing (Topcuoglu, Hariri, & Min-You Wu, IEEE Transactions on Parallel and Distributed Systems, 2002).

  3. Web Services Business Process Execution Language (WS-BPEL) 2.0 (OASIS Standard, 2007).

2. Ignoring all failure modes Using LLMs to facilitate the authors' target tasks is fine, but the novelty is low. The serious issue is that LLM itself suffers from several well known shortcomings. Without addressing these know issues, the resulting system may be full of bugs. Some well know shortcomings are:

  1. Cyclical or Invalid Graphs: Generates circular dependencies (A -> B -> A) or drops prerequisites.

  2. Permission Leaks: Leaks restricted multi-user data through prompt injection or context bleed.

  3. Constraint Violations: Approves decisions that breach policy, such as spending caps or required sign-offs.

Questions for authors
  1. Please discuss what the traditional workflow could not support the two features your system was designed to faclitate?

  2. Please address how you plan to deal with LLMs' intrinsic shortcomings enumerated above?

Reviewer_rXP5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Org-Agent is a framework for a shared LLM agent serving multiple users. It decomposes pending requests into a dynamically updated task dependency graph, schedules nodes topologically, and executes them with evidence-acquisition and memory-management tools. The framework targets both cross-user coordination and question answering over multi-user histories. Experiments on MUSES-Bench and GroupMemBench report higher aggregate scores than the evaluated baselines, with additional ablations, backbone comparisons, and token-cost analysis.

Critical strengths
  1. The paper identifies a useful shared structure across multi-user coordination and memory tasks: actions depend on evidence, permissions, and decisions distributed across users. The integration of dependency planning with metadata-aware retrieval is coherent and addresses a practically relevant setting.
  2. The evaluation includes two complementary benchmarks and multiple backbones, and the results show meaningful gains in some important tasks. With GPT-4o-mini, meeting success increases from 37.04% to 60.19%, and GroupMemBench accuracy increases from the strongest evaluated baseline’s 38.79% to 44.03%. The dependency ablations provide positive evidence for the scheduling mechanism on both benchmarks; the build-plus-query token-cost analysis is also helpful.
Critical weaknesses
  1. The aggregate improvement obscures a substantial failure on a central organizational constraint. For Qwen3-32B, Privacy drops from 64.28% to 47.45%, while GPT-4o-mini loses 8.44 percentage points of authorized-access Utility (Table 2). Since access permissions are part of the paper’s core motivation, an unqualified average over these metrics is insufficient to support improved constraint handling. The method describes LLM-mediated constraint-aware execution, but does not establish that the relevant permissions are enforced reliably or explain these regressions.
  2. The applicability of the dynamic DAG to tasks that invalidate completed work is not sufficiently characterized. Section 4.2 allows updates to the remaining plan and graph reconstruction, so I do not assume that the framework cannot handle iterative negotiation or feedback. However, a revoked approval or a changed availability constraint can invalidate an already completed decision and its downstream results. It remains unclear how the framework detects and propagates such invalidation, revisits completed nodes, and keeps execution memory consistent. The evaluation does not separately establish performance on these cases. This is an insufficiently validated applicability boundary, rather than a demonstrated inability of DAG-based planning to handle dynamic tasks.
Questions for authors
  1. What failure modes cause the 16.83-point Privacy regression for Qwen3-32B and the Utility regression for GPT-4o-mini? Are access rules enforced by a deterministic layer or only represented in the LLM context? Please report the privacy–utility trade-off separately from the aggregate score.
  2. When a new user input invalidates an already completed subtask (e.g., approval is withdrawn or a participant changes availability), how are affected downstream decisions and memory entries invalidated and recomputed? Please distinguish this from updating only the pending plan.
  3. Do the benchmarks include iterative negotiation or revisions that invalidate completed work? Please identify such cases and report their outcomes or provide a focused evaluation, including stale-decision failures. This would clarify the framework's applicability boundary.
Reviewer_tmwmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ORG-AGENT, a constraint-centric framework for language model agents serving multiple users in organizational settings. It addresses cross-user interaction and decision-making, as well as cross-user memory and knowledge use. The framework decomposes tasks into a dependency graph, schedules subtasks through topological sorting, and executes them using constraint-aware retrieval and memory-management tools. Experiments on MUSES-Bench and GroupMemBench demonstrate improvements over the evaluated baselines, supported by ablation and scalability analyses.

Critical strengths
  1. The paper addresses an important challenge in extending personal assistants to organizational settings, where agents must handle multiple users with different authorities, information access permissions, and potentially conflicting requirements.
  2. The combination of task dependency modeling, dependency-aware scheduling, and constraint-aware execution provides a clear and practical approach to organizing complex multi-user workflows.
  3. ORG-AGENT improves overall performance across both benchmarks and multiple LLM backbones. The ablation studies, user-scaling analysis, and cost evaluation further support its practical effectiveness.
Critical weaknesses
  1. The core components, including LLM-based task decomposition, DAG scheduling, metadata-aware retrieval, and memory management, are largely established techniques. The paper does not sufficiently demonstrate what is fundamentally new beyond their integration into a unified framework.
  2. Despite emphasizing organizational constraints, ORG-AGENT does not consistently improve privacy and access control. In particular, privacy drops by 16.83 percentage points with Qwen3-32B, while access utility decreases by 8.44 points with GPT-4o-mini. These regressions raise concerns about the reliability of constraint-aware execution.
  3. On MUSES-Bench, the framework is compared only against a vanilla agent, making it difficult to distinguish the benefits of the proposed dependency graph from those of additional planning, tool use, and structured prompting.
  4. The TDG and its dependencies are generated by an LLM, but the paper does not directly evaluate their correctness or the reliability of dynamic graph updates. The relatively small gains from dependency modeling in the ablation study further motivate a more detailed analysis.
Questions for authors
  1. How does ORG-AGENT differ from existing LLM planning frameworks that combine task decomposition, dependency scheduling, and tool-augmented execution? Could the authors compare against stronger planning baselines under matched inference budgets?
  2. Why does privacy performance decrease substantially with Qwen3-32B, and access utility decline with GPT-4o-mini? Could the authors analyze these failures and clarify how organizational constraints are enforced during execution?
  3. How accurately does the LLM identify task dependencies, and how robustly are they updated when new user inputs arrive?
  4. Could the authors further separate the effects of metadata filtering, relation traversal, execution memory, and dependency scheduling to clarify which components drive the observed improvements?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.