- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AtxServe is a system designed to deal with wasted work across AI agents working and editing the same shared files. It avoids outdated agent edits that cause GPU time to be spent on a discarded and regenerated version by putting a gatekeeper in front of the infrerence server that tracks conflicts. When requests conflict, they wait. When they do not, multiple non-conflicting requests are admitted at once, starting from the least conflicting ones. The key idea is to decide conflicts before spending any GPU time, thereby allowing to reduce wasted work and achieve better GPU packing. The authors implement AtxServe and demonstrate up to 50% increase in throughput and zero stale or wasted edits.
Critical strengths
(S1) Thank you for the submission. Allowing agents to update shared documents efficiently is a real and important problem in practice. I think asking the question at the serving layer is an interesting (albeit some unconventional) take on the problem.
(S2) The paper is overall well written and is clear about its scope, in particular with respect to declaration requirements and exclusion of external effects.
Critical weaknesses
(W1) I think the paper asks an interesting question: how can a shared serving layer help avoid inference waste on edit conflicts? However, it does not convince that the problem shows up in practice in the form it describes. The authors do not evaluate on real agent frameworks and instead focus on synthetic tasks. Similarly, a number of obvious alternatives are not considered, e.g., merging, prompting again with changed inputs, or using finer-grain blocks. The paper also assumes that the read and write sets predeclared before inference. Hoewver, the experiments replay fixed sets so it is not clear that agents can automatically produce them correctly. Overall, I find the question intriguing, but I am not convinced that serving is the right layer or that the specific problem appears in practice outside of the paper's own limited setup.
(W2) I feel the novelty of the paper is relatively limited compared to the broader area of currency control. Much of the design has direct precedence in distributed systems and database design, such as contention-aware lock scheduling, reordering operations using predeclared sets, conflict graphs, etc. The paper claiming to “reformulate concurrency control as a GPU scheduling problem” is a weird overstatement since vLLM remains unmodified and the proposed solutions run on top of it. Similarly, calling what the system does “transactions” is somewhat confusing as it does not deal with conflicts across the task stages agents go through, i.e., there is no atomicity or isolation for an agentic task. As far as I can tell, the equivalent primitive to AtxServe in databases would be read-modify-write, which is absolutely not equivalent to a transaction.
(W3) The paper's benefits are measured and reported mostly against the wrong baseline. Optimistic concurrency control (OCC) loses because it regenerates stale edits. This mostly comes from the fact that two edits to the same file will conflict even if they touch different locations and the rejected edit is therefore regenerated in full, which means that simple locking beats OCC under contention. However, on simple locking, AtxServe only achieves benefits because it packs request batches better as shown in Table 2. In other words, if the goal is to not waste GPU on stale edits, simple locking is a good enough solution and better batch packing is a distinct contribution that can be completely separated from the locking aspects.
(Minor) The fit is potentially questionable for ICLR. My score does not reflect this, but in the event the paper is not accepted at ICLR, I would advise the authors to consider systems or database venues as a potential resubmission target. I feel they would be a more natural home for this kind of work.
Questions for authors
Please see the critical weaknesses above for context.
(Q1) Can you better support your serving-layer approach, e.g., through an argument or comparisons with alternative designs?
(Q2) Please explain the choice of terminology related to transactions.
(Q3) Are there any benefits beyond better packing when comparing with the right baseline?
3 comments
Response to Reviewer Bfhj (2 of 3)
We address the alternative designs in three parts.
1. Text merging does not ensure that read dependencies remain valid. Even when patches merge cleanly, the state on which they were generated may have changed. For example, in the Commit0-derived workload, commit 139dc11 reads, through an import, a module modified by a87422f. We therefore consider both textual mergeability and dependency-version validity.
2. Reprompting and repair must be evaluated together with their recovery costs. Repair can preserve unaffected work, but may require additional model calls and tool operations. Our CoAgent/MTPO comparison [4] includes agent-guided repair and inverse tool operations, with recovery costs included in the evaluation. On Commit0-derived (Focused), CoAgent and AtxServe achieve UsefulTP values of 1,063.4 and 1,219.2 ops/GPU-hour, with total input-token counts of 1.073M and 0.782M, respectively. This provides an overall efficiency comparison between coordination before inference and repair after conflicts.
3. Fine-grained protection can be combined with conflict-aware admission. All four methods in the Jinja experiment use the same Python AST region granularity. As the worker count increases from 4 to 32, OCC's conflict rate still rises from 12.61% to 43.63%, indicating that shared-state conflicts require coordination even at this granularity. AtxServe constructs a conflict graph from these declarations to schedule compatible stages concurrently before main inference.
Q2 / W2: Transaction Terminology and Novelty
Transaction terminology and protection boundaries. We use stage transaction to mean one independently committed stage of an agent task. If a later stage aborts, earlier stages remain committed. Within a stage, read–modify–write (RMW) describes the computation. The transaction boundary specifies how its effects become visible: all of the stage’s writes to the managed workspace are published atomically, and an aborted stage publishes none. Under the conditions of Proposition 3, including enforced declarations covering planning and execution dependencies, atomic snapshot binding and publication, and valid grants held from successful binding through commit, the committed history of managed stages is conflict-serializable. Atomicity and isolation are scoped to a single stage; a long-running task proceeds through multiple independently committed stages.
Technical contribution. AtxServe uses predeclared access sets, conflict graphs, and logical protection to coordinate model inference, turning planning dependencies into enforceable admission constraints. The system combines three complementary design choices motivated by inference cost.
- Coordinate conflicts before main inference to reduce repeated computation after invalidation. Retries caused by invalidated dependencies consume shared model-service resources again. In the high-contention experiment with 1,024 operations, as concurrency increases from 8 to 32, OCC's runtime rises from 594s to 997s, whereas AtxServe's rises from 592s to 634s (Table 22). These results demonstrate the efficiency benefit of coordination before inference as concurrency scales.
- Use conflict structure to select compatible requests and preserve concurrency. In an ablation with reservation disabled and identical declaration resolution, validation, grants, and capacity constraints, minimum-degree selection improves UsefulTP over PCC-FIFO by 4.9% on RMWBench (High) and 12.3% on Commit0-derived (Focused) (Table 11). This ablation isolates the benefit of candidate selection under the same protection mechanism.
- Validate planning observations at binding to connect planning with execution. When dependencies change before binding, validation at binding reduces tokens per completed task by 18.9% relative to validation only at commit (Fig. 6d). After successful validation, the grant protects both planning and execution dependencies until commit.
Response to Reviewer Bfhj (3 of 3)
Q3 / W3: Are There Benefits Beyond Packing Compared with the Appropriate Baseline?
AtxServe's core contribution is to incorporate shared-state dependencies into inference admission to improve useful inference throughput under consistency constraints. Dependency protection orders stages with declared conflicts, while conflict awareness uses the conflict structure to select compatible requests and order admission.
Comparisons under the same protection mechanisms quantify the scheduling contribution. PCC-FIFO already allows compatible requests to run concurrently and skips blocked requests. Table 11 fixes declaration resolution, binding validation, grants, and capacity settings, and disables reservation, changing only the candidate selection rule. Minimum-degree selection prioritizes stages with fewer conflicts to preserve subsequent opportunities for compatible execution. On RMWBench (High) and Commit0-derived (Focused), random compatible selection improves UsefulTP over PCC-FIFO by 1.1% and 2.1%, whereas minimum-degree selection yields gains of 4.9% and 12.3%. Across all 10 paired repetitions over these two workload settings, AtxServe's UsefulTP exceeds PCC-FIFO's. Within each setting, the mean useful-completion counts are equal and mean token counts differ by less than 1.2% (Table 13; Appendix D.4).
Scheduling gains increase with concurrency. In the high-contention experiment, AtxServe's UsefulTP gain over PCC-FIFO grows from 4.2% to 27.9% as concurrency increases from 8 to 32 (Table 22). The Jinja experiment with online declarations provides a further comparison: at 32 workers, PCC-FIFO and AtxServe have mean E2E runtimes of 32.73 and 24.32 minutes, respectively (Table R3-1).
Shared dependencies can also inform engine-side request priorities. For example, under mixed traffic or limited KV capacity, the current waiting relationships could be used to assign higher request priorities, at submission time, to grant holders blocking multiple stages. Such a scheme could use vLLM's priority scheduling without modifying the engine code. The approach would prioritize requests that block more subsequent work, illustrating the value of connecting state coordination with engine scheduling at the inference-admission point.
Minor Concern: Fit for ICLR
Thank you for this suggestion. ICLR's call for papers explicitly includes infrastructure for learning systems, and LLM-serving work such as Preble [5] was published at ICLR 2025. Our work studies coordination between dependencies arising from agent planning and shared inference resources. The added Jinja experiment further demonstrates end-to-end behavior with online declarations.
We would welcome further discussion of the paper and the opportunity to improve it with your feedback. The relevant changes are marked in blue in the uploaded revised PDF for your reference.
References
[1] Nicholas Carlini. Building a C compiler with a team of parallel Claudes. Anthropic, 2026. URL https://www.anthropic.com/research/building-c-compiler
[2] Xin Yang, Letian Li, Zimo Ji, Terry Jingchen Zhang, and Wenyuan Jiang. Position: Multi-Agent Systems Should Prioritize Concurrency Control. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 172725–172740, 2026.
[3] Jinja task definitions. Anonymous supplementary materials. URL https://anonymous.4open.science/r/atxserve-1FDD/data/jinja/Jinja_Feature_Definitions.md
[4] Hongtao Lyu, Dingyan Zhang, Mingyu Wu, Xingda Wei, and Haibo Chen. CoAgent: Concurrency Control for Multi-Agent Systems. arXiv preprint arXiv:2606.15376, 2026.
[5] Vikranth Srivatsa, Zijian He, Reyna Abhyankar, Dongming Li, and Yiying Zhang. Preble: Efficient Distributed Prompt Scheduling for LLM Serving. In International Conference on Learning Representations, 2025.
Response to Reviewer Bfhj (1 of 3)
Overall Response
Thank you for recognizing the value of the problem and for your comments on the system's positioning, transaction terminology, baselines, and alternative designs. Our starting point is to use the conflict structure of shared state as a scheduling signal for inference services, coordinating conflicts before main inference while preserving opportunities for compatible requests to execute in parallel. Below, we address these concerns by explaining the design rationale, stage-level protection boundaries, and the incremental gains from conflict-aware scheduling under the same protection mechanisms.
Q1 / W1: Why Inference Admission? Does the Problem Arise in Practice?
Here, the admission layer refers to the decision point before main inference and the information available at that point. For a stage, coordination mechanisms can be grouped as follows.
Admission is the point at which declarations are available but the GPU cost of main inference has not yet been incurred. Admission decisions can use both the conflict structure, which is available to the state layer but not the engine, and engine state, which is available to the engine but not the state layer. Both constrain the admission decision. AtxServe combines dependency protection with compatible-request selection at this point, allowing conflict information to directly inform inference execution. In the submitted version, this coupling is reflected in tying admission capacity to the engine's sequence slots and expressing the objective in terms of GPU time.
Practical relevance. The answer is yes! Whared-state conflicts have been observed in practical multi-agent collaboration. Anthropic's parallel compiler project (2026) [1] reports frequent merge conflicts and agents overwriting one another's changes, and uses task lock files to coordinate task claims. The ICML 2026 paper Position: Multi-Agent Systems Should Prioritize Concurrency Control [2] further notes that long LLM inference durations amplify anomalies such as stale reads and lost updates, and advocates explicit concurrency control.
End-to-end development task. We added an end-to-end development experiment in Jinja, comprising 30 feature requirements in the same codebase decomposed into 100 acceptance subtasks. Qwen3-14B agents autonomously search the source code, generate access declarations online, edit, and test, with conflicts arising during execution. We evaluate 4–32 workers, with three runs per setting. The Jinja experimental materials and task definitions are available in the anonymous repository [3]. Full results are reported in Table R3-1.
Table R3-1. End-to-end development results in Jinja.
Table R2-2. OCC conflict rate and token usage on the Jinja end-to-end development workload. Token counts are reported in millions (M).
As concurrency increased from 4 to 32 workers, the OCC conflict rate rose from 12.61% to 43.63%, and OCC token usage increased from 9.67M to 13.55M. Across the four settings, AtxServe used 36.0%–48.1% fewer tokens than OCC, with the largest reduction at 16 workers.