- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents AgentReplay, a trace record-and-replay framework for agent serving. Agent serving interleaves model inferences and tool calls. AgentReplay records input/output tokens, expert selections and request dependencies for the model inference part, as well as tool durations for the tool call part. In addition, it separates trajectory generation from performance generation. This enables smaller models to replay long-horizon traces collected from more capable models. The evaluation compares AgentReplay with greedy decoding and length-wise replay, and shows the benefits of AgentReplay.
Critical strengths
- AgentReplay is a record-and-replay framework for agent serving. It records both the model inference part and the tool call part. It is useful for evaluating serving optimizations.
- The evaluation compares AgentReplay with greedy decoding and length-wise replay. The results show AgentReplay can reduce workload variation.
Critical weaknesses
- The paper does not discuss and evaluate the replay overhead. It would be good to discuss and evaluate the serving performance with and without replay.
Questions for authors
-
The introduction says that "AgentReplay, a configurable trace record-and-replay framework for vLLM (Kwon et al., 2023)". Is the solution general? Does AgentReplay only work for vLLM? Can the design also be used for other inference engines such as SGLang?
-
Can you clarify the difference between AgentReplay and XPerf (Wang et al., 2026)? XPerf "records input and output token IDs, invocation dependencies, and tool durations". AgentReplay "records input/output tokens, expert selections and request dependencies and tool durations for the tool call part". Is the difference only whether to record expert selections?
-
What is the overhead of replay? Does running vLLM with replay introduce any performance overhead compared to running vLLM without replay? Does the overhead affect the evaluation of serving performance optimizations?