- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
StateBundle selects compact telemetry bundles for operational agents using hierarchical representations, root-cause supervision, and complementary evidence retrieval. The paper also introduces DC-Bench, a deterministic simulator coupling facility and IT dynamics, with 72 tasks spanning 18 fault mechanisms. Experiments across agent systems report substantial token savings and improved aggregate success, although some individual task results decline.
Critical strengths
- Organizing fragmented telemetry while controlling context costs is practically relevant to autonomous data-center operations.
- DC-Bench and its generated dataset extend evaluation to interactions between facility infrastructure and the IT stack.
- StateBundle reduces token consumption across all evaluated agents while improving observed aggregate success.
Critical weaknesses
-The introduction describes telemetry fragmentation broadly but lacks a clear incident walkthrough demonstrating why existing representations fail and how StateBundle resolves that failure.
- Figures 1 and 2 contain small labels and densely packed components, making the operational scenario and training-to-inference relationships difficult to follow.
- Evaluation relies on deterministic dynamics and single injected faults. The 72-task suite provides limited evidence of robustness to concurrent incidents, stochastic behavior, and production telemetry imperfections.
- Root-cause labels, propagation supervision, and embedding similarity can support predictive relevance, but do not establish that selected observations identify causal mechanisms. Interventions or tests that disrupt spurious associations would strengthen the cause-centered claim.
- Despite aggregate gains, GPT-5.6 Sol’s localization success falls from 98.1% to 92.6%, and Claude Opus 5’s from 92.6% to 88.9%. These regressions warrant uncertainty estimates and analysis of whether selection removes evidence useful to capable models.
Questions for authors
Please refer to the Weakness section above.
3 comments
Response to Reviewer 1eQk. Q3
Response to Reviewer 1eQk
[W3&Q3]
Concern Summarization Root-cause labels, propagation supervision, and embedding similarity can support predictive relevance, but do not establish that selected observations identify causal mechanisms. Interventions or tests that disrupt spurious associations would strengthen the cause-centered claim.
Response:
We thank the reviewer for this insightful comment. To demonstrate StateBundle selects evidence associated with the actual fault origin rather than spurious associations, we added an interventional test: we move the injected fault to a different target while holding everything else fixed, and check whether StateBundle's anchors move with it. If selection relied on a spurious association, such as a learned preference for particular racks, servers, or tenants, the anchors would stay where they were.
Fault-target relocation. We use the ten mechanisms whose faults have a specific rack, server, or tenant as target. For each, we build paired scenarios that differ only in the faulted target ( or ), with identical topology, workload, mechanism, severity, and injection time, and run five trials per mechanism. To isolate evidence selection from agent behavior, we extract the canonical observations at the same post-injection time in both scenarios, before any agent action. Each observation is assigned to a target through its observable resource identifiers.
Metrics. Fault-Target Anchor Share (FTAS) is the fraction of anchors on the injected target :
In addition, we introduce Paired Target-Switch Effect (PTSE), which explicitly measures how anchor allocation changes when the injected fault moves between two targets. Let be the share of anchors on target when target is faulted.
Table 2. Fault-target relocation results. Results are reported as macro-averages.:
| Selection | FTAS(%) | PTSE |
|---|---|---|
| StateBundle anchors | 24.48% | 0.2448 |
As shown in Table 2 , StateBundle achieves an FTAS of 24.48% and a PTSE of 0.2448. The equality between PTSE and FTAS indicates that virtually no anchors are selected from the opposite fault target, suggesting that anchor allocation responds to changes in fault location rather than reflecting a fixed target preference.
For comparison, we also report the FIAS of the full canonical observation which is 4.78%. Despite the diversity regularization that discourages semantically redundant anchors, StateBundle increases the fault-target observation share by approximately 5.12 times. These results provide evidence that StateBundle preferentially selects observations associated with the fault origin and adapts its anchor selection when the fault target changes.
Manuscript revision:
We added a new appendix subsection describing the relocation test, the two metrics, and the results above.
Response to Reviewer 1eQk. Q4
Response to Reviewer 1eQk
[W4&Q4]
Concern Summarization Despite aggregate gains, GPT-5.6 Sol’s localization success falls from 98.1% to 92.6%, and Claude Opus 5’s from 92.6% to 88.9%. These regressions warrant uncertainty estimates and analysis of whether selection removes evidence useful to capable models.
Response:
We thank the reviewer for highlighting the localization regressions. We agree that these differences warrant further investigation. To address this concern, we conducted uncertainty analysis of localization performance.
Uncertainty estimates. The table gives each agent's difference (StateBundle − Full-Canonical) with its 95% confidence interval. All ten intervals include zero, so there is no clear evidence of a consistent localization advantage for either setting.
| Agent | Full-Canonical | StateBundle | Difference (pp) | 95% CI (pp) |
|---|---|---|---|---|
| DeepSeek-V4-Flash-0731 | 83.3% | 74.1% | −9.3 | [−24.1, +7.4] |
| GPT-5.6 Luna | 88.9% | 83.3% | −5.6 | [−14.8, +5.6] |
| Gemini 3.1 Flash-Lite | 87.0% | 90.7% | +3.7 | [−7.4, +14.8] |
| Claude Sonnet 5 | 94.4% | 96.3% | +1.9 | [−3.7, +7.4] |
| GPT-5.6 Sol | 98.1% | 92.6% | −5.6 | [-11.1, +1.9] |
| Claude Opus 5 | 92.6% | 88.9% | −3.7 | [−18.5, +11.1] |
| GPT-6 Astra | 96.3% | 98.1% | +1.9 | [−5.6, +11.1] |
| Codex (GPT-5.6 Luna) | 85.2% | 92.6% | +7.4 | [−5.6, +20.4] |
| Codex (GPT-5.6 Sol) | 87.0% | 90.7% | +3.7 | [−7.4, +14.8] |
| Codex (GPT-6 Astra) | 88.9% | 88.9% | 0.0 | [−7.4, +7.4] |
In addition, given the high success rates under both observation settings—487 of 540 episodes (90.2%) for Full-Canonical and 484 of 540 episodes (89.6%) for StateBundle—it can be inferred that localization is relatively well handled by the evaluated agents, leaving limited room for further improvement on many tasks. We further investigate the shared failure mode and find that the dominant failure mode is the same under both settings: agents identify a component affected by fault propagation rather than the component where the fault originates. This failure mode is concentrated in four mechanisms: storage I/O saturation (41 of 60 episodes across both settings), placement-policy misconfiguration (19/60), load-balancer misconfiguration (16/60), and autoscaler misconfiguration (11/60), which together account for 87 of the 109 failures (79.8%). The remaining 22 failures account for only 2.0% of all 1,080 localization episodes.
Overall, it can be inferred that, because the localization task is relatively easy, both Full-Canonical and StateBundle handle it well except for this common failure mode. The success-rate difference can be mainly attributed to randomness. However, these results indicate that, for tasks that are handled well, StateBundle maintains similar average performance while substantially reducing the telemetry presented to agents.
Manuscript revision:
We added the appendix subsection "Uncertainty Analysis of Localization Performance", containing the above data and analysis.
Response to Reviewer 1eQk. Q1 and Q2
Response to Reviewer 1eQk
[W1&Q1]
Response:
We thank the reviewer for this helpful feedback. We have revised both figures to improve readability by enlarging all labels and reducing the number of components in each panel.
Manuscript revision:
Figures 1 and 2 in the revised manuscript have been replaced with the redrawn versions.
[W2&Q2]
Response:
We thank the reviewer for this important point. We agree that the original suite does not test these conditions, so we added three stress-test suites (i.e. stochastic behavior, telemetry imperfections, and concurrent incidents), with the original StateBundle weight to evaluate its robustness to stress conditions. All experiments use Codex (GPT-5.6 Luna), the agent from our ablation studies.
Stochastic dynamics. We rerun the 72 original tasks with temporally correlated workload fluctuations. A disturbance is sampled at every simulator tick of the healthy, fault-active, and recovery phases and propagates through the existing utilization, power, thermal, and service-performance dependencies:
Dt=max(0,Dˉt(1+ϵt)),ϵt=ρϵt−1+σ1−ρ2ξt,ξt∼N(0,1),
where Dˉt is the original demand, ρ=0.95, and σ=0.05.
Telemetry imperfections. We rerun the 72 original tasks with random observation loss and delayed delivery. Each eligible canonical observation is independently dropped with probability pdrop=0.05 and independently delayed with probability pdelay=0.05, in which case it reaches the agent 1–10 ticks late.
Concurrent incidents. We build 72 new tasks with each contains two injected faults from the original 18 fault mechanisms. To ensure balanced coverage, we randomly permute the 18 mechanisms using a specific seed and pair each mechanism with its successor in the permutation. Success requires handling both faults: identifying both mechanisms with evidence (detection), both targets (localization), each mechanism with its target and evidence (RCA), and recovery from both faults over the stability window (mitigation). The agent is told that one or two faults may be present, but not how many.
Table 1. Success rates (%) over 72 tasks × 3 trials and average model tokens per episode:
Stochastic dynamics and imperfect telemetry. StateBundle maintains its performance advantage over Full-Canonical under both stochastic system dynamics and telemetry imperfections, suggesting robustness to these perturbations. Notably, under telemetry imperfections, both methods exhibit only minor performance changes, with Full-Canonical declining by 1.9 percentage points and StateBundle maintaining its original success rate. A plausible reason for it is that a fault usually leaves evidence in several channels, allowing agents to retain sufficient diagnostic information even when a small proportion of observations is missing or delayed.
Concurrent incidents. For this setting, the gap between StateBundle and full-canonical observations narrows to 1.9 points. A likely reason is that StateBundle is trained only on single-fault incidents, which can concentrate the bundle on one root cause. Nevertheless, StateBundle maintains a higher task success rate than Full-Canonical while consuming fewer model tokens, suggesting that its evidence-selection strategy remains beneficial under concurrent-fault conditions.
Manuscript revision:
We added the appendix subsection "Robustness Stress Tests", which describes the three suites and reports the results in Table R1.