- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies deployment evaluation when a controller affects its environment through both its actions and the computational resources consumed in producing those actions. It formalizes the error of reconstructing coupled deployment value from three separate evaluation arms as an execution–actuation factorial interaction. The paper derives an exact Bellman decomposition, discount- and mixing-dependent bounds with asymptotically sharp constants, and a lower bound showing that complete trajectories from the three separate arms may not identify even the sign of deployment benefit. An analytically tractable queueing example characterizes false deployment rejection, while synthetic queue experiments and a CPU-sharing microbenchmark demonstrate substantial interaction and several decision reversals.
Critical strengths
1.Strong theoretical treatment of interaction and non-identifiability. The Bellman decomposition clearly explains why locally additive transition perturbations need not yield additive long-horizon value. The sharp propagation bounds and the three-arm trajectory indistinguishability result are technically substantive.
2.Careful experimental methodology and reporting. The four interventions are defined explicitly, paired randomness is used appropriately, and the paper carefully distinguishes closed-loop from frozen-tape estimands. The authors are also transparent about post-hoc comparisons, null results, hardware uncertainty, and the limits of the queueing divergence result.
Critical weaknesses
-
The practical target is insufficiently established. The paper is motivated as addressing an important deployment-evaluation practice, but it does not demonstrate that researchers commonly estimate joint deployment value using or an equivalent transport assumption. Direct end-to-end evaluation of , separate reporting of performance and computation, and Pareto evaluation are not subject to the same reconstruction error. Without stronger evidence about the prevalence of the targeted workflow, the practical significance of the theory remains unclear.
-
The experiments have limited external validity for learned policies. The empirical evidence consists of synthetic threshold/periodic controllers, a queue simulator designed around action–capacity coupling, and one short-window Python CPU microbenchmark whose active controller consumes roughly 30% of the measurement window. There is no trained policy, neural inference workload, production system, or small-footprint hardware experiment. The experiments convincingly demonstrate possibility, particularly near capacity boundaries, but do not show that the effect materially alters representative RL evaluations.
Questions for authors
-
What concrete existing evaluation workflows use the three-cell additive reconstruction, or make an equivalent assumption that the execution effect measured under suppressed actuation transfers unchanged to the actuated regime? Please distinguish this from papers that merely report task performance and runtime separately, or that directly measure the complete deployed system. Evidence that the targeted practice is common would materially strengthen the paper’s motivation and impact.
-
Do the authors have evidence that the reported interaction remains decision-relevant for a trained policy and a realistic inference footprint?
-
What is the intended practical use of the proposed bounds when , , , and the mixing coefficient cannot be reliably estimated from ordinary deployment logs? Is there a realistic setting in which these quantities can be certified more easily than directly evaluating , or should the bounds mainly be understood as structural rather than operational results?