Policies Are Programs: A Factorial Theory of Resource-Coupled Control
Abstract
A deployed controller changes its environment through both its actions and the resources used to compute them. Reconstructing its value from separate action and execution evaluations assumes that these channels combine additively. We characterize the resulting error by an execution–actuation factorial contrast. An exact Bellman identity separates local interaction from cross-channel propagation: even an additive one-step kernel can have nonadditive long-horizon value. Discount- and mixing-dependent bounds have asymptotically sharp leading constants, attained by two-state constructions. A four-state lower bound further shows that arbitrarily many complete trajectories from the three separate evaluation arms can leave the sign of deployment benefit unidentified, even with additive kernels. An exact stationary queue law identifies when the additive model rejects a beneficial deployment, including families with vanishing execution loss and unbounded rejection regret. Reanalysis of held-out synthetic queue records finds a predicted gain of versus an actual gain of at the same operating point. A randomized-block CPU control independently exhibits interaction with action-release latency removed. Together, these results characterize the accuracy, information requirements, and decision consequences of additive deployment evaluation.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.