acceptodds
Published as a conference paper at NeurIPS 2026

Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Risk

Tim Woydt, Paul-David Zuercher

Abstract

Critical sequential decisions are rarely single-timescale: a strategic decision causally shapes the context in which every subsequent tactical choice is made; standard bandit and reinforcement-learning theory does not capture this causal coupling between timescales. We formalise the problem class as *Nested Contextual Causal Bandits* (NCCBs), a hierarchical SCM where each level's action sets the next level's context distribution, and propose *Nested Causal Thompson Sampling* (NCTS), which draws one mechanism-factorised belief per episode and acts recursively under it. Our main theoretical result is a causal PAC-Bayesian excess-risk bound that certifies any candidate deployment policy from historic data alone, off-policy and anytime, answering the deployment question: *can we trust this agent here, and at what risk?* Experiments on a hierarchical SCM show that, against a matched RFF-GP joint regression on the same function class, the factorised SCM-mechanism posterior transfers *significantly* better zero-shot under exogenous distribution shifts, the recursive meta-to-inner commit *significantly* dominates the joint-commit alternative in distribution, and the certificate *significantly* contracts as offline data accumulates. Combining these results, we establish *progressive certified handover*, a safe-deployment method: each timescale flips from a legacy controller to NCTS when gains can be certified, independently of the others.

Related papers

See more related submissions when the market opens.