Synthetic-Powered Directional Testing

Many data-driven decisions rely on tests of composite hypotheses, where alternatives may depart from the null in multiple directions and thus a uniformly most powerful test may not exist. We propose synthetic-powered directional testing (SPDT), a framework that safely leverages synthetic data to increase power in composite hypothesis testing problems. The key idea is to use synthetic data to identify a plausible direction in which to concentrate power, and then combine a directional test with an existing base test. We show that SPDT can yield substantial power gains when the chosen direction is informative, while retaining base power even when it is misleading. At the same time, the procedure inherits the validity guarantees of its component tests within a prespecified total type-I error budget. We demonstrate the benefits of SPDT through simulations and applications spanning LLM safety and treatment effect heterogeneity detection.