Quantifying Canary Score Dependence in One-Run Unlearning Audits

One-run privacy audits for machine unlearning embed $k$ canaries in a forget set, apply an unlearning operator, and aggregate per-canary membership scores into a confidence statement, assuming the scores contribute independent information about the run. We show this assumption fails in a structured way. Adapting the intraclass-correlation (ICC) decomposition from biostatistics, the marginal ICC(1,1) is near zero for conservative operators (NPO, RMU) while the conditional ICC(3,1), on canary-centered residuals, is substantial—$0.045$–$0.217$ for conservative operators, $0.069$–$0.713$ for gradient ascent, all converged: a marginal diagnostic is blind to the dependence that matters for per-canary procedures. Within the dependence that survives canary centering, the run-level share dominates, with a smaller entity-block component (permutation $p=0.000999$); the mechanism is not identified by this statistic. At $k=400$ the conditional design effect reaches ${\mathrm{DEFF}}_{\mathrm{c}}\approx 286$ for converged gradient ascent—400 canaries carry only $\sim 1.4$ independent observations (scalar) or $\sim 1.18$ (eigenvalue bound)—and independence-assuming mean-MIA intervals under-cover (marginal tier, $\sqrt{{\mathrm{DEFF}}_m}$ convention), while the per-canary-count statistic mildly under-covers at $87.8$–$93.9\%$ naive but is over-conservative ($\ge 99.5\%$) once corrected; for conservative operators a fresh canary draw expects near-zero transferable dependence (ICC(2,1): NPO $0.020$, RMU $0.002$). At the highest coupling strength (converged GA), the correctly scaled Level-1 correction reaches full coverage on this cell ($1.000$, $30/30$; unchanged under residual-run resampling), so the auditor proceeds rather than declining. The framework requires $R\ge 20$–$30$ calibration runs for both detection and magnitude estimation, so it is a *diagnostic* that tells an auditor when the one-run regime is valid.