AVERT: An Action-Wise Eligibility Funnel for Offline Clarification Policies

Offline evaluation can promote a clarification policy when its estimated value is driven by actions for which the log contains little effective support. We present Advance-or-refuse Validation under Effective support for Rare Typed actions (AVERT), an action-wise eligibility funnel for a frozen family of clarification policies. AVERT screens every typed clarification action using effective sample size, candidate–logger propensity alignment, and an unclipped importance-weight-tail diagnostic before applying clipped doubly robust ranking. The method separates selection from execution: selection either advances the lowest-loss eligible policy or returns a typed refusal that identifies the action and failed support condition; execution then applies an advanced fixed policy as a one-step value-of-clarification controller. We formalize this interface and give a finite-family conditional concentration result for the clipped ranking objective. The resulting object makes missing action-specific evidence an explicit decision outcome rather than an implicit property of an offline winner.