Refine or Preserve? Acceptance-Gated Correction for Autoregressive Image Generation

Post-hoc refinement can repair degraded autoregressive image samples, but the same intervention can damage outputs that are already well tuned. We therefore ask a prior question: should a generated sample be refined at all? We introduce Acceptance-Gated Refinement (AGR), a two-level correction policy for frozen autoregressive image generators. An image-level router chooses between exact preservation and refinement, while a token-level gate proposes masked-token replacements and commits only edits that pass a refiner-score test. The router is derived from the gate's own initial refiner pass and calibrated per domain using tuned and severely degraded validation samples; no degradation label is required at test time. Across FFHQ, CelebA-HQ, and ImageNet, AGR bypasses at least 98.5% of tuned samples, with observed FID changes of only +0.06, +0.08, and −0.01, while reducing severe-input FID from 102.71 to 47.36, 81.26 to 30.17, and 96.25 to 58.09. Mild inputs excluded from router fitting also improve. A router fitted only on temperature shifts improves unseen 4-bit quantized samples while bypassing top-$k$ and top-$p$ truncated samples for which forced refinement is harmful. At matched edit rates, random editing degrades mild inputs whereas score-gated editing improves them. Across thirteen settings in three domains, refinement gain increases with the input's score deficit under the refiner. These results show that a refiner can provide not only candidate corrections, but also a practical signal for deciding when to refine and when to preserve.