AEGIS: Adaptive Efficient Generative Inference Scheduling for Structured Latent Models

Diffusion and flow-matching models have become the dominant paradigm for high-fidelity generation across images, video, and 3D content. However, acceleration methods that work well in image or video often fail to preserve 3D geometric consistency, while existing 3D inference approaches still rely on fixed reuse schedules and cannot adapt computation to instance difficulty or trajectory-specific demands. We propose AEGIS, a training-free framework that reframes test-time acceleration as quality-constrained budget allocation solved by an online feedback rule. AEGIS maintains a two-dimensional runtime state that tracks sample difficulty and accumulated correction debt, and turns it into per-step skip and refresh quotas that any cache-style executor can realize. It generalizes the corresponding open-loop schedule and provides an explicit upper bound on cumulative reuse drift. Extensive experiments show that AEGIS improves inference efficiency while better balancing throughput and quality.