HALo: Enabling Self-Improving Image Generation via A Hierarchical Agentic Loop

Real-world image-generation queries are demanding and often underspecified. They often entail complex specifications, such as spatial layouts and rendered text, while requiring external knowledge that is beyond the generator's internal knowledge. Conventional feed-forward models struggle to handle these queries. Recent methods therefore make image generation agentic by adding reasoning, retrieval and repeated revision. However, they often employ a single loop to plan and revise the whole prompt. This design introduces two key limitations: first, requirements may be omitted or facts lost during planning, prior to image generation; second, correcting a single failed requirement necessitates rewriting the entire prompt, potentially disturbing constraints that have already been satisfied. We introduce HALo, a hierarchical agentic loop with three layers: layout specifies what the image should contain, draft prepares supporting material for constraints that a textual prompt alone cannot enforce, and render generates the image. HALo improves generation at two levels. An inter-layer loop establishes a bidirectional flow. Plans and supporting materials propagate upward from layout to rendering, while unresolved failures are routed downward to the drafting stage for local repair. Crucially, satisfied constraints remain fixed throughout this process, preventing regression in subsequent revisions. Within each layer, an inner loop checks and refines its output in a local context. Intermediate trial-and-error remains local, thus avoiding any rewriting of outputs already established by other layers. Consequently, failed constraints are rectified by updating only their associated supporting material, while the global plan and satisfied constraints remain invariant. Across four benchmarks spanning world knowledge, compositional generation, knowledge-intensive generation and image editing, HALo achieves the best overall performance among the compared methods.