RAVEL: Rare Concept Generation and Editing via Graph-driven Relational Guidance
Abstract
Despite rapid advances in text-to-image (T2I) generation, current models often struggle with rare, compositional, and culturally grounded concepts where training data is sparse and visual priors are weak or absent. Increasing model capacity or prompt detail alone does not consistently resolve these challenges, particularly in long-tail settings where attribute grounding remains unstable due to attention skew, semantic collapse, and needle-in-a-haystack retrieval dynamics. We introduce RAVEL, a training-free framework that improves rare concept generation, context-driven image editing, and self-correction by integrating graph-based retrieval-augmented generation (RAG) into T2I pipelines. Unlike prior RAG and LLM-enhanced approaches that rely on visual exemplars, static captions, or implicit model knowledge, RAVEL leverages structured knowledge graphs to retrieve targeted compositional, symbolic, and relational context, enabling more precise grounding even in the absence of visual priors. To further refine generation quality, we propose SRD, a self-correction module that iteratively updates prompts using multi-aspect alignment feedback, improving attribute accuracy, narrative coherence, and semantic fidelity. Our framework is model-agnostic and is compatible with leading diffusion and autoregressive models, including Stable Diffusion XL, Flux, DALL-E 3, GLM-Image, and Janus-Pro. We conduct extensive evaluations across three newly proposed benchmarks: MythoBench, Rare-Concept-1K, and NovelBench, and show consistent improvements over SOTA baselines across perceptual, alignment, and LLM-as-a-Judge metrics, while also demonstrating stable and interpretable refinement trajectories for rare concepts across diverse model families. These results position RAVEL as a simple and robust approach for controllable and interpretable T2I generation in long-tail domains. Our code is available at: https://anonymous.4open.science/r/ravel-54E5/
Related papers
See more related submissions when the market opens.