RRA-Avatar: Representation Responsibility Allocation for Gaussian Avatars with Loose Garments

Animatable Gaussian avatars typically use a single body-guided representation for both body-supported surfaces and off-body structures such as loose garments, hair, and accessories. This creates a representation-responsibility ambiguity: regions with substantially different levels of body-prior support are forced to share geometry, deformation, and optimization capacity. We introduce RRA-Avatar, a support-aware dual-layer Gaussian avatar framework that assigns representation responsibility according to whether foreground geometry is adequately explained by the fitted body prior, rather than according to garment semantics. The body-aligned layer preserves stable SMPL-guided modeling, while the residual-surface layer provides dedicated capacity for unexplained off-body structures. To instantiate this decomposition without garment templates or semantic labels, we recover residual-surface seeds from temporally fused multi-view foreground evidence. We further introduce a responsibility-allocation mechanism that coordinates layer-specific supervision, gradient flow, and topology evolution, reducing inter-layer competition while retaining joint refinement near uncertain boundaries. Experiments on DNA-Rendering, I3D-Human, and ZJU-MoCap demonstrate substantially improved reconstruction of loose and off-body structures while preserving competitive body-region and novel-pose performance. On six DNA-Rendering subjects, RRA-Avatar reduces LPIPS from 30.91 to 22.98 relative to the strongest compared baseline under the same protocol.