TriRelVLA: Triadic Relational Structure for Generalizable Embodied Manipulation

Vision-language-action (VLA) models perform well on training-seen robotic tasks but struggle to generalize to unseen scenes and objects. A key limitation is their implicit visual representations, which entangle object appearance, background, and scene layout, making policies sensitive to visual variations. Prior work improves transferability through structured intermediate representations, but these mainly capture scene semantics rather than action-relevant relations. We argue that manipulation actions are fundamentally governed by the object–hand–task relational structure, which defines interactions among task requirements, robot states, and object properties. Based on this observation, we propose TriRelVLA, a triadic relational VLA framework for generalizable embodied manipulation. Our approach consists of three components: 1) We construct explicit object–hand–task triadic representations from multimodal inputs as relational primitives. 2) We build a task-grounded relational graph. Task-guided cross-attention forms nodes, and a relation-aware graph transformer models interactions among them. 3) We perform relation-conditioned action generation. The relational structure is compressed into a bottleneck space and projected into the LLM for action prediction. This triadic relational bottleneck reduces reliance on appearance statistics and enables transfer across scenes, objects, and task compositions. We further introduce a real-world robotic dataset for fine-tuning. Experiments show strong performance on fine-tuned tasks and clear gains in cross-scene, cross-object, and cross-task generalization. Our code will be released upon acceptance.