DORIC: Digital-Gene-based Object Understanding from a single RGBD Image via Chain-of-Thought Reasoning

Understanding 3D object knowledge and functionality in the physical world is a fundamental challenge for embodied AI. The recently proposed Digital Gene offers a promising explicit representation of objects via executable programs for object structure and function understanding. However, three major practical obstacles remain: 1) inaccurate gene grounding, 2) missing task-oriented reasoning for function utility, and 3) inflexible functional programs. To address these, we introduce DORIC, a framework that pairs MLLM's semantic and image reasoning with our proposed gene grounding modules for precise physical and numerical analyses. DORIC decomposes object-understanding requests into sub-tasks, each focusing on a single object and processed in a four-stage Chain-of-Thought (CoT) pipeline: I. Object and Structure Identification, II. Gene Selection and Grounding, III. Part-Level Program Selection, and IV. Object-Level Program Synthesis. By integrating multi-modal reasoning with specialized grounding modules, our contributions are four-fold: 1) a hybrid architecture for precise gene grounding involving MLLM and specialized modules, 2) a task-oriented four-stage CoT pipeline for function utility reasoning, 3) an in-context functional program synthesis paradigm, and 4) superior effectiveness and robustness demonstrated by extensive experiments.