- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CoRe3D extends a unified 3D language model to connect semantic planning with spatial construction: it generates a textual plan describing an object’s attributes and relations, then autoregressively predicts 512 spatially indexed 3D tokens conditioned on that plan. Its main contributions are semantic–geometric co-attention, which gives geometric predictions continued access to the plan; an octant-based token representation for local construction; and Co-GRPO, a GRPO-based training framework combining preference, attribute correctness, text–3D alignment, and physical-coherence rewards. The rewards reference each candidate’s semantic plan, while adaptive sampling balances generation and understanding updates.
Critical strengths
The paper is well motivated: explicit semantic cot planning offers a reasonable way to guide 3D generation and connect linguistic intent with geometric construction.
- Clear and intuitive methodology: Semantic–geometric co-attention allows each geometric decoding step to access the semantic plan, providing continued guidance during construction. Co-GRPO combines complementary preference, semantic alignment, attribute correctness, and physical-coherence rewards to jointly train generation and understanding. Together, these components form an easy-to-follow framework addressing both semantic faithfulness and geometric quality.
- Strong presentation: The paper is well organized and readable, with polished figures, effective layouts, and informative qualitative comparisons that clearly illustrate the framework and its capabilities.
Critical weaknesses
Figure 1 may mislead readers about the actual generation process. It appears to suggest that successive semantic CoT stages guide the construction of corresponding object parts, potentially through explicit stage-to-region supervision. However, Section 3.3 describes generating the complete semantic trace first, followed by autoregressive geometric decoding that accesses this trace through co-attention. The method does not specify interleaved semantic–geometric generation, stage-to-part assignments, or explicit supervision of these correspondences. The authors should distinguish conceptual guidance from the implemented decoding procedure and revise Figure 1 to reflect the actual temporal flow.
The paper does not directly show whether using the semantic trace to compute rewards improves performance. Table 5 tests different critics, but does not test whether those critics benefit from reading the trace. Removing Semantic CoT in Table 6 also changes how the object is generated, so it cannot isolate the effect on reward computation. The missing comparison is straightforward: keep the semantic trace available during generation, and only vary whether the reward evaluators use it.
In Table1 and Table3, The baseline comparisons could be more up to date. Keeping ShapeLLM-Omni makes sense because it is the starting point, but newer models such as Qwen3-VL-8B and Qwen3.5-9B would provide useful comparisons for captioning and language reasoning. A particularly informative baseline would use these models to expand the prompts while keeping the 3D generator fixed. This would help show whether the gains come from the proposed joint framework or simply from stronger semantic planning.
Questions for authors
Evidence for trace-aware rewards: What evidence isolates the benefit of using the semantic trace in reward computation? Can the authors provide a controlled comparison that keeps semantic-trace generation, co-attention, critics, and training budget fixed, and only changes whether rewards use the original prompt alone or also the generated trace? This would directly test the claimed contribution of trace-aware evaluation.
Evidence beyond the current baselines: Have the authors evaluated comparable-size recent models, such as Qwen3-VL-8B or Qwen3.5-9B, on the tasks in Tables 1 and 3 under matched evaluation conditions? In particular, is there evidence that CoRe3D outperforms using a stronger model to produce semantic plans while keeping the 3D generator fixed? This would help separate the benefit of the proposed framework from improvements due to stronger semantic planning.
Clarification of evaluation settings: Why does the full model achieve a METEOR score of 24.98 in Tables 1, 4, and 5, but only 17.03 under the default settings in Tables 6–8? Were the same datasets, splits, checkpoint, and evaluation protocol used? If not, please specify the differences and indicate which results are directly comparable.