- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper addresses sketch-based 3D shape retrieval. It introduces an additional modality, textual description, and makes the retrieval task more complex: the method must retrieve not only the correct shape but also the correct texture. The authors propose the StructAlign framework, which includes a 3D shape encoder, a texture encoder, a sketch encoder, and an MLLM text encoder. The goal is to encourage the feature encoders to focus on the sketch strokes themselves while ignoring the blank background regions. The method appears to use contrastive learning to learn shape and texture representations, and it uses GRPO to train the MLLM, specifically with LoRA on Qwen-8B.
Critical strengths
- The authors provide a video demo, which is valuable because it allows reviewers to directly observe the effect of the proposed method.
- The attempt to use contrastive learning to address sketch- and text-based 3D shape and texture retrieval is workable.
- The authors claim to provide a dataset of around 15,000 sketch-text-textured 3D shapes, although the shape categories are limited to two.
Critical weaknesses
I believe there is a clear rejection signal, and I will explain my assessment as follows.
Writing and presentation
The writing has significant problems. In the abstract and introduction, the paper does not disclose core implementation details, such as the use of Qwen-8B, LoRA, and another key component, the Retrieval KL. If readers expect to understand the core logic from the abstract alone, they will be seriously misled or confused. In multimodal and reinforcement learning papers, the base model and the training method, such as LoRA, are critical information. From the perspective of the idea, the paper appears to apply GRPO to sketch-based 3D shape retrieval, but the difference from existing GRPO applications is not clear, and I do not see significant novelty.
Figure quality
The figures do not meet the standard of a research paper.
Regarding Figure 1, I am not satisfied with its presentation. It is not solid and is prone to ambiguity. The middle part of Figure 1 summarizes existing works on sketch and text semantic encoding and shows failure cases from existing works. I would like to know which specific works belong to the summarized encoding ways. The authors should provide actual results and applicable citations. In addition, for the encoding ways summarized in the figure and the proposed encoding way mentioned in the right subfigure, the paper states that existing encoding methods pay weaker attention to the foreground (sketch) than to the background (blank regions). I am skeptical of this assumption. I cannot infer from the fusion way drawn in Figure 1 that this would necessarily prevent attention to the sketch. Since this cannot be inferred from the method logic, the authors need to provide solid failure cases. Otherwise, readers will view Figure 1 as not solid and will question the entire paper.
Figure 2 is the framework overview of the proposed method and is very important. However, the figure contains highly refined content, and I suspect it was generated by an auto research tool or a similar AI tool. Such high-level conceptual description is characteristic of AI generation, and I am concerned about the authenticity of the paper. I have the following specific concerns about this figure. First, the content of Figure 2 does not correspond to the relevant descriptions in the main text. I suggest adding the mathematical notation mentioned in the main text to Figure 2 to improve readability. Second, in subfigure 1, the output of Qwen is labeled as "K candidate semantic completions." This description is extremely inaccurate. What is being completed? Third, in subfigure 2, the output of Qwen is described as "Qwen hidden features," which is inconsistent with the notation in subfigure 1. This inconsistency is critical. Fourth, for the token operations, they are connected only by simple arrows, and I do not know what the specific operations are. Fifth, in subfigure 3, because this is retrieval, features need to be extracted for every item in the entire dataset and then compared with the query to obtain the top-k candidates. I can understand this operation, but how the query is input is not shown. Overall, the figure only shows inputs and outputs without details, and it is unclear what it is trying to convey.
Methodological issues
There are several methodological flaws. One of the contributions is the Structural KL, but this KL constraint is not described at all. What is KL? I assume it refers to Kullback-Leibler Divergence. The only relevant section is 3.6, "FOREGROUND STRUCTURAL CONSISTENCY LOSS," where Eq. 9, 10, and 11 are meaningless. They simply modify existing general formulas into a form that fits the notation of this paper. The main text explicitly states, "We further introduce two KL constraints," namely Structural KL and Retrieval KL. These are mentioned vaguely in the figure and introduction. The section related to Retrieval KL is 3.7, "RETRIEVAL-DRIFT-CALIBRATED ADAPTIVE GRPO." It is very difficult to locate the Retrieval KL here. After careful reading, I found it in Eq. 14. The readability is too low and does not meet the standard of a research paper. In addition, both KL constraints are applications of existing theory, and I do not see any novelty.
Some details of the method are also puzzling. For example, in eq.7 the term "joint" is mentioned. What does "joint" refer to here? Even considering the context, I cannot understand the meaning of "joint." Does it refer to a vertex in geometry? At least it should be related to shape and texture in the context. In addition, the description of contrastive learning in this paper is disastrous. It does not explain how contrastive learning operates in this context, and I find it difficult to understand the mechanism.
Experimental issues
The experimental dataset has too few categories, only chairs and lamps. This means the proposed method cannot handle other categories, and its generalization ability is weak. The amount of data is also small. I suspect the learned network will overfit. Can such a small amount of data be used to fine-tune an LLM? I am very doubtful. In addition, the implementation description is inaccurate. The paper mentions using PointNet++ as the geometric encoder, but the original text says, "while the 3D branch employs PointNet++." This description is inaccurate. I can only infer that PointNet++ is the geometric encoder of this paper. There is also a texture encoder, but what is it? The paper does not mention it. Furthermore, the paper does not clearly describe why LoRA is used. In summary, this does not meet the standard of a qualified paper.
Questions for authors
Please see in "Critical Weaknesses"