- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper targets multimodal product retrieval by extracting product attributes from multimodal product information. However, they found out that existing attributes can be missing, conflicting, and hallucinated, leading to poor retrieval performance. To address this, the paper proposes CMAC, a framework that uses cross-modal attribute consistency to improve product retrieval. CMAC identifies attribute faithfulness as a three-view consistency problem, where the model independently extracts attributes under image-only, text-only, and multimodal input. Then, they define TRIAD, a reliability score that is calculated by penalizing conflicts on shared attributes, rewarding attributes unique to one view, and utilizing the multimodal view to help with conflicts. TRIAD is then used in three training stages: weighted contrastive learning, conflict-driven adversarial supervision, and TRIAD-GRPO. Weighted contrastive learning downweights samples that have more conflicting attributes, while TRIAD-GRPO uses the consistency score as GRPO reward to improve attribute extraction.
Critical strengths
- The paper defines an interesting viewpoint on attribute faithfulness, namely the three-view consistency problem. It discusses agreement and conflict under image-only view, text-only view, and multimodal views.
- The paper proposes the CMAC framework for product retrieval and demonstrates that their proposed framework outperforms several baseline models.
Critical weaknesses
- The structure and clarity of the paper can be largely improved. For example, Mode II appears in both Introduction and the main workflow figure, but the Method section only briefly states that Mode II does not support a performance claim so it is deferred to future work, without sufficient explanation. This can be misleading and should be revised. In addition, the main workflow figure (Figure 2) can also be more information. The current workflow figure does not fully reflect the function of each component. The text inside the boxes, especially the grey ones, can be ambiguous and needs more clarification.
- Limited baseline comparisons. The paper compared against AFMRL and several other baselines such as CLIP and BLIP. Broader and more recent baseline comparisons are not covered.
Questions for authors
- Could the authors provide experimental results compared to other baselines?
- Could the authors clarify how coefficients in TRIAD score (equation 3) are tuned and selected? Since TRIAD score is a crucial component of the workflow, the coefficients in equation (3) need more justification. Please also provide sensitivity analysis to show how changing those coefficients affects the model performance if possible.
- Could the authors explain in equation (3), why the coefficient for complementarity (0.2 x 0.2) is much smaller than the coefficient for conflict-free rate (0.8)?
- Could the authors provide several qualitative examples of product retrieval as well as failure cases?