acceptodds
Withdrawn from ICLR 2027

CMAC: Cross-Modal Attribute Consistency as Label-Free Supervision for Multimodal Product Retrieval

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal representation is crucial for e-commerce tasks such as identical prod- uct retrieval. Attribute-enhanced models define the state of the art on this task, yet nothing inside the pipeline checks whether the extracted attributes are faith- ful: our bad-case analysis of 1,918 retrieval queries finds 48.7% of failures in- volve missing, conflicting, or hallucinated entities. To address this, we propose CMAC, which defines attribute faithfulness as a three-view consistency problem. It runs one extractor under independent image-only, text-only, and multimodal input masks, then uses TRIAD, a closed-form reliability score whose conflict penalty is adjusted by multimodal arbitration and whose complementarity term retains view-specific information. CMAC uses TRIAD for consistency-weighted contrastive learning, conflict-driven adversarial supervision, and a dense, policy- dependent GRPO reward—whose pure form needs no retriever, oracle, or extra annotation in the reward loop. This paper evaluates the contrastive and RL stages; the adversarial stage is specified and its calibrated evaluation is deferred. After reinforcement learning, the generator improves on the fixed evaluation sample: at- tribute consistency rises from 0.6265 to 0.6916, parse failures drop from 3.41% to 0.36%, and response length falls from 697 to 339 tokens. Under the strongest pub- lished baseline’s own evaluation protocol, the CMAC encoder consuming identi- cal entity inputs outperforms it by +5.98 Recall@1 (95% paired-bootstrap CI [+4.91, +7.08], ≈5× the protocol’s 2σ resampling-noise floor), following the first faithful external reproduction of that baseline. These results establish three-view consistency as an evidence-grounded source of label-free supervision—first for the retrieval encoder that consumes attributes, then for the generator that produces them.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
multimodal representation learning, cross-modal consistency, label-free supervision, multimodal product retrieval, attribute faithfulness, contrastive learning, reinforcement learning (GRPO), e-commerce search

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RB3POpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper targets multimodal product retrieval by extracting product attributes from multimodal product information. However, they found out that existing attributes can be missing, conflicting, and hallucinated, leading to poor retrieval performance. To address this, the paper proposes CMAC, a framework that uses cross-modal attribute consistency to improve product retrieval. CMAC identifies attribute faithfulness as a three-view consistency problem, where the model independently extracts attributes under image-only, text-only, and multimodal input. Then, they define TRIAD, a reliability score that is calculated by penalizing conflicts on shared attributes, rewarding attributes unique to one view, and utilizing the multimodal view to help with conflicts. TRIAD is then used in three training stages: weighted contrastive learning, conflict-driven adversarial supervision, and TRIAD-GRPO. Weighted contrastive learning downweights samples that have more conflicting attributes, while TRIAD-GRPO uses the consistency score as GRPO reward to improve attribute extraction.

Critical strengths
  • The paper defines an interesting viewpoint on attribute faithfulness, namely the three-view consistency problem. It discusses agreement and conflict under image-only view, text-only view, and multimodal views.
  • The paper proposes the CMAC framework for product retrieval and demonstrates that their proposed framework outperforms several baseline models.
Critical weaknesses
  • The structure and clarity of the paper can be largely improved. For example, Mode II appears in both Introduction and the main workflow figure, but the Method section only briefly states that Mode II does not support a performance claim so it is deferred to future work, without sufficient explanation. This can be misleading and should be revised. In addition, the main workflow figure (Figure 2) can also be more information. The current workflow figure does not fully reflect the function of each component. The text inside the boxes, especially the grey ones, can be ambiguous and needs more clarification.
  • Limited baseline comparisons. The paper compared against AFMRL and several other baselines such as CLIP and BLIP. Broader and more recent baseline comparisons are not covered.
Questions for authors
  • Could the authors provide experimental results compared to other baselines?
  • Could the authors clarify how coefficients in TRIAD score (equation 3) are tuned and selected? Since TRIAD score is a crucial component of the workflow, the coefficients in equation (3) need more justification. Please also provide sensitivity analysis to show how changing those coefficients affects the model performance if possible.
  • Could the authors explain in equation (3), why the coefficient for complementarity (0.2 x 0.2) is much smaller than the coefficient for conflict-free rate (0.8)?
  • Could the authors provide several qualitative examples of product retrieval as well as failure cases?
Reviewer_pjceOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CMAC, a framework for multimodal product retrieval. The main idea is to use attribute consistency across image-only, text-only, and multimodal views as a supervision signal. The authors introduce the TRIAD score and use it for contrastive learning and GRPO training. The paper reports improvements over AFMRL on M5Product and shows better attribute consistency after RL training.

Critical strengths

The paper studies an important problem in multimodal product retrieval. Using attribute consistency to improve training is an interesting idea, especially for products that differ only in a few attributes. The authors also make an effort to reproduce AFMRL and compare the models using the same entity inputs. The reported retrieval improvement is encouraging.

Critical weaknesses
  1. The baseline comparison is not sufficient. The experiments mainly compare CMAC with AFMRL and older models such as VLM2Vec. However, several stronger and more recent multimodal embedding models are not included, such as Qwen3-VL-Embedding and MOON3.0, which are already mentioned in the paper. I understand that these models may use different evaluation protocols, but the authors could still evaluate them on the same M5Product benchmark.
  2. The main idea of attribute consistency needs stronger evidence. The paper assumes that agreement across three views indicates reliable attributes. However, all three views are produced by the same extractor, and they may share the same errors. Agreement does not always mean that an attribute is correct. I would like to see a more direct evaluation of attribute accuracy. The current results mainly show that the TRIAD score improves, but this does not fully prove that the generated attributes become more accurate. Also, the paper lacks a clear ablation showing how much of the retrieval improvement comes from TRIAD weighting itself.
  3. Some experiments are unfinished, and negative results are not fully reported. This is one of my main concerns. Appendix G is titled Backfill Registry (Pending Experiments). The authors state that four completed title-only experiments perform worse than AFMRL, but the actual results are not included in the paper. The authors plan to include them only after completing an improved run. I do not think this is a good way to report experimental results. Negative results are still useful, especially when they may show the limitations of the method. Moreover, Mode II has a calibration problem and still needs to be retrained. Mode III only reports generator-level improvements without showing whether the new attributes improve downstream retrieval. As a result, the complete framework has not been fully evaluated. I appreciate that the authors disclose these issues, but I think the current submission is not yet complete. It would be better to finish the experiments and report both positive and negative results before submission.
  4. The evaluation is limited to one dataset, and the query analysis is not detailed enough. The main retrieval experiments are all based on M5Product. Although the paper analyzes 1,918 queries and reports that many retrieval failures involve attribute problems, it is still unclear whether these attribute errors are the actual reasons for the failures. I would like to see which types of queries benefit from CMAC, especially whether the improvements mainly come from queries with attribute conflicts. Evaluation on another dataset would also help show whether the method generalizes.
  5. The presentation needs improvement. Some figures are difficult to read because the text is too small, especially Figures 1 and 2. Figure 1 also contains product images with visible brand names and trademarks. The authors should clarify whether these images are used under the dataset's license or other suitable permission. The organization of Section 4 could also be improved. In particular, the title of Section 4.2, AFMRL Table 2 Reproduction and Head-to-Head Comparison, is too specific to a table from another paper. A more general section title would make the paper easier to follow.
Questions for authors
  1. Why are recent multimodal embedding models such as Qwen3-VL-Embedding not included in the comparison? Does TRIAD still help when using a stronger backbone?

  2. Can the authors provide an ablation study of TRIAD weighting? Does a higher TRIAD score really mean better attribute accuracy?

  3. Appendix G reads more like an internal experiment tracking note than a section of a finished research paper. Why are the completed negative results not fully reported, and why is the unfinished Mode II evaluation still part of the proposed framework?

  4. Can the authors show whether CMAC mainly improves queries with attribute conflicts? Does the RL-trained generator also improve downstream retrieval performance?

Reviewer_bKTGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CMAC, a label-free way to check whether product attributes extracted by a multimodal LLM are faithful. It extracts attributes 3 times, from the image alone, the text alone and both together, and scores how much the 3 views agree with a hand-designed score called TRIAD. That score is then used to reweight contrastive training of a retrieval encoder, which beats the reproduced AFMRL baseline by +5.98 R@1 on M5Product. It is also used as a GRPO reward for the attribute generator, which raises consistency and halves output length, but the paper does not test whether this improves retrieval.

Critical strengths
  • The paper identifies a meaningful limitation of attribute-enhanced multimodal retrieval: errors introduced during attribute extraction can propagate to the retrieval encoder. The analysis of 1,918 queries, together with the large-scale three-view attribute decomposition, provides empirical motivation for addressing this issue.

  • TRIAD provides a closed-form consistency signal that can be integrated into both contrastive training and GRPO without requiring additional human annotations in the reward loop. The distinction between conflicting and complementary attributes is particularly useful, since information available in only one modality should not automatically be considered unreliable.

Critical weaknesses
  1. The paper assumes that agreement between image-only, text-only, and multimodal extractions indicates attribute faithfulness. However, since all three views use the same extractor, they may share similar errors or hallucinations. High TRIAD scores could therefore reflect consistent but incorrect attributes. A comparison with independently verified attributes would help validate this assumption.

  2. TRIAD relies on a relatively simple scoring function, but the choice of penalties and weights is not sufficiently justified. Also, multimodal agreement does not guarantee correctness, and attributes found only in the multimodal view do not seem to directly affect the score. More ablations and comparisons with simpler consistency measures would help clarify the contribution of each component.

  3. The paper reports improvements in retrieval and generator quality separately, but does not show whether the GRPO-trained generator improves downstream retrieval. Moreover, Mode II is proposed but not evaluated. This leaves the effectiveness of the complete CMAC framework unclear.

  4. Since TRIAD-GRPO directly optimises consistency, improvements in the same metric do not necessarily indicate better attribute extraction. The substantial reduction in output length also raises concerns that the model may simply generate fewer or more generic attributes. Additional evaluations of precision, recall, completeness, and hallucination rates would help validate the reported improvements.

Questions for authors

How do you know that agreement between the three views reflects correct attributes and not shared errors of the same extractor? Have you checked TRIAD scores against any independently verified attributes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.