acceptodds
Withdrawn from ICLR 2027

Toward Detail-Intensive E-commerce Image-Text Retrieval: Hierarchical Refinement of Query-Relevant Representations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

E-commerce image-text retrieval (E-ITR) aims to rapidly retrieve relevant products from large-scale image corpora in response to user textual queries, supporting everyday shopping search, recommendation, and assistance. As living standards improve and product choices expand, E-commerce images become increasingly detail-intensive, containing densely packed elements such as product logos and usage instructions. Moreover, real-time response requirements necessitate retrieval from low-resolution E-commerce images, making it more challenging to accurately identify query-relevant products. To address the dilemma, we contribute EComClutter, a detail-intensive E-commerce image-text retrieval database with 94,481 query-image pairs, which are sourced from 35 E-commerce categories (e.g., fashion and electronics). Relevance is annotated through an ensemble of chain-of-thought reasoning outputs from multiple advanced MLLMs, followed by rigorous human verification. Based on EComClutter, we further propose EHiRe, a novel E-ITR framework that hierarchically refines query-relevant representations through Detail Region Rearrangement, Important Tokens Modulation, and Rel- evant Embedding Selection. EHiRe achieves state-of-the-art retrieval accuracy under low-resolution inference settings. Extensive experiments and comparisons demonstrate the superiority of the proposed method.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Detail-Intensive,E-commerce Image-Text Retrieval, Low-Resolution, Query-Relevant Representation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DupuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes EHiRe for detail-intensive e-commerce image-text retrieval with low-resolution inputs. It combines Detail Region Rearrangement (DRR), which rearranges local details with global context; Important Tokens Modulation (ITM), which uses high-resolution teacher attention and CoT guidance during training; and Relevant Embedding Selection (RES), which guides embedding updates through task and teacher rewards. It also introduces EComClutter, containing 94,481 query-image pairs across 35 categories with three relevance levels. Extensive experiments on EComClutter demonstrate the approach's effectiveness for detail-intensive product retrieval across different backbones and input resolutions. Experiments on MMEB-V2 further support its transferability to broader image-text retrieval tasks.

Critical strengths

1.The method addresses the loss of fine-grained information in low-resolution retrieval by combining local visual evidence with query-relevant representation learning. High-resolution teacher guidance and graded-relevance supervision encourage the model to retain details needed to distinguish partial from complete query matches.

2.Experiments across different backbones and input resolutions demonstrate the approach's effectiveness for detail-intensive retrieval. Results on MMEB-V2 also support its transfer to a broader range of retrieval tasks.

Critical weaknesses
  1. Unclear attribution of performance gains. The comparisons do not adequately account for the extra training and domain-specific fine-tuning used by EHiRe. Section 5.1 gives EHiRe four extra epochs after two SFT epochs, while Table 6 compares its 0.846 average with 0.783 for the model trained only with SFT. Table 4 compares GME Original at 0.719 with GME plus EHiRe at 0.809, although fine-tuned GME* already reaches 0.774 in Table 2. Without these controls, the experiments do not convincingly show that EHiRe offers an advantage over standard fine-tuning.

2.Unverified data separation and label reliability. The paper does not explain whether training and test sets share images or products, and it provides too little evidence about the reliability of fully relevant labels. Section 5.1 states that the queries in the two sets are different, but gives no image or product overlap statistics. Section 3.2 reports that an average of 96% of annotators agreed with the MLLM consensus on 1,000 unanimous predictions, but gives no separate audit results for fully relevant pairs, which make up 9.27% of the dataset. Fully relevant labels determine Recall@K, and their reliability cannot be judged from the overall agreement rate. These gaps weaken confidence in the retrieval results, and the experiments do not provide convincing evidence of generalization to unseen products.

Questions for authors
  1. Please clarify the novelty of EHiRe and its fundamental distinctions in design rationale from existing detail-aware retrieval, high-resolution teacher distillation, and relevance-supervised learning methods.

  2. How many training epochs did each fine-tuned baseline in Table 2 receive? Are comparisons using the same total number of training epochs as EHiRe already available?

  3. Do the training and test sets contain identical or nearly identical images, or images of the same products? If so, how much overlap is there?

  4. In the 1,000 audited pairs, what are the sample counts and average annotator agreement with the MLLM consensus for each relevance level?

  5. In Table 5, does DRR enlarge regions from the original images or from images already reduced to the stated resolution? How does this affect the interpretation of the claimed detail-preservation benefit?

Reviewer_oKDxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper consider image-text retrieval for detail-intensive product images. The authors propose a dataset EComClutter. They further propose a method, EHiRe. This method uses a high-resolution teacher and CoT supervision to improve low-resolution representations. Experiments on six MMEB-V2 retrieval tasks show the method effectiveness.

Critical strengths
  • The authors considers the problem of image-text retrieval for detail-intensive product images is practically relevant. Addressing this limitation has value for e-commerce applications.

  • The dataset could be useful. EComClutter invloves multiple product categories and provides graded relevance annotations.

  • The improvements are substantial. The authors provide experiments across multiple backbones. These results suggest that the proposed approach is effective.

Critical weaknesses
  • It would be good to describe how the CoT is designed in the dataset construction part.

  • As the large model is fine-tuned in this work, a dedicated subsection in experiments is suggested to show that the fine-tuned model improves retrieval performance.

  • Only a Chinese dataset is provided, without a matching English dataset, and all image samples are in Chinese.

  • Some references on Image-Text Retrieval are missing.

  • The dataset and code are not publicly available.

Questions for authors
  • Is DRR applied at inference? If so, how does the system support offline indexing of image embeddings? Could you report end-to-end latency?

  • How are the two ablated variants trained? What loss is used without RES?

Reviewer_jNwzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates detail-intensive e-commerce image-text retrieval under low-resolution settings. The authors first introduce EComClutter, a dataset containing 94,481 query-image pairs from 35 e-commerce categories, with three-level relevance annotations obtained through multiple multimodal large language models (MLLMs) and human verification. Based on this dataset, the authors propose EHiRe, a framework that progressively refines query-relevant visual representations through three modules: (1) Detail Region Rearrangement (DRR), which preserves important visual details in low-resolution inputs through local region cropping and rearrangement; (2) Important Tokens Modulation (ITM), which adjusts the importance of visual tokens using attention information from a high-resolution teacher model; and (3) Relevant Embedding Selection (RES), which optimizes image-text representations by combining task rewards and teacher ranking rewards. Experimental results show that EHiRe outperforms several existing multimodal retrieval methods on EComClutter and achieves competitive results across six image-text retrieval tasks on MMEB-V2.

Critical strengths
  1. Practical problem. The paper addresses a practical problem involving e-commerce images that contain dense textual information, product descriptions, and fine-grained visual details. It highlights the difficulty faced by conventional image-text retrieval methods in identifying query-relevant information under low-resolution settings.
  2. New dataset contribution. The authors introduce EComClutter, a dataset comprising 94,481 query-image pairs with three-level relevance annotations, designed to evaluate retrieval performance on detail-intensive e-commerce images.
  3. Experimental validation. EHiRe achieves promising results across multiple retrieval metrics. The ablation studies demonstrate that DRR, ITM, and RES each contribute to the overall performance improvements.
Critical weaknesses
  1. Limited evaluation scale. The paper emphasizes the importance of large-scale e-commerce image-text retrieval. However, each query is associated with only 9.74 candidate images on average. The current evaluation setting is therefore closer to relevance ranking over a small candidate set than to retrieval from a large-scale product database in real-world scenarios. Given the limited number of candidate images, the reported Recall@3 and Recall@5 results may not adequately demonstrate the advantages of the proposed method in large-scale retrieval settings.
  2. Scalability concerns with query-dependent image rearrangement. The image rearrangement process in DRR depends on both the user query and the candidate image, meaning that the processed representation of an image may vary across different queries. This potentially requires generating different visual representations for the same product image depending on the query. However, practical large-scale image-text retrieval systems typically rely on precomputing and storing image embeddings to enable efficient similarity search.
  3. Insufficient reporting of language distribution. Many product images and user query examples in the paper contain Chinese text, but the authors do not explicitly report the language distribution of EComClutter. This creates a barrier for readers who do not understand Chinese. Even if the original data is in Chinese, the relevant textual content in the figures should be presented in English, with appropriate clarification in the figure captions or main text.
Questions for authors

Q1. This paper refers to the appendix for some important implementation details of DRR, but the appendix is not included in the current manuscript.

Q2. DRR preserves more visual details within a fixed input resolution by cropping and enlarging local regions from the original images. Why not extract textual information and other fine-grained details using OCR or similar techniques during the indexing stage, and store them in a structured format for subsequent retrieval?

Q3. Could the authors provide more details about the original data annotation process? For example, what proportion of samples received consistent annotations from all three MLLMs? How frequently did MLLM annotations disagree with human annotators? Additional statistics and examples would help readers better assess the reliability of the dataset.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.