acceptodds
Desk rejected at ICLR 2027

FIVE: Frequency-Aware and Intent-Guided Visual Evidence Learning for Remote Sensing Composed Image Retrieval

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Remote Sensing Composed Image Retrieval (RSCIR) is an emerging retrieval paradigm that retrieves target images from complex scenes with multi-scale objects guided by diverse modification intents. However, existing methods still suffer from limitations in both visual representation granularity and modification intent modeling. On the one hand, small objects are prevalent in remote sensing images, yet their visual cues are often overwhelmed by large background regions in the resulting image representations, limiting fine-grained retrieval. On the other hand, most methods inject the query text into the retrieval representation as a unified condition, lacking explicit modeling of the visual evidence selection and semantic transfer mechanisms required by different editing operations. To address these issues, we propose Frequency-aware and Intent-guided Visual Evidence Learning (FIVE), a novel framework for RSCIR that improves the discriminability of retrieval representations from two perspectives: object-level visual representation enhancement and intent-guided semantic selection. Specifically, we design a Frequency-aware Object Enhancement module that introduces the Fast Fourier Transform during visual feature modeling to explicitly capture frequency-domain cues, thereby enhancing the representation of contours, textures, and local structures of small objects. Furthermore, we develop an Intent-guided Visual Evidence Selection module that leverages modification intents to guide the selection and reorganization of object regions, attribute cues, and spatial relations in the reference image, generating intent-aware query representations. Experimental results on two RSCIR datasets demonstrate that our method outperforms existing approaches, and ablation studies further verify the effectiveness of frequency-domain representation enhancement and modification intent modeling.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Remote Sensing Composed Image Retrieval, Modification Intent Modeling, Frequency-domain Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.17 over 6 reviews, above 49% of the others.
Reviewer_bi7oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FIVE, a framework for remote sensing composed image retrieval that combines Frequency-aware Object Enhancement (FOE) and Intent-guided Visual Evidence Selection (IVES) to improve small-object representations and modification-intent modeling. The authors also introduce PatternCIRRS, a dataset containing 17,469 reference-dependent retrieval triplets. Experiments on PatternCIR and PatternCIRRS report improvements over existing methods, supported by ablation studies and qualitative comparisons.

Critical strengths
  • The paper addresses two important challenges in remote sensing composed image retrieval: fine-grained small-object representation and explicit modification-intent modeling.
  • The proposed framework achieves competitive retrieval performance across two datasets, and the newly constructed PatternCIRRS benchmark provides a potentially useful resource for evaluating reference-dependent retrieval.
Critical weaknesses
  • TisCIR is originally designed for zero-shot retrieval without labeled triplets, whereas FIVE is trained using supervised retrieval triplets. Comparing these methods without clearly distinguishing their supervision settings weakens the claimed superiority and makes the large performance gap difficult to interpret.
  • Removing the object-aware mask causes only marginal performance drops (e.g., 0.26 and 0.29 points in average global recall on the two datasets), despite its central role in both FOE and IVES.
  • FOE selects reference-image patches according to the semantics of the edited object, while Section 3.4 explicitly states that the object is absent from the reference image for addition operations.
  • Section 3.2 introduces an object–intent–constraint semantic graph, but the subsequent formulation does not explicitly utilize its graph edges or relational structure. Consequently, it is unclear what the proposed graph contributes beyond independently encoded semantic attributes.
  • Table 3 reports average global and subset recall of 71.84/81.97 for the full model on PatternCIRRS, whereas Figure 9 reports different results under the stated default hyperparameters. These unexplained discrepancies raise concerns about experimental consistency and reproducibility.
  • The Qwen3-4B instruction parser lacks sufficient training and accuracy details, despite being essential for selecting the correct editing branch. The ablations do not establish whether the gains stem from accurate intent understanding or simply increased model complexity.
Questions for authors

Please see the Weaknesses.

Reviewer_9MYCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a framework, FIVE, for Remote Sensing Composed Image Retrieval. FIVE tries to address "tiny object problem" in RSCIR by introducing frequency-aware object enhancement. Additionally, Intent-Guided Visual Evidence Selection is introduced to explicitly model how different modification intents reshape the reference image. To support this research, PatternCIRRS is curated.

However, I believe that remote sensing is inherently an application-driven discipline, and its technological evolution should serve real-world Earth observation and practical demands, and this is a defining characteristic of the field. Yet, since the PatternCIRRS dataset proposed in this paper is essentially generated from PatternCIR via large model annotations, the actual practical value of this dataset remains open to discussion.

Critical strengths

This paper clearly outlines two major issues that need to be addressed: the submergence of small target clues in the background, and the injection of query text as a uniform condition without distinguishing between different editing operations. Moreover, both issues have corresponding module designs. Especially, the frequency domain design part is intuitive. Additionally, a number of existing methods are compared.

Critical weaknesses

The “tiny object problem” exists in both the general domain and remote sensing, and frequency-domain modeling is a general design for it, although the two settings differ. The overall design of the method does not reflect any specificity to small remote sensing targets. I therefore believe that validating the method on more that two general-domain CIR datasets would make the paper more valuable.

As far as I can tell, PatternCIRRS does not really introduce new data. Page 6, line 309 states that “PatternCIRRS retains the official train/validation/test partitions of PatternCIR,” and the authors do not appear to report any new data source from other real RSCIR scenarios. If so, the data produced by the Triplet Construction process would be regarded merely as an intermediate product for implementing the FIVE method. This may weaken the contribution of the dataset.

In addition, having reviewed the PatternCIR series of papers, I noticed that the RSCIR field is still in its early stages. Whether the setting of PatternCIRRS aligns with potential practical applications or research value remains an open question. eg, while searching for a specific small car with a distinct appearance appearing or disappearing in a target region may be challenging, technical difficulty and problem significance are two different matters. This is more like artificially creating some meaningless problems. So I am not sure about the practical value of this dataset for remote sensing, and that is the reason why I want to know the method performance on general domain.

Additional comments (areas the authors could improve; I do not treat the writing issues as a reason for my reject)

The paper defines a large number of symbols, and most of them are used only once. This distracted me while reading. The writing style is not that comfortable for a human reader. Some definitions also use sub-subscripts, such as h_{o_i}, so the authors could improve how the equations are arranged. In addition, G in Figure 2 is not used consistently with the main text, and Eq. (2) should end with a comma “,”.

Questions for authors

I am sorry that, based on this year's 4-point rating scale, I can only give you a score of 2. However, except for improving the paper writing, I may consider change my score if the authors do:

  1. evaluate your method on more general domain dataset if possible.

  2. based on the definition of input "input consists only of a reference image and a semantic modification description", whether your method can be applied to other dataset?

  3. is this a real problem or a human-made problem? By computing image features (ResNet152) from the existing remote sensing image library (PatternCIR), different images with similarity greater than 0.85 are selected, and then a large model is used to construct a description of the differences between the images in the later stage. Does the dataset constructed using this method, especially if the descriptive differences do not come from actual business scenarios, have substantive significance? Or, which remote sensing scenarios with actual CIR requirements the dataset can cover?

  4. From the perspective of feature change editing, real-world RSCIR faces far more complex scenarios with much broader spatial search ranges. IVES limits modifications to just four categories: Deletion, Addition, Spatial Transfer, and Attribute Editing. If real-world demands require combinations of these operations, does the proposed method still hold practical value?

In summary, unless the authors can adequately address the practical utility of their formulation, the article's persuasiveness hinges on proving the algorithm's effectiveness in general CIR/CV benchmarks. Without this, the paper suffers from obvious limitations and risks addressing merely a highly engineered, niche problem.

Reviewer_La5QOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper tackles the composed image retrieval problem for remote sensing. The authors propose the benchmark PatternCIRRS, which is a refinement of PatternCIR but with more complementary text queries. They also propose FIVE, a CIR method that uses both spatial and frequency-domain features for the image modality, in order to increase performance on results with very small objects. FIVE is reported to have state-of-the-art performance in PatternCIRRS and PatternCIR

Critical strengths
  1. The paper has a benchmark contribution. It is a refinement of a pre-existing dataset, but the need was motivated quite well in Figure 3
  2. The idea of combining spatial and frequency information is well motivated as well, through the rationalization of very small target objects, and it sounds quite appealing in terms of the possible orthogonal information the two image modalities can carry
  3. The proposed method seems to be working well
Critical weaknesses
  1. It was not clear to me which loss was training which component. I would like to see a cleaner summary of that
  2. I would like to see some more experiments and information about the method and the competitors to further understand the contribution of the paper (mainly latency vs. performance and number of weights)
  3. Some information is missing on how Qwen was used
Questions for authors
  1. In line 161, you mention a LoRA-finetuned Qwen. I couldn't find enough information in the paper to understand everything I needed. Please tell me if you fine-tune Qwen yourself, how it is fine-tuned, on which dataset, and what targets/losses.
  2. Please report the number of weights your full method needs in evaluation and the source of these weights (eg is Qwen needed in evaluation?), as well as the number of weights of each competitor in Table 1
  3. I would like to see a latency vs. performance trade-off of all methods in Table 1
  4. Table 3: It is not very clear to me how the MLP of FOE is trained in ablation #7. Is it trained end-to-end with L_Q? Please provide a table with the different trainable weights and which loss in the total objective (L_Q, L_FOE, L_IVES) they are included in.
  5. I see in Table 3 that some variations are very close to the full FIVE (eg #6 and #8). I would like to see the average of 5 seeds and the std of the table 3. Also I believe the full K version of Table 3 would be a nice addition to your supplementary
1 comment

I thank the authors for the replies. I will keep my positive rating

Reviewer_jw6uOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the issues of the visual features of small targets being easily obscured by complex backgrounds in remote sensing combined image retrieval (RSCIR), and the lack of explicit modeling for the text modification intention. To address these problems, the FIVE framework is proposed. This method designs a frequency-aware target enhancement module (FOE), which enhances the fine-grained visual representation through spatial target selection and Fourier frequency domain feature extraction. At the same time, an intention-guided visual evidence selection module (IVES) is proposed, which realizes intention-driven visual evidence reorganization by modeling operations such as deletion, addition, spatial migration, and attribute editing. Additionally, the authors constructed a PatternCIRRS dataset containing 17,469 triples, and achieved superior retrieval performance on PatternCIR and PatternCIRRS compared to existing baselines.

Critical strengths
  1. Focusing on the two key challenges of small target identification and modification intention modeling in remote sensing images has practical research value.
  2. FOE combines spatial and frequency domain features to attempt to enhance the fine-grained target representation in complex backgrounds.
  3. IVES explicitly distinguishes different editing operations and introduces a structured semantic modeling mechanism for combined retrieval.
Critical weaknesses
  1. FOE performs FFT on low-resolution visual tokens and retains only the mean amplitudes of four radial frequency bands. This may result in the loss of spatial position and direction information of small targets.
  2. Equation (5) directly aligns the object features of the reference image with the global features of the target image. In scenarios of object deletion or addition, it may introduce unreasonable supervision signals.
  3. IVES only defines four types of single editing operations and does not clearly specify how to handle the widespread replacement, quantity changes, and composite editing instructions in the dataset.
  4. The LoRA fine-tuning data, structural analysis accuracy, and error propagation analysis of Qwen3-4B are insufficient, making it difficult to determine the actual reliability of the intent modeling.
Questions for authors
  1. Can the author provide a comparison of FFT and DCT, spatial high-pass filtering, etc. as alternative methods, and conduct group evaluations based on the actual size of the target to prove that radial spectrum aggregation can indeed enhance the discriminative features of small targets?
  2. For the deletion operation, the objects referenced in the figure do not exist in the target image; how does the author explain the rationality of Equation (5) in bringing the object features closer to the target image features? Has the intention-conditioned contrastive supervision been considered?
  3. For compound instructions such as "replace multiple objects" or "add and delete objects simultaneously", how does IVES map and combine multiple operation branches? Can it provide independent performance analysis for different editing types?
  4. Can the author explain the source of fine-tuning data and training strategy for the Qwen3-4B parser, and report the accuracy of parsing objects, intentions, and constraints, as well as the impact of parsing errors on the final retrieval performance?
Reviewer_1vsROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FIVE for remote sensing composed image retrieval. The method has good novelty and a sound approach to solving the problem, but the conclusion lacks sufficient evidence and its generalizability is uncertain.

Critical strengths
  1. The issues discussed in this article have significant practical significance and are valuable research.
  2. The baseline of the experiment is comprehensive and the experimental sample size is sufficient.
Critical weaknesses
  1. The dataset proposed in the paper is highly aligned with FIVE, but its generalization ability is worrying. This evaluation may be more inclined to reward methods that are consistent with the data generation logic, rather than necessarily representing the general combined retrieval capability.
  2. FIVE additionally uses Qwen3-4B and structured semantic graphs, while most baselines only handle the original text-image features. Does this lead to unfair results? The authors can add some ablation/comparison experiments.
  3. The existing ablation experiments simultaneously remove the complete frequency domain branch and its representation capacity, which can only prove that adding this branch is useful, but cannot prove that the FFT itself is effective.
  4. PatternCIRRS and PatternCIR share the underlying images and partitions, and the visual distribution faced by the models is basically the same. The paper lacks independent image sources, different sensors, different spatial resolutions, or migration results on different geographical regions.
  5. Figure 2 looks nice, but the font is too small, which is not conducive to reading.
  6. The paper does not report the number of parameters, FLOPs, single-query delay, memory usage, and index overhead. For large-scale remote sensing archive retrieval, the additional inference cost is also very important.
Questions for authors

I have no further questions. I hope the author can provide their responses to the Critical Weaknesses.

Reviewer_wad1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies remote sensing composed image retrieval and proposes FIVE, combining Frequency-aware Object Enhancement (FOE) with Intent-guided Visual Evidence Selection (IVES). A fine-tuned Qwen3-4B parser extracts objects, intents, and constraints. FOE uses object-weighted patch features and radial FFT amplitude summaries, while IVES models deletion, addition, spatial transfer, and attribute editing. These features augment a BLIP-2/Q-Former query representation. The paper also introduces PatternCIRRS, containing 17,469 automatically generated and manually verified triplets. Experiments on PatternCIRRS and PatternCIR report improved retrieval performance, with module ablations and small-object analyses.

Critical strengths
  1. The paper addresses two clear challenges in remote sensing composed image retrieval: preserving small-object details and understanding modification intents. FIVE improves the reported aggregate metrics on both datasets, while the component ablations and small-object analyses support the usefulness of its design.
  2. PatternCIRRS provides queries that describe changes from a reference image to a target image. The dataset includes automatic and manual verification, and the comparisons with image-only and text-only retrieval show the benefit of using both inputs.
Critical weaknesses
  1. The experiments do not show whether FFT is better than simpler spatial processing. Table 3 shows that removing the frequency branch reduces averaged global recall on PatternCIRRS from 71.84 to 64.02. However, removing a whole branch only shows that the branch is useful. It does not show that FFT is necessary. The paper should compare FFT with spatial pooling or multi-scale aggregation using the same object weights and a similar number of feature tokens.
  2. It is unclear how IVES selects the correct object instance for editing. Equation (1) combines object relevance scores into one mask, and the deletion branch in Eq. (7) does not explicitly use location constraints. For an image containing two storage tanks, how does it distinguish “remove the left tank” from “remove the right tank”? If location information is already included in the object embeddings, the paper should explain this.
  3. The paper does not report the cost of processing a query. FIVE adds a Qwen3-4B parser and several feature-processing components to the BLIP-2 retrieval model, but reports no query latency or memory usage. It also does not compare these costs with the Q-Former baseline. Since the method targets large remote sensing image collections, the paper should report the time and memory required for a complete query, including instruction parsing, and separate this from offline gallery encoding.
Questions for authors
  1. Does the FFT branch outperform spatial pooling or multi-scale aggregation when using the same object weights and a similar number of feature tokens?
  2. How does IVES distinguish between “remove the left tank” and “remove the right tank”? Where does location information enter the deletion operation in Eq. (7)?
  3. What are the end-to-end query latency and peak GPU memory usage, including instruction parsing, compared with the Q-Former baseline? Please report offline gallery encoding separately.

Comments on the paper

1 comment

Dear Reviewers wad1, 1vsR, jw6u, and bi7o,

The author–reviewer discussion closes today, Nov 18 (AoE). Please read the authors’ response to your review and briefly reply: which concerns are resolved, which are not, and whether your rating changes.

Thank you to La5Q and 9MYC for already engaging.

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.