- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes Textual Residual Prompt to learn the fine-grained attributes for fine-grained open vocabulary detection.
Critical strengths
- The paper is well-written and easy to follow. 2.The observation of fine-grained attributes is informative.
Critical weaknesses
1.The proposed TRP method appears closely related to compositional zero-shot learning (CZSL), which aims to recognize novel attribute–object compositions. The FG-OVD benchmark primarily evaluates fine-grained object attributes, such as color, material, and pattern, rather than fine-grained taxonomic categories such as dog breeds or bird species. The assumption that semantic displacement vectors are transferable across base categories is closely related to the attribute–object compositionality studied in prior CZSL research, including Misra et al. (2017) and Geng et al. (2025). However, the paper does not sufficiently discuss these connections or clearly distinguish its contribution from existing CZSL approaches.
[1]Ishan Misra, Abhinav Gupta, and Martial Hebert. 2017. From red wine to red tomato: Composition with context. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1160–1169.
[2]Geng, Yuxia, et al. "Graph-guided cross-composition feature disentanglement for compositional zero-shot learning." Findings of the Association for Computational Linguistics: ACL 2025. 2025.
2.Given the conceptual similarity between TRP and CZSL, the paper would benefit from comparisons with representative CZSL methods adapted to the FG-OVD setting. Such comparisons could help establish whether the proposed semantic residual modeling provides advantages over existing attribute–object composition techniques. The authors should also discuss the differences between CZSL classification and FG-OVD localization when designing these comparisons.
3.Minor
- TCR appears to be one of the strongest baselines in the experimental results, but its original reference is not clearly provided.
- ? in line 140
Questions for authors
1.Please discuss the distinction between TRP and CZSL 2.Please provide experimental comparisons between TRP and CZSL 3.Please indicate the reference for TCR
8 comments
W2 & Q2: Lack of Experimental Comparisons with Representative CZSL Methods
1. Fundamental differences between CZSL classification and FG-OVD localization.
CZSL focuses on generalization to unseen compositions, whereas FG-OVD focuses on the accuracy of fine-grained semantic discrimination and object localization. CZSL primarily investigates zero-shot generalization to attribute–object compositions, namely recognizing unseen compositions based on attributes and objects seen during training. FG-OVD primarily investigates matching fine-grained textual descriptions to specific object regions in images and localizing them. It requires not only recognizing object categories but also distinguishing differences in attributes such as color, material, and pattern within the same category, while accurately predicting object bounding boxes.
2. How to reasonably adapt CZSL methods originally designed for classification to object detection to ensure fair experimental comparisons.
We draw inspiration from L-Adapter’s approach of introducing trainable residual modules within a frozen text encoder. Considering that the computational and maintenance costs of its global compositional graph and GNN may grow with the vocabulary and composition scale, directly applying it to real-time open-vocabulary detection with dynamic queries presents certain challenges. Therefore, we omit components such as the compositional graph and GNN, insert only a shared residual MLP after the final CLIP text Transformer block and before the final LayerNorm. We name this simplified text-tower adaptation baseline T-Adapter to distinguish it from the original L-Adapter.
The mean ± standard deviation of mAP (%) across three random seeds is shown below (the results in the table differ slightly from those in the paper because they summarize results from three random seeds):
| Benchmark | T-Adapter | TCR | TRP |
|---|---|---|---|
| Hard | 48.45 ± 0.26 | 46.31 ± 0.48 | 50.14 ± 0.38 |
| Medium | 56.44 ± 0.21 | 54.90 ± 0.53 | 57.13 ± 0.20 |
| Easy | 59.72 ± 0.27 | 58.56 ± 0.68 | 59.81 ± 0.35 |
| Trivial | 58.59 ± 0.22 | 58.54 ± 0.39 | 59.38 ± 0.10 |
| Color | 45.75 ± 0.12 | 43.53 ± 0.09 | 48.05 ± 0.72 |
| Material | 53.70 ± 0.32 | 51.47 ± 0.52 | 54.07 ± 0.10 |
| Pattern | 55.32 ± 1.16 | 52.76 ± 1.02 | 55.60 ± 0.96 |
| Transparency | 63.05 ± 1.00 | 61.18 ± 1.52 | 63.24 ± 1.30 |
| Overall | 55.13 ± 0.38 | 53.41 ± 0.60 | 55.93 ± 0.26 |
Each benchmark is first averaged over valid N values, and Overall is the macro-average across the eight benchmarks. TRP’s Overall is 0.80 percentage points higher than T-Adapter, with gains of 1.69 and 2.30 percentage points on Hard and Color, respectively. We attribute the improvement to the semantic displacement conditioning used by TRP, which explicitly models the relationship between base and fine-grained descriptions, rather than directly introducing a residual adapter inside the text encoder. In addition, because CLIP is an integrated pretrained model and the T-Adapter module is located inside the text tower, it cannot directly reuse the standard CLIP final text embedding cache. Consequently, T-Adapter cannot directly reuse the precomputed final text embeddings during training and must execute the text encoder at each iteration, introducing additional computational overhead. We further report parameter efficiency and training overhead (the results in the table differ slightly from those in the paper because they summarize results from three random seeds):
| Method | Updated component | Trainable params (M) | Model ratio (%) | F+B time (ms/step) | FG-OVD mAP (%) | COCO mAP (%) |
|---|---|---|---|---|---|---|
| TRP | Textual adapter | 0.328 | 2.51 | 163.2 | 55.93 ± 0.26 | 37.3 |
| T-Adapter | Text-tower adapter | 0.329 | 2.51 | 518.4 | 55.13 ± 0.38 | 37.3 |
| Head-FT | Detection head | 2.317 | 17.71 | 139.2 | 55.38 ± 0.24 | 2.6 |
| Full-FT | Backbone, neck and head | 12.76 | 97.49 | 223.1 | 54.45 ± 0.21 | 2.5 |
TRP can directly reuse precomputed frozen text embeddings, avoiding repeated execution of the complete text tower during training. Based on the values in the table, T-Adapter’s forward–backward time per step is approximately 3.18 times that of TRP.
Revision: We have clarified the differences between CZSL classification and FG-OVD localization, introduced T-Adapter as a simplified baseline inspired by L-Adapter, and added comparisons of detection performance across three random seeds, parameter efficiency, and training overhead (Appendix B.5).
W3 & Q3: Other weaknesses
1. Sources of the comparison baselines
TCR is a comparison baseline inspired by CoCoOp and constructed for fine-grained open-vocabulary detection. Lines 291–292 of the original manuscript explicitly state this design inspiration. Therefore, TCR is not a direct reproduction of an existing method with the same name, nor is there an original paper that directly proposes TCR. Specifically, we draw on CoCoOp’s conditional prompt learning approach, conditioning on fine-grained text embeddings and predicting textual residuals through a shared generator to construct a comparison method suitable for detection tasks. Since research on parameter-efficient fine-tuning for real-time open-vocabulary detectors remains limited, our work also includes introducing multiple published parameter-efficient fine-tuning strategies into the same detection framework for comparison with TRP. Similarly, TPT is a shared textual residual baseline inspired by TaskRes. Lines 290–291 of the original manuscript explicitly use “Inspired by TaskRes” to indicate its source, rather than presenting it as a direct reproduction of TaskRes.
2. Formatting issue
We thank the reviewer for pointing this out. The issue in line 140 was caused by a LaTeX formatting error and has now been corrected.
We sincerely thank the reviewer for the positive evaluation and constructive feedback. We appreciate the reviewer’s recognition that our paper is well-written and easy to follow, and that our observations on fine-grained attributes are informative. We take the reviewer’s concerns regarding the soundness of our method seriously and address them point by point below.
W1 & Q1: TRP is closely related to compositional zero-shot learning (CZSL), but the paper does not sufficiently discuss their connections and differences
1. Connections and differences between TRP and compositional CZSL
We agree that, at a higher level, TRP and the studies by Misra et al. (2017) and Geng et al. (2025) all involve transferable compositional structures between attributes and objects. However, the specific patterns identified and the modeling approaches adopted by the three studies differ considerably.
① Misra et al. (2017) found that the visual appearance of the same attribute across different objects is strongly context-dependent. For example, the color red has different visual appearances in red wine and *red tomato*.
② Geng et al. (2025) proposed that the visual features of the same attribute vary significantly across different compositions.
③ TRP focuses on the observation that, in the text embedding space, the semantic displacements induced by the same attribute across different base categories exhibit a high degree of directional consistency, unlike the preceding two studies, both of which involve interactions between visual information and textual semantics.
It should be emphasized that TRP’s contribution is not to introduce an entirely new attribute–object compositional learning paradigm, but to identify and exploit the directional consistency of these semantic displacements for parameter-efficient adaptation of real-time open-vocabulary detectors.
2. The modeling structures of existing CZSL methods are not suitable for direct transfer to real-time open-vocabulary detectors
Although TRP is related to existing CZSL methods at a higher level, the attribute–object modeling structures adopted by existing methods are not suitable for direct application to real-time open-vocabulary detectors.
① Geng et al. (2025) perform cross-composition disentanglement of textual and visual features through L-Adapter and V-Adapter, respectively. L-Adapter constructs a global compositional graph containing attribute, object, and composition nodes. The total number of nodes is N = A + O +C, where A denotes the attribute set, O denotes the object category set, and C denotes the set of attribute–object compositions currently involved in graph construction.
However, open-vocabulary detection involves numerous attributes and object categories, resulting in a large number of combinations that can cause the compositional graph to grow rapidly. Geng et al. (2025) explicitly acknowledge this issue in the Limitations section on page 9 of their paper. Therefore, this modeling approach, which relies on a global compositional graph, is not suitable for direct transfer to real-time open-vocabulary detectors.
② Misra et al. (2017) rely on pretrained, fixed attribute and object visual classifiers for composition. Their modeling approach is restricted to a predefined set of visual classifiers, making it difficult to apply directly to query categories dynamically specified through natural language in open-vocabulary detection.
3. TRP’s research objective and core contribution
We further emphasize that our research objective is to improve fine-grained open-vocabulary detection, rather than to validate compositional zero-shot recognition capabilities(Examples include recognizing “green tomatoes” without training samples of “green tomatoes” after seeing “red tomatoes” and “green apples”; or learning the attribute “red” and the category “tomato” separately and then recognizing the combination “red tomatoes,” which was absent during training).
TRP’s distinctive contribution is not to introduce a new attribute–object compositional learning paradigm, but to identify the high directional consistency of textual semantic displacements induced by the same attribute across different base categories and apply this prior to parameter-efficient adaptation for real-time open-vocabulary detection. Our research focuses on comparisons with other parameter-efficient fine-tuning methods in the same family, rather than comparisons with CZSL methods in compositional zero-shot recognition capabilities.
[1] Misra I, Gupta A, Hebert M. From red wine to red tomato: Composition with context[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 1160-1169.
[2] Geng Y, Zhu R, Chen J, et al. Graph-guided cross-composition feature disentanglement for compositional zero-shot learning[C]//Findings of the Association for Computational Linguistics: ACL 2025. 2025: 2678-2690.