W1: Applying flow matching to one-dimensional scalar linear interpolation appears to be severe over-engineering that lacks core ablation justification. Moreover, given that the correction path is constructed via a deterministic linear combination zm(tk) with piecewise constant velocity um(t) derived from simple linear interpolation, it remains conceptually unjustified why an auxiliary velocity field vϕ must be trained to predict derivatives and integrated via Euler approximation (Eq. 4) at every epoch to recover a scalar, rather than directly regressing the target similarity s^m(tn) using the exact same three-layer MLP. Finally, the ablation in Figure 4(b) merely compares SimFlow against Direct Trajectory, thereby conflating the incorporation of relation context cm with the necessity of the velocity field itself, rendering the claimed 3.4–3.9 RSum gain uninformative regarding the true benefit of flow matching.
W2: There exists a severe representation covariate shift between the velocity field training and the flow-guided adaptation phases. According to Appendix A.1, the image and text representations in the relation context cm are extracted by the auxiliary encoders during velocity field training. However, during the subsequent adaptation epochs, the auxiliary encoders are discarded and vϕ remains frozen, while the context cm is dynamically constructed at each epoch using representations produced by the continuously updated backbone encoders Fv and Ft. The authors merely assume compatibility based on identical architecture and initialization. Consequently, the evolving feature space of the backbone encoders inevitably diverges from that of the auxiliary encoders, introducing unmanaged distribution drift at the input of the frozen velocity field.
W3: Section 3.2.1 states that reference relations are constructed by retrieving clean counterparts via the auxiliary encoders, while simultaneously asserting that their similarity trajectories {smref(tk)}k=0K−1 are tracked and recorded during the training of these very auxiliary encoders. This creates an unaddressed causal dilemma regarding how the reference relations could be retrieved and tracked from t=0 before the auxiliary encoders are even trained. Second, while Appendix A.1 specifies that clean and noisy relations are dynamically re-identified at the beginning of each of the eight adaptation epochs, the integral for deriving the target similarity in Eq. (4) strictly relies on the initial similarity sm(0), leaving it entirely ambiguous how sm(0) is defined or retrieved if an image-text pair is newly detected as noisy only at an intermediate epoch (e.g., epoch 5). Third, candidate FN relations are ephemeral, non-diagonal image-text combinations (Xi,Yj) (i=j) mined exclusively via mutual nearest neighbors within individual mini-batches of size B=512. Consequently, due to standard epoch-wise data shuffling, an off-diagonal pair formed in one epoch will almost certainly not co-occur in the same batch in subsequent epochs, making it practically impossible to maintain a persistent initial similarity sm(0) or track a continuous evolution trajectory across adaptation epochs for these fleeting relations.
W4: The greedy low-similarity batch construction during auxiliary encoder training distorts the reference trajectory distribution and overlooks computational feasibility. According to Section 3.2.1, in order to reduce false negatives during the training of the auxiliary encoders Fvaux and Ftaux, each batch is constructed greedily by selecting relations whose images and texts exhibit low similarity to those already chosen. While this heuristic minimizes the incidence of false negatives, it artificially eliminates informative hard negative pairs within each batch. In cross-modal contrastive learning, the absence of hard negatives depresses the repulsive force between unmatched samples, which inevitably inflates cosine similarities across unpaired instances and introduces a systematic upward bias into the collected reference trajectories {smref(tk)}. Furthermore, the authors provide no explanation or empirical analysis regarding the computational complexity and scalability of executing such greedy similarity-based batch selection on large-scale datasets.
W5: The total objective in Eq. (8) exhibits a severe loss weighting imbalance with λflow=5 and λstruct=25. Yet, neither the main text nor the appendix provides sensitivity analyses for either λflow or λstruct. Furthermore, the manuscript only reports an isolated adaptation speed on the smallest dataset (Flickr30K) that omits the prerequisite overhead of auxiliary encoder training, nearest-neighbor retrieval, and velocity field optimization. This fundamentally masks the true end-to-end wall-clock training time required for full datasets such as MS-COCO and CC140K, where repeated GMM re-fitting and Euler integrations are executed.
Official Response to Reviewer eg5N - Comment#1
We thank the reviewer for the thoughtful feedback and for recognizing our approach as an "interesting perspective" that formulates noisy correspondence correction as a trajectory learning problem. We also appreciate the acknowledgement that our "experiments are comprehensive" and that the paper is "easy to follow with clear writing". We appreciate these encouraging comments and will address all remaining concerns in the response below. All clarifications and supplementary analyses from our response are included in the revised manuscript and its Appendix.
1. Directly Using Reference Trajectories.
We evaluated the mentioned alternative as the Direct Trajectory variant in Figure 4(b) of Appendix B. This variant uses the reference trajectory of each noisy relation as its correction target: at the current adaptation progress, the target is obtained by linearly interpolating the two nearest recorded similarities of the reference relation. On Flickr30K at 20% noise, this variant is lower than SimFlow by 3.4 to 3.9 RSum for K∈{2,5,10,20}.
The performance drops because the reference relations cannot fully represent the noisy relations. Although each reference relation resembles its noisy relation, the reference relations are constructed from a limited set of clean samples and cannot cover all the noisy relations newly detected during adaptation. As a result, the reference similarities start from a different value and do not reflect the content of each noisy relation. In contrast, the velocity field starts from the initial similarity of the noisy relation and is conditioned on its own image-text representations and noise type, which provides a target specific to that relation.
For additional comparisons with other simpler alternatives, i.e., target similarities directly computed by the auxiliary encoders and direct similarity regression with the same MLP as the velocity field, please refer to our response to Reviewer 2f4y (2-1. Comparison with Simpler Options for Obtaining Target Similarities). The results show that the proposed method consistently outperforms both alternatives, indicating the benefit of deriving target similarities along a coherent correction trajectory, rather than from fixed or independently predicted targets.