- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies sparse cross-modal retrieval from a geometric alignment perspective. The main idea is to treat independently learned image and text representations as two latent spaces that need to be aligned. Based on this, the authors propose SLNA, which first uses sparse verified pairs to estimate a global Procrustes alignment, and then models the remaining local mismatch with RBF-based non-rigid corrections. An optional soft refinement further makes use of unverified candidates. Experiments on Flickr30k and MSCOCO show clear improvements over rigid alignment and modest but consistent gains over direct nonlinear mapping, together with additional analyses on landmark efficiency, structural consistency, and robustness.
Critical strengths
- I find the geometric formulation of sparse cross-modal alignment quite intuitive, especially the separation between global coordinate mismatch and local non-rigid deformation.
- SLNA combines Procrustes-based global alignment with regularized local residual modeling, making the overall pipeline easy to follow and implement.
- I also appreciate that the evaluation goes beyond retrieval accuracy and includes analyses of structural consistency, landmark efficiency, geometry preservation, robustness, and synthetic deformation recovery, which makes the geometric interpretation more convincing.
- The method works with fixed unimodal encoders and only a small number of verified pairs, which is a meaningful setting when paired cross-modal data is limited.
Critical weaknesses
- The motivation for using both the coordinate and angular branches is reasonable, but their complementary roles could be explained more clearly. In particular, the different fusion weights selected for Flickr30k and MSCOCO are interesting and deserve a short discussion.
- SLNA-soft helps on some metrics but leads to small drops on others. It would be useful to hear the authors’ view on when the soft refinement is likely to be beneficial, and when the base SLNA model may be a safer choice.
- The main evaluation is based on independent 400-sample test sets, and an identity-matched comparison with SUE is not available. It would be useful to clarify how representative these subsets are of the full datasets.
- Since both the global alignment and the local residual model rely on sparse landmarks, I wonder how much performance depends on landmark coverage rather than just the number of landmarks. Some discussion of the role of spatial or semantic coverage in the spectral space would be helpful.
- The synthetic deformation experiment shows that the residual model can recover non-rigid distortions. However, the connection between the synthetic deformation and the residual mismatch observed on Flickr30k and MSCOCO could be made clearer.
- The paper uses for all spectral representations, but the effect of this choice is not discussed in detail. A brief explanation of why this dimensionality is appropriate, and whether the method is expected to be sensitive to moderate changes in , would improve the technical clarity.
Questions for authors
- Could the authors clarify how sensitive SLNA is to imperfect global alignment? Since the local residual field is fitted after the Procrustes stage, it would be useful to understand whether moderate errors in the estimated global transform can be effectively absorbed by the local RBF correction, or whether they noticeably degrade performance.
- Could the authors provide more intuition on the dataset-dependent fusion between the coordinate and angular branches? In particular, the selected differs substantially between Flickr30k and MSCOCO. Is this mainly due to differences in the geometry of the learned spectral spaces, or simply an outcome of cross-validation?
Official Comment by Authors
Dear Reviewer,
We sincerely appreciate the reviewer's thoughtful and constructive feedback, which has helped us further clarify our methodology, strengthen our experimental analysis, and improve the overall quality of our manuscript. We hope that our detailed responses and additional experimental evidence have adequately addressed your concerns, and we would be grateful to know whether any questions remain.
We also welcome any further questions or suggestions you may have and will be happy to provide additional clarifications and engage in further discussion.
Best regards,
SLNA Authors