- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper delivers a causal-theoretic treatment of compositional reasoning vulnerabilities in CLIP and downstream Mllms. The authors construct a token-aware SCM of image-text generation, establishing that while contrastive alignment successfully identifies modality-invariant latents at the token level, it suffers from an intrinsic composition nonidentifiability: there exist "pseudo-optimal" text encoders that achieve near-zero alignment loss yet remain invariant to semantic perturbations such as SWAP. Leveraging the constant modality gap, the authors show this blindness propagates to visual representations, culminating in an algorithm-independent impossibility theorem showing that no downstream projector, LLM capacity, or instruction tuning can elevate an MLLM above random chance on affected queries. Guided by this diagnosis, the paper develops compound hard-negative data curation schemes (Multi-Calling / Needle-Tracking Multi-Calling) with paired synthetic negative images, yielding improvements across CLIP and MLLMs on SugarCREPE and MMVP while safeguarding zero-shot generality.
Critical strengths
Problem has a good merit, this is a long-standing problem associated with CLIP.
Rigorous theoretical foundation are provided.
Critical weaknesses
The theoretical proofs rely on multiple strong structural assumptions. I don't know how applicable they are, looks like its easy to doubt its meaningfulness. (I didnt check all Lemmas and I am not an expert in theory).
Proposed fix solution is quite expensive and involves multiple models (MLLM, T2I).
Baselines are very old. No good open-source MLLM verification (Qwen3.5/3.8 27B, Gemma...) More like 2024-2025 paper. Evaluation is also very limited. only 3 benchmarks are reported in MLLM part.
minor:
Please check lines 1907 and 1977 for wrong references.
Questions for authors
please see weaknesses
Confidential note regarding Submission 25031 and the novelty concern raised by Reviewer KWW4
Dear Area Chair,
We thank you and the three reviewers, d2qV, KWW4, and 8ixv, for their time and careful reading of our submission. We have studied every comment closely. Most of the technical points are well taken, in particular the issues around Eq. (75), the construction of g** in Theorems 3–5, the scope of Theorem 8, and the inconsistencies in Tables 5–6. We agree they need substantive revision, and we will address them fully in the next version of the manuscript.
However, we have decided not to post a rebuttal in this cycle. One main reason concerns Reviewer KWW4's central novelty claim, which we explain confidentially below.
1. The "previously published" claim of KWW4 is factually incorrect.
Reviewer KWW4 states that most of our theoretical contributions (Theorems 1 and 3–5) have "already been published" at ICLR 2026, citing arXiv:2510.26302. That work is in fact an earlier version of this submission by the same author(s). [It was submitted to ICLR 2026 and was not accepted (OpenReview ID: jA7y8mTUsN).] It has never appeared in any peer-reviewed archival venue. It therefore does not count as prior publication under the ICLR dual-submission policy.
We acknowledge that the arXiv PDF carried the header "Published as a conference paper at ICLR 2026". This was a template artifact from preparing the de-anonymized preprint, and we have now corrected it (arXiv vX). We regret any confusion it may have caused.
2. Our concern with how the claim was reached.
The two documents share most of their wording, notation, and formatting nearly verbatim. That alone strongly suggests they are versions of the same work, not a case of independent prior work. In such a situation, we believe a reviewer should first verify whether the cited work was actually accepted and published. That check takes a single click on OpenReview. Reviewer KWW4 instead presented the overlap as established prior publication. This claim shaped the "Poor" novelty assessment and the overall score of 1, and it turned a question about the relationship between two versions into a de facto originality concern. We are concerned that this level of verification falls short of what the ICLR Reviewer Guidelines expect, particularly for a claim that directly determines the score.
We also note that the review rates Soundness as "2: minor or fixable concerns" while concluding that the theorems do not hold. We respectfully ask the AC to consider this internal inconsistency when weighing the review.
Based on these, we are unsure whether the overlap assessment relied on automated AI tools. We can accept if you make AI-assisted review, but should have verified the output by yourself before you submitted your review.
3. Why we are not rebutting due to KWW4‘s concern.
We cannot answer this point publicly without revealing our identity, which would violate the double-blind policy. Combined with the substantive revisions the other comments require, we have concluded that the most responsible course is to revise the work thoroughly and resubmit it, rather than give a partial response now.
We remain grateful to all reviewers for their constructive technical feedback, which will substantially improve the paper.
Sincerely,
The Authors of Submission 25031