- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper showcases an analysis of different errors across VLM models concerning the nature of answer changes given controlled edits. Authors report different results for various well-known models and train a LoRA module in order to mitigate these errors.
Critical strengths
None
Critical weaknesses
The paper is close to, if not, unreadable.
- The introduction section includes prior work explanation alluding to what would be included in a "Related Work" section. "Repair" and "rollback" (line 35-36) are introduced but not defined at all until the first part of section 2. The authors also mentioned that they study these errors "jointly" without explaining why the joint study is useful, or how they study these errors. They mention that "changing this pairing" can reverse the verification value (which is also not defined on how it is calculated or what it alludes to) without mentioned how this pairing is changed. From the introduction, I do not understand what the paper is about, what the authors have done or what I will see below.
- There are no examples of questions/answers here, or how the queried scene is defined or what T_u actually is. On Line 91, it is never explained what an "ordinary" request actually is. On Line 95, it is not explained HOW the source is labeled as "pre-edit" and if "pre-edit" is simply something that is added to the prompt or some other way of conditioning the editing generation. Uncertainty matching is never defined or cited or explained. Same as for "non-state-contradiction" and "identity-two-pass" effects, or "state-applicability" or "generic rechecking. The Table Correct/Poor distinction is never explained, nor how the applicability is calculated in any way in the main text. The second part goes into "Related Work" again for some reason without any explanation.
- Its never defined on Line 128 how these states are defined. Are they graphs, are they text, are they something else, I do not understand. Equation 1 is defined as an Optimal Transport trajectory between X and O but its never explained why we are assuming a linear interpolation. On Line 140, "opportunity" is introduced without any explanation of "opportunity" is, with first being introduced as "repair" instead of "correction". Still no explanation of how the scene states are defined in the end of part 1. Proposition 2 introduces yet another term with "dependence" never explained again, despite probably being statistical dependence, as well as "neural moments" and "neural intervention" in Lines 198-200. Section 3.3 talks about improvements as well as introducing ey another term on "raw-source movement" which is again never defined.
- Section 4 introduces "displacement" without explaining what "displacement" is. Proposition 3 does not explain why the end result has anything to do with the obsolete displacement. Section 4.2 is the first time anything related to artificial intelligence is introduced, but includes phrases such as "Terminal reading clears caches and preserves public options." which is not explained at all. Section 4.3 introduces even more terms for no apparent reason. By the end of Section 4, it still has not been explained what the authors have introduced or proposed or trained in any shape or form.
- Section 5.1 still contains jargon and no explanation of what is used from each VLM or what the setup actually includes. All the rest of the sections include more, unintroduced terms, zero specifications of how the authors have actually conducted this experiment. By the end of Section 5, I still cannot tell what the authors have done. It is only when someone goes into the Appendix it is mentioned that the "reader" is the proposition and is a trained LoRA module. Across all 5 sections, intro and abstract included, the authors never mention what they propose, only showcase random math with zero explanation and new terminologies that have no basis in literature or in their own analysis.
Questions for authors
My concerns are about presentation rather than specific missing details. I do not believe they can be resolved through clarifications in a rebuttal, they would require a significant rewrite and recontextualization of the entire manuscript.