- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines text-for-vision substitution in diagram reasoning through a fixed-solver input ladder, validity recovery, question-relevant fidelity, and graph interventions. On FlowGen, the learned–gold accuracy gap survives validity recovery and widens with structural difficulty.
Critical strengths
-
Reserved charts, prespecified tests, and chart-level uncertainty estimates support the reported descriptive trends.
-
Validity recovery and recorded examples clearly distinguish syntactic validity from preservation of answer-critical structure.
Critical weaknesses
-
The main comparison includes source-specific targets that are not printed in the images. On 124 connectedTo questions, gold answers 123 correctly while every image-based condition answers none. Although disclosed, this mismatch leaves the share of the headline gap attributable to loss of visually available evidence unresolved.
-
The fixed-solver input ladder does not isolate acquisition failure from difficulty using the acquired representation. Learned inputs use entity IDs and relation objects, whereas gold uses labeled source triplets. Holding the solver fixed does not control this encoding difference, so high gold accuracy does not establish that the solver can use equally correct structure in the learned encoding.
-
Relevant edits are chosen to change the answer, irrelevant edits to preserve it, and both are scored against the original answer. A solver that correctly follows each edited graph would therefore produce the same qualitative contrast, so the result mainly confirms sensitivity to deliberately answer-changing inputs.
-
The exact-match comparison leaves the explanation of its predictive gain unresolved. Question-relevant support changes both which elements are evaluated and how many must match, so the observed improvement does not distinguish the value of answer relevance from the effect of reducing the scope of exact matching.
Questions for authors
See Weakness