acceptodds
Under review as a conference paper at ICLR 2027

CVIF: A Criticality-Driven Visual Intervention Framework for Geometric Diagram Understanding in MLLMs

Abstract

Despite significant progress in visual tasks by Multimodal Large Language Models (MLLMs), geometric diagram understanding remains challenging due to the presence of sparse visual cues and ambiguous symbol-primitive associations. MLLMs may therefore rely on textual priors, producing interpretations that conflict with visual evidence. We introduce the training-free Criticality-Driven Visual Intervention Framework (CVIF), an inference-time method that localizes critical layers and executes visual interventions during the transition from evidence aggregation to semantic decoding. At these layers, a Geometry-Constrained Local Relation Reconstruction (GCLR) module selects and weights vertex-centered visual evidence, while an Adaptive Visual Steering Operator (AVSO) redistributes attention mass toward the selected tokens. Experiments on PGPS9K and PGDP5K show that CVIF raises Overall F1 from 77.85 to 85.58 and from 75.23 to 82.84, respectively, establishing a novel inference-time visual intervention paradigm.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Geometric Diagram Understanding, Geometric Diagram Parsing, Criticality-Driven Visual Intervention

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_piapOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CVIF, a training-free inference-time intervention framework for geometric diagram parsing. It selects critical intervention layers using layer-wise predictive entropy, employs Geometry-Constrained Local Relation Reconstruction (GCLR) to select visual evidence based on vertex locations and historical attention, and uses an Adaptive Visual Steering Operator (AVSO) to redistribute attention. The authors report improved parsing performance on PGPS9K and PGDP5K and evaluate the utility of generated diagram descriptions for downstream geometry reasoning on MathVerse.

Critical strengths

S1. The paper addresses a well-defined and practically relevant problem. Associating symbols and annotations with the correct geometric entities is an important source of diagram interpretation errors. Separating diagram parsing from downstream reasoning makes it easier to evaluate perceptual performance directly.

S2. The reported improvements in parsing performance are substantial. CVIF increases Overall F1 from 77.85 to 85.58 on PGPS9K and from 75.23 to 82.84 on PGDP5K. Improvements in Perfect Parsing Rate further suggest that the gains extend beyond individual predicates to the completeness and correctness of entire diagram parses.

Critical weaknesses

W1. The connection between predictive entropy and visual criticality could be clarified. Table 4 supports the usefulness of the selected layers, although vocabulary-level entropy alone does not distinguish visual grounding from textual priors or establish sensitivity to visual correction. It also remains unclear how the question-end diagnostic applies to later predictions. Specifying the fitting and decoding-time update rules, together with comparisons under matched intervention budgets, would help establish whether the diagnostic captures the proposed criticality beyond providing an effective layer-selection heuristic.

W2. GCLR's topology-specific contribution would benefit from a clearer explanation. The vertex-centered design is well motivated, and the Top-K ablation supports its effectiveness. However, radial proximity alone cannot distinguish equally distant tokens associated with different edges; this distinction still relies on historical attention and contextualized features. It would be helpful to explain how GCLR corrects an already biased attention preference rather than reinforcing it, and what it contributes beyond spatial reweighting. Defining the reference location in Equation (5) would also make this mechanism easier to assess and reproduce.

W3. The scope of the reasoning-preservation claim could be more carefully qualified. AVSO's head-level gating provides selective intervention, but Equation (13) still scales textual context uniformly within an affected head, including potentially useful constraints and intermediate conclusions. Normalization alone does not establish that these functions are preserved. Table 5 offers encouraging downstream results, although its decoupled setup neither directly tests AVSO during reasoning nor isolates CVIF's benefit over base-parser descriptions. Given the single intervention backbone, the evidence currently supports parsing improvements more directly than general reasoning preservation.

Questions for authors

Q1. Could authors elaborate on how predictive entropy identifies layers receptive to visual correction?

Clarifying the fitting procedure and whether layer choices are reused or updated during decoding would be helpful. Evidence linking the diagnostic to intervention effectiveness across generation positions, with intervention budgets matched, would strengthen the proposed interpretation of criticality.

Q2. Could authors clarify the modules' geometric contribution and how their parameters transfer across settings?

In particular, how is the reference location in Equation (5) computed, and how can GCLR recover the correct association when historical attention is already biased? Moreover, sensitivity results or transfer to another backbone without retuning would help assess how broadly the design applies.

Q3. Could authors clarify the scope of the reasoning-preservation claim and how useful textual context is retained?

It would be helpful to distinguish evidence from applying AVSO during reasoning from evidence obtained through the decoupled pipeline. For the latter, comparing base-parser and CVIF descriptions under the same reasoning and conversion settings would make the downstream contribution clearer. If direct preservation has not been evaluated, a more qualified claim would better reflect the current evidence.

If the authors can adequately address these concerns, I would consider raising my score.

9 comments

We thank the reviewer for the detailed feedback and constructive suggestions.

Q1. Could authors elaborate on how predictive entropy identifies layers receptive to visual correction?

Clarifying the fitting procedure and whether layer choices are reused or updated during decoding would be helpful. Evidence linking the diagnostic to intervention effectiveness across generation positions, with intervention budgets matched, would strengthen the proposed interpretation of criticality.

One point worth clarifying is that our work is not aimed at identifying layers that are sensitive to visual intervention or at using predictive entropy to measure that sensitivity. "Sensitivity to visual perturbations" alone provides limited guidance on where to intervene, because high sensitivity can also mean that the model’s current prediction is easily disrupted. When layers are selected solely for their sensitivity, even correct geometric cues can disrupt an otherwise reasonable prediction, or cause the model to generate erratic token predictions along its existing (and potentially inherently biased) semantic trajectory. CVIF seeks to identify the stage at which relevant visual evidence can still redirect the model’s interpretation. We call this state representational criticality. The model has already integrated multimodal cues but has yet to form a stable preference for a particular interpretation, so introducing relevant geometric cues can still change the direction of subsequent predictions.

To locate this transition, we trace the normalized predictive entropy of the question-end token across layers. The entropy trajectory reflects three successive stages: multimodal evidence aggregation, semantic decoding and prediction formation, and consolidation and refinement of candidate predictions. The trajectory changes nonlinearly in Stage 2. The trends on either side are clearer: entropy decreases approximately linearly in Stage 1 and approaches a low plateau in Stage 3. We use these two trends as references to identify the start and end of Stage 2. Specifically, we use nivariate least squares regression, with the layer index as the predictor and normalized predictive entropy as the response. We fit a declining linear trend for Stage 1 and approximate the low-entropy plateau in Stage 3 with a zero-slope line. The point at which the entropy curve persistently departs from the fitted Stage 1 trend marks the start of Stage 2. The point at which it enters the Stage 3 plateau marks the end. These two boundaries guide the selection of the intervention layer set L\mathcal{L}.

For each input, we compute L\mathcal{L} once from the question-end token’s entropy trajectory before decoding and reuse the layer indices at subsequent generation steps. At each step, we recompute the current attention Acurr(l)\mathbf{A}_{\mathrm{curr}}^{(l)}, the smoothed historical attention Aˉ\bar{\mathbf{A}}, and the target set and topology-aware weights produced by GCLR using the current query. Thus, each step uses the same intervention layers while selecting geometric cues relevant to its own query.

We tested intervention placement on 100 representative PGPS9K images. For each image, we shifted intervention layers by the same offset while keeping the intervention budget and all other settings fixed. Negative offsets move the intervention to earlier layers. An offset of zero corresponds to the entropy-selected layers, which serve as the reference for the F1 changes.

Layer offsetPrecision (%)Recall (%)Overall F1 (%)F1 change
−482.6978.4180.50−6.37
−282.5981.6482.11−4.76
−185.1479.6282.29−4.58
086.7487.0086.870.00
+182.7478.2880.45−6.42
+277.0378.6977.85−9.02
+479.4778.0278.74−8.13

Experiments show that the F1 score exhibits a downward trend as the intervention shifts. An early intervention can lose influence as the visual features propagate through subsequent layers. A late intervention supplies visual cues after the model has settled on an interpretation, making those cues harder to incorporate and potentially disrupting the existing prediction. With the intervention budget matched, the entropy-selected layers outperform all shifted configurations, supporting the choice to intervene at the representational transitions identified by the entropy trajectory.

We will clarify the entropy fitting and layer reuse in the revised paper and include the matched-budget layer-offset results in the appendix.

Q2. Could authors clarify the modules' geometric contribution and how their parameters transfer across settings?

In particular, how is the reference location in Equation (5) computed, and how can GCLR recover the correct association when historical attention is already biased? Moreover, sensitivity results or transfer to another backbone without retuning would help assess how broadly the design applies.

GCLR uses vertices as spatial anchors to recover local geometric cues that the model attended to in earlier layers but may overlook at a critical layer. This helps the model associate each vertex with nearby edges, intersections, and annotations relevant to the current prediction.

In Eq. (5), the prediction-relevant reference location is the position of the detected vertex with the highest base relevance score. Let pk\mathbf{p}_k denote the position of vertex kk, and let pt\mathbf{p}_t denote the center of the image region corresponding to visual token tt. The distance between them is

dk(t)=∥pt−pk∥2.d_k(t) = \lVert \mathbf{p}_t - \mathbf{p}_k \rVert_2.

GCLR computes the base relevance of vertex kk by weighting the smoothed historical attention Aˉ(t)\bar{\mathbf{A}}(t) with the vertex-centered topological prior mtopo(k)(t)m_{\mathrm{topo}}^{(k)}(t), which depends on dk(t)d_k(t).

=\sum_{t\in\Omega_{\mathrm{vis}}} \bar{\mathbf{A}}(t)\\,m_{\mathrm{topo}}^{(k)}(t).$$ The highest-scoring vertex defines the reference location. Each vertex’s reference distance is its Euclidean distance to that location. $$k^{\star} = \arg\max_{u=1,\ldots,N_v} s_{\mathrm{base}}^{(u)}, \qquad \mathbf{p}_{\mathrm{ref}} = \mathbf{p}_{k^{\star}}, \qquad d_k^{\mathrm{ref}} = \lVert \mathbf{p}_k - \mathbf{p}_{\mathrm{ref}} \rVert_2.$$ Eq. (5) adds a distance-decaying term to each vertex’s base score: $$s_v^{(k)} =s_{\mathrm{base}}^{(k)} +\eta\\,s_{\mathrm{base}}^{(k^\star)} \exp\\!\left(-\frac{d_k^{\mathrm{ref}}}{\tau}\right).$$ Nearby vertices with weaker base scores can receive a boost that decreases with distance. This helps retain related vertices whose visual neighborhoods jointly provide evidence for angles, edge connectivity, and intersections. GCLR selects target tokens only around vertices whose scores meet the adaptive threshold $s_v^{(k)}\geq\mu_v+\alpha_v\sigma_v$. Here, historical attention is the smoothed attention over preceding network layers in the same forward pass. During evidence aggregation, earlier layers attend to multiple vertices, annotations, and associated line segments. As the model forms an interpretation, attention can narrow to a few regions, reducing the weight of cues that were previously attended to and are still relevant. Even when the historical attention favors an incorrect region, useful cues can retain some weight across those earlier layers. GCLR pools this evidence across layers and uses the vertex-centered spatial prior and distance-based adjustment in Eq. (5) to bring relevant nearby cues into the candidate set. Recovery depends on the earlier layers retaining some usable signal for the correct local structure. We evaluated hyperparameter sensitivity on a fixed subset of 100 representative PGPS9K images. For each sweep, we varied one hyperparameter and held the others, the prompt, the layer-selection procedure, and the evaluation settings fixed. Each entry shows the parameter value → Overall F1 (%), ordered by increasing parameter value. Bold entries indicate the parameter values used in CVIF. | Hyperparameter | Setting 1 | Setting 2 | Setting 3 | Setting 4 | | :------------------------ | -----------: | ---------------: | --------------: | -----------: | | $\rho$ | 0.1 → 82.28 | **0.2 → 85.90** | 0.3 → 81.19 | 0.4 → 79.26 | | $\sigma_g$ | 0.1 → 80.18 | **0.2 → 85.90** | 0.3 → 81.37 | 0.4 → 80.36 | | $\sigma_l$ | 0.1 → 80.31 | 0.3 → 77.56 | **0.5 → 85.90** | 0.7 → 80.53 | | $\eta$ | 0.2 → 81.67 | 0.4 → 72.60 | **0.6 → 85.90** | 0.8 → 83.79 | | $\tau_{\mathrm{ratio}}$ | 0.1 → 80.94 | **0.2 → 85.90** | 0.3 → 82.17 | 0.4 → 81.51 | | $\alpha_v$ | 0.0 → 84.18 | 0.2 → 81.74 | **0.5 → 85.90** | 1.0 → 83.90 | | $\alpha_{\mathrm{tok}}$ | 0.5 → 71.34 | 1.0 → 79.69 | **1.5 → 85.90** | 2.0 → 79.76 | | $\gamma$ | 0.2 → 82.33 | **0.4 → 85.90** | 0.6 → 82.21 | 0.8 → 81.53 | | $\alpha$ | 0.1 → 83.67 | 0.3 → 81.25 | **0.5 → 85.90** | 0.7 → 80.56 | | $\tau_{\mathrm{down}}$ | 0.01 → 80.25 | **0.02 → 85.90** | 0.03 → 82.20 | 0.04 → 81.52 | | $\tau_{\mathrm{up}}$ | 0.05 → 81.66 | **0.10 → 85.90** | 0.15 → 77.86 | 0.20 → 83.04 | | $\lambda$ | 1.0 → 81.57 | **2.0 → 85.90** | 3.0 → 81.94 | 4.0 → 84.02 |

The parameters used in CVIF achieved the highest Overall F1 among the tested settings. For example, lowering the token-selection threshold coefficient αtok\alpha_{\mathrm{tok}} from 1.5 to 0.5 makes token selection less restrictive and reduces Overall F1 from 85.90 to 71.34. We use the same Qwen3-VL-32B hyperparameter configuration on PGPS9K and PGDP5K without dataset-specific tuning.

We further transferred CVIF to other backbone models (Qwen3-VL-4B and Qwen3-VL-8B) without retuning, and evaluated them on 1000 representative PGPS9K images. We report precision (P), recall (R), and F1 (%) for structural, semantic, and overall predicates.

ModelStructural PStructural RStructural F1Semantic PSemantic RSemantic F1Overall POverall ROverall F1
Qwen3-VL-4B72.2444.7655.2763.8663.7763.8263.6353.7458.27
Qwen3-VL-4B + CVIF71.9456.9263.5573.3677.5575.4067.3166.6666.99
Qwen3-VL-8B72.9646.4956.7977.9367.9172.5773.4256.6063.92
Qwen3-VL-8B + CVIF75.0263.5368.8077.7878.3178.0473.1370.5171.79

Experiments show that CVIF improves Overall F1 by 8.72 points on Qwen3-VL-4B and 7.87 points on Qwen3-VL-8B. Both models improve in structural and semantic F1, with structural recall rising by 12.16 and 17.04 points, respectively. These results demonstrate the strong generalization capability of CVIF across various backbone models.

In the revised paper, we will specify how the reference vertex in Eq. (5) is selected and report the sensitivity analysis and the 4B/8B results in the appendix.

Reviewer_5vqgOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose a CVIF, is a training-free framework for geometric diagram parsing that selects intervention layers using predictive entropy, selects vertex-centered visual evidence through GCLR, and redistributes attention through AVSO. Experiments on two geometric diagram datasets show substantial parsing improvements over the Qwen3-VL-32B backbone.

Critical strengths
  • Improvements extend from predicate F1 to whole-diagram Perfect Parsing Rate on both datasets, providing consistent evidence of better parsing.

  • The two-stage control and GCLR/AVSO replacement ablations support the complete design beyond a change in output format.

Critical weaknesses
  • Vocabulary uncertainty at the question-end token differs from the sensitivity that defines representational criticality, and superiority to one fixed layer schedule does not distinguish the proposed mechanism from a favorable choice of intervention depth.

  • GCLR's gains do not isolate its claimed reconstruction of local topology from improved spatial focus. Although contextualized vertex tokens can encode geometric relations, GCLR selects and weights them using distance priors and historical attention, while the Top-K replacement removes the spatial guidance as well.

  • CVIF's advantage over established training-free interventions remains unclear. PAI already strengthens image-token attention and penalizes text-only priors, whereas the reported comparisons use unmodified MLLMs and simplified replacements. These comparisons do not establish the additional benefit of CVIF's proposed mechanisms over an existing intervention method.

  • The downstream experiment establishes the utility of added diagram descriptions, but it does not isolate reasoning gains attributable to CVIF's improved parsing. Direct answering is compared with description-augmented inputs, so added textual evidence and parser quality remain entangled. These results also do not establish AVSO's claimed preservation of native logical reasoning, since the effect of attention intervention on reasoning is not isolated.

Questions for authors

See Weakness

2 comments

We thank the reviewer for the careful analysis and the thoughtful suggestions.

Response to W1: Predictive Entropy and Critical Layer Selection

CVIF focuses on identifying the stage at which relevant visual evidence can still redirect the model’s interpretation. Sensitivity to visual perturbations alone provides limited guidance on where to intervene, because high sensitivity can also mean that the current prediction is easy to disrupt. When layers are selected solely based on sensitivity, even correct geometric cues can disrupt an otherwise reasonable prediction or lead to less stable token predictions along the model’s existing, potentially biased interpretation.

We use representational criticality to describe a state in which the model has integrated multimodal cues but has yet to form a stable preference for a particular interpretation. At this point, introducing relevant geometric cues can still influence the interpretation the model forms next. Layerwise predictive entropy provides a reference for how the model organizes its representations for the task across network depth. Representational criticality is a high-level property governed primarily by the model’s architecture and parameters and is relatively stable for a given task. We therefore use the question-end token, which has access to the full image and question context, to observe this process before generation begins. Its layerwise predictive entropy helps distinguish evidence aggregation, prediction formation, and prediction consolidation, allowing us to locate the representationally critical state.

To evaluate CVIF’s layer-selection mechanism, we used 100 representative PGPS9K images. For each sample, we shifted all entropy-selected layers by 1, 2, or 4 layers toward either shallower or deeper positions and compared performance against the unshifted setting. All comparisons used the same intervention budget and varied only the intervention layer positions.

Layer offsetPrecision (%)Recall (%)Overall F1 (%)F1 change
−482.6978.4180.50−6.37
−282.5981.6482.11−4.76
−185.1479.6282.29−4.58
086.7487.0086.870.00
+182.7478.2880.45−6.42
+277.0378.6977.85−9.02
+479.4778.0278.74−8.13

The entropy-selected layers achieved higher Overall F1 than all six shifted configurations under the same intervention budget. This result provides empirical support for the effectiveness of the entropy diagnostic. We will clarify the diagnostic role of predictive entropy in the revised paper and include the layer-offset experiments in the appendix.

Response to W2: Spatial Focusing and Local Topology Reconstruction

We thank the reviewer for the careful analysis. In this paper, local topology reconstruction refers to organizing cues about line segments, intersections, and annotations that are distributed across visual tokens around vertices relevant to the current prediction, and assigning weights to these cues for subsequent attention intervention. Spatial focusing is one step in this reconstruction process.

GCLR implements local topology reconstruction by assigning weights to a set of target tokens selected jointly by geometric constraints and historical attention. At the vertex level, GCLR combines historical attention with distance-based priors to identify the vertices and neighborhoods relevant to the current prediction. At the visual-token level, it uses spatial location and historical attention to select and weight cues within each neighborhood. The ablation that replaces GCLR with Top-K attention selection shows that the tokens and weights produced by GCLR provide more effective guidance about local structures than Top-K selection based on attention alone. We will clarify the meaning of “local topology reconstruction” in the revised paper.

Response to W3: Comparison with PAI

We compared the original model, PAI, and CVIF on the same set of 200 representative PGPS9K images, using Qwen3-VL-32B as the backbone.

ModelStructural PStructural RStructural F1Semantic PSemantic RSemantic F1Overall POverall ROverall F1
Qwen3-VL-32B80.3873.7476.9178.9674.8676.8578.7074.2676.42
Qwen3-VL-32B + PAI81.7071.0275.9982.5578.9580.7178.9074.7276.75
Qwen3-VL-32B + CVIF90.4988.0489.2584.6884.7584.7185.5886.5086.04

PAI primarily addresses object-level hallucinations in natural-image descriptions, such as when language priors lead a model to describe an object that is absent from the image. Strengthening the model’s existing attention to the image and suppressing text-only priors is a reasonable approach to this problem. Geometric diagram understanding requires the model to infer abstract structures from sparse points, lines, and annotations, then organize the local relations into a self-consistent description that adheres to the geometric logic of the diagram.

Simply increasing image attention or weakening textual priors can still leave the model focused on salient letters and numbers while overlooking the thin lines and intersections that determine geometric relations. CVIF uses historical attention and vertex-based geometric constraints to recover these structural cues and directs the model to use them while its interpretation is still forming, so that they can guide the resulting prediction. The two methods were designed for tasks with different challenges. This comparison under the same conditions tests which intervention mechanism is better suited for understanding abstract structures within geometric diagrams.

Response to W4: Downstream Reasoning Gains and the Scope of Reasoning Preservation

To separate the benefit of adding diagram descriptions from the gains attributable to CVIF’s improved parsing, we additionally report downstream results using the base Qwen3-VL-32B parser. For each downstream model and representation format, we keep the reasoning settings fixed and vary only the source of the diagram description. In the natural-language setting, we use the descriptions directly. In the formal-language setting, we use the same Qwen3-VL-32B converter for descriptions from both parsers. The table reports answer accuracy (%).

Input settingQwen3-VL-32BQwen3-VL-8BInternVL3.5-8BQwen2.5-VL-7BQwen3-VL-4B
Direct answer53.7345.1041.1842.1641.18
+ Formal language (base parser)53.1444.9045.6946.0839.41
+ Formal language (CVIF)53.7349.6145.2946.0841.37
+ Natural-language description (base parser)54.3147.6546.0844.7142.35
+ Natural-language description (CVIF)58.6348.2447.0646.4742.94

Replacing the base parser’s natural-language descriptions with CVIF’s descriptions improves accuracy across all five downstream models, with gains ranging from 0.59 to 4.32 percentage points. This comparison isolates the effect of the description source and shows that CVIF’s improved diagram descriptions provide more useful geometric information for downstream problem solving.

The phrase “without disrupting native logical reasoning” in the introduction refers to attention-head gating during diagram parsing. As specified in Eq. (11) in Section 4.3, AVSO intervenes only in heads that already assign sufficient attention to visual tokens but assign insufficient attention to the target geometric tokens. The downstream experiments evaluate how the parsing outputs help solve geometry problems. We will revise the introduction to tie the statement explicitly to this gating mechanism and include the downstream comparison with the base parser in the revised paper.

Reviewer_x7kVOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, the authors propose CVIF, a training-free inference-time intervention framework for geometric diagram parsing in MLLMs. Layer-wise normalized predictive entropy is used to locate "critical layers" at which the model transitions from evidence aggregation to semantic decoding; at those layers, a GCLR module selects vertex-centered visual tokens with topology-aware weights, and an AVSO operator reshapes multi-head attention toward them while compensating the attention mass taken from text tokens. On PGPS9K and PGDP5K, CVIF raises Overall F1 of Qwen3-VL-32B-Instruct from 77.85 to 85.58 and from 75.23 to 82.84.

Critical strengths
  1. The diagnosis in Sec. 1 is concrete and well illustrated (Figure 1): visual evidence attended in shallow layers loses influence in deeper layers, and attending to a vertex label does not by itself recover the incident-edge topology. This is a more specific failure account than the usual "models ignore the image" framing.

  2. Using geometric vertices as the organizing anchor for token selection is well matched to the task, since vertices are simultaneously the visual junctions and the referents of the formal language (BD, ∠ABC). The diagram-level metrics in Table 3 (Sample Accuracy, Perfect Parsing Rate) are a good addition, as category-level F1 can hide incoherent whole-diagram parses.

Critical weaknesses
  1. The method is validated on exactly one backbone (Qwen3-VL-32B-Instruct). For a training-free intervention whose main selling point is general applicability, this is the central missing experiment. Table 1 already contains Qwen3-VL-8B-Instruct, Qwen2.5-VL-32B-Instruct, and InternVL3.5 variants; applying CVIF to at least two of them would show whether the mechanism generalizes or whether it is tuned to one model's layer dynamics.

  2. "Training-free" overstates the setting in two ways. First, GCLR requires geometric vertex coordinates from an external detector (Sec. 4.1, citing Jocher et al., 2023), i.e. a model trained on geometric diagrams; the framework is parameter-free with respect to the MLLM but not annotation-free. Second, the method introduces at least twelve hyperparameters (ρ, σg, σl, η, τ, αv, αtok, γ, α, τdown, τup, λ). How and on which split were these selected, and how sensitive is Overall F1 to them? Without this, the comparison against zero-configuration baselines is not like-for-like.

  3. The dependence on vertex detection is never evaluated. What is the detector's precision/recall on PGPS9K and PGDP5K, and how does CVIF degrade when vertices are missed or spurious? Since every downstream quantity (Eqs. 4–8) is anchored on Pvertices, a failure analysis conditioned on detection quality is necessary.

  4. Generated descriptions are translated into formal language by Qwen3-VL-32B (end of Sec. 4.1) — the same model that serves as the intervention backbone. The accuracy of this translation step is not reported, yet it sits between every system and its score, including the closed-source baselines. Please report translation fidelity on a human-verified subset, and ideally use a translator from a different family.

Questions for authors

No

3 comments

We thank the reviewer for the detailed feedback and constructive suggestions for strengthening the experimental evaluation.

Response to W1: Generalization Across Backbones

To test whether our method generalizes to other backbones, we applied CVIF to Qwen3-VL-4B and Qwen3-VL-8B on the same set of 1,000 representative PGPS9K samples. The table below compares the diagram-parsing performance of each model with and without CVIF. All metrics are reported as percentages.

ModelStructural PStructural RStructural F1Semantic PSemantic RSemantic F1Overall POverall ROverall F1
Qwen3-VL-4B72.2444.7655.2763.8663.7763.8263.6353.7458.27
Qwen3-VL-4B + CVIF71.9456.9263.5573.3677.5575.4067.3166.6666.99
Qwen3-VL-8B72.9646.4956.7977.9367.9172.5773.4256.6063.92
Qwen3-VL-8B + CVIF75.0263.5368.8077.7878.3178.0473.1370.5171.79

CVIF improves Overall F1 by 8.72 and 7.87 percentage points, respectively. Both backbones improve in structural and semantic F1. These results demonstrate the generalization capability of CVIF across various backbone models. We will include both sets of results in the revised paper.

Response to W2: Hyperparameter Selection and Sensitivity

We selected the hyperparameters for the entire task, refining them using representative samples. The tuning samples came from PGDP5K, and we used the same configuration to evaluate PGDP5K and PGPS9K without dataset-specific tuning.

We further evaluated hyperparameter sensitivity on a fixed set of 100 PGPS9K images. We varied one hyperparameter at a time while keeping all other settings fixed. Each entry shows the parameter value → Overall F1 (%). Bold entries indicate the parameter values used in CVIF.

HyperparameterSetting 1Setting 2Setting 3Setting 4
ρ\rho0.1 → 82.280.2 → 85.900.3 → 81.190.4 → 79.26
σg\sigma_g0.1 → 80.180.2 → 85.900.3 → 81.370.4 → 80.36
σl\sigma_l0.1 → 80.310.3 → 77.560.5 → 85.900.7 → 80.53
η\eta0.2 → 81.670.4 → 72.600.6 → 85.900.8 → 83.79
τratio\tau_{\mathrm{ratio}}0.1 → 80.940.2 → 85.900.3 → 82.170.4 → 81.51
αv\alpha_v0.0 → 84.180.2 → 81.740.5 → 85.901.0 → 83.90
αtok\alpha_{\mathrm{tok}}0.5 → 71.341.0 → 79.691.5 → 85.902.0 → 79.76
γ\gamma0.2 → 82.330.4 → 85.900.6 → 82.210.8 → 81.53
α\alpha0.1 → 83.670.3 → 81.250.5 → 85.900.7 → 80.56
τdown\tau_{\mathrm{down}}0.01 → 80.250.02 → 85.900.03 → 82.200.04 → 81.52
τup\tau_{\mathrm{up}}0.05 → 81.660.10 → 85.900.15 → 77.860.20 → 83.04
λ\lambda1.0 → 81.572.0 → 85.903.0 → 81.944.0 → 84.02

The configuration used in CVIF achieved the highest Overall F1 (85.90) among the tested settings. For example, lowering αtok\alpha_{\mathrm{tok}} from 1.5 to 0.5 makes token selection less selective and reduces Overall F1 to 71.34. We will include the hyperparameter sensitivity results in the revised paper.

Respose to W3: Vertex Detection Accuracy and Robustness to Detection Errors

We added a vertex detection evaluation and a diagram-parsing analysis grouped by detection outcome. We evaluated the vertex detector on PGDP5K and PGPS9K. The detector performs well on both datasets, indicating that it provides sufficient vertex coordinate information for CVIF. Precision exceeds 98% on both datasets. Recall is lower on PGPS9K, where missed vertices are the main source of detection errors.

DatasetVertex Precision (%)Vertex Recall (%)Vertex F1 (%)
PGPS9K98.3490.3394.16
PGDP5K99.7797.7898.76

Among samples with detection errors, cases with one missed vertex, two missed vertices, and one extra vertex account for 61.01%, 21.82%, and 11.37%, respectively. We selected 50 images from each of these three error categories, along with an additional 50 images where detections were correct, then compared CVIF’s parsing performance across the four groups. All metrics are reported as percentages.

Vertex detection outcomeStructural PStructural RStructural F1Semantic PSemantic RSemantic F1Overall POverall ROverall F1
Correct90.5089.1689.8389.3891.3390.3489.2690.0889.67
One missed vertex91.2290.0390.6284.7782.4883.6185.5786.3085.94
Two missed vertices80.4775.5677.9475.0065.4369.8976.8170.3873.46
one extra vertex85.7886.6086.1983.2576.5479.7583.3381.9682.64

The results show that vertex detection errors reduce CVIF’s parsing performance. CVIF remains relatively robust to the most common error, a single missed vertex. Performance can drop more substantially in the less common cases where multiple vertices are missed. We will include the vertex detection metrics and this group-wise analysis in the revised paper.

Response to W4: Formal-Language Translation Fidelity

We manually verified the translation fidelity of Qwen3-VL-32B on 50 representative images from the PGPS9K dataset, and obtained a human reference translations. Furthermore, we utilized DeepSeek V4.1 Flash to perform translations and compared the translation fidelity between the two models.

TranslatorStructural FidelitySemantic FidelityOverall Fidelity
Qwen3-VL-32B96.26%91.18%93.78%
DeepSeek V4.1 Flash97.20%91.18%94.26%

The results indicate that Qwen3-VL-32B preserves most of the geometric information in the natural-language descriptions, and the translator from a different model family achieves comparable fidelity.

Tthe large F1 drops under some parameter changes leave me concerned about how much tuning CVIF needs, so I keep my score of 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.