- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CVIF, a training-free inference-time intervention framework for geometric diagram parsing. It selects critical intervention layers using layer-wise predictive entropy, employs Geometry-Constrained Local Relation Reconstruction (GCLR) to select visual evidence based on vertex locations and historical attention, and uses an Adaptive Visual Steering Operator (AVSO) to redistribute attention. The authors report improved parsing performance on PGPS9K and PGDP5K and evaluate the utility of generated diagram descriptions for downstream geometry reasoning on MathVerse.
Critical strengths
S1. The paper addresses a well-defined and practically relevant problem. Associating symbols and annotations with the correct geometric entities is an important source of diagram interpretation errors. Separating diagram parsing from downstream reasoning makes it easier to evaluate perceptual performance directly.
S2. The reported improvements in parsing performance are substantial. CVIF increases Overall F1 from 77.85 to 85.58 on PGPS9K and from 75.23 to 82.84 on PGDP5K. Improvements in Perfect Parsing Rate further suggest that the gains extend beyond individual predicates to the completeness and correctness of entire diagram parses.
Critical weaknesses
W1. The connection between predictive entropy and visual criticality could be clarified. Table 4 supports the usefulness of the selected layers, although vocabulary-level entropy alone does not distinguish visual grounding from textual priors or establish sensitivity to visual correction. It also remains unclear how the question-end diagnostic applies to later predictions. Specifying the fitting and decoding-time update rules, together with comparisons under matched intervention budgets, would help establish whether the diagnostic captures the proposed criticality beyond providing an effective layer-selection heuristic.
W2. GCLR's topology-specific contribution would benefit from a clearer explanation. The vertex-centered design is well motivated, and the Top-K ablation supports its effectiveness. However, radial proximity alone cannot distinguish equally distant tokens associated with different edges; this distinction still relies on historical attention and contextualized features. It would be helpful to explain how GCLR corrects an already biased attention preference rather than reinforcing it, and what it contributes beyond spatial reweighting. Defining the reference location in Equation (5) would also make this mechanism easier to assess and reproduce.
W3. The scope of the reasoning-preservation claim could be more carefully qualified. AVSO's head-level gating provides selective intervention, but Equation (13) still scales textual context uniformly within an affected head, including potentially useful constraints and intermediate conclusions. Normalization alone does not establish that these functions are preserved. Table 5 offers encouraging downstream results, although its decoupled setup neither directly tests AVSO during reasoning nor isolates CVIF's benefit over base-parser descriptions. Given the single intervention backbone, the evidence currently supports parsing improvements more directly than general reasoning preservation.
Questions for authors
Q1. Could authors elaborate on how predictive entropy identifies layers receptive to visual correction?
Clarifying the fitting procedure and whether layer choices are reused or updated during decoding would be helpful. Evidence linking the diagnostic to intervention effectiveness across generation positions, with intervention budgets matched, would strengthen the proposed interpretation of criticality.
Q2. Could authors clarify the modules' geometric contribution and how their parameters transfer across settings?
In particular, how is the reference location in Equation (5) computed, and how can GCLR recover the correct association when historical attention is already biased? Moreover, sensitivity results or transfer to another backbone without retuning would help assess how broadly the design applies.
Q3. Could authors clarify the scope of the reasoning-preservation claim and how useful textual context is retained?
It would be helpful to distinguish evidence from applying AVSO during reasoning from evidence obtained through the decoupled pipeline. For the latter, comparing base-parser and CVIF descriptions under the same reasoning and conversion settings would make the downstream contribution clearer. If direct preservation has not been evaluated, a more qualified claim would better reflect the current evidence.
If the authors can adequately address these concerns, I would consider raising my score.
9 comments
Q2-1
Q2. Could authors clarify the modules' geometric contribution and how their parameters transfer across settings?
In particular, how is the reference location in Equation (5) computed, and how can GCLR recover the correct association when historical attention is already biased? Moreover, sensitivity results or transfer to another backbone without retuning would help assess how broadly the design applies.
GCLR uses vertices as spatial anchors to recover local geometric cues that the model attended to in earlier layers but may overlook at a critical layer. This helps the model associate each vertex with nearby edges, intersections, and annotations relevant to the current prediction.
In Eq. (5), the prediction-relevant reference location is the position of the detected vertex with the highest base relevance score. Let denote the position of vertex , and let denote the center of the image region corresponding to visual token . The distance between them is
GCLR computes the base relevance of vertex by weighting the smoothed historical attention with the vertex-centered topological prior , which depends on .
=\sum_{t\in\Omega_{\mathrm{vis}}} \bar{\mathbf{A}}(t)\\,m_{\mathrm{topo}}^{(k)}(t).$$ The highest-scoring vertex defines the reference location. Each vertex’s reference distance is its Euclidean distance to that location. $$k^{\star} = \arg\max_{u=1,\ldots,N_v} s_{\mathrm{base}}^{(u)}, \qquad \mathbf{p}_{\mathrm{ref}} = \mathbf{p}_{k^{\star}}, \qquad d_k^{\mathrm{ref}} = \lVert \mathbf{p}_k - \mathbf{p}_{\mathrm{ref}} \rVert_2.$$ Eq. (5) adds a distance-decaying term to each vertex’s base score: $$s_v^{(k)} =s_{\mathrm{base}}^{(k)} +\eta\\,s_{\mathrm{base}}^{(k^\star)} \exp\\!\left(-\frac{d_k^{\mathrm{ref}}}{\tau}\right).$$ Nearby vertices with weaker base scores can receive a boost that decreases with distance. This helps retain related vertices whose visual neighborhoods jointly provide evidence for angles, edge connectivity, and intersections. GCLR selects target tokens only around vertices whose scores meet the adaptive threshold $s_v^{(k)}\geq\mu_v+\alpha_v\sigma_v$. Here, historical attention is the smoothed attention over preceding network layers in the same forward pass. During evidence aggregation, earlier layers attend to multiple vertices, annotations, and associated line segments. As the model forms an interpretation, attention can narrow to a few regions, reducing the weight of cues that were previously attended to and are still relevant. Even when the historical attention favors an incorrect region, useful cues can retain some weight across those earlier layers. GCLR pools this evidence across layers and uses the vertex-centered spatial prior and distance-based adjustment in Eq. (5) to bring relevant nearby cues into the candidate set. Recovery depends on the earlier layers retaining some usable signal for the correct local structure. We evaluated hyperparameter sensitivity on a fixed subset of 100 representative PGPS9K images. For each sweep, we varied one hyperparameter and held the others, the prompt, the layer-selection procedure, and the evaluation settings fixed. Each entry shows the parameter value → Overall F1 (%), ordered by increasing parameter value. Bold entries indicate the parameter values used in CVIF. | Hyperparameter | Setting 1 | Setting 2 | Setting 3 | Setting 4 | | :------------------------ | -----------: | ---------------: | --------------: | -----------: | | $\rho$ | 0.1 → 82.28 | **0.2 → 85.90** | 0.3 → 81.19 | 0.4 → 79.26 | | $\sigma_g$ | 0.1 → 80.18 | **0.2 → 85.90** | 0.3 → 81.37 | 0.4 → 80.36 | | $\sigma_l$ | 0.1 → 80.31 | 0.3 → 77.56 | **0.5 → 85.90** | 0.7 → 80.53 | | $\eta$ | 0.2 → 81.67 | 0.4 → 72.60 | **0.6 → 85.90** | 0.8 → 83.79 | | $\tau_{\mathrm{ratio}}$ | 0.1 → 80.94 | **0.2 → 85.90** | 0.3 → 82.17 | 0.4 → 81.51 | | $\alpha_v$ | 0.0 → 84.18 | 0.2 → 81.74 | **0.5 → 85.90** | 1.0 → 83.90 | | $\alpha_{\mathrm{tok}}$ | 0.5 → 71.34 | 1.0 → 79.69 | **1.5 → 85.90** | 2.0 → 79.76 | | $\gamma$ | 0.2 → 82.33 | **0.4 → 85.90** | 0.6 → 82.21 | 0.8 → 81.53 | | $\alpha$ | 0.1 → 83.67 | 0.3 → 81.25 | **0.5 → 85.90** | 0.7 → 80.56 | | $\tau_{\mathrm{down}}$ | 0.01 → 80.25 | **0.02 → 85.90** | 0.03 → 82.20 | 0.04 → 81.52 | | $\tau_{\mathrm{up}}$ | 0.05 → 81.66 | **0.10 → 85.90** | 0.15 → 77.86 | 0.20 → 83.04 | | $\lambda$ | 1.0 → 81.57 | **2.0 → 85.90** | 3.0 → 81.94 | 4.0 → 84.02 |Q2-2
The parameters used in CVIF achieved the highest Overall F1 among the tested settings. For example, lowering the token-selection threshold coefficient from 1.5 to 0.5 makes token selection less restrictive and reduces Overall F1 from 85.90 to 71.34. We use the same Qwen3-VL-32B hyperparameter configuration on PGPS9K and PGDP5K without dataset-specific tuning.
We further transferred CVIF to other backbone models (Qwen3-VL-4B and Qwen3-VL-8B) without retuning, and evaluated them on 1000 representative PGPS9K images. We report precision (P), recall (R), and F1 (%) for structural, semantic, and overall predicates.
| Model | Structural P | Structural R | Structural F1 | Semantic P | Semantic R | Semantic F1 | Overall P | Overall R | Overall F1 |
|---|---|---|---|---|---|---|---|---|---|
| Qwen3-VL-4B | 72.24 | 44.76 | 55.27 | 63.86 | 63.77 | 63.82 | 63.63 | 53.74 | 58.27 |
| Qwen3-VL-4B + CVIF | 71.94 | 56.92 | 63.55 | 73.36 | 77.55 | 75.40 | 67.31 | 66.66 | 66.99 |
| Qwen3-VL-8B | 72.96 | 46.49 | 56.79 | 77.93 | 67.91 | 72.57 | 73.42 | 56.60 | 63.92 |
| Qwen3-VL-8B + CVIF | 75.02 | 63.53 | 68.80 | 77.78 | 78.31 | 78.04 | 73.13 | 70.51 | 71.79 |
Experiments show that CVIF improves Overall F1 by 8.72 points on Qwen3-VL-4B and 7.87 points on Qwen3-VL-8B. Both models improve in structural and semantic F1, with structural recall rising by 12.16 and 17.04 points, respectively. These results demonstrate the strong generalization capability of CVIF across various backbone models.
In the revised paper, we will specify how the reference vertex in Eq. (5) is selected and report the sensitivity analysis and the 4B/8B results in the appendix.
Q1
We thank the reviewer for the detailed feedback and constructive suggestions.
One point worth clarifying is that our work is not aimed at identifying layers that are sensitive to visual intervention or at using predictive entropy to measure that sensitivity. "Sensitivity to visual perturbations" alone provides limited guidance on where to intervene, because high sensitivity can also mean that the model’s current prediction is easily disrupted. When layers are selected solely for their sensitivity, even correct geometric cues can disrupt an otherwise reasonable prediction, or cause the model to generate erratic token predictions along its existing (and potentially inherently biased) semantic trajectory. CVIF seeks to identify the stage at which relevant visual evidence can still redirect the model’s interpretation. We call this state representational criticality. The model has already integrated multimodal cues but has yet to form a stable preference for a particular interpretation, so introducing relevant geometric cues can still change the direction of subsequent predictions.
To locate this transition, we trace the normalized predictive entropy of the question-end token across layers. The entropy trajectory reflects three successive stages: multimodal evidence aggregation, semantic decoding and prediction formation, and consolidation and refinement of candidate predictions. The trajectory changes nonlinearly in Stage 2. The trends on either side are clearer: entropy decreases approximately linearly in Stage 1 and approaches a low plateau in Stage 3. We use these two trends as references to identify the start and end of Stage 2. Specifically, we use nivariate least squares regression, with the layer index as the predictor and normalized predictive entropy as the response. We fit a declining linear trend for Stage 1 and approximate the low-entropy plateau in Stage 3 with a zero-slope line. The point at which the entropy curve persistently departs from the fitted Stage 1 trend marks the start of Stage 2. The point at which it enters the Stage 3 plateau marks the end. These two boundaries guide the selection of the intervention layer set L.
For each input, we compute L once from the question-end token’s entropy trajectory before decoding and reuse the layer indices at subsequent generation steps. At each step, we recompute the current attention Acurr(l), the smoothed historical attention Aˉ, and the target set and topology-aware weights produced by GCLR using the current query. Thus, each step uses the same intervention layers while selecting geometric cues relevant to its own query.
We tested intervention placement on 100 representative PGPS9K images. For each image, we shifted intervention layers by the same offset while keeping the intervention budget and all other settings fixed. Negative offsets move the intervention to earlier layers. An offset of zero corresponds to the entropy-selected layers, which serve as the reference for the F1 changes.
Experiments show that the F1 score exhibits a downward trend as the intervention shifts. An early intervention can lose influence as the visual features propagate through subsequent layers. A late intervention supplies visual cues after the model has settled on an interpretation, making those cues harder to incorporate and potentially disrupting the existing prediction. With the intervention budget matched, the entropy-selected layers outperform all shifted configurations, supporting the choice to intervene at the representational transitions identified by the entropy trajectory.
We will clarify the entropy fitting and layer reuse in the revised paper and include the matched-budget layer-offset results in the appendix.