acceptodds
Under review as a conference paper at ICLR 2027

DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoning

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a leading paradigm for enhancing visual reasoning in Multimodal Large Language Models (MLLMs). However, existing RLVR methods optimize primarily for the reasoning outcome and overlook how that outcome is produced: as a chain-of-thought unfolds, some tokens must extract evidence from the image while others must anchor to the preceding textual context, yet a single sequence-level reward provides no signal about whether each token draws on the modality its role demands. Token-level analyses and controlled interventions reveal that tokens frequently under-attend to their designated modality, and that this coordination breakdown is causally linked to reasoning failures. Motivated by these findings, we propose DyCo-RL, which integrates dynamic cross-modal coordination into RLVR optimization. Specifically, DyCo-RL uses the Fisher–Rao geodesic distance to measure within-modality attention shifts, assigning tokens to either visually-oriented or text-oriented functional roles. It then evaluates the alignment between a token's actual attention allocation and its assigned role, leveraging this score for alignment-guided advantage reweighting during policy optimization. Extensive experiments demonstrate that the algorithm-agnostic DyCo-RL, when applied to Qwen2.5-VL-3B/7B and Qwen3.5-27B, improves the average performance of four representative RLVR algorithms across seven benchmarks spanning visual-centric and mathematical reasoning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Reasoning, Reinforcement Learning, Cross-Modal Coordination, Multimodal Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uWCSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies dynamic coordination between visual information acquisition and textual reasoning in multimodal reinforcement learning. It proposes DyCo-RL, a token-level advantage-reweighting method for RLVR. The method uses Fisher–Rao geodesic distance between consecutive within-modality attention distributions to assign visual, textual, or neutral roles, and adjusts token advantages according to the alignment between the assigned role and actual attention allocation. Extensive experiments using 11K examples for RL training demonstrate that the algorithm-agnostic DyCo-RL can improve the average performance of all four evaluated RLVR algorithms across seven benchmarks.

Critical strengths
  1. The empirical improvements are consistent across all configurations. Table 1 reports average improvements across all 12 combinations of three backbone configurations and four RL algorithms. The three-seed GRPO experiments in Table 18 improve from 47.4 to 49.2 on 3B and from 55.8 to 58.1on 7B. Table 20 also reports gains across six additional benchmarks.
  2. The ablations provide meaningful evidence for role-conditioned reweighting. Random or reversed role assignments and fixed visual-only or text-only scoring underperform the full method as shown in table3 and 4.
Critical weaknesses
  1. The gap remains between the automatic role estimator and the proposed causal interpretation. Table 11 offers useful annotation-based validation, but its 74.0% accuracy applies only to the 26.3% of annotated tokens receiving non-neutral predictions, using erroneous trajectories from the 3B model. Moreover, Fisher–Rao’s balanced accuracy of 72.8% does not exceed KL’s 73.5%. These limitations reduce confidence in the semantic interpretation of the assignments and the specific advantage attributed to Fisher–Rao.
  2. The incremental value over closely related methods and under matched computational budgets remains insufficiently established. The experiments establish benefits at equal training-data exposure, but do not determine whether those benefits remain competitive when the baseline receives the same computational budget.
Questions for authors

The authors manually annotated 200 incorrect responses to evaluate the accuracy of automatic token-role assignment. Was the annotated diagnostic set used to select the role-assignment metric or threshold? Could you report coverage and role accuracy separately for correct and erroneous spans, and clarify whether role reliability has been evaluated on fully correct trajectories or larger backbones?

4 comments

Response to uWCS (Part 1/3): Role Diagnostics and Matched Coverage

Due to the comment length limit, our response is provided in three parts.

We sincerely thank the reviewer for the careful and constructive assessment. We appreciate your recognition of the consistent improvements across configurations, the three-seed results and out-of-domain evaluations, and the evidence from role-conditioned ablations. Below, we clarify the diagnostic protocol and provide additional role validation and matched-budget comparisons to address your concerns.

Point 1: W1 & Q1 — Role Reliability Across Span Correctness, Matched Coverage, Scale, and Causal Interventions

Clarification on metric and threshold selection (Q1). No—the 200-sample human-annotated set was strictly used for post-hoc diagnostic evaluation and was never used to choose the distance metric or threshold τ\tau. Fisher–Rao was motivated a priori as a symmetric, bounded ([0,π][0, \pi]) geodesic metric on the probability simplex, with τ=0.05\tau = 0.05 set as a small stability margin to filter near-tie transitions. Crucially, this single configuration was held strictly fixed without retuning across all 12 backbone–optimizer combinations (Table 1) and 6 held-out out-of-domain benchmarks (Table 20), with post-hoc ablations (Tables 3 and 13) confirming robustness across every tested τ∈{0,0.05,0.1}\tau \in \{0, 0.05, 0.1\}.

(1) Separate evaluation on locally correct vs. erroneous spans (Q1 & W1). Splitting the 6,875 annotated tokens in Table 7 into locally correct spans (Pr∪Rr\mathcal{P}^r \cup \mathcal{R}^r, 5,157 tokens) and locally erroneous spans (Pe∪Re\mathcal{P}^e \cup \mathcal{R}^e, 1,718 tokens) at τ=0.05\tau = 0.05 reveals why Fisher–Rao outperforms KL:

Table U1. Role agreement (%) on locally correct vs. erroneous spans (3B, τ=0.05\tau = 0.05).

MeasurementCorrect Cov.Correct Acc.Correct Bal. Acc.Erroneous Cov.Erroneous Acc.Erroneous Bal. Acc.
Attention24.360.455.231.960.857.9
Cosine Similarity18.664.167.616.468.368.1
Entropy24.867.356.324.165.958.7
KL Divergence29.273.674.325.364.566.4
Ours (Fisher–Rao)28.075.374.121.168.669.5

Fisher–Rao achieves the highest accuracy in both groups (75.3% on correct spans; 68.6% on erroneous spans). Notably, while KL is comparable on correct spans, its reliability drops sharply on erroneous spans, where Fisher–Rao leads KL in both accuracy (+4.1 points) and balanced accuracy (+3.1 points: 69.5% vs. 66.4%). Furthermore, on tokens misclassified by the Attention-magnitude baseline, Fisher–Rao (τ=0.05\tau = 0.05) covers 57.9% of tokens with 65.8% accuracy, directly correcting 38.1% of Attention's role errors.

(2) Superior accuracy and balanced accuracy under matched coverage (W1). In Table 11, a fixed τ=0.05\tau = 0.05 yielded slightly different coverages across differently scaled metrics (e.g., 28.2% for KL vs. 26.3% for Fisher–Rao). Matching non-neutral coverage at 25%, 50%, and 75% via per-response top-k% ranking removes scale differences:

Table U2. Role Accuracy / Balanced Accuracy (%) at matched non-neutral coverage (3B).

Measurement25% Coverage50% Coverage75% Coverage
Attention62.9 / 61.463.5 / 58.455.1 / 56.8
Cosine Similarity65.8 / 60.766.9 / 52.761.1 / 54.0
Entropy66.8 / 58.061.8 / 55.063.8 / 56.4
KL Divergence71.0 / 62.469.3 / 62.862.3 / 59.1
Ours (Fisher–Rao)73.6 / 68.272.8 / 63.565.4 / 59.6

Under matched selectivity, Fisher–Rao achieves both the highest accuracy and the highest balanced accuracy at every coverage level.

(3) Reliability beyond the 26.3% subset (W1). Varying τ\tau across {0,0.025,0.05,0.1}\{0, 0.025, 0.05, 0.1\} confirms that agreement with human roles is not limited to a narrow subset. At τ=0\tau = 0, the observed non-neutral coverage is 100.0% on this diagnostic set:

Table U3. Fisher–Rao role agreement across thresholds τ\tau (3B).

τ\tauCoverage (%)Accuracy (%)Balanced Accuracy (%)3B RL Avg. (Table 13)
0100.063.961.548.6
0.02552.970.267.0—
0.0526.374.072.849.3
0.15.480.567.748.7

At 52.9% coverage (τ=0.025\tau = 0.025, over 2×2\times the default coverage), Fisher–Rao retains 70.2% accuracy.

Response to uWCS (Part 2/3): Model Scale and Automatic-Role Interventions

Point 1: W1 & Q1 (continued)

(4) Larger backbones and trajectory correctness (Q1). While fully correct trajectories were not annotated as a separate set, our 3B diagnostic set includes locally correct reasoning spans. Within this group, Fisher–Rao achieves 75.3% accuracy on non-neutral predictions (Table U1). To test larger backbones, we newly annotated 100 rollouts (4,370 annotated tokens) from Qwen2.5-VL-7B following Appendix F:

Table U4. Fisher–Rao (τ=0.05\tau = 0.05) role agreement across model scales.

BackboneCoverage (%)Accuracy (%)Balanced Accuracy (%)
Qwen2.5-VL-3B (Table 11)26.374.072.8
Qwen2.5-VL-7B34.770.966.2

On 7B, Fisher–Rao achieves 70.9% accuracy (again the highest among all five evaluated measurements at τ=0.05\tau = 0.05, ahead of KL's 68.1%, Attention's 62.4%, Cosine's 59.2%, and Entropy's 59.1%), supporting transferability to larger models.

(5) Bridging the automatic estimator and causal interpretation (W1). To directly test whether automatic predictions carry the causal utility of human role labels, we replace human roles with step-wise Fisher–Rao predictions (τ=0.05\tau = 0.05) at the exact same annotated error spans, keeping intervention strengths λ\lambda, replay budgets, and decoding settings fixed (Appendix H). Neutral steps skip enhancement, and all 200 responses remain in the denominator.

Table U5. Answer recovery at visually-oriented erroneous spans (%).

λ\lambdaHuman rolesOur predicted rolesNo-labelRandom locationWrong-Role
0.00.00.00.00.00.0
0.17.04.54.05.54.0
0.215.514.016.011.57.5
0.313.015.514.510.59.5
0.421.518.513.015.08.0
0.532.525.511.520.06.5
0.623.527.07.014.56.5
0.728.021.510.514.50.5
0.811.010.52.08.50.5
Avg. (λ>0\lambda > 0)19.017.19.812.55.4

Table U6. Answer recovery at text-oriented erroneous spans (%).

λ\lambdaHuman rolesOur predicted rolesNo-labelRandom locationWrong-Role
0.00.00.00.00.00.0
0.114.513.011.09.07.5
0.219.021.515.012.54.0
0.326.022.021.519.58.5
0.423.022.513.518.09.0
0.521.519.517.019.07.5
0.626.521.514.018.54.0
0.724.520.07.517.02.5
0.816.012.51.514.01.0
Avg. (λ>0\lambda > 0)21.419.112.615.95.5

(Bold under "Our predicted roles" marks strengths outperforming all three non-oracle controls.)

Automatic-role interventions beat all three controls at 6 of 8 active strengths for visual errors and 7 of 8 for textual errors, retaining ~90% of the average recovery rate of human annotations (within 1.9 and 2.3 points) and providing complementary functional evidence for the automatic role assignments.

Response to uWCS (Part 3/3): Training Cost and Matched Budget

Point 2: W2 — Incremental Value Over Related Methods Under Matched Compute Budgets

(1) Comparable compute cost to VPPO with nearly double the gain over GRPO. Measuring training cost on the same 8×A100 server (11,031 samples) shows that VPPO incurs nearly the same rollout/processing cost as DyCo-RL:

Table U7. Measured 1-epoch training cost on Qwen2.5-VL-3B (8×A100 GPUs).

MethodThroughput (samples/s)Wall-clock time (hours)GPU-hoursAvg. Score
GRPO0.4476.8554.847.4
VPPO0.3379.0972.748.4
GRPO + DyCo-RL0.3279.3775.049.3

DyCo-RL requires 9.37 hours versus 9.09 hours for VPPO—only a 3.1% increase (+0.28 h, +2.24 GPU-hours), with zero extra parameters or runtime cost at inference. Over the shared GRPO baseline (47.4), where PAPO-G gains +0.5 (47.9) and VPPO gains +1.0 (48.4), DyCo-RL gains +1.9 points (49.3, outperforming PAPO-G on all 7 benchmarks and VPPO on 6 of 7) at virtually the same training budget.

(2) Incremental gains persist when baselines receive the full compute budget. We independently retrain GRPO and VPPO from the same initial checkpoint for the full 9.37 hours (75.0 GPU-hours) under identical hardware and evaluation settings:

Table U8. Performance at equal data exposure (1 epoch) vs. matched compute budget (9.37 hours).

MethodTime (h)WeMathMathVisionMathVerseMMBenchLogicVistaHallusionBenchMMEAvg.
GRPO (1 epoch)6.8523.021.434.472.839.860.879.947.4
VPPO (1 epoch)9.0926.922.035.875.139.661.278.548.4
GRPO (Matched Budget)9.3725.223.435.169.338.761.879.447.6
VPPO (Matched Budget)9.3727.322.036.472.437.461.079.147.9
GRPO + DyCo-RL (1 epoch)9.3725.922.436.076.241.262.480.749.3

At matched training compute (9.37 h), DyCo-RL remains ahead of GRPO and VPPO by +1.7 and +1.4 average points, respectively (and +0.9 points above VPPO's 1-epoch score of 48.4), supporting the value of dynamic cross-modal coordination under the matched compute budget.

Dear Reviewer uWCS,

We hope this message finds you well.

We have carefully addressed your questions and concerns, including conducting additional experiments on role reliability across different settings, matched-coverage evaluations, automatic-role interventions, and matched-compute comparisons, and have provided detailed responses in our rebuttal.

As the discussion deadline is approaching, we would sincerely appreciate it if you could share your updated thoughts based on our rebuttal. If you have any additional questions or concerns, please do not hesitate to let us know, and we will respond promptly.

Thank you again for your thoughtful review and valuable feedback, which have helped us improve the quality of our work.

Kind regards,

Authors of Submission 18709

Reviewer_7SGjOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DyCo-RL adds token-level modality guidance to multimodal RLVR. It assigns visual, textual, or neutral roles using Fisher-Rao distances between consecutive within-modality attention distributions, then reweights outcome advantages using role-aligned attention. The paper motivates this design through annotated error analysis and attention interventions. Experiments train on ThinkLite-hard-11K and evaluate seven benchmarks using Qwen2.5-VL-3B/7B and Qwen3.5-27B with GRPO, DAPO, SAPO, and GSPO. The authors report average-score improvements in all twelve backbone-optimizer combinations, alongside comparisons with PAPO-G and VPPO, component controls, repeated GRPO runs, and held-out evaluations.

Critical strengths
  1. The method addresses both visual extraction and textual inference, with supporting diagnostics. The annotated error analysis and role-targeted interventions examine a more specific problem than simply increasing visual attention. Wrong-role, random-location, and no-label controls strengthen the case that modality and intervention location both matter.

  2. The practical gains extend across optimization settings and model scales. Table 1 reports higher seven-benchmark averages in all twelve backbone–optimizer combinations. This supports the usefulness of the coordination signal across different post-training configurations.

Critical weaknesses
  1. The trade-off between role coverage and assignment accuracy is incompletely characterized. In Appendix I, Table 11, the automatic rule assigns explicit roles to 26.3% of annotated tokens, with 74.0% agreement with human labels. The authors motivate neutral assignments as filtering ambiguous roles, and Table 13 shows lower task scores at a threshold of 0 than at 0.05. However, coverage and agreement with human labels are not reported across thresholds. It is therefore unclear whether the lower task scores at a threshold of 0 are accompanied by less accurate role assignments as more tokens are classified. This limits understanding of how role reliability and selection coverage contribute to the method's gains.

  2. Training efficiency relative to VPPO remains unclear. Table 2 reports a 0.9-point average improvement over VPPO in the 3B setting. Appendix L, Table 19 reports throughput of 0.327 samples/s for DyCo-RL versus 0.447 for GRPO, corresponding to approximately 37% more time per sample relative to GRPO. However, VPPO's throughput or training cost is not reported, nor is performance compared at a similar training-time budget. This leaves the additional gain over VPPO difficult to assess against its computational cost.

Questions for authors
  1. For the thresholds evaluated in Appendix I, Table 13 (0, 0.05, and 0.1), what are the role coverage and agreement with human annotations? Is the lower task performance at a threshold of 0 accompanied by lower role-assignment accuracy?

  2. What are VPPO's training throughput or GPU-hours in the Table 2 setting? If existing timing records and checkpoints permit a comparison at a similar training-time budget, does DyCo-RL retain its average-score advantage?

3 comments

Response to 7SGj

We sincerely thank the reviewer for the thoughtful and constructive feedback. We are glad that you recognized the value of examining both visual extraction and textual inference, the supporting controlled interventions, and the gains across model scales and optimization settings. Below, we provide threshold-wise role diagnostics, measured training costs, and matched-budget comparisons to address your questions.

Point 1: W1 & Q1 — Characterizing the Coverage–Accuracy Trade-off Across Thresholds

Following the reviewer's request, we report role coverage and human-annotation agreement across thresholds on the diagnostic set, alongside the corresponding downstream 7-benchmark averages from Table 13. At τ=0\tau = 0, the observed non-neutral coverage is 100.0% on this diagnostic set.

Table S1. Role-assignment coverage, human agreement, and downstream task performance across thresholds τ\tau.

τ\tauCoverage (%)Accuracy (%)Balanced Accuracy (%)3B Task Avg. (Table 13)
0100.063.961.548.6
0.02552.970.267.0—
0.0526.374.072.849.3
0.15.480.567.748.7

Yes—lowering τ\tau from 0.05 to 0 increases coverage to 100.0% by forcing ambiguous transitions into binary roles, which reduces role-assignment accuracy from 74.0% to 63.9% (and balanced accuracy from 72.8% to 61.5%) alongside a lower task average (48.6 vs. 49.3 on 3B, and 57.2 vs. 58.3 on 7B in Table 13). Conversely, raising τ\tau to 0.1 increases accuracy to 80.5% but reduces coverage to 5.4% and balanced accuracy to 67.7%, limiting token-level supervision and lowering task scores (48.7 on 3B; 57.4 on 7B). Thus, τ=0.05\tau = 0.05 achieves the best performance by jointly balancing role reliability and selection coverage.

Point 2: W2 & Q2 — Training Cost Relative to VPPO and Comparison at Matched Training Time

(1) Comparable training throughput and GPU-hours to VPPO. Following Q2, we report throughput, wall-clock time, and total GPU-hours for the Table 2 configurations on the same 8×A100 server (11,031 samples):

Table S2. Measured 1-epoch training cost on Qwen2.5-VL-3B.

MethodThroughput (samples/s)Wall-clock time (hours)GPU-hours
GRPO0.4476.8554.8
VPPO0.3379.0972.7
GRPO + DyCo-RL0.3279.3775.0

VPPO achieves 0.337 samples/s (72.7 GPU-hours). The additional cost of DyCo-RL over VPPO is only 0.28 wall-clock hours (2.24 GPU-hours, +3.1%), with zero extra parameters or runtime cost at inference.

(2) Preserved advantage under a matched training-time budget. We independently retrain GRPO and VPPO from the same initial checkpoint for 9.37 hours on the same hardware and evaluate with identical settings:

Table S3. Performance at 1 epoch vs. matched training-time budget (9.37 hours).

MethodTime (h)WeMathMathVisionMathVerseMMBenchLogicVistaHallusionBenchMMEAvg.
GRPO (1 epoch)6.8523.021.434.472.839.860.879.947.4
VPPO (1 epoch)9.0926.922.035.875.139.661.278.548.4
GRPO (Matched Budget)9.3725.223.435.169.338.761.879.447.6
VPPO (Matched Budget)9.3727.322.036.472.437.461.079.147.9
GRPO + DyCo-RL (1 epoch)9.3725.922.436.076.241.262.480.749.3

At matched training time (9.37 h), DyCo-RL retains average-score gains of +1.7 points over GRPO and +1.4 points over VPPO (while also remaining +0.9 points above VPPO's 1-epoch score of 48.4), confirming that extra baseline training time does not close the performance gap.

Thanks for the additional analyses and experiments. The threshold diagnostics and matched-budget results address my main concerns. I'll keep my weak accept rating and raise my confidence.

Thank you for your prompt reply and positive feedback. We truly appreciate your time and support!

Reviewer_fK4TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DyCo-RL, which assigns visual or textual roles to generated tokens based on changes in within-modality attention distributions, and adjusts token-level advantages according to the alignment. The authors evaluate the method across three model configurations, supported by role annotations, attention intervention experiments, and ablation studies. The results show that the proposed method can improve the performance of existing algorithms.

Critical strengths
  1. The paper addresses coordination between visual information extraction and text-based reasoning. The proposed method requires no additional training.
  2. The experiments cover multiple model scales and reinforcement learning algorithms. Results across three random seeds on the 3B and 7B models further support the performance improvements over GRPO.
  3. The authors provide extensive ablation studies, including human role annotations, controlled attention interventions, and a centered reweighting variant that removes the common shift in token-level advantages.
Critical weaknesses
  1. The central assumption underlying role assignment requires further validation. The paper interprets larger within-modality attention changes as indicating that the current token depends more strongly on that modality. However, attention redistribution does not necessarily reflect a token’s functional role. When the model’s attention is misallocated, it is unclear whether changes in that attention can reliably indicate which modality the token should be. In Table 11, Fisher–Rao achieves a role prediction accuracy of 74%, but this is measured only on the 26.3% of annotated tokens assigned non-neutral roles. Moreover, the diagnostic set consists of incorrect responses from the 3B model.
  2. The ablations do not adequately separate the contribution of the distance metric from that of token selection. Appendix I uses the same threshold, as τ=0.05\tau=0.05 for different metrics, even though Fisher–Rao distance, KL divergence, and entropy operate on different numerical scales. An identical threshold therefore does not imply a comparable degree of selectivity. The current comparisons change both the criterion for role assignment and the set of selected tokens.
  3. The gains over related methods are modest, particularly given the additional training cost. In Table 2, DyCo-RL improves the average score over VPPO by 0.9 points. Meanwhile, Table 19 reports a decrease in training throughput from 0.447 to 0.327 samples/s, implying longer training time for the same number of samples. The experiments match the amount of training data but not the computational budget. It remains unclear whether DyCo-RL would retain its advantage if the baselines used the same additional budget for more training updates.
Questions for authors
  1. How does the recovery rate in the attention intervention experiments change when automatically predicted roles replace human-annotated roles, while keeping intervention locations and the number of attempts fixed?
  2. Does Fisher–Rao retain a consistent advantage when thresholds are tuned separately for each metric or when the coverage of non-neutral tokens is matched across methods?
5 comments

Response to fK4T (Part 1/4): Role Reliability and Model Scale

Due to the comment length limit, our response is provided in four parts.

We sincerely thank the reviewer for the detailed and constructive feedback. We appreciate your recognition of our evaluation across model scales and RL algorithms, the three-seed results, and the extensive ablations. Below, we provide additional role diagnostics, automatic-role interventions, matched-coverage comparisons, and evaluations under matched compute budgets to address your concerns.

Point 1: W1 & Q1 — Validating Role Assignment Under Misallocated Attention, Broader Coverage, Larger Scale, and Automatic Interventions

(1) Why within-modality shifts remain informative under misallocated attention. Mechanistically, the failures in Fig. 2 stem from under-allocating total cross-modal attention mass rtmr_t^m to the required modality. Because Eq. (6) normalizes attention within each modality (pt,jm=at,j/rtmp_{t,j}^m = a_{t,j}/r_t^m), Fisher–Rao captures intra-modality focus shifts independently of suppressed cross-modal mass rtmr_t^m. Empirically, we evaluate Fisher–Rao (τ=0.05\tau = 0.05) both on annotated tokens misclassified by the Attention-magnitude baseline (τ=0.05\tau = 0.05, N=712N = 712) and separately across locally erroneous (Pe∪Re\mathcal{P}^e \cup \mathcal{R}^e) versus locally correct (Pr∪Rr\mathcal{P}^r \cup \mathcal{R}^r) spans from Table 7:

Table F1. Fisher–Rao (τ=0.05\tau = 0.05) on misclassified tokens and erroneous vs. correct spans (3B).

Evaluated SubsetNon-Neutral Coverage (%)Role Accuracy (%)Attention Errors Corrected (%)
Attention-misclassified tokens (N=712N = 712)57.965.838.1
Locally erroneous spans Pe∪Re\mathcal{P}^e \cup \mathcal{R}^e (N=1,718N = 1{,}718)21.168.6—
Locally correct spans Pr∪Rr\mathcal{P}^r \cup \mathcal{R}^r (N=5,157N = 5{,}157)28.075.3—

Even when raw attention magnitude assigns the wrong modality, Fisher–Rao assigns explicit roles to 57.9% of those tokens with 65.8% accuracy, directly correcting 38.1% of Attention's errors. Moreover, on locally erroneous spans where model attention is actively misallocated, Fisher–Rao achieves 68.6% role accuracy (69.5% balanced accuracy, outperforming KL's 64.5% / 66.4% and Attention's 60.8% / 57.9%). Within locally correct spans, Fisher–Rao achieves 75.3% accuracy on non-neutral predictions.

(2) Role accuracy across broader coverage and its link to downstream RL. Varying τ\tau from 00 to 0.10.1 on the 3B diagnostic set shows that useful role accuracy extends well beyond the default 26.3% subset. At τ=0\tau = 0, the observed non-neutral coverage is 100.0% on this diagnostic set:

Table F2. Fisher–Rao role agreement across thresholds τ\tau and downstream RL scores (3B).

τ\tauCoverage (%)Accuracy (%)Balanced Acc. (%)Downstream 7-Bench Avg. (Table 13)
0100.063.961.548.6
0.02552.970.267.0—
0.05 (Default)26.374.072.849.3
0.15.480.567.748.7

At 52.9% coverage (τ=0.025\tau = 0.025, over 2×2\times the default coverage), Fisher–Rao retains 70.2% accuracy. Lowering τ\tau to 00 forces ambiguous or hybrid transitions into binary roles, reducing role accuracy to 63.9% and downstream RL performance from 49.3 to 48.6 (while still beating GRPO's 47.4). Conversely, τ=0.1\tau = 0.1 raises accuracy to 80.5% but reduces coverage to 5.4% and balanced accuracy to 67.7%, slightly reducing the RL score to 48.7.

(3) Extending beyond 3B erroneous responses. Within locally correct spans in our 3B diagnostic set, Fisher–Rao achieves 75.3% accuracy on non-neutral predictions (Table F1). We further evaluate Fisher–Rao (τ=0.05\tau = 0.05) on newly annotated Qwen2.5-VL-7B rollouts (100 rollouts, 4,370 annotated tokens) following the exact protocol in Appendix F:

Table F3. Fisher–Rao (τ=0.05\tau = 0.05) role agreement across model scales.

Diagnostic SubsetCoverage (%)Accuracy (%)Balanced Acc. (%)
Qwen2.5-VL-3B erroneous rollouts (Table 11, N=6,875N = 6{,}875)26.374.072.8
Qwen2.5-VL-7B erroneous rollouts (N=4,370N = 4{,}370)34.770.966.2

On 7B, Fisher–Rao achieves 70.9% accuracy (also the highest among all five measurements at τ=0.05\tau = 0.05, ahead of KL's 68.1%, Attention's 62.4%, Cosine's 59.2%, and Entropy's 59.1%), supporting that the role signal extends beyond the 3B model.

Response to fK4T (Part 2/4): Automatic-Role Interventions

Point 1: W1 & Q1 (continued)

(4) Causal validation via automatic-role interventions (Q1). Following the reviewer's request, we replace human role labels with online step-wise Fisher–Rao predictions (τ=0.05\tau = 0.05) at the exact same annotated error spans, keeping intervention strengths λ\lambda, replay budgets, stopping rules, and greedy decoding fixed per Appendix H. Steps predicted as neutral skip enhancement, and all 200 responses remain in the evaluation denominator.

Table F4. Answer recovery at visually-oriented erroneous spans (%).

λ\lambdaHuman rolesOur predicted rolesNo-labelRandom locationWrong-Role
0.00.00.00.00.00.0
0.17.04.54.05.54.0
0.215.514.016.011.57.5
0.313.015.514.510.59.5
0.421.518.513.015.08.0
0.532.525.511.520.06.5
0.623.527.07.014.56.5
0.728.021.510.514.50.5
0.811.010.52.08.50.5
Avg. (λ>0\lambda > 0)19.017.19.812.55.4

Table F5. Answer recovery at text-oriented erroneous spans (%).

λ\lambdaHuman rolesOur predicted rolesNo-labelRandom locationWrong-Role
0.00.00.00.00.00.0
0.114.513.011.09.07.5
0.219.021.515.012.54.0
0.326.022.021.519.58.5
0.423.022.513.518.09.0
0.521.519.517.019.07.5
0.626.521.514.018.54.0
0.724.520.07.517.02.5
0.816.012.51.514.01.0
Avg. (λ>0\lambda > 0)21.419.112.615.95.5

(Bold under "Our predicted roles" marks strengths outperforming all three non-oracle controls.)

Intervening via automatically predicted roles beats all three controls at 6 of 8 active strengths for visual errors and 7 of 8 for textual errors, retaining ~90% of the average recovery rate of human annotations (17.1% vs. 19.0%, and 19.1% vs. 21.4%). This shows that Fisher–Rao-predicted roles provide effective modality guidance at fixed erroneous spans.

Response to fK4T (Part 3/4): Matched-Coverage Comparisons

Point 2: W2 & Q2 — Decoupling the Distance Metric from Token Selection via Matched Coverage

(1) Offline role-assignment accuracy under matched non-neutral coverage. To eliminate numerical scale differences across metrics, we match non-neutral coverage across all five measurements at 25%, 50%, 75%, and 100% (τ=0\tau = 0) using per-response top-k% ranking. At τ=0\tau = 0, the observed non-neutral coverage is 100.0% on this diagnostic set. Token filtering is therefore completely removed, so all methods evaluate the exact same set of tokens.

Table F6. Role Accuracy / Balanced Accuracy (%) at matched non-neutral coverage (3B).

Measurement25% Cov.50% Cov.75% Cov.100% Cov. (τ=0\tau = 0)
Attention62.9 / 61.463.5 / 58.455.1 / 56.854.5 / 60.9
Cosine Similarity65.8 / 60.766.9 / 52.761.1 / 54.056.6 / 54.2
Entropy66.8 / 58.061.8 / 55.063.8 / 56.459.9 / 56.0
KL Divergence71.0 / 62.469.3 / 62.862.3 / 59.161.3 / 59.8
Ours (Fisher–Rao)73.6 / 68.272.8 / 63.565.4 / 59.663.9 / 61.5

Fisher–Rao achieves the highest accuracy and balanced accuracy across every coverage level. Mechanistically, Entropy only captures single-step concentration rather than temporal shifts, while KL divergence DKL(ptm∥pt−1m)D_{\text{KL}}(p_t^m \parallel p_{t-1}^m) is asymmetric and unbounded whenever newly attended tokens have near-zero prior mass (pt−1,jm≈0p_{t-1,j}^m \approx 0), creating cross-modality scale spikes. Being symmetric and strictly bounded in [0,π][0, \pi], Fisher–Rao makes visual and textual shift magnitudes directly comparable.

(2) Downstream RL performance under matched coverage (Q2). To verify that this metric advantage translates to RLVR optimization (Table 3) after controlling non-neutral coverage, we evaluate Qwen2.5-VL-3B + GRPO under two controlled regimes: (i) 100% coverage (τ=0\tau = 0), where all tokens receive roles without neutral filtering; and (ii) matched ~29.7% training coverage (matching Fisher–Rao's non-neutral ratio in Table 12 via per-response top-k% ranking).

Table F7. Downstream 7-benchmark average on Qwen2.5-VL-3B under matched selectivity.

Role CriterionFixed τ=0.05\tau = 0.05 (Table 3)100% Coverage (τ=0\tau = 0)Matched ~29.7% Coverage
GRPO Baseline47.447.447.4
+ Entropy48.247.548.4
+ KL Divergence48.648.048.7
+ Ours (Fisher–Rao)49.348.649.3

Holding token coverage strictly fixed—either at 100% (no selection) or at ~29.7%—Fisher–Rao consistently outperforms both Entropy and KL across the seven benchmarks, showing that its advantage is not explained by differences in token-selection coverage.

Comments on the paper

1 comment
Area_Chair_eNKqOpenReview ↗

Please engage in the discussion

Dear Reviewers,

Thank you for your thoughtful reviews. The authors have posted their rebuttal, please read it and share any updated assessments.

As you know, we have only 3 reviews in total. Each piece of your opinions is valuable and may have crucial impact on the final decison. So please take it seriously.

Thanks again for your contribution!

Best Regards,

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.