- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims to solve the problem that early visual misinterpretations may mislead the reasoning and get wrong answers. The authors proposed a Visual Board centered harness for improving visual reasoning accuracy. The harness makes task-relevant visual premises, uncertain details, and intermediate reasoning explicit without requiring an exhaustive image description. Empirical experiments show that solver training improves both model-only and harness-based inference.
Critical strengths
-
The paper addresses an important problem in visual reasoning, where incorrect visual observations can persist and mislead the following reasoning process. The proposed Visual Board provides a structured and revisable workspace for maintaining visual facts, uncertainties, and intermediate reasoning, while retaining access to the original image.
-
The proposed harness improves performance over model-only baselines across multiple model configurations and benchmarks.
Critical weaknesses
-
The proposed framework primarily combines existing techniques, including structured textual memory, visual inspection tools, history compression, and iterative solver–critic interaction, and the incremental contribution is unclear. Moreover, the main component ablations are conducted on only 50 Euclid 30K examples with a 32B solver, limiting the evidence that the Visual Board is consistently responsible for the improvements across different benchmarks and model scales.
-
The comparison is unfair. The model-only baselines do not have access to the additional visual tools, critic feedback, and multiple reasoning rounds used by the proposed harness. As stated in Appendix A.1, the 4B configuration relies on Qwen3-VL-Flash for visual-tool calls and Qwen3-Max-Preview as the critic. So, the reason of reported gains are unclear that whether they attribute specifically to the proposed framework or the additional model capabilities and inference budget. Although the comparison with ProReason is useful, it is limited to one benchmark and does not establish a computation-matched comparison.
Questions for authors
-
Do the benefits of the Visual Board remain consistent across different benchmarks and model scales? The current component ablations are limited to only 50 examples with a 32B solver.
-
How often does the Visual Board actually help correct previously mistaken visual observations? A quantitative analysis of successful and unsuccessful premise revisions would help to understand why it works.
-
Why there are hashing-like strings in this paper (Line 790, Line 830)? Could the authors clarify this?
-
The released
cold_start/review_euclid30k_8b.pyrecords Codex-based answer-equivalence judgments that revise correctness labels and recompute accuracy, while Section 6 states that AI tools were used only for language polishing and code debugging. Were these judgments used in the reported Euclid30K results, and if so, how did they affect accuracy and how does this align with the AI Use Statement?
5 comments
Structured Board vs. shared notes.
| Configuration | Default minus free-form accuracy (pp) | 95% paired bootstrap CI (pp) | Comparison scope |
|---|---|---|---|
| All-4B | +13.05 | [+8.43, +17.67] | 244/498 vs. 179/498 correct |
| All-8B | +9.35 | Not reported | Aggregate scoring sets |
| Qwen3.8-Flash | +6.67 | Not reported | Aggregate scoring sets |
All-4B uses 498 common questions, counts run failures as incorrect, and uses 100,000 paired bootstrap resamples. There are 105 default-only correct and 40 free-form-only correct questions. Its difference is calculated from unrounded counts. pp denotes percentage points.
The results favor structured visual evidence over free-form shared notes in these Euclid configurations, with paired statistical support for All-4B.
Cross-benchmark comparisons after SFT. We additionally compare inference settings for the same Qwen3-VL-4B Solver after 200 SFT steps on 1,556 trajectories from the Euclid train split. Euclid testing uses the separate evaluate split; ChartQA uses the 826-question bar-chart subset, with no ChartQA examples used for SFT. Both Solver-only tool-use settings below retain a Board and omit the Critic; they differ in whether the Board uses free-form notes or the prescribed structure.
| Euclid30K inference setting | Board | Critic | SFT 4B accuracy (%) |
|---|---|---|---|
| Direct QA | None | No | 42.80 |
| Solver-only + Tools | Free-form | No | 55.20 |
| Solver-only + Tools | Default | No | 55.80 |
| Full harness, heterogeneous | Default | Yes | 57.03 |
ChartQA before and after SFT: separating training gains from Board-format differences. We additionally evaluate the base Solver under the same ChartQA inference settings. Within each row below, tool access, visual-tool backend, Board format, and Critic setting are held fixed between the base and SFT Solvers.
| ChartQA Inference Setting | Board | Critic | Base 4B (%) | SFT 4B (%) | SFT Gain (pp) |
|---|---|---|---|---|---|
| Direct QA (no tools) | None | No | 57.75 | 70.58 | +12.83 |
| Solver-only + Tools | Free-form | No | 77.24 | 83.78 | +6.54 |
| Solver-only + Tools | Default | No | 78.45 | 82.81 | +4.36 |
| Full harness (heterogeneous) | Default | Yes | 78.69 | 84.87 | +6.18 |
Base means no additional SFT; all SFT rows use the same 200-step checkpoint.
Training gains within a fixed Board format. Read each ChartQA row from Base to SFT: accuracy improves under both Board formats without Critic feedback, as well as under direct QA and full-harness inference. This supports geometry-to-chart transfer across inference settings.
In the free-form setting, the Solver itself decides what information to store and how to organize it, without the Board's prescribed structure. Our interpretation is that training on harness rollouts may help the Solver internalize Board-related knowledge about selecting, organizing, and using visual evidence, which it can then apply through self-organized notes. The improvement over the base Solver under the same free-form setting is consistent with this interpretation and with transfer beyond geometry.
2. W2: Helper-Model Capacity and Inference Resources
We distinguish stronger helper models from additional inference. In the original heterogeneous system, "4B" denotes the Solver: Qwen3.0-VL-Flash serves the visual tools and Qwen3-Max-Preview serves as Critic.
To address helper-model capacity, we added an All-4B configuration using Qwen3-VL-4B-Instruct for both the Solver and visual tools, and Qwen3-4B for the Critic:
| Configuration | Euclid30K (%) | SolidGeo (%) | ChartQA (%) |
|---|---|---|---|
| Standalone Qwen3-VL-4B-Instruct | 39.40 | 32.11 | 57.75 |
| All-4B harness | 49.00 | 40.84 | 77.97 |
| Improvement (percentage points) | +9.60 | +8.73 | +20.22 |
Substantial gains therefore persist without larger helper models. This is a matched-size system comparison, not an equal-compute comparison.
Resource trade-offs within each configuration. The table compares default Board against each reference using the accuracy and token statistics in Section 1.
| Configuration | Reference | Accuracy difference (pp) | Token change (%) |
|---|---|---|---|
| All-4B | No Board / retry | +19.00 | -7.9 |
| All-4B | Free-form Board | +13.05 | +2.2 |
| All-8B | No Board / retry | +7.35 | -24.4 |
| All-8B | Free-form Board | +9.35 | +4.4 |
| Qwen3.8-Flash | No Board / retry | +8.47 | -38.2 |
| Qwen3.8-Flash | Free-form Board | +6.67 | -21.7 |
The Board improves observed accuracy with fewer tokens than retry in all three configurations. Relative to free-form notes, it uses slightly more tokens for All-4B and All-8B and fewer for Qwen3.8-Flash.
We thank the reviewer for the constructive feedback. We address the concerns with expanded 500-question component comparisons across three configurations, Board-format comparisons on Euclid30K and ChartQA, including ChartQA evaluations before and after SFT, an All-4B system, Board-update analysis, and evaluation clarification.
1. W1 & Q1: Core Contribution and Expanded Ablations
Our key contribution is the Visual Board, a structured linguistic representation of image content, rather than a general-purpose store of dialogue history. It separates image-grounded observations and unresolved visual uncertainties from downstream reasoning, maintaining a compact representation through updates rather than unrestricted accumulation of reasoning traces. Although its persistent, shared implementation may resemble memory, the central design choice is how visual evidence is represented and maintained to support subsequent reasoning, not persistence or sharing alone.
To evaluate the value of this design, we expanded the original 50-example pilot to 500 Euclid30K questions with All-4B(Qwen3VL-4B for solver, Qwen3-4B for critic), All-8B(Qwen3-VL-8B for solver, Qwen3-8B for critic), and Qwen3.8-Flash(Qwen3.8-VL-Flash for solver and critic) configurations. Within each model group, the shared free-form scratchpad (
board_free), no-Board retry baseline (no_board), and default Board retain the same Solver, visual-tool backend, and Critic.In
board_free, the model writes its own free-form textual notes as shared memory, without the Visual Board's prescribed structure.The comparisons use the same maximum of 3 rounds, tool-call limits, stopping criteria, and maximum output-token setting. Actual reasoning-token consumption varies, so we report resource use rather than describe the runs as strictly total-token-budget matched.
Accuracies use the retained scoring sets, which can be smaller than the sampled count. The All-4B default-versus-free-form comparison uses 498 common questions, with run failures counted as incorrect. Resource figures are original run aggregates.