- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes clue-aware RL for visual jigsaw puzzles using structural and semantic clues. Through training with clues, the model performs better on visual jigsaw tasks. Experimental results show that ClueJigsaw also benefits performance across various visual benchmarks.
Critical strengths
-
The motivation for using clues is clear and intuitive.
-
Training on ClueJigsaw alone transfers to general visual benchmarks.
Critical weaknesses
-
Missing comparison with standard jigsaw training such as [1, 2]
-
It is unclear whether clues help general visual tasks. Does the model perform better than jigsaw training alone?
Reference:
[1] Visual Jigsaw Post-Training Improves MLLMs, ICLR 26
[2] Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles, TMLR 26
Questions for authors
Does the model also train on standard RL data?
3 comments
As the discussion period draws to a close, we would like to check whether our response has adequately addressed your concerns. We would be happy to clarify any remaining points while there is still time.
With about 24 hours left in the discussion period, we would appreciate your feedback on our response. Please let us know if any concerns remain. Thank you!
We thank the reviewer for helping strengthen our paper and clarify the baseline comparisons and training data. We hope these responses address your concerns and would appreciate your reconsideration of the score.
Q1. Benefits Beyond Jigsaw Training Alone
To address W1, we compare reconstruction performance with Visual Jigsaw (Wu et al., 2026) and Jigsaw-R1 (Wang et al., 2025), retraining both methods with the same backbone and shared reconstruction data and evaluating their released checkpoints as supplementary baselines. To address W2, we extend these comparisons to the six general visual benchmarks to examine whether clue-guided training provides additional transfer benefits beyond jigsaw training alone.
Our primary comparison retrains both approaches using the same Qwen3-VL-8B-Instruct backbone and shared full-restoration training data. These baselines optimize answer correctness and output format without explicit clue supervision or clue-use rewards. We additionally evaluate the authors' released checkpoints. The experimental settings are detailed below.
Visual Jigsaw. Following the original method, we retrain Visual Jigsaw-8B* using GRPO without an SFT cold start. The accuracy reward is 1 for exact reconstruction, 0.2 times the correct-position fraction for other valid permutations, and 0 for invalid ones. Correct formatting earns an additional 0.2, and KL regularization is disabled. We also directly evaluate the released Visual Jigsaw-7B checkpoint.
Jigsaw-R1. Following the original method, we retrain Jigsaw-R1-8B* using GRPO without an SFT cold start. The accuracy reward is the correct-position fraction, with an additional format reward of 0.5 and a KL coefficient of 0.04. We also directly evaluate the released Thinking 3B checkpoint, denoted as Jigsaw-R1-3B.
The following table reports reconstruction accuracy and general visual benchmark scores (%), with bold and underlining indicating the best and second-best results, respectively.
\hline \text{Setting} & \text{Method} & \rlap{\text{Jigsaw}} & & \rlap{\text{General Visual Benchmarks}} & & & & & \\\\ & & 2\times2 & 3\times3 & \text{MME} & \text{RWQA} & \text{MMVP} & \text{BLINK} & \text{Hall} & \text{MMMU} \\\\ \hline \text{Released} & \text{Visual Jigsaw-7B} & 55.4^\dagger & 41.4^\dagger & 46.7 & 65.2 & 52.0 & 55.0 & \underline{65.4} & 52.1 \\\\ & \text{Jigsaw-R1-3B} & 0.4 & 0.0 & 45.0 & 60.8 & 39.3 & 47.0 & 59.8 & 43.2 \\\\ \hline \text{Retrained} & \text{Visual Jigsaw-8B}\ast & \underline{71.2} & \underline{51.7} & \underline{48.5} & 67.3 & \underline{60.0} & \underline{59.9} & 61.5 & 52.8 \\\\ & \text{Jigsaw-R1-8B}\ast & 44.8 & 20.6 & 48.2 & \underline{69.5} & 59.3 & 59.3 & 60.7 & \underline{53.4} \\\\ \hline \text{Ours} & \mathbf{ClueJigsaw\text{-}8B} & \mathbf{97.4} & \mathbf{64.1} & \mathbf{50.3} & \mathbf{71.2} & \mathbf{66.0} & \mathbf{62.1} & \mathbf{67.6} & \mathbf{56.9} \\\\ \hline \end{array}$$ $^\dagger$ Since some test source images appear in Visual Jigsaw's public training data, we report its $2\times2$ and $3\times3$ accuracies on the non-overlapping subset. **Reconstruction Performance.** ClueJigsaw achieves higher exact reconstruction accuracy than the jigsaw-specific baselines, supporting the effectiveness of our framework relative to existing jigsaw post-training approaches. **Additional Transfer Benefits.** ClueJigsaw also achieves gains on general visual reasoning benchmarks over correctness-only jigsaw training, suggesting that our clue-guided post-training framework provides additional transferable benefits beyond jigsaw training alone. ### Q2. Post-Training Data Our additional SFT and RL stages use only jigsaw-related data, without mixing in general-purpose RL or VQA data. The six general visual benchmarks are used solely for evaluation, with no benchmark-specific fine-tuning. The resulting gains highlight the transfer benefits of clue-guided jigsaw post-training without additional general-purpose training data.