- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces ReDiS, a post-training framework for remote sensing visual question answering. A Reasoner samples multiple evidence–answer candidates, and a Diagnoser evaluates those candidates and selects one or rejects the group. The two roles use separate LoRA adapters attached to a shared frozen Qwen3.5 backbone. The Reasoner is optimized with answer-correctness-gated rewards that include diagnostic feedback. The authors also propose Failure-Triggered Recovery (FTR): when an entire sampled group is incorrect and group-relative advantages provide no useful signal, answer-value tokens receive a supervised correction. Experiments on LHRS-Bench, MME-RS, and the multiple-choice subset of CHOICE show improved answer accuracy across 0.8B, 2B, and 4B models.
Critical strengths
-
The motivation makes sense to me. In visual question answering, a model can give the right answer while using questionable visual evidence, and ordinary outcome rewards do not tell us where the reasoning went wrong. I can see why the authors introduce a separate Diagnoser and let its feedback influence training. Gating the reward by answer correctness is also a reasonable precaution.
-
FTR is a useful part of the paper. When all sampled answers are wrong, the group-relative RL update has little to work with; giving a supervised correction on the answer tokens is a practical fix. The full method improves the 0.8B average score by about 6 points and also improves the 2B and 4B models. It is good that the extra diagnostic role is used during training and does not need to run at inference time.
Critical weaknesses
-
I was not convinced that the Diagnoser has learned to judge visual evidence. In Section 3.3, the diagnostic targets are largely constructed from whether the final answer is correct. But a correct answer can still come with a made-up or irrelevant visual explanation. I may miss but I did not find an evaluation where evidence quality is checked independently of answer correctness. This is the part I would most like to see, because evidence diagnosis is central to the motivation. Also, It is difficult to tell how much of the improvement comes from the online Diagnoser.
-
I am not sure why this method is presented specifically as a remote sensing approach. The paper mentions dense small objects and difficult spatial relations, but the Reasoner–Diagnoser loop and FTR seem applicable to general VQA almost unchanged. That is fine if the training method itself brings a sufficiently new result. In that case, though, the connection to earlier generator–verifier work such as RL Tango and hybrid RL/SFT training such as ReLIFT should be much clearer. If the contribution is primarily to remote sensing, I would like to know which of its harder question types benefit most. The category breakdown does not make that clear against the corresponding untuned Qwen3.5 models.
Questions for authors
-
Could the authors evaluate the Diagnoser on a held-out set of responses with correct final answers but differing visual-evidence quality? Independent checks on a modest subset would be useful. I would substantially revise my view of the central contribution if the Diagnoser can consistently identify unsupported evidence under this control.
-
The same training procedure appears applicable to general visual question answering. Which challenges of remote-sensing imagery are addressed especially well by ReDiS? A category-wise improvement analysis against each matched Qwen3.5 backbone, particularly for counting, localization, and spatial relations, could make this much clearer.