acceptodds
Under review as a conference paper at ICLR 2027

TerraEvo: Skill-Guided Self-Evolution of Earth Observation Vision-Language Models

Abstract

Remote-sensing vision-language models (VLMs) remain heavily dependent on human-annotated supervision, constraining their scalability across increasingly diverse Earth observation (EO) imagery. Self-evolution offers a promising alternative, yet sustaining the reliability and diversity of self-generated supervision remains challenging. We introduce TerraEvo, a self-evolving framework that enables Proposer and Solver roles to co-evolve from unlabeled EO imagery without additional human annotations or an external judge. At its core, skill-guided Socratic dialogue transforms the Solver's answer disagreement into targeted follow-up questions. Quality supervision and adaptive rewards encourage informative, visually grounded questioning, while skill balancing and knowledge preservation support sustained learning across diverse EO tasks. A composite label-free learning signal further integrates answer consistency and reasoning quality to guide Solver improvement. Across five remote-sensing benchmarks, TerraEvo achieves the highest average accuracy among the evaluated general-purpose and domain-specific models, while ablation studies demonstrate the effectiveness of its key components. Our code, data, and models will be released publicly.

Primary area:
reinforcement learning
Keywords:
Self-evolving, Remote sensing vision-language models, GRPO, reinforcement learning, Earth observation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LZjPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TerraEvo, a self-evolution framework for earth-observation VLMs that trains a Proposer and a Solver jointly from unlabeled remote-sensing imagery, with no human annotations and no external judge. Specifically, TerraEvo uses a skill-guided Socratic dialogue to turn the Solver's answer disagreement into targeted follow-up questions, while skill balancing, quality supervision, adaptive rewards, and knowledge-preservation mechanisms keep the self-generated data grounded and diverse. On five RS benchmarks, the Qwen3-VL-4B-based model reaches 54.2% average accuracy, the highest among all. Ablations on GeoBench-VLM also support each component in TerraEvo.

Critical strengths
  1. The motivation of the paper is clear, and the performance of TerraEvo is good.
  2. The component ablations are quite comprehensive.
Critical weaknesses
  1. The experiments are currently conducted only with Qwen3-VL-4B-Instruct. There is no evaluation across different model families (e.g., Llama, Gemma, etc.), nor are larger-scale models or more recent models from the Qwen series tested. This limited scope makes it difficult to assess whether the observed findings generalize beyond this single configuration.
Questions for authors

Please see weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer LZjP: results on a larger model (Qwen3-VL-8B)

We thank the reviewer for the summary, for recognizing the motivation, performance, and component ablations, and for encouraging evaluation beyond the 4B model. The submitted paper reported the 4B experiments, and the 8B experiment was still running at that time. We now report its completed results below and in Table 1 of the revised manuscript.

Weakness 1 and Question 1: larger model. Question 1 refers to the weaknesses, so this reply answers both. Accuracy (%) on five EO benchmarks. Average is the unweighted mean of the five scores.

ModelGeoBench-VLMRSHR-BenchXLRS-BenchMME-RealWorld-RSLHRS-BenchAverage
Qwen3-VL-4B-Instruct39.354.249.651.264.151.7
Qwen3-VL-8B-Instruct42.253.850.157.162.453.1
TerraEvo 4B43.655.050.455.966.154.2
TerraEvo 8B43.754.551.458.764.854.6

Gain over the base model (points).

ModelGeoBench-VLMRSHR-BenchXLRS-BenchMME-RealWorld-RSLHRS-BenchAverage
TerraEvo 4B vs. base 4B+4.3+0.8+0.8+4.7+2.0+2.5
TerraEvo 8B vs. base 8B+1.5+0.7+1.3+1.6+2.4+1.5

What the results show. TerraEvo improves over its base model on all five benchmarks at both sizes. The average gain is +2.5 points for 4B and +1.5 points for 8B. The 8B base model is already stronger than the 4B base model, which may leave less room for gains. Compared with TerraEvo 4B, TerraEvo 8B is higher on GeoBench-VLM, XLRS-Bench, and MME-RealWorld-RS, and lower on RSHR-Bench and LHRS-Bench. Its average is higher (54.6 vs. 54.2). Both models are trained for 3,000 steps with seed 42.

Scope. We agree that broader testing would further show how well TerraEvo generalizes. In the discussion period, we did not evaluate models larger than 8B, other model families such as Llama or Gemma, or more recent Qwen models. Our evidence therefore covers Qwen3-VL at two sizes. All other experiments and analyses in the paper use Qwen3-VL-4B-Instruct.

Changes in the manuscript: Sections 5.1 and 5.2 and Table 1.

Thank you for responses. I will maintain my positive score.

AuthorsOpenReview ↗

Thank you to Reviewer LZjP

Dear Reviewer LZjP,

Thank you for reading our responses and for keeping your positive score. We appreciate your feedback and your time throughout the review process. We have included the results on the larger model (Qwen3-VL-8B) in the revised manuscript.

Authors

Reviewer_eBkTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses self-evolving vision-language models for Earth observation imagery, where unconstrained self-evolution can produce poorly grounded questions, narrow skill coverage, and forgetting of previously learned skills. The authors propose TerraEvo, a Proposer–Solver framework that co-evolves both roles from unlabeled EO imagery without additional human annotations or an external judge. TerraEvo combines skill-targeted generation with multi-turn Socratic dialogue conditioned on Solver answer disagreement, quality supervision and validity checks, an adaptive question reward, and a composite label-free Solver reward over answer agreement, trajectory agreement, and CoT quality. It further introduces skill-adaptive KL regularization and skill-balanced replay for knowledge preservation. Experiments on five remote sensing VQA benchmarks report the highest average accuracy among the evaluated models, improving over the Qwen3-VL-4B backbone from 51.7% to 54.2%, with ablations supporting the main components.

Critical strengths
  1. TerraEvo integrates skill balancing, Solver-conditioned multi-turn dialogue, question-validity supervision, composite Solver rewards, and knowledge-preservation mechanisms. The ablation studies examine these components individually and in combination, providing reasonable empirical support for their contributions.
  2. The framework improves an EOVLM using unlabeled imagery without human annotations or an external judge, and achieves the best average accuracy among the evaluated models on five remote sensing benchmarks. The quality audit, implementation details, and per-task results in the appendix also improve transparency.
Critical weaknesses
  1. TerraEvo relies heavily on answer agreement and heuristic reasoning-quality rewards, which do not necessarily reflect factual correctness. The quality audit reports 93% answer agreement but only 76% human-verified answer correctness, raising concerns about reinforcing shared errors during iterative self-evolution.
  2. While the paper demonstrates improvements in final benchmark accuracy, it provides limited analysis of how the Proposer and Solver capabilities evolve throughout training. Without tracking question quality, Solver performance, and their interactions across training iterations, it remains unclear whether the framework achieves sustained mutual improvement.
Questions for authors
  1. Given the gap between answer agreement and human-verified correctness reported in Appendix A.5, could the authors further analyze the relationship between self-consistency rewards and actual answer correctness? In particular, how effectively do the proposed quality supervision and composite rewards prevent shared errors from being amplified during iterative self-evolution?
  2. Could the authors provide training dynamics showing how Proposer question quality and Solver performance evolve over successive training steps? In particular, does the Proposer continuously generate more informative and challenging questions as the Solver improves, and do both roles exhibit sustained improvement rather than early saturation? Such analysis would help substantiate the central claim of mutual self-evolution.
  3. TerraEvo involves multi-turn Socratic dialogue, multiple Solver rollouts, quality verification, and replay-based training, which may introduce substantial computational overhead. Could the authors report the overall training cost (e.g., GPU hours or training FLOPs) and provide compute-matched comparisons with simpler self-play or self-evolution baselines? This would help clarify whether the performance improvements justify the additional computational cost.
7 comments
AuthorsOpenReview ↗

Response to Reviewer eBkT (1/2): agreement, correctness, and shared errors

We thank the reviewer for the accurate summary and for recognizing the framework components, the ablations, and the value of learning from unlabeled imagery. We appreciate the reviewer's concern and explain below what each mechanism does and what our experiments show. Unless otherwise stated, the analyses in this response use Qwen3-VL-4B-Instruct, and all changes are in blue in the revised manuscript.

Part 1 of 2: Weakness 1 and Question 1 (agreement, correctness, and shared errors)

We agree that responses can agree with each other and still be wrong.

What the two numbers measure. The 93% agreement in Appendix A.5 measures whether the responses to a question describe the same visible content, as judged by an external model. The 76% correctness measures whether the final answer is correct according to a human judge. Agreement differs from the answer agreement used in training, which depends on how many Solver responses give the same answer. Neither measure shows that an answer is correct, because several responses can make the same visual mistake.

What TerraEvo adds beyond agreement (Section 4.3.1). Question validation checks that a question is supported by the image and answerable. Answer verification checks that the majority answer is supported by the image. After warmup, a response enters the replay buffer only if both checks and the skill-specific agreement gate pass, so high agreement alone is not enough. A question that fails validation loses the Proposer's validity bonus but is still used in the regular Solver update. The CoT reward encourages explicit, specific reasoning and does not check the image. These mechanisms aim to reduce repeated learning from unreliable examples.

Evidence. We audited 100 randomly sampled training questions from each method, with the same human judge and criteria. A question is valid if it is well posed, unambiguous, and has one defensible answer.

SettingValid questionsCorrect answers
TerraEvo77%76%
TerraEvo without question validation53%59%
Naive self-play47%52%
EvoLMM51%49%

TerraEvo has fewer invalid questions and fewer incorrect answers than both baselines. The ablated run differs from TerraEvo only in the question validation check. Without it, valid questions fall from 77% to 53% and correct answers from 76% to 59%. This shows that the check reduces flawed supervision. The audit covers 100 questions per setting and uses one judge, and answer verification is not ablated separately.

The accuracy ablations point the same way (Table 2). Removing question validation lowers GeoBench-VLM accuracy from 43.6% to 41.8%, removing trajectory agreement gives 41.9%, and removing the CoT reward gives 42.0%. Accuracy also rises during training, from 41.44% at step 1,000 to 43.60% at step 3,000 (checkpoint table in our next comment).

Limitation. These results support cleaner supervision, but they are only indirect evidence about shared errors. We do not track whether individual shared errors become more or less frequent during training. The checks use the Solver itself, so shared mistakes can still pass them. We therefore call the checks safeguards and not guarantees. Human judgments are used only to audit the generated data and never as training rewards or labels.

Changes in the manuscript: Section 4.3.1 and Appendix A.5.

AuthorsOpenReview ↗

Response to Reviewer eBkT (2/2): training dynamics and computational cost

Part 2 of 2: Weakness 2 and Question 2 (training dynamics), and Question 3 (computational cost)

Training dynamics: Solver performance (new). We evaluated checkpoints saved during a TerraEvo run on the same GeoBench-VLM questions.

Step1,0001,5002,0002,5003,000
Average accuracy (%)41.4442.4243.0343.2343.60

Accuracy rises at every checkpoint, from 41.44% to 43.60%, with the largest gains early in training.

Training dynamics: question quality and skill coverage. Figure 3 and Appendix A.4.1 show that TerraEvo keeps broader skill coverage. Across steps 400 to 3,000, its mean normalized skill entropy is 0.937, compared with 0.727 for the generic Proposer. The human evaluation finds 77% valid questions for TerraEvo, compared with 47% for naive self-play and 51% for EvoLMM.

Training dynamics: Proposer and Solver interaction. Freezing the Proposer lowers GeoBench-VLM accuracy from 43.6% to 42.6% (Table 2). Removing Solver-conditioned follow-up questions lowers it to 42.1% (Table 3). Both adapting the Proposer and using Solver disagreement help the Solver.

Scope. These measures cover different things. Checkpoint scores track the Solver over time, and skill entropy tracks the balance of question types. The human evaluation samples questions from the whole training run, so it shows overall question quality and not how it changes over time. Together they support better Solver performance, maintained skill coverage, and useful Proposer adaptation. They do not show directly that questions become harder or more informative at every stage.

Computational cost (Question 3). We report the estimated time for 3,000 training steps on four NVIDIA A100 80GB GPUs. All configurations use Qwen3-VL-4B-Instruct, and accuracy is the GeoBench-VLM average from Tables 2 and 3.

ConfigurationEstimated GPU hoursAccuracy (%)
EvoLMM317.6640.1
iReasoner317.6740.3
TerraEvo, Kmax = 1 (single turn)719.4341.9
TerraEvo, Kmax = 3715.0842.3
TerraEvo, Kmax = 5 (full)920.9743.6
TerraEvo, Kmax = 71,269.3242.7
TerraEvo, Kmax = 5, without Solver-conditioned follow-ups879.3242.1

Full TerraEvo needs about 920.97 GPU hours in total, which is about 230 hours of wall clock time on four GPUs. This is about 2.9 times the cost of EvoLMM and iReasoner, which cost about the same. The extra cost comes from the larger work in each step. TerraEvo generates 14 Solver responses per question, up to 512 tokens each, while the iReasoner timing run uses five responses of up to 256 tokens. TerraEvo updates the Proposer every two steps instead of every five. Its regular Solver update trains on the seven highest-reward responses in one group update, while the iReasoner implementation we inspected makes five separate updates. Replay and quality checks stay active even with one dialogue turn. More turns are not always better: five turns give the highest accuracy (43.6%), while seven turns cost more (1,269.32 GPU hours) and give lower accuracy (42.7%). This supports our choice of five turns.

Changes in the manuscript: Section 5.3 (cost table and discussion) and Appendix A.4.1 (training progress).

Thank you for the detailed response and the additional experiments.

The new human audit comparing validity-check settings, the checkpoint accuracy curve, the GPU-hour report, and the added implementation details are helpful. Two main concerns remain:

  1. Shared errors are still not directly addressed. The audit shows that the validity check reduces flawed questions, but the authors acknowledge that this is only indirect evidence, since the checks are performed by the Solver itself and can share its visual errors. The audit is also small (100 questions per setting, one judge). In addition, questions that fail validation are still used in the regular Solver update, so a substantial share of invalid questions remains in training. The relationship between the self-consistency rewards and actual correctness, and whether shared errors are amplified over training, is not tracked directly.

  2. The evidence for sustained co-evolution is still limited. The new results track only the Solver’s accuracy on GeoBench-VLM, and the per-500-step gains shrink over training, which may indicate saturation. For the Proposer, there is no evolution over training of question quality, difficulty, or validity. Skill entropy measures the balance of question types, but it does not show that questions become more informative or challenging. The authors acknowledge that the current analysis does not show this directly, so the central claim of mutual improvement between the two roles remains insufficiently supported. Tracking Proposer-side metrics (e.g., validity rate, agreement, and difficulty over steps) would help.

Since these two concerns are only partly resolved, I keep my rating at 2.

Reviewer_H3SpOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TerraEvo explores how to train VLMs through self-evolution, utilizing only unlabeled remote sensing imagery. Based on the Qwen3-VL-4B-Instruct model, the framework employs separate LoRA adapters to establish a Proposer and a Solver. The Proposer generates questions targeting under-represented skills and initiates multi-turn follow-up inquiries based on disagreements in the Solver’s responses. The Solver, in turn, learns via rewards derived from answer consistency, the coherence of reasoning text, and heuristic Chain-of-Thought (CoT) processes. The system further integrates mechanisms for question validity verification, skill-adaptive KL regularization, and skill-balanced experience replay. The paper reports an improvement in average accuracy across five remote sensing benchmarks—from 51.7% to 54.2%—and an increase from 39.3% to 43.6% on GeoBench-VLM. While these results hold practical value, the current evidence is insufficient to fully substantiate the claims regarding reliable visual supervision, enhanced reasoning capabilities, and the sustained retention of knowledge.

Critical strengths
  1. The most valuable aspect of the design is the integration of skill-coverage control with follow-up questioning based on answer discrepancies, allowing question generation to remain responsive to both the training distribution and the current state of the solver. This represents a logical system extension beyond simple, single-turn self-questioning. While the core training paradigm and multi-objective reward structure have clear precedents, the innovation lies in the specific combination of mechanisms and their adaptation to the remote sensing domain.
  2. The experiments cover five remote sensing benchmarks and provide comprehensive task-level results. Tables 2–4 present ablation studies on factors such as question validity, skill guidance, replay mechanisms, reward components, dialogue length, and feedback conditions. The appendix includes an audit of the generated data quality. The main text is clearly structured, with framework diagrams, pseudocode, and validation prompts that facilitate an understanding of the overall process.
Critical weaknesses
  1. The reward function remains primarily composed of answer frequency from the same model, similarity in reasoning text, and superficial formatting features; it has not been demonstrated that these signals can effectively identify shared visual errors. Furthermore, several aspects of the trajectory reward—such as the “self-contained center” approach, the calculation of intra-group dominance following a filtering step, and the use of historical rewards for experience replay—introduce optimization issues that require further clarification.
  2. The separation of images between the training and evaluation sets is not sufficiently explained; the paper also lacks information regarding random seeds, error intervals, and the use of an independent validation set. Key implementation details are missing, including the source of reasoning embeddings, rules for answer normalization, the full prompts used for generation, the construction of feedback, sampling temperature, and image resolution settings.
  3. Existing related work has not been adequately contextualized or compared. The reward designs in EvoLMM and iReasoner are highly relevant to this study, and the extent to which the current approach builds upon or modifies these methods should be clarified. While VisPlay is cited, it is not included in the experimental comparisons. Additionally, for the mechanism claimed to preserve knowledge, there is a lack of direct evaluation regarding catastrophic forgetting.
Questions for authors
  1. How can the discrepancy between overall performance gains and the results on certain reasoning tasks be explained? While the paper reports an average performance improvement across five benchmarks, some reasoning-related results—detailed in the appendix—do not show corresponding gains; for instance, performance on the reasoning subset of RSHR-Bench shows a slight decline, and results on the reasoning category of LHRS-Bench remain on par with the base model. How do the authors account for this variation? Based on the current experiments, which specific capabilities benefit most from this approach, and how should the claim that “reasoning ability has improved” be precisely defined?
  2. To what extent does the existing evidence demonstrate that quality control effectively reduces erroneous supervision? The paper outlines check for question validity and answer verification, while acknowledging that internal validation may fail to detect errors shared by the models themselves. Although the quality audit in the appendix assesses the data generated by the full method, it lacks a corresponding baseline for comparison using the same evaluation criteria. Can the authors provide further analysis or experimental evidence to confirm that the inclusion of these checks actually leads to a reduction in flawed questions or incorrect answers? While the ablation study in Table 2 suggests that quality checks contribute to final accuracy, the claim that this occurs specifically through the reduction of erroneous supervision requires further clarification.
  3. Do the skill-adaptive KL penalty and skill-balanced replay help preserve the model’s existing capabilities? In Section 4.4.2, the authors employ these two mechanisms for the purpose of knowledge preservation, and the ablation study in Table 2 ind5icates that they contribute to higher final accuracy. However, an increase in accuracy alone does not distinguish between the acquisition of new capabilities and a reduction in the degradation of existing ones. Could the authors clarify which experimental results support the interpretation that these mechanisms facilitate knowledge preservation? Is the primary function of these designs to promote learning, or to mitigate forgetting?
7 comments
AuthorsOpenReview ↗

Response to Reviewer H3Sp (1/2): reward signals, quality control, and trajectory reward

We thank the reviewer for the careful review and for recognizing the skill-coverage control, the follow-up questions, and our ablations. We agree that our contribution is the combination of these mechanisms for Earth observation, not the agreement reward itself. All analyses use Qwen3-VL-4B-Instruct, and all changes are in blue in the revised manuscript.

Part 1 of 2: Weakness 1 (reward signals) and Question 2 (quality control)

1. Can the rewards detect shared visual errors? We agree that answer agreement alone cannot tell a shared mistake from a correct answer. TerraEvo adds two checks (Section 4.3.1). Question validation checks that a question is supported by the image, clearly written, answerable, and consistent with its skill. Answer verification checks that the majority answer is supported by the image. After warmup, a response enters the replay buffer only if both checks and the skill-specific agreement gate pass. A failed validation also removes the Proposer's validity bonus, but the question is not automatically excluded from the regular Solver update. Both checks use the Solver itself, so they can share its visual errors. The CoT reward scores the presence, specificity, consistency, and length of the reasoning text, and the trajectory reward compares reasoning texts. Neither checks the image.

2. Do the checks reduce flawed supervision (Question 2)? We audited 100 randomly sampled training questions per setting, with the same human judge and criteria.

SettingValid questionsCorrect answers
TerraEvo77%76%
TerraEvo without question validation53%59%
Naive self-play47%52%
EvoLMM51%49%

TerraEvo has fewer invalid questions and fewer incorrect answers than both baselines. These methods differ in many components, so the key test is the ablated run, which differs from TerraEvo only in the question validity check. Without the check, valid questions fall from 77% to 53% and correct answers fall from 76% to 59%. This controlled comparison supports the contribution of question validation to reducing flawed supervision. GeoBench-VLM accuracy also drops from 43.6% to 41.8% (Table 2). Agreement is not correctness, since our audit shows 93% response agreement but only 76% correct answers. The revised manuscript describes the checks as safeguards and not as guarantees. Human judgments are used only to audit the generated data and never as training rewards or labels.

3. Trajectory center and self-inclusion. We take the responses that give the majority answer and have nonempty reasoning. Each reasoning text is turned into an embedding (a vector for comparing texts) with unit length. The normalized average of these embeddings is the trajectory center, defined in Appendix A.2.1, Equation 12, and every response is compared with this same center. Each response is also part of the center. A normalized embedding has similarity 1 with itself, so this adds the same amount to every response in the group. The ranking therefore depends only on how well a response's reasoning agrees with the others. The score values can still shift, which changes the weight of the trajectory term in the composite reward. Removing the trajectory reward lowers GeoBench-VLM accuracy from 43.6% to 41.9% (Table 2). The trajectory reward measures agreement between reasoning texts and not correctness.

4. Majority group proportion and filtering. The proportion dd is the fraction of all MM sampled responses that give the majority answer. It is computed before any filtering. For example, if 10 of 14 responses give the majority answer, d=10/14d = 10/14. If two of them have empty reasoning, they are left out of the trajectory comparison, but dd stays 10/1410/14, because their answers still count for answer agreement. We then compute the composite rewards and keep the seven highest-reward responses for the regular Solver update. Their advantages use the reward mean and standard deviation of these seven responses (Eq. 1), and they receive one update. Neither step changes dd, so filtering does not inflate majority agreement.

5. Historical rewards in replay (Section 4.4.2). Replay reuses stored responses without generating or scoring them again. Each update uses one stored response, and its saved composite reward is used directly as the advantage, with no group normalization. The saved reward is the score given when the response was stored. It acts as a fixed training weight, not a new score from the current Solver. Each update also uses the current skill-adaptive KL penalty. Skill-balanced replay reaches 43.6% accuracy, compared with 42.9% for uniform replay (Table 2).

Changes in the manuscript: Sections 4.3.1 and 4.4, Appendices A.2 and A.5.

AuthorsOpenReview ↗

Response to Reviewer H3Sp (2/2): implementation details, related work, reasoning results, and forgetting

Part 2 of 2: Weaknesses 2 and 3, Questions 1 and 3

Weakness 2: data separation and implementation details. We added these details to the revised manuscript.

  • Data: training images are randomly selected from the training splits of the source train datasets. We checked that none of them overlap with the test benchmark images. We use only the images, not their labels.
  • Seeds and validation: all accuracy results use seed 42 and checkpoints at 3,000 steps. We use no validation set. Each configuration is trained with one seed, so we report no uncertainty intervals.
  • Reasoning embeddings: taken from the Solver's own input embedding layer. For each text, we average the embeddings of up to 64 tokens and scale the result to unit length.
  • Answer normalization: we first search for the answer marker followed by a letter from A to E. If this fails, we look for a trailing answer letter, and then for an isolated answer letter within the final 20 characters. If no letter is found, the answer is "unknown", and it is included when counting answer frequencies.
  • Generation: temperature 1.0 and top-p 1.0 for both roles. The Proposer generates at most 256 new tokens and the Solver at most 512.
  • Images: we convert images to RGB and pass them to the Qwen3-VL image processor. We do not resize images ourselves. The processor keeps the aspect ratio and keeps each image between 65,536 and 16,777,216 pixels, with patch size 16 and spatial merge size 2. These are its default settings for the 4B and 8B models.
  • Feedback and prompts: When the Solver answers disagree, the next turn gives the Proposer feedback that names the majority answer and the minority answers from the previous turn. The Proposer is asked to write a follow up question that helps tell these answers apart. The feedback does not give a known correct answer. All prompts are reported in the updated manuscript (Appendix A.7).

Weakness 3: related work, VisPlay, and forgetting. Naive self-play, EvoLMM, and iReasoner use answer agreement as the learning signal, and iReasoner also rewards agreement between reasoning steps. TerraEvo also controls which skills are practiced and refines questions using Solver disagreement. We trained EvoLMM and iReasoner with their released implementations, adapted to Qwen3-VL-4B-Instruct, on the same images. GeoBench-VLM accuracy is 40.2% for naive self-play, 40.1% for EvoLMM, 40.3% for iReasoner, and 43.6% for TerraEvo (Table 2). We could not find released VisPlay weights. A fair comparison needs retraining with the same backbone and images, which we cannot finish in the discussion period. VisPlay rewards uncertain and diverse questions, while TerraEvo tracks coverage across EO skills and uses Solver disagreement for targeted follow-ups. Related Work now compares these methods (Section 2.2).

Question 1: reasoning results and scope of the claim. We agree that a higher average does not mean every task improves. The clearest gains are in counting. General vehicle counting on GeoBench-VLM improves from 20.7% to 34.7%, and counting on MME-RealWorld-RS improves from 16.6% to 25.1%. On RSHR-Bench, perception improves from 35.5% to 36.3% and multi-turn from 72.4% to 74.3%, while reasoning drops from 54.8% to 54.2%. On LHRS-Bench, reasoning stays at 84.8%. The gains on several tasks outweigh the small drops on others, so the average rises. Our training signals reward skill coverage, follow up questions, and consistent answers and reasoning. They can raise average accuracy without improving every capability. More explicit reasoning text does not by itself mean better planning or causal reasoning, and our experiments do not show the cause of each change. We therefore limit our claim to this. TerraEvo improves average accuracy across the evaluated EO benchmarks, with clear gains on several counting and perception tasks and mixed results on reasoning tasks.

Question 3: do KL and replay preserve capabilities? Both mechanisms aim to keep earlier capabilities while the Solver improves. Removing skill-adaptive regularization lowers GeoBench-VLM accuracy from 43.6% to 42.1%, and uniform replay gives 42.9% (Table 2). These ablations do not separate new learning from retention. We therefore compared a saved TerraEvo run on the same 3,211 GeoBench-VLM questions at steps 1,000 and 3,000. Of the 1,604 questions correct at step 1,000, 1,532 are still correct at step 3,000 (95.51% retention). 120 incorrect answers become correct and 72 correct answers become incorrect, a net gain of 48. This analysis covers the full framework, so it does not show the separate effect of KL or replay. We describe them as supporting learning while aiming to preserve earlier capabilities, without claiming that either one alone explains the retention.

Changes in the manuscript: Sections 2.2, 4.4.2 and 5.1 to 5.3, the abstract, the Conclusion, and Appendices A.1, A.2.1, and A.7.

AuthorsOpenReview ↗

Additional Response to Reviewer H3Sp: quality checks and shared errors

We thank the reviewer again for this thoughtful comment. We agree that the effectiveness of quality control should be evaluated beyond the internal reward signals, particularly because validation performed by the same model may fail to detect shared visual errors. To provide further evidence, we analyzed the training log of our 3,000-step run (Qwen3-VL-4B-Instruct, seed 42), examined Proposer-side metrics throughout training, and conducted an additional human audit. No additional training was performed.

Human audit of the validation check

Two human judges reviewed 200 training questions sampled from the run, 100 that passed validation and 100 that failed it. The questions were presented in mixed order, without revealing their validation status. Among the questions that passed, 84 were judged valid and 16 invalid. Among those that failed, 56 were judged invalid and 44 valid. Overall, the validation check agreed with the human judgments on 140 of 200 questions (70%).

GroupQuestionsValid (%)Invalid (%)
Passed validation1008416
Failed validation1004456

This human evaluation provides direct evidence that the validation check separates valid from invalid questions, since questions that pass are valid more often (84%) than questions that fail (44%).

Failed questions during training

Across the 3,000-step run, 30.9% of selected questions (926 of 3,000) failed validation. Although these questions receive a lower Proposer reward and cannot enter replay, the regular Solver update was still applied in 91% of the steps involving failed questions. The share of selected questions that failed fell from about 36% in the first 1,000 steps to 23.4% in the final 500 steps.

Training stepsFailed validation (%)
0 to 50032.1
500 to 1,00039.2
1,000 to 1,50030.6
1,500 to 2,00027.8
2,000 to 2,50032.2
2,500 to 3,00023.4
Entire run30.9

Failed questions are invalid more often (56%) than passed ones (16%), so this fall suggests that fewer invalid questions reached the Solver update.

Proposer and Solver evolution

We also tracked Proposer-side metrics over training, from the log of the same run, for the question selected at each step. The last column is the Solver's GeoBench-VLM average at the end of each interval.

StepsPassed validation (%)Mean answer agreementSolver disagreement (%)Proposer reward rqr_qGeoBench-VLM average
0 to 50067.90.84650.90.45440.95
500 to 1,00060.80.85450.80.42241.44
1,000 to 1,50069.40.85252.60.45642.42
1,500 to 2,00072.20.85053.80.46543.03
2,000 to 2,50067.80.81760.00.48243.23
2,500 to 3,00076.60.83254.10.50443.60

The share of questions that pass validation rises from 67.9% in the first 500 steps to 76.6% in the last 500 steps, and the Proposer reward rises from 0.454 to 0.504. The Solver's answers differ on 51% of the selected questions in the first 1,000 steps and on between 54% and 60% in the last 1,000 steps, so the questions remain challenging. The Solver's accuracy rises at every checkpoint, from 40.95 at step 500 to 43.60 at step 3,000.

What the results show

TerraEvo is a self-evolving method and uses no labels during training, so shared errors cannot be verified directly. The human audit and the Proposer's quality checks give indirect evidence. The validation check agrees with the human judges on 70% of the audited questions, and fewer failed questions reach the Solver update. Over training, the Proposer produces more valid questions and more challenging questions for the Solver in later steps, and the Solver keeps improving, with its accuracy rising at every checkpoint. As in our first reply, we describe the checks as safeguards and not as guarantees.

Changes in the manuscript: Appendices A.4.1 and A.5.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their constructive feedback and for appreciating the clear motivation, the combination of skill coverage control and Solver-conditioned follow-up questions, learning from unlabeled imagery, the strong results on five benchmarks, and the comprehensive ablations and quality audit. We ran several additional analyses and experiments to address the main concerns. All changes are in blue in the revised manuscript. All analyses use Qwen3-VL-4B-Instruct unless stated otherwise.

  • Quality of supervision (H3Sp, eBkT). We audited 100 sampled questions per setting with a human judge. TerraEvo has 77% valid questions and 76% correct answers. Naive self-play has 47% and 52%, and EvoLMM has 51% and 49%. Removing only the question validation check lowers valid questions from 77% to 53% and correct answers from 76% to 59%. The checks use the Solver itself, so we call them safeguards and not guarantees.
  • Training dynamics (eBkT). GeoBench-VLM accuracy rises from 41.44% at step 1,000 to 43.60% at step 3,000. TerraEvo's questions also cover skills more evenly than those of a generic Proposer (skill entropy 0.937 vs. 0.727, where higher means more even).
  • Knowledge preservation (H3Sp). Of the 1,604 questions answered correctly at step 1,000, 1,532 (95.51%) are still correct at step 3,000. This result is for the full framework, so it does not show the separate effect of KL regularization and replay.
  • Computational cost (eBkT). We report estimated GPU hours for all configurations. Full TerraEvo costs about 2.9 times more than EvoLMM and iReasoner and has higher accuracy (43.6% vs. 40.1% and 40.3%).
  • Clarifications (H3Sp, eBkT). We added implementation details and all prompts (Appendices A.1 and A.7). We explained the trajectory center, the majority group proportion, and replay rewards. We also clarified our reasoning claim, since gains are clearest in counting and mixed on reasoning tasks.
  • Larger model (LZjP). TerraEvo trained from Qwen3-VL-8B-Instruct improves over its base model on all five benchmarks. The average gain is +1.5 points, compared with +2.5 points for 4B.
  • Remaining limitations. We use one seed (42) and no validation set, so we give no uncertainty intervals. We did not test other model families. We could not compare with VisPlay because we found no released weights.

Detailed replies are in our separate comments to each reviewer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.