acceptodds
Under review as a conference paper at ICLR 2027

RISE: Reliable Improvement in Self-Evolving Vision-Language Models

Abstract

Vision-language models (VLMs) have achieved strong multimodal reasoning capabilities, but further improving them still relies heavily on large-scale human-constructed supervision for post-training. Such supervision is costly to obtain, especially for reasoning-intensive multimodal tasks where questions, answers, and feedback signals must be carefully designed. This motivates self-evolving learning, where a model improves itself through a dual-role closed loop: a questioner autonomously poses questions and a solver learns to solve them. However, we observe that current VLM self-evolving methods still face three major challenges: coarse-grained role alternation delays the interaction between question generation and solver adaptation; generated questions can progressively degrade in quality; and question types may collapse toward a narrow distribution. These issues limit the efficiency and reliability of self-evolution. Thus, we propose RISE, a reliable self-evolving framework for vision-language models. RISE is built on three complementary designs: fine-grained role alternation, which shortens the feedback loop between the questioner and the solver to improve efficiency; a quality supervisor, which improves question validity and pseudo-label reliability; and skill-aware dynamic balancing, which mitigates mode collapse and maintains broad skill coverage during evolution. Together, these components enable more reliable and effective self-evolution from unlabeled images. Experiments on two VLM backbones across seven benchmarks show that RISE consistently improves the base models, yielding broad and sustained gains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Vision-Language Models, Self-Evolution, Pseudo-Supervision, Multimodal Reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_dDK1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RISE, a self-evolving training framework for vision-language models, aiming to address delayed feedback updates, degradation in generated question quality, and insufficient task diversity in existing dual-role self-evolving methods. The framework leverages unlabeled images to construct self-supervised training signals through alternating optimization of a questioner and a solver. Specifically, Fine-grained Role Alternation divides longer role-training phases into multiple shorter cycles to accelerate feedback between question generation and solving capabilities. Quality Supervisor reuses the current solver as a supervision model to assess the validity of generated questions and the reliability of pseudo-labels. Skill-aware Dynamic Balancing adjusts rewards according to the generation frequency of different skill categories and maintains training data diversity through stratified sampling. The authors conduct experiments on Qwen2.5-VL-7B and Qwen3-VL-8B using the Vision-47K image collection, optimizing the questioner and solver separately through GRPO. Across seven multimodal understanding and reasoning benchmarks, after 60 steps of self-evolving training, the weighted average scores of the two models improve from 48.17 to 52.26 and from 53.38 to 57.37, respectively. The paper further validates the contributions of the proposed components through ablation studies, generated data analysis, and repeated training experiments.

Critical strengths
  1. Table 2 in Section 5.3 shows that, on Qwen2.5-VL-7B, when fine-grained role alternation is applied without quality supervision, the average score decreases from 50.43 at Step 20 to 44.85 at Step 60, even falling below the initial model's score of 48.17. After introducing quality supervision, the same setting achieves a score of 51.71 at Step 60. Together with the changes in generated question quality and skill distribution shown in Figure 3, these results provide concrete evidence of quality degradation during self-evolving training and demonstrate the practical role of quality control in maintaining training stability. The paper designs corresponding mechanisms around problems observed during training, establishing a meaningful connection between the research motivation and experimental analysis.

  2. Table 1 shows that RISE achieves an overall improvement of approximately 4 percentage points on two different versions of Qwen-VL models, with gains on multimodal reasoning tasks such as MathVerse and MathVision. Appendix C.4 further validates the contributions of the three components on Qwen3-VL-8B, while Appendix C.5 reports an average score of 57.44 with a standard deviation of 0.28 across five independent runs. In addition, the authors reproduce VisPlay as a controlled baseline and examine part of the experimental results using benchmark-specific evaluation rules in Appendix C.6. These experiments provide relatively strong support for the effectiveness of the method and the stability of the reported results.

Critical weaknesses
  1. Section 4.4.1 constructs the question difficulty reward based on the consistency of the solver's answers, but this design cannot reliably distinguish genuinely challenging reasoning questions from ambiguous ones. According to Equations (9) and (10), when 5 out of 10 sampled responses produce the same answer, the difficulty score reaches its maximum value of 0.5. However, such disagreement may arise either because a question is near the model's capability boundary or because the question is poorly specified or lacks sufficient visual evidence. More importantly, Equation (11) only incorporates question validity as a reward term with a weight of 0.2, without imposing a hard constraint on semantically invalid questions. This means that an invalid question with a high difficulty score may still receive a higher reward than a simple but valid question. Although the subsequent data filtering process checks question validity, it cannot completely eliminate the incorrect incentives introduced during questioner training. Therefore, more direct validation is needed to determine whether the proposed reward mechanism genuinely encourages the model to generate high-quality questions near its capability boundary, rather than merely questions with high answer uncertainty.

  2. Section 4.3.1 constructs the Quality Supervisor by reusing the current solver to verify generated questions and pseudo-labels, but this design may introduce correlated errors between the solver and the supervisor. According to Section 4.4.2, pseudo-labels are obtained through majority voting over 10 sampled solver responses, while Equation (13) allows samples with consistency scores between 0.3 and 0.8 to proceed to subsequent verification. This means that some candidate answers may receive support from only 3 sampled responses and still have a chance to enter the training data after passing supervisor verification. When the solver systematically misjudges certain types of visual questions, the same model acting as the supervisor may also approve incorrect answers, allowing erroneous supervision signals to be repeatedly reinforced during subsequent self-evolution. Although Appendix C.1 evaluates the supervisor using Qwen3.5-Plus as an external reference, the average precision is only 0.72 and the recall is 0.885, while this validation still relies on judgments from another model. The paper does not further report independently human-verified accuracy for the final retained pseudo-labels or analyze error accumulation across different skill categories. Therefore, the existing evidence supports the effectiveness of quality filtering to some extent, but is insufficient to demonstrate that this shared-parameter self-verification mechanism can reliably suppress long-term error propagation.

  3. The Skill-aware Dynamic Balancing mechanism in Section 4.3.2 primarily computes rewards based on the six skill categories declared by the questioner and their recent generation frequencies, and constructs the training set through approximately uniform sampling. This approach can adjust the number of samples across categories, but a more balanced category distribution does not necessarily imply richer reasoning content in generated questions or more balanced improvements in model capabilities. In particular, the difficulty levels and task formats within each of the six skill categories may vary substantially, and controlling category frequencies alone cannot prevent the generation of many similar questions within the same category. Furthermore, the existing experiments ablate the skill-frequency reward and training-stage stratified sampling together as a single component, without separately analyzing their contributions. Table 2 shows that introducing the complete skill-balancing mechanism improves the average scores by only 0.56, 0.09, and 0.55 percentage points at Steps 20, 40, and 60, respectively, compared with Fine-grained + Sup. Although these results indicate that the mechanism provides some benefits, the paper does not further report actual changes in solving capabilities across different skills, nor does it provide an independent comparison with a simple uniform sampling strategy. Therefore, it remains difficult to determine whether the performance improvements arise from dynamic skill regulation or from more conventional balancing of the training data distribution.

  4. Section 4.2 presents Fine-grained Role Alternation as an important design for improving self-evolving efficiency and conducts comparisons under the same number of parameter update steps. However, equal numbers of update steps do not necessarily imply equal computational costs. More frequent switching between the questioner and solver increases the number of question generation, pseudo-label construction, and supervisor verification operations. In fact, Appendix C.8 reports that, under the same backbone model, image collection, and update budget, the complete RISE training process takes 73 hours, compared with 65 hours for the reproduced VisPlay, representing a 12.3% increase. Although RISE improves the average score from 49.96 to 52.26, this does not directly demonstrate an advantage in actual computational efficiency. Furthermore, Appendix C.3 shows that when the alternation cycle is further shortened to b=1b=1, the score at Step 20 reaches 50.94, higher than the 50.64 achieved by the default b=5b=5. However, at Step 60, the score instead decreases to 50.71, below the 51.71 achieved by b=5b=5. This indicates that more frequent alternation does not always lead to better results. The paper should further distinguish parameter-update efficiency from actual training efficiency, clarify the rationale for selecting a particular alternation frequency, and examine whether the performance advantage remains under matched computational costs.

  5. Section 5 mainly validates RISE through comparisons with different role alternation strategies and component ablations, but its substantive incremental contribution over existing self-evolving methods remains insufficiently established. The three core components respectively shorten alternation cycles, filter low-quality samples based on model judgments, and adjust the sampling distribution according to category frequencies. These designs are closer to a combination of improvements to existing training procedures. Although Appendix C.9 provides reference comparisons with Vision-Zero, EvoLMM, and MM-Zero, and also reproduces VisPlay, the results for the other methods come from different training settings. The authors also explicitly acknowledge that these cross-paper results cannot support a rigorous ranking of methods. The only controlled reproduction, VisPlay, corresponds to a coarse-grained alternation setting without quality supervision or skill balancing. Therefore, the advantage of the complete model simultaneously includes the benefits of multiple training strategies. The existing ablations demonstrate that these components have individual effects, but comparisons with other simple self-training strategies under matched computational budgets are still lacking, such as using a fixed questioner, applying only quality filtering, or adopting static skill balancing. Consequently, it remains unclear how much of RISE's advantage comes from the proposed dual-role coordination mechanism and how much could be achieved through conventional data filtering and training scheduling. This limits my assessment of the methodological novelty and actual incremental contribution of the work.

  6. Section 5.2 describes RISE as achieving stable and sustained self-evolving improvements, but the current experiments provide limited evidence for long-term evolution and cross-model generalization. The main experiments use only two models from the same Qwen-VL family, Qwen2.5-VL-7B and Qwen3-VL-8B, and a single image source, Vision-47K. These settings are insufficient to demonstrate that the method would have similar effects across different model architectures, pretrained capabilities, and image distributions. Appendix C.7 further extends Qwen3-VL-8B training to 100 steps. Although the average score increases from 57.37 at Step 60 to 58.05 at Step 100, the additional 40 training steps yield only a 0.68-percentage-point improvement, while performance on several benchmarks fluctuates or declines. Meanwhile, Appendix C.2 shows that Qwen2.5-VL experiences substantial performance degradation without the supervisor, whereas the stronger Qwen3-VL can still maintain some improvement. This suggests that training stability may depend considerably on the base model's own ability to generate and recognize correct answers. Therefore, the current experiments support the effectiveness of RISE in improving self-evolution within the tested models and training range, but further evidence is needed to establish whether it can reliably sustain improvements over longer training periods and under more diverse model conditions.

Questions for authors

Please refer to Critical Weaknesses.

10 comments
AuthorsOpenReview ↗

Response to dDK1 (Part 1)

Thank you for your careful reading and constructive feedback. We address your questions and concerns point by point below and will incorporate the relevant supplementary results into the revised manuscript.

W1: Whether the difficulty and validity rewards encourage high-quality, learnable questions.

A: Thank you for your detailed analysis of the reward design. First, we would like to clarify that λv=0.2\lambda_v=0.2 is the additive coefficient of the validity term; it does not mean that validity accounts for only 20% of the total reward. Its actual influence must be understood in relation to the numerical scales of the other reward terms. During training, we observed that the mean difficulty reward is typically below 0.1 in the early stages and approximately 0.2–0.3 in the later stages as self-evolution stabilizes. In comparison, every question that passes the validity check receives an additional reward of 0.2. The validity term therefore provides a strong preference for quality early in training and remains comparable in magnitude to the difficulty reward later on. It is not a negligible correction relative to the difficulty term.

The motivation for this setting is to promote basic question quality early in training through a strong validity incentive. Once the questioner can satisfy the validity requirements relatively consistently, the difficulty term provides an additional optimization preference, encouraging it to explore valid questions that are more challenging for the current solver. We agree that an additive reward does not impose a strict priority for validity, so some invalid questions may still receive higher rewards than simple valid questions, although such cases represent a small proportion. Our goal is to reduce the tendency to generate low-quality questions and their proportion in the training data, rather than to claim that this soft constraint eliminates every instance of misaligned incentives. During data construction, candidates must also pass both validity and pseudo-label correctness checks before being used for solver training.

Regarding the more direct validation you requested, the valid-and-correct rate (VCR) in Figure 3 is specifically designed to assess whether questions meet quality requirements while retaining uncertainty for the solver. Here, valid requires a clear question, consistency between its task type and content, relevance to the image, and an answer that depends on sufficient information provided in the image; correct requires the pseudo-label used for training to be the correct answer. These properties are assessed by a strong external model that is not involved in training. Thus, VCR goes beyond measuring disagreement among the solver's answers to examine the quality of both the questions and their supervision targets. We regard samples that pass the [0.3, 0.8] consistency window and satisfy both valid and correct as learnable candidates that pose a moderate challenge to the current model. Figure 3 shows that removing quality supervision causes the proportion of high-quality samples to decline substantially during training, falling below 10% in the later stages for Qwen2.5-VL. With quality supervision, this proportion remains at a higher level. Downstream results are consistent with this change: under the same fine-grained alternation setting, with skill balancing disabled in both cases, adding quality supervision improves the Step-60 average score from 44.85 to 51.71. Together, these results support the contribution of the full quality-control mechanism in terms of both sample quality and learning outcomes.

AuthorsOpenReview ↗

Response to dDK1 (Part 2)

W2: Does the shared-parameter supervisor lead to compounding errors? (Part 1)

A: Thank you for raising the concern about correlated errors arising from parameter sharing. We agree that the supervisor, instantiated from the current solver, is not an independent external judge and therefore cannot completely eliminate self-confirmation errors. For example, the solver and supervisor may overlook the same visual detail or agree on an internally consistent but incorrect answer. We adopt this design to provide effective quality checks on generated samples without deploying an additional supervisor model, rather than to guarantee that every accepted pseudo-label is correct.

Mechanistically, sharing parameters does not mean that generation and verification perform the same task. The solver generates an answer from the image and question; the supervisor additionally receives the candidate answer and uses a separate verification prompt to check whether it is supported by the visual evidence and satisfies the question's requirements. Providing the candidate answer allows the model to check specific inconsistencies, giving it an opportunity to identify errors in its own generated answers. We use this judgment as a filtering signal: candidates that fail verification are rejected, while passing verification is not treated as proof of correctness. Acceptance instead depends jointly on this check, question-validity assessment, and consistency filtering. The lower bound of the consistency window only determines whether a candidate enters subsequent checks; it does not imply that a small number of supporting answers is sufficient to guarantee label reliability.

Furthermore, verification responses are not themselves the GRPO training targets, and the solver receives no direct reward for approving its own answers. Thus, the mechanism does not directly optimize the supervisor's tendency to accept its own outputs. Nevertheless, the shared parameters change as the solver is updated. Once an incorrect pseudo-label passes the checks and enters training, it can indirectly influence generation and verification in the next round. We therefore acknowledge that an indirect feedback path remains, and its actual effect needs to be assessed through quality audits during training. Our existing process statistics do not show a sustained deterioration in overall supervision quality during self-evolution.

Table: Qwen2.5-VL supervisor precision and recall

Training step51015202530354045505560Mean
Precision0.680.600.620.650.690.660.740.650.700.680.670.660.667
Recall0.700.750.730.670.690.770.720.700.650.730.740.670.710

The external audit in Appendix C.1 and the additional Qwen2.5-VL audit above show fluctuations and differences across backbones in the supervisor's detection capability, but no sustained downward trend.

We also conducted an additional external audit using the stronger Qwen3.8-max. For each backbone, we sampled 600 finally accepted questions from each of six training stages: Steps 0–10, 10–20, 20–30, 30–40, 40–50, and 50–60. We assessed question validity and pseudo-label correctness separately and specifically measured the answer error rate among valid questions. Thus, the denominator in the tables is the number of questions judged valid by the external evaluator within the corresponding stage or skill category. Step denotes the final step of each training stage.

Table: Pseudo-label error rates by training stage

BackboneStep 10Step 20Step 30Step 40Step 50Step 60
Qwen2.5-VL-7B20.43%24.37%26.08%23.95%22.20%25.22%
Qwen3-VL-8B18.63%17.13%17.31%17.19%17.44%19.53%

These results show that residual errors remain, but no sustained increase is observed across the examined stages, even for the weaker model. The VCR curves in Figure 3 also show that including quality supervision better maintains training-data quality than removing it.

AuthorsOpenReview ↗

Response to dDK1 (Part 3)

W2: Does the shared-parameter supervisor lead to compounding errors? (Part 2)

A: We further provide the analysis of error accumulation across skills that you requested. As above, these are pseudo-label error rates among valid questions.

Table: Qwen2.5-VL-7B --- pseudo-label error rates by skill

SkillStep 10Step 20Step 30Step 40Step 50Step 60
Coarse Perception15.08%3.57%10.53%15.92%13.16%5.88%
Fine-grained Perception18.39%25.49%29.29%30.77%27.89%24.32%
Instance Reasoning21.50%18.18%28.57%22.73%8.33%21.43%
Logical Reasoning16.98%16.67%18.18%26.32%16.67%11.11%
Math & Counting20.83%27.55%25.77%25.00%22.99%29.63%
Science & Technology22.37%32.73%30.51%23.81%33.33%25.00%

Table: Qwen3-VL-8B --- pseudo-label error rates by skill

SkillStep 10Step 20Step 30Step 40Step 50Step 60
Coarse Perception16.67%15.34%13.27%16.05%13.76%15.00%
Fine-grained Perception23.16%18.81%17.24%12.50%16.67%10.71%
Instance Reasoning11.65%16.04%23.53%14.04%9.09%5.00%
Logical Reasoning7.34%13.64%15.19%6.35%10.00%17.85%
Math & Counting16.52%16.96%10.84%13.77%23.81%23.04%
Science & Technology38.64%20.00%26.32%28.03%17.50%20.00%

These results show fluctuations across stages and skills, but no uniform, sustained increase in error rates during training. Thus, we do not observe the pattern of compounding deterioration you are concerned about.

To assess the reliability of the external evaluations, we additionally performed a manual check. We sampled 50 finally accepted questions across training stages, manually examined question and pseudo-label quality, and checked agreement with the stronger model gpt6astra. Human judgments agreed with Qwen3.5-Plus on 46 questions, yielding 92% agreement, and agreement with gpt6astra was 88%. Disagreements mainly concerned answer granularity rather than clearly invalid questions or unequivocally incorrect answers. This check provides supplementary support for the external evaluations, although its sample size is limited and the results are for reference only.

We also conducted a frozen-supervisor control experiment. The evaluation results for a frozen supervisor and the shared supervisor updated alongside the solver on Qwen2.5-VL are shown below.

Model / stageMMMUMMVetRealWQAChartQAMathVerseMathVisionMathVistaAVG
Base Model49.7151.8357.1279.4840.4823.2158.7048.17
Frozen supervisor — Step 2053.5654.5958.0481.2042.6327.1859.8050.61
RISE — Step 2054.0357.8057.5281.6843.6527.3660.4051.20
Frozen supervisor — Step 4054.8457.3459.2282.4043.7328.1360.1051.68
RISE — Step 4054.0356.8858.3081.8444.1928.7360.1051.74
Frozen supervisor — Step 6054.7355.9660.5282.3244.1628.4760.2051.94
RISE — Step 6055.3157.8060.3982.0844.9528.3261.1052.26

The two settings yield similar results, and the differences may partly reflect training randomness. The frozen supervisor shows no performance advantage in this comparison. These results complement the audits during training and support the feasibility of reusing the current solver as the supervisor over the tested training range, while avoiding an additional frozen supervisor model copy.

Our method cannot completely eliminate erroneous samples, but the sample-quality analyses and ablation results support its practical contribution: under the same fine-grained alternation setting, adding quality supervision raises Qwen2.5-VL's Step-60 average score from 44.85 to 51.71 and substantially mitigates the degradation of intermediate training-data quality.

Reviewer_C1KyOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses improving vision-language models from unlabeled images without human-annotated supervision. Prior work such as R-Zero and VisPlay casts this as a dual-role loop where a questioner proposes image-grounded questions near the solver’s capability boundary and the solver learns from majority-voted pseudo-labels. The authors argue these methods alternate roles in long phases that delay feedback, and that difficulty-driven rewards let question quality degrade and question types collapse toward counting and estimation. RISE keeps the same loop but splits the update budget into shorter alternation cycles, reuses the current solver as a prompt-based supervisor that filters invalid questions and unverified pseudo-labels, and adds a frequency-based bonus plus stratified sampling over six skill categories. Validation consists of 60-step training on Qwen2.5-VL-7B and Qwen3-VL-8B evaluated on seven benchmarks with an LLM judge (about +4 weighted average on each), component ablations on both backbones, an alternation-granularity sweep, an external audit of supervisor precision and recall, five-seed variance, rescoring under benchmark-native evaluators, and a cost comparison against the authors’ own VisPlay reproduction.

Critical strengths
  1. The paper documents a concrete, previously unreported failure mode of VLM self-evolution: fine-grained role alternation alone accelerates collapse (Table 2, AVG 50.43 to 44.85 by Step 60, below the base model), and Figure 3 and Figure 4 show the mechanism, namely difficulty-only rewards driving the questioner toward ambiguous estimation questions. This matters because it turns a design choice that looks like a free efficiency gain into one that requires explicit quality control, which is useful guidance for anyone building these loops regardless of whether RISE itself is adopted.

  2. The experimental reporting is unusually complete for this line of work: five independent runs with standard deviations (Table 5), a controlled rescoring of fixed outputs under benchmark-native evaluators (Table 6) showing the gain direction survives the scorer change, 100-step extended training, and matched wall-clock cost. Self-evolution papers are frequently single-seed and judge-dependent, so this level of reporting makes the core claim of consistent improvement over the base model credible even where the comparison to prior methods is not.

Critical weaknesses
  1. The only controlled baseline is the authors' own ablation. The "reproduced VisPlay" in Table 10 is simply the Step 60 "Coarse-grained w/o Sup." row of Table 2, i.e., RISE with two components removed, not VisPlay's released code or recipe. Whether its reward design, filtering rules, and hyperparameters match the original VisPlay is not stated. All other comparisons (Vision-Zero, EvoLMM, MM-Zero) are cross-paper gains under different image sets, budgets, base checkpoints, and scorers; the authors' own Appendix C.6 shows that changing the scorer alone shifts the aggregate gain from +4.86 to +7.16, which exceeds the margins in Table 10. The paper therefore provides no evidence that RISE outperforms any prior method under matched conditions.

  2. The difficulty reward does not identify the capability boundary. A self-consistency of 0.5 is equally produced by (a) questions the solver is on the verge of learning, (b) ambiguous or unanswerable questions where responses split into two camps, and (c) answer-format variation. The reward cannot distinguish these, and the paper delegates the distinction entirely to the supervisor, whose precision is 0.72. No analysis checks, with an external model, what fraction of the retained "boundary" questions are genuinely learnable versus ill-posed, nor how this fraction evolves across cycles.

  3. The shared supervisor is unconstrained and only partially audited. Because the supervisor is the solver's current weights, every GRPO update to the solver silently changes judging behavior with no reward governing it, and errors the solver makes systematically are the ones it is least able to reject. The paper acknowledges correlated errors but the external check (Appendix C.1) is reported only for Qwen3-VL, the strong backbone; the Qwen2.5-VL curve, where correlated errors should be most damaging, is absent. The natural control (a frozen external or initial-base supervisor) is not run.

  4. The [0.3, 0.8] consistency threshold is fixed, unexplained, and coupled to both M and model capability. No sensitivity analysis or per-cycle statistics on how many samples this filter removes are given. Since the solver's consistency rises as it improves, the window selects a different difficulty band at each stage and likely shrinks the usable pool over time, which may be an unexamined cause of the diminishing returns attributed in Section 5.2 to maj@n. On a new domain or distribution the window would need to be retuned.

Questions for authors
  1. Does the "reproduced VisPlay" in Table 10 run VisPlay's released code and reward/filter configuration, or is it the Step 60 "Coarse-grained w/o Sup." row of Table 2 obtained by removing two RISE components? If the latter, please state which VisPlay design choices differ. A matched run of at least one prior method (VisPlay or MM-Zero, same backbone, image set, budget, and Qwen3-Max scoring) showing RISE still ahead would raise my score; confirmation that the only controlled baseline is RISE's own ablation would lower it.

  2. Please report the supervisor precision and recall curve (Appendix C.1 protocol) for Qwen2.5-VL, where correlated solver-supervisor errors should be most harmful. Also report, for a single checkpoint, the fraction of accepted samples whose pseudo-label is wrong according to Qwen3.5-Plus, since the current false-negative rate is normalized over problematic candidates rather than over accepted data.

  3. How was the consistency window [0.3, 0.8] chosen? Please give, per alternation cycle, the number of candidates removed by this filter and the resulting consistency histogram. If the usable pool shrinks as the solver stabilizes, this would be an alternative explanation for the diminishing gains attributed to maj@n in Section 5.2.

6 comments
AuthorsOpenReview ↗

Response to C1Ky (Part 1)

Thank you for your careful reading and constructive feedback. We address your questions and concerns point by point below and will incorporate the relevant supplementary results into the revised manuscript.

W1/Q1: VisPlay Baseline and Evaluation

A: The VisPlay result in Table 10 comes from the Step-60 experiment corresponding to "Coarse-grained w/o Sup." in Table 2. We constructed this baseline using VisPlay's open-source implementation, and RISE was developed on the same codebase. The baseline follows the training schedule in the released 7B scripts: three alternation rounds, with 20 updates per role in each round. Both roles use GRPO, retaining the consistency-based difficulty reward for the questioner, majority-voted pseudo-labels, and the [0.3, 0.8] consistency filtering window. RISE's quality supervisor and skill-balancing module are disabled in the baseline. During training, we observed that the BLEU-based repetition penalty introduced by VisPlay had a very small magnitude, so we removed it from the reported baseline. Comparisons with and without this penalty also yielded similar performance gains. Both RISE and our reproduced VisPlay omit this penalty, so this modification relative to the original VisPlay implementation is shared by both sides of our controlled comparison.

In our controlled comparison, both models are initialized from the same Qwen2.5-VL-7B-Instruct checkpoint and use the same Vision-47K image set, rollout batch size, and update budget of 60 steps per role. Both models are evaluated on the same seven benchmarks using the same Qwen3-Max protocol. RISE improves the weighted average score from 49.96 to 52.26 (+2.30 percentage points), with higher scores on all seven benchmarks. We believe this reproduction faithfully follows the key settings of the original VisPlay method, including role alternation, the difficulty reward, and pseudo-label construction. The backbone, reward and filtering configurations, image collection, training budget, and Qwen3-Max evaluation conditions you specified are all matched. This baseline was precisely the starting point from which we identified the key problems discussed in our paper: delayed feedback, degradation in generated-question quality, and collapse of the skill distribution.

Appendix C.6 fixes the outputs of the Qwen3-VL base model and RISE and rescores them using Qwen3-Max and the benchmark-specific rule-based evaluators, respectively. The direction of improvement is consistent across all five benchmarks, with weighted average gains of +4.86 and +7.16 percentage points, respectively. Rule-based evaluation on some benchmarks relies on exact matching or predefined answer-normalization rules, which have limited flexibility in handling semantically equivalent answers expressed differently. One purpose of using an LLM judge is to reduce the risk of apparent gains arising merely because the model's output better matches these rules. Therefore, all primary benchmark results in the paper, including the direct comparison between RISE and the reproduced VisPlay, use the same Qwen3-Max LLM-judge protocol. The published results of Vision-Zero, EvoLMM, and MM-Zero come from their respective data, training budgets, and evaluation settings and are included only as background references.

AuthorsOpenReview ↗

Response to C1Ky (Part 2)

W2: The proportion of learnable questions.

A: We agree that answer consistency alone cannot distinguish reasoning difficulty, question ambiguity, and differences in answer formulation. The current solver serves as the supervisor during training, and its judgment capability is indeed limited. Our goal is therefore to minimize the number of problematic samples entering solver training. The precision of 0.72 in Appendix C.1 means that 72% of the samples rejected by the supervisor are judged problematic by the external reference model. The corresponding recall of 0.885 indicates that the supervisor identifies most of the problematic samples identified by the external model. These results demonstrate that the filtering is useful, although it also rejects some good questions and misses some problematic ones. Its practical contribution is also evident in Figure 3 and Table 2. Under the same fine-grained alternation setting, removing the supervisor causes Qwen2.5-VL's valid-and-correct rate to fall below 10% in the later stages, with a weighted average score of 44.85 at Step 60. Adding the supervisor improves data quality and raises the Step-60 average score to 51.71. We therefore consider its contribution clear, while recognizing that its verification capability still has room for improvement.

Regarding the "learnable questions" you ask about, we assess two aspects separately in the paper: valid and correct. Valid assesses whether the question itself is well-formed: its meaning is clear, its task type matches its content, it is relevant to the image, and answering it requires information provided in the image. Correct assesses whether the corresponding pseudo-label is the correct answer. If a question passes both the valid and correct checks, and the current solver's answer consistency falls within [0.3, 0.8], we regard it as a learnable question near the current solver's capability boundary. Such a question provides a clear and correct training target, while the solver still does not answer it correctly consistently. Accordingly, the valid-and-correct curves in Figure 3 report the proportion of questions retained during self-evolution that satisfy these criteria, as assessed by a stronger external model. This is precisely the proportion you are asking about. If you have further perspectives on the concept of learnability, we would be happy to provide additional analysis.

AuthorsOpenReview ↗

Response to C1Ky (Part 3)

W3/Q2: Supervisor reliability and errors in accepted pseudo-labels (Part 1).

A: Following the same protocol as Appendix C.1, where positive examples are problematic samples (invalid questions or questions with incorrect answers), we additionally report the supervisor's precision and recall across training stages on Qwen2.5-VL-7B, using Qwen3.5-Plus as the external reference.

Training step51015202530354045505560Mean
Precision0.680.600.620.650.690.660.740.650.700.680.670.660.67
Recall0.700.750.730.670.690.770.720.700.650.730.740.670.71

The mean precision is approximately 0.67, and the mean recall is approximately 0.71, both lower than the corresponding values of 0.72 and 0.885 for Qwen3-VL, with a particularly pronounced gap in recall. This is consistent with your concern about weaker backbones and indicates that the supervisor's ability to identify problematic samples is constrained by the current model's capabilities.

To further locate this limitation, we separately analyzed the supervisor's question-validity assessment and answer-correctness checking capabilities. We randomly sampled 20 cases per round and measured recall for each task against the external evaluation. Qwen2.5-VL's mean recall was 78% for question-validity assessment and 59% for answer-correctness checking. This indicates that it can reasonably identify whether a question is meaningful, image-relevant, and answerable from the image, as this task is relatively easier. Answer-correctness checking is the more pronounced weakness. Verifying a candidate answer requires reliable image understanding and the corresponding reasoning, making this stage more susceptible to the base model's limitations and more likely to miss systematically incorrect answers generated by the model itself. Nevertheless, this check is useful: it does identify some problematic samples containing the model's own incorrect answers. During questioner training, we only assess question validity, which provides a basic level of reliability for supervision at this stage.

We assessed the quality of the finally accepted samples using Qwen3.5-Plus. Figure 3 shows that Qwen2.5-VL's valid-and-correct rate at Step 60 is 54%, meaning that 46% of the accepted samples fail to satisfy both question validity and pseudo-label correctness. This proportion includes invalid questions and incorrect answers.

To further report the pseudo-label error rate you requested, we also conducted an external audit using the stronger Qwen3.8-max. For each backbone, we sampled 600 finally accepted questions from each of six training stages: Steps 0–10, 10–20, 20–30, 30–40, 40–50, and 50–60. We assessed question validity and pseudo-label correctness separately and specifically measured the answer error rate among valid questions. Thus, the denominator is the number of questions judged valid by the external evaluator within the corresponding stage or skill category. The step in the table denotes the final step of each training stage.

BackboneStep 10Step 20Step 30Step 40Step 50Step 60
Qwen2.5-VL-7B20.43%24.37%26.08%23.95%22.20%25.22%

These results show that some incorrect pseudo-labels remain after quality filtering. Across the audited checkpoints, the error rates fluctuate but do not exhibit a sustained upward trend.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to PGpU (Part 1)

Thank you for your careful reading and constructive feedback. We address your questions and concerns point by point below and will incorporate the relevant supplementary results into the revised manuscript.

W1: When quality filtering provides useful supervision, particularly in weaker domains.

A: Thank you for raising this key question. We believe that whether self-generated supervision provides an effective learning signal depends first on whether the model can already produce useful, correct solutions to the target problems. When the correct answer has a higher generation probability than competing answers, majority voting can reduce the randomness of individual samples and turn correct but unstable responses into more stable supervision targets. Thus, the role of Maj@n is to extract a more reliable consensus from the model's existing answer distribution, rather than to assume that an answer is correct merely because the responses agree.

Building on this, the quality filter further improves the composition of the accepted data. Its effectiveness does not require every judgment by the supervisor to be correct. The key condition is that the benefits from reliable samples outweigh the harm from problematic samples. The ability to distinguish these samples can increase the proportion of valid-and-correct examples in the filtered data. RISE separately checks whether questions are well-defined and supported by sufficient visual evidence, and whether candidate answers are supported by the image and question, thereby reducing the invalid questions and incorrect answers that consistency filtering alone may retain.

For skills in which the base model is weaker, two situations should be distinguished. If the model already possesses some relevant capability but produces correct solutions inconsistently, multiple sampling, candidate-answer verification, and quality filtering can still provide useful training signals. Weaker problem-solving ability does not mean that the model is entirely unable to assess question quality: identifying whether a question relates to the image, whether sufficient information is available, or whether an answer contains an obvious contradiction are different tasks from generating a complete, correct solution. The filter therefore still has an opportunity to reject some unreliable samples. Conversely, if the model persistently makes the same systematic error on a class of problems and also fails to recognize it during verification, majority voting may stabilize that error rather than correct it. In this case, high consistency cannot substitute for genuine supervision quality.

The experiments on the two backbones reflect this capability dependence. Qwen2.5-VL exhibits substantial degradation in the later stages without the supervisor, whereas adding quality supervision raises its Step-60 average score from 44.85 to 51.71. The stronger Qwen3-VL still benefits from some high-quality samples even without the supervisor and improves further when the supervisor is included. These results indicate that models differ in their tolerance for noisy supervision, while quality control provides practical benefits at different levels of foundational capability.

Our focus is therefore on whether the filtered data provide more reliable and useful learning signals than the original candidates, rather than requiring every pseudo-label to be completely correct or relying on a single accuracy threshold that applies to all models and skills. The validity of the finally accepted samples, their answer error rates, and how these measures change during training provide direct evidence for assessing this condition. Downstream ablations further test whether these quality improvements translate into actual learning gains.

AuthorsOpenReview ↗

Response to PGpU (Part 2)

W1/Q1: Errors in accepted pseudo-labels across backbones, training stages, and skills.

A: Thank you for your question. We agree that this is an informative analysis.

We conducted an external audit using the stronger Qwen3.8-max. For each backbone, we sampled 600 finally accepted questions from each of six training stages: Steps 0–10, 10–20, 20–30, 30–40, 40–50, and 50–60. We assessed question validity and pseudo-label correctness separately and specifically measured the answer error rate among valid questions. Thus, the denominator in the tables is the number of questions judged valid by the external evaluator within the corresponding stage or skill category. Step denotes the final step of each training stage.

Table: Pseudo-label error rates by training stage

BackboneStep 10Step 20Step 30Step 40Step 50Step 60
Qwen2.5-VL-7B20.43%24.37%26.08%23.95%22.20%25.22%
Qwen3-VL-8B18.63%17.13%17.31%17.19%17.44%19.53%

Table: Qwen2.5-VL-7B --- pseudo-label error rates by skill

SkillStep 10Step 20Step 30Step 40Step 50Step 60
Coarse Perception15.08%3.57%10.53%15.92%13.16%5.88%
Fine-grained Perception18.39%25.49%29.29%30.77%27.89%24.32%
Instance Reasoning21.50%18.18%28.57%22.73%8.33%21.43%
Logical Reasoning16.98%16.67%18.18%26.32%16.67%11.11%
Math & Counting20.83%27.55%25.77%25.00%22.99%29.63%
Science & Technology22.37%32.73%30.51%23.81%33.33%25.00%

Table: Qwen3-VL-8B --- pseudo-label error rates by skill

SkillStep 10Step 20Step 30Step 40Step 50Step 60
Coarse Perception16.67%15.34%13.27%16.05%13.76%15.00%
Fine-grained Perception23.16%18.81%17.24%12.50%16.67%10.71%
Instance Reasoning11.65%16.04%23.53%14.04%9.09%5.00%
Logical Reasoning7.34%13.64%15.19%6.35%10.00%17.85%
Math & Counting16.52%16.96%10.84%13.77%23.81%23.04%
Science & Technology38.64%20.00%26.32%28.03%17.50%20.00%

These results show fluctuations across training stages and skills, but no uniform, sustained increase in error rates during training. Thus, we do not observe the pattern of compounding deterioration you are concerned about. Qwen2.5-VL's error rate increases initially and then mainly fluctuates across stages, while Qwen3-VL's overall trajectory is relatively stable. The skill-level results further show improvements in some skills during the later stages. Math & Counting has relatively higher error rates in the later stages. One possible explanation is that these question types offer more opportunities for reward hacking: the questioner can obtain high difficulty rewards by eliciting answer disagreement even when the resulting pseudo-labels are unreliable. This mismatch between difficulty rewards and supervision quality may contribute to the higher pseudo-label error rates in this category. Other question types, such as coarse perception and logical reasoning, tend to have lower error rates, indicating that errors also depend on question type. Overall, residual errors vary substantially across skills, rather than showing a shared deterioration in supervision quality across all skills.

These fluctuations should also be interpreted in light of changes in the training-data distribution. Dynamic skill regulation changes the focus of question generation across stages, so the audits do not examine exactly the same task distribution at every stage. When challenging skills occupy a larger share of a stage's data, the aggregate error rate can change even if the within-skill error rates remain unchanged. Image content and question difficulty may also vary within the same skill. Thus, some stage-level fluctuations may reflect changes in the composition of generated tasks. Additionally, the comparison between the two backbones further indicates that pseudo-label reliability depends on both backbone capability and the specific skill.

Taken together, these audits directly connect the reliability analysis to the samples actually retained for training. Across the examined stages, the error rates among valid accepted questions do not exhibit a consistent, system-wide pattern of progressive amplification. These observations address your concern about a deteriorating feedback loop and support the stability of pseudo-label quality among valid accepted questions over the tested stages.

AuthorsOpenReview ↗

Response to PGpU (Part 3)

Q2: A dynamically updated shared supervisor versus a frozen supervisor.

A: We added the control you suggested: using the same initial base model as the solver to instantiate the supervisor, keeping its weights frozen throughout training, and continuing to update the questioner and solver. The results on Qwen2.5-VL are shown below.

Model / stageMMMUMMVetRealWQAChartQAMathVerseMathVisionMathVistaAVG
Base Model49.7151.8357.1279.4840.4823.2158.7048.17
Frozen supervisor — Step 2053.5654.5958.0481.2042.6327.1859.8050.61
RISE — Step 2054.0357.8057.5281.6843.6527.3660.4051.20
Frozen supervisor — Step 4054.8457.3459.2282.4043.7328.1360.1051.68
RISE — Step 4054.0356.8858.3081.8444.1928.7360.1051.74
Frozen supervisor — Step 6054.7355.9660.5282.3244.1628.4760.2051.94
RISE — Step 6055.3157.8060.3982.0844.9528.3261.1052.26

Both settings improve the base model's performance and yield similar overall results. The frozen supervisor shows no clear advantage in downstream performance. The shared version achieves slightly higher average scores at all three stages, but we primarily interpret these results as indicating comparable practical performance between the two approaches; the small differences may also reflect training randomness.

Our initial choice of the shared design was primarily motivated by model deployment costs. During questioner training, the questioner undergoes GRPO updates, while the current solver generates multiple answers to provide the difficulty signal. Deploying an additional independent frozen supervisor would require maintaining three sets of model weights simultaneously: the questioner, the current solver, and the frozen supervisor, increasing GPU memory requirements and deployment overhead. The shared design reuses the already deployed solver to check question validity and answer correctness through different verification prompts, saving the resources needed for an additional supervisor model copy.

Furthermore, during the initial design stage, we compared a frozen supervisor and a supervisor updated alongside the solver on the same set of questions. We observed similar judgment performance, with no clear advantage for the frozen version. The present training comparison further shows that the shared, updated supervisor can maintain learning outcomes comparable to those of the frozen version. Thus, in response to your concern about error amplification, the above experiments and analyses of questions generated during training show no sustained error amplification associated with shared updates over the examined training range.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.