acceptodds
Under review as a conference paper at ICLR 2027

ConfAL-WM: Confidence-Guided Active Learning for Action-Conditioned World Models

Abstract

Action-conditioned world models have become an important foundation for embodied prediction, planning, and synthetic data generation, but their errors under new task and scene distributions are often concentrated in localized spatiotemporal regions such as robot arms, manipulated objects, contact areas, and occluded objects. This paper presents ConfAL-WM, a confidence-guided active learning framework for post-training embodied world models. Built upon EVAC, we attach a lightweight confidence probe to UNet decoder features and predict dense confidence maps in the latent space. These maps are aggregated into task-, frame-, and patch-level scores, enabling both efficient data selection and localized training enhancement. Our pipeline first retrains the confidence probe and warms up EVAC with a small subset of target-domain data, then performs task-level prescreening to allocate sampling budgets, and finally applies selected-data retraining with optional frame or patch weighted data enhancement. Experiments on RoboTwin2.0 show that confidence-guided selection improves post-training efficiency, while dense frame and patch weighting further enhances prediction quality and embodied trajectory consistency compared with scalar reward, progress, and judge-based scoring baselines. A quick visual overview of this work is available at https://ConfAL-WM.github.io.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Action-Conditioned World Model, Active Learning, Confidence

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NTGdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an active learning method for fine-tuning a UNET based video world model called EnerVerse-AC (EVAC), which is trained on robotics data. Authors propose to train a confidence predictor on top of the frozen model. This confidence map is then used to select episodes comprising the most difficult examples (lowest confidence), and for increasing weights on most difficult frames and patches at training time. Experiments on a new robot dataset show some performance gains compared to the original model, as well as the model trained on a randomly sample dataset.

Critical strengths
  • Actively selecting the most relevant samples with a fixed budget is a great experiment towards making better use of compute and training data, in situations were budgets are restricted. Choosing samples where models are most uncertain could be a valuable strategy.
  • The confidence estimation procedure is sensible, and experiments suggest some gains to the sample selection approach.
  • Authors carried a very large set of experiments to evaluate their approach in great detail, included results across multiple seeds, and comparison to several state of the art models.
Critical weaknesses
  • The confidence computation section in 3.1 essentially follows the methodology described in the C3 paper (Mei et al, 2026). The main differences claimed in the introduction are the use of a U-Net vs a DiT (the main needed change here seems to be deciding where embeddings are taken) and an EMA-based threshold supervision (which shows no clear advantage in experiments). While the introduction does mention that confidence estimation follows C3, the original work should be credited accurately in the methods section, and the core changes required should be highlighted clearly.
  • As a result, the novelty is limited: the main innovation relates to leveraging confidence information for pre-selection of training samples, which is a narrower contribution. Results show incremental gains compared to a EVAC v1 model and frame/patch weighting yield mixed results. For example, frame weighting tends to have worse results than no weighting at all.
  • I have concerns over using confidence as a way of measuring episode difficulty or areas to focus training on. This does not seem to take into account potential stochastic behaviours, which could lead to low confidence if multiple outcomes are equally valid, yet will be valid predictions. Isn’t there a risk to over learn on specific outcomes (especially with a fixed selection budget) and obtain poorly calibrated models?
  • In section 4.2, authors use pixel/latent error as a ground truth reference to evaluate their confidence/risk prediction. However, those measure two separate quantities (thinking again about stochastic events), and while pixel error can be useful for context, it should not be used as a ground truth for confidence. I am not sure that this section answers the “why confidence?” question, but rather verifies that confidence estimation is sensible.
  • Minor: The related work could be structured better, with different approaches grouped together with no clear connections.
Questions for authors
  • I find it surprising that the Random baseline does worse than EVAC v1, since it is trained on a larger dataset. Is there a particular selection process for the v1 dataset? Is the assumption that training the conference probe in parallel helps?
    How were training budgets chosen? 6,248 episodes for v1 and 7,298 for retraining seems very specific. It would also be good to understand how the approach compared to a model finetuned on the full dataset. Is there a remaining gap? Is focusing on hard examples only better?
  • Are stochastic events taken into account in the method design? Have authors verified impact of their approach on world model calibration?
  • Sample selection is carried out with a different model than the one eventually evaluated. Why not add new samples during training instead of doing this in two stages? This could also enable curriculum type approaches.
Reviewer_XPHzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose an active data selection method that uses dense confidence maps for world model post-training. The method builds off an uncertainty quantification framework that uses strictly proper scoring rules to train video world models for correctness and calibration. For the main results, the authors evaluate the correlation between confidence scores and prediction errors across different scales (task, frame, patch) and whether confidence-guided selection and weighing improve reconstruction quality, semantic agreement, and trajectory consistency.

Critical strengths
  1. Using interpretable uncertainty quantification scores for localized confidence-guided post-training is a well-motivated and practical approach.
  2. The authors provide a rigorous analysis of the key design choices, including confidence-prob features, adaptive threshold supervision, and risk aggregation strategies.
Critical weaknesses
  1. While the experiments show that confidence is useful for estimating prediction error, they do not establish that confidence identifies the training examples with the greatest marginal learning value. The evaluation is also limited to offline video-generation and trajectory-consistency metrics. It is not clear that improvements in these proxy metrics translate into better controllability, planning, or policy performance. Downstream evaluation would help establish the practical value of the proposed active-learning method.
  2. The statistical evidence for superiority over other methods is insufficient. Table 5 reports paired improvements over EVAC-v1 rather than direct comparisons against other baseline acquisition strategies.
Questions for authors

Refer to weaknesses.

Reviewer_xPPzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ConfAL-WM, which attaches a lightweight confidence probe to the UNet decoder of an action-conditioned world model (EVAC) and uses the predicted dense risk for task-level data selection and for frame/patch-weighted retraining. ConfAL-WM leads seven of nine metrics among selection-only methods on RoboTwin2.0 at the 40% budget (Table 2), and in particular raises Scene Consistency to 0.9196 against 0.9057 for the strongest baseline (GVL).

Critical strengths

S1. The motivation is concrete. The paper starts from the observation that world-model errors are spatially and temporally localized (Section 1, paragraph 1), and the choice of a dense confidence map follows directly from that observation.

S2. The experiments are organized around two separable questions. Section 4.2 checks whether the confidence tracks the oracle error before Section 4.3 checks whether using it helps retraining, so the two claims can be judged independently.

Critical weaknesses

W1: The dense confidence map is barely used for data selection. The main claim is that dense, patch-level confidence provides information that scalar reward, progress, or judge models cannot (Section 1, paragraph 5). However, selection in Section 3.2 reduces the map to one task-level mean risk estimated from a few representative episodes, followed by random sampling within each task. The scalar baselines in Table 2 pass through the same task-quota allocator, so the selection-level comparison tests only the quality of one scalar per task, not the dense map. In my view, episode-level selection with S_mean or S_tail over the full candidate pool is the natural test of the claim. The ablation of selection criteria (Table 3) also compares only aggregations of the same scalar and does not address this point.

W2: The reported gains are small relative to run-to-run variation and come with metric trade-offs. In Table 2, confidence-guided selection improves over the strongest baseline by roughly 0.01 on PSNR (0.6746 vs. 0.6630) and SSIM (0.7692 vs. 0.7603), while the seed-42 run in Table 3 differs from the three-seed average in Table 2 by 0.0092 on PSNR and 0.0132 on Traj-HSD. Frame-and-patch weighting lowers Scene Consistency (0.8942) below EVAC-v1 (0.9047) and below selection-only confidence (0.9196). Table 2 reports no standard deviation or confidence interval, so it is unclear which of the differences between methods are reliable. Reporting per-metric standard deviations in Table 2, together with the same three-seed protocol for Table 3, would make the main comparison interpretable.

Questions for authors

Q1. In Table 2, Random selection-only retraining scores below Base EVAC (Warmup v1) on PSNR (0.5968 vs. 0.6446), SSIM (0.6849 vs. 0.7309), and Logics (0.3554 vs. 0.5537), although it is trained on more episodes (7,298 vs. 6,248). How should this be interpreted? Does retraining from the original checkpoint on randomly selected target data hurt, and if so, how much of the gain of each method comes from the selection signal rather than from avoiding this degradation?

Q2. Do the three seeds (42, 3407, 123) change the selected episodes, or only the optimizer?

1 comment

The response has resolved all of my concerns. I am therefore raising my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.