- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how optimization experience can be reused across heterogeneous world models. VERDI characterizes each model with paired inference-time probe responses, retrieves compatible operators using Interventional Repair Geometry (IRG), and confirms their effects through target-side experiments under a frozen verifier with protected-metric checks. Reliable opposite outcomes for nearby models trigger additional probes, admitted through held-out replay. The paper reports leave-one-backbone-out transfer, selector and verification ablations, and model-specific video/control improvements, including a small real-robot study. Its central contribution is the proposed co-evolution of reusable operators and the behavioral representation used to select them.
Critical strengths
-
The separation between transfer prioritization and target-side verification is well motivated. A favorable source prediction does not establish a target improvement, and utility gains cannot compensate for protected regressions. The paired confirmation, explicit support conditions, and retention of null/harmful outcomes make the proposed optimization loop more informative than simply collecting successful recipes. The mixed Ctrl-World and Cosmos-Predict 2 outcomes in Appendix C are useful examples of this distinction.
-
Using cheap intervention responses to guide expensive operator trials is a promising design. The reported raw-response, metadata, fixed-executor, and verification controls provide useful initial evidence, while the simulated control results connect selected operators to downstream utility. I also appreciate that the small Franka experiment is presented with uncertainty and that the combined RL + self-forcing result is not attributed to either component alone.
Critical weaknesses
-
The scope of the evidence supporting the main transfer claims is inconsistent. Table 3 and Section 4.5 describe 216 decisions over 27 campaigns, whereas Appendix B.12 explicitly attributes the same Table 3 point estimates and their confidence intervals to 72 decisions over nine campaigns on three backbones. Likewise, Table 5 is labeled as a nine-backbone study, but Table 8 presents the same ablation values on the three-backbone subset. Section 4.1 and Figure 5 also describe nine-backbone evaluation, while Appendix B generally assigns archive-prefix replay to the detailed subset. The paper needs to distinguish these results and their actual bootstrap units. Tables 6-7 list all nine targets, but the numerical per-target breakdown in Table 9 covers only three. This matters because transfer between heterogeneous interfaces is a central claim; the current reporting does not let me assess how much of the headline evidence comes from the other six targets.
-
The experiments do not yet isolate the distinctive benefit of contradiction-driven representation refinement. Figure 7 compares four fixed probes, fourteen freely added probes, and seven gated probes; it therefore changes probe count and admission policy together. It supports careful admission, but does not establish that verified contradictions select better probes than another selection strategy using the same candidate pool, budget, and held-out gate. The three extra directions in Figure 8 and the locality example in Appendix C.8 do not resolve this comparison.
Questions for authors
-
Which results in Tables 3/5 and Figures 5/6 use all nine backbones, and which use the three-backbone subset? Please reconcile Appendix B/B.12 with the main text and provide the missing per-target numerical results, together with the actual cluster counts used for each confidence interval.
-
Does contradiction-driven probe selection improve held-out operator-selection regret over a control that uses the same candidate probes, admits the same number, and applies the same replay/calibration gate without selecting from verified opposing outcomes? Please specify the discovery, promotion, calibration, and final evaluation split sizes used for Figure 7 and report uncertainty for the comparison.
-
How is the learned-embedding baseline in Table 3 trained and calibrated? Can you compare IRG with a response-informed transfer surrogate, such as RGPE or a Bayesian task embedding adapted to the compatible compiled-operator pool, while holding the archive, executor, verifier, and probing budget fixed? A risk comparison at matched transfer coverage would also help identify the contribution of the selection representation.
6 comments
Response to Reviewer ekCZ (2/3): Matched-coverage comparison and the RGPE baseline
- Question 3: How is the learned-embedding baseline trained and calibrated, and how does IRG compare with response-informed RGPE at matched coverage?
Answer: Thank you for requesting the baseline details and a matched-coverage comparison. At matched coverage of 102/216 decisions, IRG has 5.9% negative-transfer risk, compared with 11.8% for response-informed RGPE and 14.7% for the learned embedding.
(1) The comparison holds the transfer and verification conditions fixed. IRG and RGPE share the initial historical archive, compatible compiled-operator pool, operator registry, probes and query budget, executor, frozen verifier, and stopping rule. Additional target observations consume the same search budget. Table 28 reports harmful counts of 6, 12, and 15 among 102 accepted transfers, sign accuracies of 179/216, 171/216, and 160/216, and trial means of 3.83, 4.25, and 4.60 for IRG, RGPE, and the learned embedding, respectively.
| Selector | Sign accuracy | Trials | Coverage | Harmful / accepted | Protected violations |
|---|---|---|---|---|---|
| IRG | 179/216 = 0.829 | 3.83 | 102/216 = 47.2% | 6/102 = 5.9% | 0.03 |
| Response-informed RGPE | 171/216 = 0.792 | 4.25 | 102/216 = 47.2% | 12/102 = 11.8% | 0.08 |
| Learned embedding | 160/216 = 0.741 | 4.60 | 102/216 = 47.2% | 15/102 = 14.7% | 0.10 |
(2) The response-informed RGPE configuration is specified in Table 30. Compatible historical source observations train up to eight source GPs per fold; a target GP uses only observations completed before the current prediction. Inputs include budgeted response descriptors, aligned operator encodings, and missing-value masks; the target is continuous utility change. Each GP uses a constant mean and ARD Matérn-5/2 kernel. Ensemble weights use minimum-ranking-loss probabilities estimated from 256 posterior samples, with leave-one-out target predictions; fewer than two target observations trigger equal source weights. Candidates are ranked by ensemble expected improvement within the same finite compatible pool. GP fitting costs are recorded separately, and the original frozen verifier determines final outcomes.
| RGPE component | Experimental setting |
|---|---|
| Source models and candidate space | One GP for each remaining source backbone with compatible historical observations, up to 8 per fold; evaluate only operator configurations within the same task-goal and response schema that pass the shared compiler's compatibility checks |
| Inputs and regression target | Budgeted response descriptors, semantically aligned operator types and parameter encodings, and missing-value masks; predict continuous utility changes under the original goal contract with consistent utility direction; scaling uses available training data only |
| GP and noise | Single-output exact GP, constant mean, ARD Matérn-5/2 kernel with an output-scale parameter, and float64; use measured repeat variance when available, otherwise learn Gaussian-likelihood noise |
| Fitting | Maximize the exact marginal log likelihood with L-BFGS-B, up to 200 iterations, and record convergence status |
| Weights | Estimate the probability of minimum ranking loss using 256 posterior samples; use leave-one-out predictions for the target model and split tied contributions equally |
| Cold start | With fewer than 2 valid target observations, combine compatible source models with equal weights without estimating weights from target ranking loss |
| Ranking | Rank by ensemble expected improvement relative to a predefined positive-utility threshold, selecting only within the fixed finite candidate pool |
| Verification and calibration boundary | The original verifier determines positive/null/harmful outcomes and protected metrics; posterior probability of exceeding the utility threshold does not itself establish confirmation, and any additional probability calibration is restricted to independent source calibration data |
| Resources and sweep | Observation queries count against the common budget and GP fitting costs are recorded separately; use scores saved before outcome revelation for an offline risk–coverage sweep at accepted counts of 43, 72, 102, 144, 180, and 216 |
Figure, table, and appendix references are to the revised manuscript.
Response to Reviewer ekCZ (3/3): Learned embedding and risk-coverage results
- Question 3 (continued): How is the learned-embedding baseline trained and calibrated, and how does IRG compare with response-informed RGPE at matched coverage?
Answer: Thank you for requesting the baseline details and a matched-coverage comparison. IRG has fewer harmful transfers at every partial-coverage operating point; all three selectors accept the same outcomes at full coverage.
(1) The learned embedding predicts utility-change direction from historical transfer metadata. Table 31 lists binned historical GPU-h cost, source family, campaign time intervals, cumulative transfer success/failure counts, and discrete backbone metadata as features. Training uses utility-change direction labels and inverse-frequency-weighted BCEWithLogitsLoss. Temperature scaling uses an 8:2 training/validation split within historical training data. All 216 held-out decisions are excluded from fitting and calibration. The logit sign predicts effect direction; the verifier's positive/null/harmful labels remain distinct from this prediction target.
| Component | Protocol |
|---|---|
| Input features | Binned historical transfer cost in GPU-h, source-family encoding, campaign time intervals, cumulative transfer success/failure counts, and discrete backbone metadata; encoded features are concatenated into a fixed-length vector |
| Training target | Predict the effect direction of an individual transfer; labels are derived from the direction of observed utility changes under the target contract, and the logit sign gives the predicted direction |
| Loss | BCEWithLogitsLoss with inverse-frequency class weights computed from the training set |
| Calibration | Temperature scaling using an 8:2 training/validation split within the training data; only a global temperature parameter is fitted |
| Data separation | Historical time-window split; all 216 held-out decisions form a separate test set, and training and calibration exclude their samples, labels, and features |
(2) IRG has fewer harmful transfers at every partial-coverage operating point. Table 29 reports accepted counts of 43, 72, 102, 144, 180, and 216. IRG harmful counts are 2, 5, 6, 18, 45, and 73; RGPE counts are 4, 7, 12, 29, 53, and 73; learned-embedding counts are 5, 11, 15, 39, 61, and 73. At full coverage, all methods accept the same 216 decisions and therefore share 73 harmful outcomes. Scores are saved before the corresponding outcomes are revealed; thresholds are not fitted to test outcomes. Accepted null outcomes remain in the risk denominator. The full protocol and comparisons appear in Appendix E.3.
| Accepted decisions | IRG harmful | RGPE harmful | Learned embedding harmful |
|---|---|---|---|
| 43 | 2 | 4 | 5 |
| 72 | 5 | 7 | 11 |
| 102 | 6 | 12 | 15 |
| 144 | 18 | 29 | 39 |
| 180 | 45 | 53 | 61 |
| 216 | 73 | 73 | 73 |
Figure, table, and appendix references are to the revised manuscript.
Response to Reviewer ekCZ (1/3): Scope, uncertainty, and matched-budget probe selection
Answer: Thank you for identifying the conflicting scope descriptions. The revised main transfer results cover all nine backbones; Appendix B.12 reports the original three-backbone subset.
(1) The conflicting descriptions have been reconciled. Tables 3/5, Figures 5/6, and Appendix Tables 8/9 use nine backbones, 27 campaign-seed units, and 216 decisions. Table 9 now reports all nine targets, including per-target correct signs, search costs, accepted outcomes, coverage, and risk. Main-study totals are 179/216 correct signs and 102 accepted transfers, comprising 68 positive, 28 null, and 6 harmful outcomes.
(2) The confidence intervals now specify their actual clusters. Figure 5 and Appendix E.1 use nine backbone clusters, with Table 25 reporting full-study intervals. Appendix B.12 retains the original three-backbone, nine-campaign, 72-decision results and their intervals. Trials/GPU-h use the hierarchical average over tasks, repeated searches, successful campaign-seed units, and finally equally weighted backbones; completion counts retain all 27 units. These rules are stated in the introduction to Appendix B.
(3) Additional backbone visualizations are directly accessible in the revised manuscript. Appendix E.5 contains DreamerV3, V-JEPA 2, DINO-WM, and TD-MPC2 in Figures 16–19, indexed in Table 33 with results in Tables 34–35. Their evaluation units are specified separately, and none is added to the 216 main-study decisions. Real-robot results appear in Section 4.2 and Figure 4; additional paired FRANKA videos are available in the project page's Real robot section.
Answer: Thank you for requesting a controlled test of contradiction-driven selection. At the same seven-probe budget, candidate pool, and replay/calibration gate, contradiction-driven selection lowers held-out regret relative to both random selection and replay-gain selection.
(1) The new comparison holds probe budget and admission conditions fixed. The original Fixed/Free/Gated settings change both probe count and admission policy. In Appendix E.2, all three selection rules use seven probes, the same candidate pool, admission thresholds, and final evaluation set. Table 26 reports regret/calibration error of 0.30/0.15 for random selection, 0.26/0.11 for replay-gain, and 0.21/0.08 for contradiction-driven selection.
(2) The regret contrasts have positive cluster-level intervals. Table 27 reports comparator-minus-contradiction differences. Relative to random selection, the regret reduction is 0.09, with 95% CI [0.078, 0.102]; relative to replay-gain, it is 0.05 [0.021, 0.070]. Calibration error improves by 0.07 [0.058, 0.087] versus random selection. Versus replay-gain, the calibration difference is 0.03 [−0.002, 0.061]; this interval includes zero. The intervals are unadjusted and resample nine backbone clusters.
(3) The revised protocol states the split sizes. Nine-fold outer LOBO allocates four source backbones to discovery, two to promotion, two to calibration, and one held-out target to final evaluation in each fold. With three seeds per backbone, the stages contain 12, 6, 6, and 3 campaign-seed units, respectively. Final evaluation spans nine backbone clusters. These details are provided in Appendix E.2 and referenced in the Figure 7 caption and main text.
Figure, table, and appendix references are to the revised manuscript; substantive revisions are marked in dark blue. Rollout videos are indexed on the anonymous project page.