- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MERC-WM, a task-free continual world model that uses paired probes, mechanism-event replay cards, and low-rank experts to handle changing physical dynamics. Experiments on three manipulation tasks show improvements in adaptation, memory retention, and mechanism recovery.
Critical strengths
1.The problem is interesting and relevant to continual robot learning.
2.The evaluation includes multiple metrics, matched baselines, and useful ablations. The improvements in false expansion and recovery are encouraging.
Critical weaknesses
-
The framework mainly combines existing techniques, including change detection, compact replay, and expert routing. I'm not fully convinced by the algorithmic novelty. Also, closely related methods such as SPREAD, ReDRAW, and LLCD are discussed but not directly compared. The success gain over the matched Residual-MoE baseline is only 1.8 points.
-
All experiments are simulation-based. The method also relies on controlled source interventions and calibrated safe probes, which may be difficult to obtain in real environments. The main evaluation involves only three physical mechanisms, so I'm not sure how well it would generalize to more complex real-world changes.
Minor Comments
1.The method relies on a fixed visual encoder, which may limit adaptation to unseen observation changes.
2.The memory-efficiency claims would benefit from clearer reporting of actual memory usage versus configured budgets.
3.Appendix Table 6 reports slightly different false-expansion and recovery values for the default configuration compared with the main results. Please clarify.
Questions for authors
1.How does MERC-WM compare against SPREAD, ReDRAW, and LLCD under matched settings?
2.Can the authors validate the method on real robots or less controlled physical changes?
3.How sensitive is the method to inaccurate probe calibration or unavailable safe probes?
2 comments
Response to Reviewer EQzK (Part 2/2): Beyond 3 Mechanisms, Probe Sensitivity, Frozen Encoder, and Memory/Table 6 Clarifications
In this second part, we address generalization to more complex/less controlled shifts (W2 & Q2), sensitivity to probe calibration and unavailable safe probes (Q3), and the three minor comments.
Response to W2 & Q2: Generalization Beyond 3 Mechanisms and Less Controlled Physical Changes
Our evaluation already extends well beyond 3 discrete, controlled mechanisms:
- Complex & Uncontrolled Stress Regimes (Appendix S23, Table 30): We explicitly evaluate 80-step gradual continuous ramps (no abrupt boundaries), simultaneous coupled visual+physical shifts, 10% action delay, and 10% sensor dropout. MERC-WM maintains
76.3–78.4%success and79.3–85.1%event F1 across all six stress suites. - 8-Mechanism Capacity Saturation (Appendix S21, Tables 26–27): Under 8 distinct mechanisms with randomized recurrence orders and appearances, MERC-WM achieves
75.0%(4 experts) and80.1%(8 experts), outperforming matched Residual-MoE by+3.9to+4.1 pp. - Real-Robot Readiness Without Controlled Source Renderings: As detailed in our response to Reviewer Mfuc, learning on hardware does not require simulator re-rendering—applying strong photometric/camera augmentations directly to real transitions achieves
10.1%visual FAR and82.6%ablation success (retaining>90%of the gain). While physical robot deployment is beyond the simulation-benchmark scope of this paper (Section 7), all components (frozen DINOv2, photometric , conservative fallback) are designed for direct hardware transfer.
Response to Q3: Sensitivity to Inaccurate Probe Calibration and Unavailable Safe Probes
- Unavailable Safe Probes (Already Evaluated in Appendix S22, Table 29): On 300 prespecified difficult states where no safe probe is feasible, our conservative fallback abstains from probing and achieves 0.0% probe violations and 6.5% task violations (vs.
14.7%probe and18.4%task violations for forced probing), while preserving66.3%success (vs.57.7%for wait-only). - Inaccurate Probe Calibration & Execution Noise: To directly test calibration robustness, Table R9 perturbs the calibrated thresholds () by and injects actuator magnitude noise into executed probes on the primary stream:
| Perturbation Condition | Success (, %) | Visual FAR (, %) | Event F1 (, %) | (, steps) |
|---|---|---|---|---|
| Nominal Calibrated MERC-WM | 80.6 1.7 | 8.6 1.2 | 87.9 1.1 | 147 62 |
| Thresholds shifted | 79.8 1.7 | 11.8 1.4 | 85.4 1.2 | 152 60 |
| Thresholds shifted | 79.9 1.6 | 7.1 1.1 | 86.1 1.2 | 164 63 |
| Probe Actuator Noise ( magnitude) | 79.4 1.8 | 10.2 1.3 | 85.8 1.2 | 158 61 |
Across all miscalibration and noise perturbations, MERC-WM degrades gracefully (79.4–79.9% success, 7.1–11.8% FAR), consistently outperforming the calibrated Residual-MoE (78.8%, 18.7% FAR). We will add Table R9 to Appendix S22.
Response to Minor Comments 1–3
- Minor 1 (Fixed Visual Encoder): Freezing the pretrained DINOv2 ViT-S/14 encoder is a deliberate architectural choice to preserve a stable coordinate system for stored cards across long streams (preventing representational drift), while our slow normalization path and nuisance-suppressed projection absorb unseen visual shifts—confirmed by
78.2%success under Severe Appearance shifts (Table 30). - Minor 2 (Actual Memory Usage vs. Configured Budgets): At the default 18.0 MB total auxiliary cap, actual peak resident usage on the 6-segment stream is 6.06 MB for MERC-WM versus 6.25 MB for Residual-MoE (detailed in Table R1 for Reviewer Q5BF and Table R5 for Reviewer Mfuc). We will report both configured caps and actual resident usage side-by-side in Table 1.
- Minor 3 (Table 6 vs. Main Table 1): Table 1 (and Table 33
r4/18) reports the primary byte-capped runs (8.6%FAR,147steps ), whereas Table 6 (r4/B16) reports the independent card-block hyperparameter grid sweep (8.3%FAR,151steps). We will add explicit protocol tags to Table 6 and Table 33 captions so the distinction is immediately clear.
We hope these responses and additional results fully address your concerns.
Response to Reviewer EQzK (Part 1/2): Algorithmic Novelty, Direct Comparison with SPREAD/ReDRAW/LLCD, and Saturation Gains
We thank Reviewer EQzK for the constructive review and for recognizing the relevance of our problem setting, our multi-metric evaluation, and the encouraging gains in false expansion and recurrence recovery. Below, we address algorithmic novelty and direct baseline comparisons (W1 & Q1); Part 2 addresses generalization beyond 3 mechanisms, probe calibration sensitivity, and the three minor comments (W2, Q2, Q3, Minor 1–3).
Response to W1: Algorithmic Novelty Beyond Modular Combination
We respectfully clarify that MERC-WM is not a straightforward assembly of off-the-shelf change detection, replay, and MoE routing. In fact, that exact modular combination—action-conditioned residual GLR/CUSUM change detection + fixed-budget replay window + matched 4-expert LoRA routing—is our Residual-MoE baseline. MERC-WM introduces a fundamentally different adaptation principle: gating dynamics updates on matched, nuisance-suppressed action-effect evidence rather than raw observation residuals, realized through three tightly coupled algorithmic mechanisms:
Our 2×2×2 factorial study (Appendix Table 21) confirms that these components exhibit strong positive synergy rather than mere additive gains: full MERC-WM (
83.3%) outperforms removing any single component by3.7–4.9 ppand outperforms the base model by+7.5 pp.Furthermore, while the gain over matched Residual-MoE is
+1.8 ppon the 3-mechanism stream (winning on10/10seeds in our expanded 10-seed study, Wilcoxon p=0.0020, while cutting visual FAR by>50%and recovery steps by42%), the advantage more than doubles to +4.1 pp (75.1%vs.71.0%) when expert capacity is actively stressed under 8 distinct mechanisms and 4 experts (see Table R7 in our response to Reviewer Mfuc).Response to Q1: Direct Comparison Against SPREAD, ReDRAW, and LLCD Under Matched Settings
Appendix S20 (Table 25) evaluated common-backbone surrogates for online LoRA, latent-context adaptation, and changepoint detection. To directly answer Q1, we adapted the core mechanisms of SPREAD (2026), ReDRAW (2025), and LLCD (2025) onto our shared frozen DINOv2 encoder, transition backbone, and identical 18 MB auxiliary budget and MPC budget on the primary 6-segment stream (5 seeds). Table R8 summarizes the matched comparison:
Because SPREAD, ReDRAW, and LLCD trigger adaptation from unprobed observation/latent residuals without nuisance-suppressed paired verification, they still incur
2.0x–3.3xhigher visual false adaptation (17.4–28.6%vs.8.6%) and1.7x–2.0xslower recurrence recovery (248–301vs.147steps). We will include Table R8 in Section 6 and Appendix S20.