- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies which predicted rewards and values suffice to preserve a world-model planner's action and preference statistics. It introduces FutureCert, which retains selected prediction-read groups, replaces the rest with fixed references, and reruns the native planner, including newly visited prediction sites. The framework supports exact minimum-certificate auditing, budgeted search and repair, and independently calibrated reliability across planning streams. The main evaluation covers four planners and six tasks.
Critical strengths
- The problem is very much well motivated, and the experiments are carefully designed. Changing reward or value predictions can change what the planner explores next, so an explanation evaluated on frozen computation might not survive actual re-planning. The paper supports this argument with both a concrete counterexample and empirical evidences. Importantly, the authors distinguish a limitation of the frozen interface from a limitation of the attribution methods themselves. This makes the evaluation substantially more informative.
- The different guarantees are carefully separated. Finding a sufficient set, proving minimum size, recovering all minimum alternatives, and establishing reliability across streams are treated as different questions.
Critical weaknesses
- The explanatory value of the richer certificates is less established than their formal validity. The certificates seems like based on prediction reads under a specific intervention instead of particular imagined futures. The sensitivity analysis is useful, but it remains unclear which audit choices yield the most meaningful explanation for a practitioner. I would view this as a limitation of the demonstrated usefulness, rather than a flaw in the stated guarantees.
- Scalability might remain a meaningful limitation. Exact family/core audits mainly cover H <= 12, with 4,096 verifier queries for the complete enumeration at H=12, plus additional calibration runs. The large-space experiments are useful, but do not provide the same complete exact ground truth. The default search also trades additional preparation cost for better recovery like what the paper transparently reports.
Questions for authors
Q1: Beyond the action/ranking/margin controls, can you separate frozen-tree failures involving substantial changes among leading actions from those caused only by reordering low-visit actions?
Q2: For the larger evidence space, how does verified certificate size trade off against total cost?
6 comments
W1 Explanatory value of the richer certificates
If a group of reads becomes unavailable, can the planner preserve this decision with the same number of other groups? For MuZero Pong decision 81, the two minimum supports are {1, 4, 6} and {3, 4, 5}, each of size 3. If unit 1 is unavailable, the second support still works. If unit 4 is unavailable, all 26 candidates of size at most 3 that avoid it fail. The smallest feasible support now has four units, {1, 3, 5, 6}.
The certificate identifies groups of prediction reads for a specified input, planning stream and replacement rule. On re-execution, retained groups use the predictions actually produced, and the remaining reads receive their reference values. The resulting action and preference statistics must pass the fidelity check. The same group also covers reads at newly visited sites, so the same support can admit different imagined trajectories. The Core consists of the groups shared by every minimum support.
Across four TD-MPC2 tasks, the 692 inputs with nonempty minimum supports yielded 8,304 single-unit loss scenarios. Of the 6,780 scenarios admitting a same-size support, the first certificate covered 6,121. The complete family covered another 659. For the remaining 1,524/8,304 scenarios (18.4%), the complete family ruled out every same-size substitute. Any feasible alternative would have to be larger.
These queries refer to the same input and audit specification. Once the family has been stored, a practitioner can select a verified substitute without another verifier call (Table R15).
Which tasks warrant a closed-loop compression test? Across five tasks, both leave-one-episode-out predictions on the development set and closed-loop measurements ranked Finger Spin first. The predicted saving was 70.3% of prediction reads, and the measured saving was 50.9%. On Cheetah Run, Cup Catch, Walker Walk and Quadruped Run, both figures were at most 4.1%.
The choice also depends on return. On Finger Spin, skipping reads according to the certificate took 0.77 times the full planning time and reduced return by 34.6. Reducing planning iterations from 6 to 4 took 0.79 times the full planning time and reduced return by 1.9. At these comparable runtimes, reducing the iteration count preserved more return.
In an exploratory DreamerV3-S training experiment, we assigned larger weights in the critic’s TD loss to imagination steps marked by a certificate. We compared equal weights and a random permutation of the same certificate weights. Each run used 100k environment steps. We averaged return over each training run, then averaged over three seeds per task.
Table WG-R1. Mean return over training
| Task | Certificate weighting | Equal weighting | Shuffled weights |
|---|---|---|---|
| Cup Catch | 844 | 775 | 789 |
| Cheetah Run | 333 | 290 | 315 |
| Hopper Stand | 254 | 178 | 199 |
Certificate weighting gave higher mean returns than either control on all three tasks (Table R6). With three seeds per task, we treat this training comparison as exploratory. The reconfiguration experiment above directly measures what the complete family adds over one certificate.
Practitioners can choose the fidelity criterion to match the quantity they need to preserve. They can constrain action deviation alone, or also constrain the first step of the optimized action mean. Across 800 inputs in four tasks, mean minimum support size under the joint criterion was 4.19, 2.73 and 1.92 units at tolerances ε = 0.1, 0.2 and 0.3, respectively (Table R12).
The groups should match the reads that can actually be disabled. For comparisons across partitions, the retained groups can be expressed in the original units. A joint support can be computed when several specified reference values must all be accommodated. Whether these choices improve practitioners’ understanding or diagnostic performance remains an open empirical question.
In the revision, §3 and Fig. 1(b) will illustrate the read groups and their alternatives. Section 5.4 gives the gains from storing alternatives. Section 5.7 brings the five applications together in Table 4 and compares predicted and observed closed-loop read savings in Table 5. Supplementary Tables R6, R12, and R15 give the corresponding comparisons.
W2 Scalability
The 24-unit native audit recovered the complete minimum family and its Core on 136/160 inputs (85.0%). On the other 24 inputs, every candidate of size 0–5 failed, establishing the lower bound . Completion ranged from 25/40 on Quadruped Run to 40/40 on Finger Spin.
We took 40 inputs at fixed indices in each of four TD-MPC2 tasks and refined the original partition into 24 units. The audit reruns native MPPI for each mask. It checks sets in increasing order of size and finishes the first layer containing a passing set, up to layer 5. On inputs it completes, this layer-by-layer audit uses mask checks, which is for fixed minimum size and in the worst case.
Across all 160 inputs, the audit averaged 18,382 checks per input (median 2,325), including 55,455 checks for each unfinished input. On shared H200 GPUs with six processes per GPU, the consistency checks and both the 12- and 24-unit exact audits together took a mean of 1,143 seconds per input (median 166 seconds), excluding model loading (Table R13).
Budgeted search returned natively verified sets on the same 160 inputs. Table WG-R2 reports recovery and Gap on the 136 inputs with known minima. Raising the cap from 384 to 768 reduced mean excess size on these inputs from 1.60 to 0.79 units. Q2 gives the sizes returned on the remaining 24 inputs.
Table WG-R2. Budgeted recovery on inputs with known minimum size
| Query cap | Inputs recovering minimum size | Mean excess units (Gap) |
|---|---|---|
| 384 | 64.7% | 1.60 |
| 768 | 77.2% | 0.79 |
A separate 12-unit experiment measures the cost of preparation. On the 600 TD-MPC2 inputs in the paper’s Table 2, Beam-4 requires no preparation. The default configuration reuses a development library for each task and recovers minimum size more often.
Table WG-R3. Preparation cost in the 12-unit experiment
| Configuration | Minimum-size recovery | Gap | Mean online queries/input | Mean preparation queries/task |
|---|---|---|---|---|
| Beam-4 | 86.50% | 0.307 | 69.44 | 0 |
| Default | 92.97% | 0.140 | 69.62 | 17,047.3 |
Table 2 weights episodes and settings equally and averages the default configuration over ten solver seeds. If we audit inputs in each of the three tasks and reuse the development library within each task, the default costs an average of queries per input. This gives 154.86 at and 86.67 at .
Certifying reliability on fresh planning streams incurs an additional calibration cost. On the original 20 Finger Spin inputs, the broader candidate pool averaged 757.5 non-full calibration executions per input, plus 64 shared full-retention executions and the cost of proposing candidates.
Section 5.3 will report the 24-unit completion rate. The 12-unit main table will give online and amortized costs side by side. Supplementary Tables R7 and R13 distinguish search preparation, exact auditing and calibration costs.
Reply to Reviewer wGPc
Thank you for your assessment of the motivation and experimental design. We agree that the value of a certificate depends on the decisions a practitioner can make with it. Tables R1–R18 are the appendix tables added during review.