- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
GaussianDream extends π0.5 with 1024 extra prefix tokens. A 12-block temporal module (TGE) builds these tokens from frozen VGGT features of three past frames. During training, two heads decode the tokens. One head predicts per-pixel Gaussians for the current frame, and the other predicts their 3D displacements for the next 1–5 steps. The heads are supervised with RGB rendering, pseudo-depth and pseudo scene flow. At inference the heads are dropped, but VGGT, TGE and the extra tokens are kept.
Critical strengths
-
Good and reliable gains over π0.5, including on a real robot. This matters because it shows the overall recipe works beyond saturated simulation benchmarks, so the paper has real practical value.
-
A practical, training-only design. The pseudo-depth and pseudo-flow targets come from off-the-shelf models, so the recipe needs no simulator access and transfers to real data. The paper also reports results that complicate its story and releases detailed code. This makes the work easy to build on.
Critical weaknesses
-
The central claim, that the Gaussian world model drives the gains, is not supported. Compared with π0.5, the model adds VGGT features, a frame history, 1024 extra tokens and the Gaussian losses. Every ablation in Table 4 keeps the first three. The variants that each drop one loss still reach 85.9–86.2% on LIBERO-Plus, against 87.0% for the full model and 84.6% for π0.5, so most of the gain survives whichever loss is removed. The remaining 0.8–1.1-point differences come from single runs and are smaller than the spread across checkpoints in Table 5 (85.6–87.0%). The flow loss carries by far the largest weight (15, against 0.1–0.2 for the other losses) but is never ablated alone. Better world modelling also does not help: the higher-PSNR checkpoint and exact simulator geometry (GaussianDream-Sim) both give lower success. This weakness is decisive because the title, method and contributions all rest on this attribution. If a VGGT prefix trained with the action loss alone matches GaussianDream, the paper's main idea is not the source of its results.
-
Described accurately, the method is close to prior work that it does not cite. The Gaussians are pixel-aligned, rendered back to a single view, and their base colours are copied from the input. The method therefore reduces to depth and scene-flow regression. The future head is not conditioned on actions and is unused at inference. Because it is trained on expert demonstrations, it learns where things move under the expert, which is closer to auxiliary action supervision than to a world model. The Robot dimension of LIBERO-Plus fits this reading: GaussianDream falls 4.5 points below π0.5 there (72.1 vs. 76.6). Under this description, the closest precedents are uncited. ManiGaussian (ECCV 2024) supervises a policy with an action-conditioned dynamic Gaussian world model. GAF (ICRA 2026) predicts feed-forward Gaussians plus per-Gaussian displacements that reuse the current appearance, which is the paper's second contribution. DreamVLA (NeurIPS 2025) adds future depth and motion prediction to a VLA on LIBERO. GAP (CVPR 2026) jointly predicts actions and future 3D geometry. This weakness matters because, without the world-model framing, the contribution over these works is not established.
Questions for authors
Requests that could change my rating:
- This question is decisive for my rating. What do the
no_future_tokenspreset (VGGT prefix, no auxiliary losses) and thebase_vlapreset achieve on LIBERO-Plus and RoboCasa, under the same training budget as the full model and with 2–3 seeds? - What do the
no_flowpreset and a direct depth-and-flow regression without Gaussians achieve? This would show whether the Gaussian parameterization matters.
Clarifications:
- In Table 2, the baselines appear to be averaged over the 24 tasks, but GaussianDream over the three categories. Computed per task, GaussianDream scores 53.8, which is on par with Being-H0.5 (53.9). Please confirm this.
- Table 1 lists Spatial Forcing at 97.6, but the original paper reports 98.5. Table 1 also leaves out Cosmos-Policy and LingBot-VA, which both reach 98.5 in Table 3. Where does each baseline number come from? Does the claim of the best LIBERO result still hold?
- What training budget did π0.5 and the ablations use? App. B says 60K steps, while the ablation script uses 30K. How was the 60K checkpoint selected?
- Why does performance drop under robot initial-state perturbations on LIBERO-Plus?
Suggestions (not required for my assessment): An action-conditioned future head would test whether world modelling, rather than motion supervision, helps. Reframing the paper as an analysis of which auxiliary signals help VLAs would match the evidence better. The depth probe in Fig. 4a should report metrics over many images, and it would ideally compare against models whose features were not trained to predict the same pseudo-depth.