acceptodds
Under review as a conference paper at ICLR 2027

VivaWorld: Long-Horizon Video World Model with Compositional 3D Priors

Abstract

Long-horizon video generation is key to interactive world modeling, physical intelligence, and spatial reasoning, yet maintaining geometric consistency and accurate camera control remains challenging without 3D grounding. We introduce VivaWorld, a framework for generating geometry- and view-consistent long videos from a single image. We first construct a compositional hybrid 3D representation that combines reconstructed point clouds, which preserve observed scene geometry and appearance, with generated textured meshes, which provide complete object geometry across novel views. We also introduce a connected-region filter that removes inconsistent floating components for robust cross-view reconstruction and propose a hybrid camera embedding that combines global and object-anchored Plucker rays to capture both scene-level camera motion and camera-object relationships, and inject it together with the visual guidance into a video generation model via a control adapter. Extensive experiments show that VivaWorld achieves state-of-the-art visual quality and camera control, while supporting controllable novel-view synthesis, long-horizon scene exploration, and physical AI.

Primary area:
generative models
Keywords:
Video Generation, Video World Model, 3D Grounding, Camera Control, Novel View Synthesis

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KMYZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VivaWorld addresses geometric inconsistency and camera drift in long video generation from a single image. It combines reconstructed point clouds with generated object meshes, introduces mesh-seeded point-cloud filtering, and augments global camera embeddings with object-relative information to guide a pretrained video model. Experiments demonstrate improved long-video fidelity and camera accuracy, together with better downstream 3D reconstruction, compared with the evaluated baselines.

Critical strengths
  • The combination of reconstructed point clouds and generated object meshes is well motivated, providing complementary guidance for observed and unseen regions. Object-relative camera embeddings further incorporate camera–object relationships.

  • The reported experiments show improvements in long-video fidelity and camera accuracy over several baselines, supported by downstream 3D reconstruction results and supplementary visual comparisons.

Critical weaknesses
  1. Unexplained differences between ablation results. Tables 3 and 4 report different results for their reference configurations despite both evaluating on DL3DV. For example, long-video LPIPS differs substantially (0.488 vs. 0.545), exceeding the Global-to-Hybrid improvement in Table 3. The paper should clarify whether these configurations share the same components, evaluation scenes, checkpoints, and sampling settings.

  2. Incomplete justification of object-anchored camera conditioning. Table 3 shows positive gains, but the explanation that global embeddings accumulate errors is unclear: the embeddings are computed analytically, while drift occurs during generation. Comparisons with simpler object-center conditioning and sensitivity analyses for anchor count and localization errors would better establish the specific benefit of the proposed representation.

  3. Limited evaluation of robustness and computational cost. The mesh-seeded filter may discard valid geometry disconnected from registered meshes, yet its effect is evaluated only qualitatively. Robustness to inaccurate meshes and registration failures is also unexplored. End-to-end runtime and memory costs should be reported, and releasing generated training assets would help address reproducibility concerns associated with proprietary mesh generation.

Questions for authors
  1. What explains the differences between the reference results in Tables 3 and 4? Do the model configurations, evaluation scenes, or sampling settings differ?

  2. How does object-anchored conditioning help reduce generation drift, and how sensitive is its benefit to the number and accuracy of the anchors?

  3. How does the method handle inaccurate meshes, registration failures, or valid geometry disconnected from mesh seeds? What are its end-to-end inference time and memory requirements?

4 comments
AuthorsOpenReview ↗

Response to reviewer (1/n)

We thank the reviewer for the insightful comments and are encouraged by their recognition of (1) our novel combination of point-cloud and mesh-based visual guidance with hybrid camera embeddings and (2) the effectiveness of our method in long-video generation and 3D reconstruction.

Q:Inconsistent Tables.

Thank you for pointing out this inconsistency. Two factors explain the differences. First, Table 1 reports frame-count-weighted averages over both DL3DV and Tanks and Temples, whereas Tables 3 and 4 evaluate DL3DV scenes only. Second, at submission, Table 3 used checkpoints trained for 80% iterations, while Table 4 used checkpoints trained for full iterations, because the ablation runs had reached different training stages by the deadline. We acknowledge that this distinction was insufficiently documented and causes misunderstanding. We have now completed the remaining ablation training and the updated results are presented in the table below. The current setting still outperforms the other visual guidance configurations and the global embedding baseline.

Guidance SignalSingle-Chunk PSNR ↑Single-Chunk LPIPS ↓Single-Chunk Cam ↑Long-Video PSNR ↑Long-Video LPIPS ↓Long-Video Cam ↑
None with Hybrid11.470.49060.3510.030.7003.17
PC20.000.17489.6612.410.58471.34
PC + Gaussian20.760.16883.9413.600.55079.95
Global Embedding20.490.14989.8413.560.51278.32
Ours (PC + Mesh + Hybrid Embedding)21.620.14590.6914.300.48880.97

Q: Object Anchor Ablation and Failure Case Analysis

Thank you for suggesting an ablation on the number of object anchors. We acknowledge that global camera embeddings are computed analytically and do not themselves accumulate drift and drift arises during video generation using global embeddings. Object-anchored conditioning explicitly encodes camera–object relationships relative to fixed 3D anchors, providing persistent spatial references that encourage consistent object placement across autoregressive chunks. In addition, our curated training data include scenes with only one or two object anchors. We pad these inputs to the required size during training, allowing the model to handle varying numbers of available anchors.

We reduce the number of anchors from 3, 2 and 1 to evaluate how anchor count affects generation quality in 5 cases. We additionally evaluate 5 test cases (mesh numbers range from 3 to 17) with inaccurate conditions by reducing geometric coverage by retaining approximately half the meshes, reducing the average from 9 to 4 objects per scene, or applying a point-cloud (PLY) confidence threshold of 0.3, which removes approximately 44.7% of the points. It shows modest performance degradation with fewer anchors while retaining the ability to generate long videos. In the table, it could be noticed that 47% removing point-cloud points substantially reduces conditioning PSNR (-2.0) but only lowers a little generation PSNR(-0.2). Shifting mesh positions only degrades performance a little as well. Using one or two anchors causes relatively small changes (since in our training dataset, some scenes only have 1-2 anchors), and retaining half the meshes has a modest impact. In terms of inaccurate meshes, registration errors, and incomplete geometry, our model is trained to handle imperfect inputs through data augmentation and diverse conditioning inputs, including sparser RecGen meshes (in the training data curation, the feed forward mesh generation might have some inaccurate results as well), denser Astra meshes (might include some coarse mesh shape from parametric code-based modelling), and both point clouds progressively reconstructed from previously generated frames and full point clouds.

ConfigurationConditioning PSNR↑PSNR↑SSIM↑LPIPS↓
1 anchor11.6614.260.36170.5161
2 anchors11.7414.360.36250.5122
Full Mesh + Full PLY (3 anchors)11.8114.410.36580.5141
Half Mesh + Full PLY11.6914.330.36160.5231
Full Mesh + conf=0.3 PLY9.49014.130.35400.5229
Mesh Position Shift of 5% + Full PLY11.7114.180.35640.5310
AuthorsOpenReview ↗

Response to review (2/n)

For mesh-seeded filtering, we evaluate conditioning quality across 5 cases by comparing renderings of progressively reconstructed point clouds with ground-truth images. The table reports mean PSNR gains over unfiltered point clouds. Confidence filtering at 0.01 improves conditioning quality by suppressing reconstruction noise, while mesh-seeded filtering achieves the largest gain with a 128 × 128 × 128 voxel grid and r = 1. We nevertheless choose r = 2, which achieves a nearly identical gain while allowing connections across a one-voxel gap, providing greater tolerance to fragmented geometry.

Conf ≥0.01Grid32/r1Grid32/r2Grid64/r1Grid64/r2Grid128/r1Grid128/r2
+0.0512+0.0028+0.0080+0.0532+0.0531+0.1402+0.1387

In addition to using 2 grids, we use multiple object meshes derived from SAM3’s “everything” mode as seeds to preserve valid regions disconnected by occlusion or sparse sampling, increasing coverage across the scene. We also acknowledge that mesh-seeded filtering parameters (voxel sizes and connectivity grid) may require adjustment for complex scenes or inaccurate results/failure cases and users could use it as an optional choice. Each filtering run takes an average of 1 seconds, making it practical to evaluate several parameter settings and select the most suitable result (+0.2421 in those 5 test cases).

Q. Runtime and Memory.

On a single NVIDIA A100 80GB GPU, our method generates 1,000 frames in 166.4 minutes with a peak memory usage of 44.76 GiB, running faster than VerseCrafter, Lyra2, NeoVerse, and Voyager but slower than Gen3C, AnyRecon in the same 832 * 480 resolution. Diffusion denoising accounts for 150.27 minutes, other chunk processing takes 7.31 minutes, and preprocessing and remaining overhead take 8.82 minutes.

MethodModel Time (min)↓Total Time (min)↓Peak Memory (GiB)
Ours157.6166.444.76
Gen3C38.941.021.98
AnyRecon50.654.140.38
VerseCrafter148.1172.654.87
Lyra2208.7225.966.73
NeoVerse239.1254.846.54
Voyager457.1514.342.28
Component / StepTime (min)Description / Details
Video diffusion denoising~150.2713 chunks, 30 steps each
Other processing across chunks~7.31Input preparation, text/VAE processing, video encoding and saving
Processing outside chunks4.06Model initialization, initial and cumulative Pi3 control updates, stitching and validation
First-frame Pi31.79Including model loading
First-frame SAM3 segmentation0.74Including model loading
First-frame RecGen1.93Model loading, mesh generation
Voxel-based filtering0.30Point-cloud filtering
Total166.40Approximately 2 hours 46 minutes
Astra: 1 iteration (optional)2.89Excluded from the total
Astra: 2 iterations (optional)6.23Excluded from the total

Q: Data Release.

We are happy to release the data curation method and plan to release some data and code.

AuthorsOpenReview ↗

Reminder to the reviewer

Thanks again for your insightful feedback on our work! We've carefully worked to address your comments/questions and update the draft and images in the revision pdf. We are happy to offer more details if helpful. Are there any further questions or concerns we should discuss?

AuthorsOpenReview ↗

Further reponse to the reviewer

Hi, we thank the reviewer for the great suggestions! Feel free to ask more questions if it helps in increasing your confidence in our work. We also added the new Table 9-13 and new Fig.14, 15, 16 and 17 in page 20-28, including more diverse cases, including driving scenes, landscapes with full/half/inaccurate mesh or point cloud conditioning, and imperfect hybrid embeddings. Please see more details in page 20-28 in the revised pdf. Thanks again for your valuable suggestions!

Reviewer_opvUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies long-video generation from a single image along a specified camera trajectory. It combines point clouds, which preserve observed scene content, with object meshes that provide more complete geometry. It also uses scene-level and object-centered Plücker embeddings for camera control. The authors compare against six methods and evaluate the approach through ablations and 3D reconstruction results.

Critical strengths
  1. Keeping object positions and scene layouts consistent over long videos is an important problem. Combining point clouds that preserve existing observations with meshes that provide geometry for unseen surfaces is well motivated.
  2. The experiments show practical gains. The reported long-video camera control is better than the compared baselines, and Figures 5 and 6 show better object layouts and reconstructions. Tables 3 and 4 also support the usefulness of the camera embedding and geometric guidance.
Critical weaknesses

The overall gains are encouraging, but the dependence on suitable objects and the source of the improved object placement are not yet clear. First, it is unclear whether the test set includes scenes without objects suitable for mesh generation and camera anchoring, or how the method performs in such scenes. Second, mesh placement still depends on the estimated point cloud. The current comparisons do not clearly separate gains from better initial geometry from gains due to reduced drift during generation.

Questions for authors

1. How much does the method depend on having suitable objects in the scene?

Table 4 shows a clear improvement from adding meshes to point-cloud guidance. Does the test set include scenes without objects suitable for mesh generation and use as camera anchors? If so, could you report comparisons with the baselines separately for scenes with and without such objects? If not, how would the method handle this situation?

The mesh-seeded BFS filter in Appendix A.1 also depends on object meshes: it removes point-cloud components that cannot be reached from the mesh seeds as floaters. But valid parts of a scene can also be disconnected because of occlusion or sparse sampling. How does the method avoid removing valid geometry in these cases? Have the authors observed this happening?

2. What makes the object positions closer to the ground truth?

In Figures 5 and 6, the object positions and layouts appear closer to the ground truth than those of the baselines. However, the positions and scales of the meshes still depend on registration to the estimated point cloud. I wonder whether the gains mainly come from more accurate initial geometry, or from better preserving that geometry and reducing drift during generation. How does the point-cloud construction setup differ from that of the main baselines, and can the existing ablations help distinguish these two factors?

3. How are new objects handled during generation?

In standard long-video generation, do the meshes always come from the initial image? Are objects that appear later added only to the point cloud, or are meshes also generated for them and used as additional camera anchors? What exactly is updated in the “Mesh and Point Cloud Update” step in Figure 2?

4. Are long-video camera errors computed over the whole sequence or over shorter segments?

Appendix B describes computing camera errors after alignment within each window. For long videos, does a window cover the entire sequence, or is the sequence split into shorter segments? If the errors are computed separately for each segment, is drift accumulated between segments also captured?

4 comments

Thank you for the detailed responses and additional experiments. The clarifications on mesh updates and long-video camera evaluation address those questions, and the new results help establish robustness to reduced mesh coverage and imperfect geometry.

My main concern remains the dependence on suitable objects. The additional experiments still assume that usable object meshes are available, while mesh guidance, object-anchored camera conditioning, and mesh-seeded filtering all rely on them. It remains unclear what advantage the method retains when suitable objects are unavailable or exploration moves beyond the initial objects.

I would appreciate further evidence of the method’s contribution beyond existing approaches and a clearer rationale for its main design choices. Until these points are clarified, I will maintain my current rating.

AuthorsOpenReview ↗

Further Reponse to the Review

Q: Mesh Condition and Update

We appreciate your time in reviewing our paper and rebuttal. While we understand your remaining concern, we would like to point out that most environments contain objects. The main idea of our method is leveraging 3D objects for more 3D consistent generation, thus dealing with scenes that do not have such objects constitutes a (rather uncommon) out-of-domain case where there is no obvious advantage to using our method. To illustrate how prevalent objects are, we processed full 8,965 scenes from DL3DV using RecGen and a subset of approximately 5,269 scenes using Astra. Astra failed to produce any object meshes for only 0.4% of the processed scenes, while the corresponding failure rate for RecGen was approximately 3.5%. Using both mesh-generation methods reduced the proportion of scenes without any meshes to less than 0.3%, which means that over 99.7% of scenes have at least one mesh available for conditioning.

Count per sceneRecGenAstra
032023 failures
14070
24434
35106
44585
552118
649125
746628
841348
940261
1039187
11–151,673802
16–201,1871,233
21–301,0071,800
31–50252811
51–10024214
101–200081
201–429023
Counted scenes8,9655,269
Mean count11.3926.94
Maximum count83429

To further demonstrate our method’s performance with unreliable mesh conditioning, we select five natural scenes from DL3DV, including lakes and mountains, where we only have coarse meshes with incorrect shapes or placements(These cases are challenging for RecGen mesh generation. In the lake and leaf examples in Fig. 15, RecGen fails to generate any meshes, and we use only Astra meshes for conditioning.). We use these imperfect Astra meshes during inference. We also evaluate five more challenging autonomous-driving scenes from OpenDV dataset, which is outside the DL3DV dataset used to train both our method and Lyra2 and we have no mesh rendering in some chunks. For these driving scenes, we use Astra to update the mesh conditioning every other chunk as the camera moves forward. We compare our method with the strongest baseline, Lyra2, across all ten scenes. As shown in the table, our method outperforms Lyra2 across all metrics, demonstrating better image quality, more accurate camera control, and greater style consistency, even when conditioned on unreliable meshes with coarse shapes or incorrect placements. We provide representative additional qualitative comparisons on landscape and driving scenes in Fig. 15 and Fig. 16 in Page 26-27 in the revised pdf.

MethodPSNR↑SSIM↑LPIPS↓Cam.↑Style↑
Ours13.68940.37970.545270.236397.4094
Lyra213.16840.37700.653458.460889.5962

Q: More Results To further demonstrate our model’s effectiveness with unusable meshes, we provide additional visual results on 2 scenes where RecGen fails to generate any meshes, using only Astra-generated meshes with incorrect shapes and placements for conditioning in the top 2 cases from the DL3DV dataset. We also evaluate 4 additional cases without mesh conditioning, using only progressively reconstructed point clouds and global camera conditioning. As shown in Fig 17 in page 28, although performance with point-cloud-only conditioning may be lower than with mesh guidance and hybrid conditioning, our model still generates realistic videos. See more details from page 24-28 for mesh conditioning and see more details of the rebuttal from page 20-28.

We are happy to solve any further concerns. Thanks for your valuable suggestions again!

Thank you for the detailed responses and additional experiments, which have alleviated most of my concerns. I appreciate the overall approach and contributions, particularly the use of object meshes to support both camera encoding and point-cloud filtering. I still have some reservations about the connectivity assumption behind mesh-seeded filtering and how meshes with positional errors affect generation. The reported runtime also suggests that efficiency remains a limitation for interactive scene exploration.

Overall, I find the work well motivated and the proposed approach reasonable. I will therefore raise my rating to 3 (Weak Acceptance) in recognition of the authors’ contributions.

Thank you for increasing your rating. Feel free to ask if you have more questions.

Reviewer_NdCCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The proposed method named VivaWorld generates camera-controlled videos of up to 1,000 frames from one or a few images. It combines reconstructed background point clouds with generated foreground meshes to reduce geometry distortion and camera drift. RGB-D renderings and camera embeddings guide a frozen Wan2.1-14B model through a VACE-based adapter. Videos are generated in 81-frame blocks, with background geometry updated after each block. Experiments show better video generation and downstream 3DGS reconstruction than Lyra 2.0, Gen3C, and NeoVerse.

Critical strengths
  1. Combining background point clouds with foreground meshes addresses missing geometry in unseen views. Previous point-based methods leave holes behind objects, while Gaussian proxies lack fine geometry and texture. The proposed generated meshes complete foreground geometry and texture, while point clouds preserve observed background details.

  2. Object-anchored camera embeddings provide a simple, lightweight way to reduce drift. They reuse global ray directions and add three relative-moment channels per object, explicitly encoding camera–object relations through a 0.26M-parameter Conv3D encoder.

  3. The long-video results and downstream 3DGS reconstructions show strong consistency. Across roughly thirteen 81-frame blocks, VivaWorld maintains a camera-control score of 79.51 and the best PSNR, SSIM, and LPIPS, while several baselines degrade sharply. After 3,000 optimization steps, its 3DGS reconstructions also show less ghosting, object splitting, and geometry blur.

Critical weaknesses
  1. The mesh-seeded filter relies on a questionable connectivity assumption and lacks quantitative tests. It removes every point-cloud region disconnected from the initial foreground meshes. This may delete valid distant or separate background geometry, remove newly explored regions after a single break in connectivity, or empty the point cloud when no mesh is available. Beyond qualitative examples, the manuscript provides no comparison with no filtering or confidence-only filtering, nor tests of the voxel resolution and connection radius.

  2. Results conflict across tables, and several evaluation choices weaken the evidence. The full model has different scores in Tables 1, 3, and 4, even though Tables 3 and 4 both use DL3DV. In Table 4, Gaussian guidance achieves better long-video camera control than meshes, and single-block camera scores also trail some baselines. Using ViPE poses as ground truth may favor a method that relies on ViPE. The "3D Scene Generation" scores use DA3-predicted Gaussians rendered along the original path, rather than the shown 3DGS results or direct geometry and held-out-view tests.

  3. Runtime, memory use, and errors from upstream models are not evaluated. The pipeline combines SAM 3, mesh generation, ViPE, Pi3, ICP, voxel filtering, two RGB-D guidance streams, and Wan2.1-14B with a 3.05B adapter. Long videos also require repeated geometry updates. Yet no end-to-end timing, memory figures, or efficiency comparisons are reported. The effects of incorrect mesh textures, segmentation errors, and poor alignment are also untested.

  4. Some claims go beyond the method, and the draft contains unfinished material. The static geometry and fixed anchors do not model moving objects or physical interaction, limiting the claims about world models and robot learning. The robot-learning example lacks simulation, policy, reward, and collision-mesh details, and the agentic example does not explain how video feedback updates meshes. The captions of Figures 12 and 13 state “Numbers denote frame indices,” but the labels show timestamps in seconds (t=6s, t=12s, ..., t=45s). The text also contains typos, such as "scene reoncstruction" (line 311-312) and "genreaiton" (line 490-491).

Questions for authors
  1. Please check and explain why the full model's PSNR, LPIPS, and Cam. scores differ across Tables 1, 3, and 4 for both single-block and long-video generation. In particular, why do Tables 3 and 4 report different results for the same full configuration on DL3DV?

  2. How does the filter avoid deleting valid background points when no foreground mesh is found, or when distant buildings, hanging lights, or newly explored rooms are disconnected from the initial meshes?

  3. Please report end-to-end runtime and peak memory for generating 1,000 frames at 832×480832\times480, with separate figures for SAM 3, RecGen/GPT-6 Astra mesh generation, ICP, per-block Pi3 reconstruction and voxel BFS filtering, RGB-D rendering, and diffusion denoising. Compare with Lyra 2.0 and Gen3C. Please also explain Figure 8's algorithm for refining the initial mesh using generated videos and Figure 1's humanoid reinforcement-learning setup.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Area Chair and Reviewers,

We sincerely thank all reviewers (Reviewers NdCC, opvU and KMYZ) for their constructive feedback and thoughtful discussion, and for recognizing (a) our novel visual guidance, (b) our novel camera embedding, and (c) our strong results in long-video generation and 3DGS reconstruction. We have addressed their concerns through additional experiments and methodological clarifications on pages 20–28 of the revised manuscript, including 9 new tables (Tables 5,6,7,8,9,10,11,12 and 13) and 4 new figures (Figs. 14,15,16 and 17). We also appreciate the reviewers’ active engagement, and are grateful that Reviewers opvU and KMYZ indicated that these additional experiments and clarifications addressed their concerns.

Q: Inconsistent tables (see Table 5, page 20).

The differences arose because Table 1 aggregates DL3DV and Tanks and Temples results, whereas Tables 3 and 4 use only DL3DV, and the original ablations used checkpoints at different training stages by the deadline. We have completed the remaining training and updated the ablations, with our configuration continuing to outperform alternative visual guidance settings and the global embedding baseline.

Q: Mesh-seeded filtering and parameter selection (see Table 6, page 20).

We evaluate filtering parameters on five scenes by comparing progressively reconstructed point-cloud renderings with ground-truth images. A 128 × 128 × 128 grid with radius r=2 provides better point-cloud reconstruction quality among the evaluated settings. We acknowledge that in some complicated case, Mesh-seeded filtering parameters may require adjustment for complex scenes or inaccurate reconstructions, but each run takes only about one second, enabling parameter selection that yields a mean PSNR gain of 0.2421 dB across five cases.

Q: 3D scene evaluation and camera metrics (see Tables 7 and 8).

Additional 3DGS optimization generally improves short-video SSIM but reduces long-video SSIM, while our method retains the best SSIM and LPIPS in both settings. Camera-score differences arise from clipping (rotation errors above 15◦ and translation errors above 0.5 are clipped to these thresholds, so further increases do not affect the score.) and nonlinear normalization. Long-video evaluation uses one continuous DROID-SLAM trajectory aligned to ViPE reference poses with a single global scale and no chunk-level resets, capturing accumulated drift.

Q: Upstream errors and robustness (see Table 9 and Figure 14).

On five test scenes, we evaluate reduced mesh coverage, incomplete point clouds, 5% mesh-position shifts, and fewer object anchors, observing modest generation-quality degradation despite substantially weaker conditioning. This tolerance is consistent with training on diverse conditioning inputs, including sparse RecGen meshes, coarse Astra meshes, and progressively reconstructed and full point clouds.

Q: Runtime and memory (see Tables 10 and 11).

Our method generates 1,000 frames at 832 × 480 resolution in 166.4 minutes on a single NVIDIA A100 80GB GPU, using 44.76 GiB of peak memory and running faster than VerseCrafter, Lyra2, NeoVerse, and Voyager. Diffusion denoising takes 150.27 minutes, other chunk processing takes 7.31 minutes, and preprocessing and remaining overhead take 8.82 minutes.

Q: Humanoid application and mesh refinement details.

Following GRAIL, we reconstruct a generated sitting interaction, retarget it to Unitree G1, and fine-tune a SONIC controller with PPO in Isaac Lab using scene-aware observations and motion-tracking rewards. For mesh refinement, Astra uses generated views to revise procedural geometry, spatial parameters, and materials, and the updated mesh is inspected from multiple viewpoints before guiding further video generation.

Q: Imperfect or missing mesh conditioning (see Table 12, 13 and Figs. 15, 16, 17).

Mesh curation leaves fewer than 0.3% of scenes without meshes when combining RecGen and Astra, and our method outperforms Lyra2 across all reported metrics on ten challenging landscape and driving scenes with unreliable or partially missing mesh conditioning. Additional examples demonstrate generation with incorrectly shaped or positioned meshes, mesh updates during inference, and realistic videos using only progressive point clouds and global camera conditioning.

We are happy to offer more details if helpful. Thanks again for your valuable suggestions and comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.