- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TTA4D is a one-shot test-time adaptation framework that uses LoRA fine-tuning (~15 mins) to turn image-to-video diffusion models into free-viewpoint video generators. It factorizes supervision into dynamic-object/fixed-camera and static-object/dynamic-camera signals, enabling arbitrary camera trajectory control while preserving object appearance and motion to outperform strong baselines on Consistent4D and DAVIS datasets.
Critical strengths
- The proposed one-shot test-time adaptation strategy that factorizes supervision into dynamic-object and dynamic-camera signals to achieve novel viewpoint control without architectural changes or external paired data.
- Because the adaptation relies solely on lightweight LoRA fine-tuning, the method is lightweight (10–15 minutes on a single GPU) and directly plug-and-playable across various image-to-video foundation backbones.
- The method shows strong empirical performance over a number of sota baselines.
Critical weaknesses
- While the evaluation is conducted on 22 DAVIS videos, could the authors clarify whether this scale is standard and sufficient relative to existing literature benchmarks? We look forward to seeing how this addresses feedback from other reviewers.
- Given a fixed monocular input video, does the method require LoRA fine-tuning for each distinct user-specified camera trajectory? If fine-tuning is not performed per trajectory, the mechanism enabling generalization to unseen trajectories requires further explanation. An ablation study examining the impact of (Figure 2) on generalization capability would help clarify this.
- Lines 105 to 109 discussing accurate geometry require revision, as the current phrasing lacks sufficient context and is difficult to follow.
- To allow reviewers and readers to properly assess reconstruction and rendering quality, we recommend providing interactive or video visualizations of the 4D Gaussians.
- How does the pipeline handle failure cases or inaccuracies in the output generated by the TRELLIS model? Could the authors explicitly discuss how sensitivity to foundational generation errors affects overall stability as a limitation of the framework?
- Conditioning the TRELLIS-based rendering exclusively on the first frame introduces potential failure modes—such as initial frame artifacts or foreground object occlusion. Furthermore, while later frames in the video may provide richer multi-view appearance cues, these could potentially conflict with the initial TRELLIS output. How do the authors address or mitigate potential inconsistencies between the single-frame reconstruction and subsequent video evidence?
- Another potential concern is that the proposed method relies heavily on pre-trained multi-view generative priors, raising questions regarding the fairness of the baseline comparisons.
Questions for authors
- I believe L85-86 (It's highly likely that VFM's training data include videos featuring moving object from moving camera) is not true. Rather they were not conditioned with geometric conditions. If authors agree, this statement has to be fixed.
- The paper's writing assumes readers are already aware of details of SS4D and VACE. There should be at least one line opener sentence describing what each is.
5 comments
[W5, W6] TRELLIS errors and consistency with later video frames
We thank the reviewer and address W5 and W6 together. TRELLIS renderings provide SODC supervision for viewpoint changes; they are not used as the reference image at inference. Instead, inference uses the real first frame, while later input frames provide appearance and motion supervision through DOFC during adaptation.
Since SODC also provides RGB supervision, TRELLIS errors can still influence the adapted model. Nevertheless, our qualitative results and visual comparisons generally show consistent appearance and motion with the input video, supporting the effectiveness of this factorized supervision in practice. We will add comparisons with later input frames, including failure cases, and explicitly discuss the remaining sensitivity to TRELLIS errors as a limitation.
[W7] Pretrained priors and fairness of comparison
Pretrained 3D and video priors individually leave gaps in object-centric 4D generation; TTA4D’s contribution is to coordinate these complementary priors through dual-signal adaptation. Using pretrained priors is also common among the related methods, as summarized below.
| Method | 3D / Video prior |
|---|---|
| L4GM | LGM |
| SS4D | TRELLIS |
| GVFDiffusion | TRELLIS |
| Lift4D | SAM3D |
| TrajectoryCrafter | CogVideo |
| GEN3C | Cosmos |
[Q1] What supervision is missing in lines 85–86?
We will make two edits at L85–86:
- Replace “supervision” with “instance-specific supervision.”
- Replace “the target configuration, a moving object viewed from a moving camera, is never observed” with “paired target videos of the test object undergoing its recorded motion from the requested novel viewpoints are unavailable.”
[Q2] Introducing SS4D and VACE
- At L98–99, we will add: “SS4D is a generative model that produces dynamic 3D objects from monocular video; we use its coarse voxel sequences to obtain inference-time depth and mask conditioning.”
- At L105–107, before the VACE comparison, we will add: “VACE is a pretrained video generation and editing framework supporting reference-image and depth-based control; we use it as an optional controllable video backbone.”
Response from Reviewer
Thanks the authors for the rebuttal. The rebuttal partially addresses my conern while other points are kept as future work or promises. For example, why not the manuscript is updated with colored texts on the fly? Moreover, the author addresss W6 by future planned experiments. I would like to see other reviewer's perspectives and discuss with the reviewer for this submission.
We thank the reviewer for the constructive feedback and for recognizing the factorized supervision and the effectiveness of test-time adaptation. We address each point below.
[W1] Scale and scope of the real-world evaluation
Our evaluation on 22 real-world DAVIS videos is consistent with established practice and adequate in scale relative to comparable object-centric 4D generation studies. The table below summarizes the real-world evaluation datasets used in representative related work, placing our evaluation scale in context.
[W2] Trajectory generalization
No per-trajectory fine-tuning is required. TTA4D learns one set of LoRA weights per input video and reuses it across camera trajectories (Sec. 3.3, [L 256]). TTA4D’s dual-signal supervision guides the pretrained video model to follow changes in conditioning caused by object and camera motion. Generalization to unseen trajectories comes from transferring this learned behavior to new conditioning sequences, rather than simply memorizing a fixed set of camera paths.
We conducted the ablation on the number of SODC trajectories using the I2V backbone. Increasing the number of SODC trajectories improves performance on held-out trajectories, but our default set of three already performs well in most cases, with limited gains from adding further trajectories.
[W3] Clarifying lines 105–109
We thank the reviewer for highlighting this clarity issue. We will revise the passage around lines 105–109:
[W4] 4D Gaussian visualizations
We agree that video is more informative than static frames for assessing motion and rendering quality. Figure 4 currently shows RGB renderings and Gaussian primitives of the fitted DAVIS camel, and the supplementary demo includes a short dynamic visualization. Due to the page limit, we will host additional interactive or video visualizations of the 4D Gaussians on the project page.