- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Real2SAM2Real renders foreground meshes lifted from a reference image into normal-map conditions for camera, human, and object motion control. It separates appearance from geometry and uses lightweight fine-tuning with normal perturbations, adapting the model on approximately 2,000 monocular clips.
Critical strengths
1)The paper combines editable, complete foreground geometry with separate appearance conditioning to provide a unified and practical framework for camera, human, and object control. This integration of explicit control with video generation is the main reason I lean toward acceptance.
2)Results across multiple tasks and component ablations provide useful empirical support. Table 2 reduces PA-MPJPE from 34.81 mm for the strongest listed baseline to 22.80 mm, and Table 3 shows substantial improvements in object control; updating only approximately 1.8% of parameters per expert further strengthens the practical value.
Critical weaknesses
1)Appendix C.1 uses output lengths ranging from 25 to 121 frames and computes RPE between consecutive frames, resulting in unmatched trajectory intervals across methods. This reduces confidence in the corresponding camera rankings and should be addressed through matched-interval evaluation and explicit GLOMAP registration-failure accounting.
2)The ablations examine injection and perturbations but do not isolate the benefit of complete geometry. Without a controlled comparison against reference-visible-only caches, the explanation of how geometry completion contributes remains incomplete, although the overall system has empirical support.
Questions for authors
1)Were camera trajectories resampled to common timestamps before computing RPE? If not, do the relative rankings hold when evaluated over matched trajectory intervals, and how are GLOMAP registration failures handled?
2)Did the Uni3C evaluation in Table 2 enable its human-animation branch and use SMPL-X motion conditions? Appendix C.2 explicitly lists only camera trajectories and scene point clouds.
2 comments
Thank you for the clarifications and the additional evaluation.
The camera-metric concern is largely addressed. The confirmation that videos were already frame-aligned, together with the denser 49-frame evaluation and the accounting for the 11 unregistered WorldForge frames, increases my confidence in the reported camera rankings. The Uni3C setup in Table 2 is also now clear: the human-animation branch was enabled with SMPL-X motion conditioning, and the appendix has been updated accordingly.
The discussion of complete versus visible-only geometry is only partly resolved. I understand that a visible-surface cache would change the method’s design and that testing this condition without retraining would introduce a training–inference mismatch. At the same time, this remains a conceptual argument rather than a controlled comparison that isolates the benefit of geometry completion, so that part of the original concern is not fully closed.
Overall, the main evaluation issues have been addressed, and I will keep my rating of 3 (Weak acceptance).
We sincerely thank the reviewer for the careful review and insightful feedback.
C1. RPE comparability across different sequence lengths
In the paper's evaluation, we indeed aligned all videos to the same frames number before computing the camera metrics by following WorldForge’s evaluation protocol (EVO library default setting). The writing description there is fixed by adding the adjective ‘frame-aligned’. sorry for the confusion. To further address your concern, we additionally evaluate all methods with 49-frame alignment, which samples the trajectories more densely. ViewCrafter is excluded because it generates only 25 frames. As shown in the table, our method still achieves good and reliable camera accuracy under this denser alignment.
GLOMAP successfully registered over 99.99% of all frames. Only 11 frames, all from WorldForge videos with severe visual inconsistency. We excluded these frames from metric computation, which has a negligible impact on the results.
C2. The necessity of comparing the complete mesh with the initial visible surfaces
Restricting our cache to the visible surfaces would reduce our method to a reconstruction-and-warping design like WorldForge. Such designs struggle in diverse cases (Fig. 2, Table 1). Our method departs from them on purpose.Therefore, we view this comparison as contrasting two design philosophies rather than necessary ablation study. Our existing evaluation therefore contrasts coarse but instance-complete geometry with pixel-aligned reconstructions containing only visible surfaces, using each method’s native representation and training strategy.
Warping the initially visible normals of our coarse proxy would remove its geometric completeness without providing the pixel alignment of a reconstruction-based proxy. This would discard the main benefit of our representation while retaining its approximation errors. Testing this modified condition without retraining would also introduce a training–inference mismatch, since our model is trained with foreground normals from the current view at each time step.
C3. Uni3C human-animation configuration
Yes. The Uni3C results reported in Table 2 were obtained with its human-animation branch enabled and SMPL-X motion conditioning. Appendix C.2 currently lists camera trajectories and scene point clouds but omits the human-motion conditioning used in this evaluation. We have revised this section to explicitly document the enabled branch and SMPL-X inputs (Highlighted in yellow).