acceptodds
Withdrawn from ICLR 2027

Free360Head: Continuous-View Full-Head Generation via 3D-Aware Guidance

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generating a continuous free-viewpoint video of a complete human head from a single portrait requires plausible completion of unseen regions, identity preservation, precise trajectory control, and cross-view consistency. Existing 3D-based approaches provide consistent rendering but are limited by the quality of single-image reconstruction, whereas generative approaches produce realistic details but may drift in geometry and identity under large viewpoint changes. We present , to the best of our knowledge the first reference-independent 3D-aware trajectory control framework, combining the strengths of both routes. Our method explicitly disentangles identity from trajectory: an arbitrary-view reference portrait specifies identity without defining any output viewpoint, while an identity-agnostic depth sequence independently controls the target trajectory. Compared with camera-pose conditioning, the depth sequence provides dense, frame-aligned cues for projected silhouette, visible surface, and depth ordering, enabling fine-grained geometric control. Built on the pretrained Open-Sora video diffusion Transformer, uses a reference-prefixed latent injection scheme and an auxiliary reference reconstruction objective to propagate identity throughout depth-guided video generation. Experiments demonstrate improved visual fidelity, identity preservation, cross-view consistency, and trajectory control over existing approaches.

Primary area:
generative models
Keywords:
Video Generation, Multiple-View Diffusion Models, Single-Image 3D Face Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4TKXOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Free360Head, a conditional 360 degree full-head video generation model built on top of Open-Sora 1.3. To synthesize continuous full-head orbit videos from an arbitrarily posed portrait, it conditions camera motion on dense depth maps rendered from a shared canonical 3D head proxy (generated by PanoHead) while prepending the reference latent as a temporal prefix token to inject identity. The model is trained and evaluated on synthetic heads from PanoHead and 3DGS reconstructions from RenderMe-360 across varying azimuth and pitch trajectories.

Critical strengths

The paper presents a clean method for generate 360 head videos without geometry or identity drifting, by conditioning the video generation on the depth information of a proxy head. This proves to be superior than basic camera-pose conditioning baselines. The experimental comparisons are well-organized, and the visual results show smooth, plausible completion of occluded hair and ear regions.

Critical weaknesses

My major concern is that the methodological contribution is incremental and potentially dresses up standard DiT conditioning tricks as novel formulations. At its core, the method simply fine-tunes a video model to "render" a head conditioned on a base geometry (via depth maps) and an image reference. Video generation with depth map condition for controllability is a well established paradigm in literature (e.g. ControlVideo [Zhang et al., ICLR 2024], and Diffusion as Shader [Gu et al., 2025])). At the architectural level, temporally prepending reference image latent for conditioning is also a standard trick. Therefore, this paper reads like a rather straightforward domain application rather than a methodological advance.

Questions for authors

Can you include a zero-shot baseline using a modern open-weight video DiT such as Wan2.2-VACE-Fun (or similar open/close weight video models) conditioned on the exact same PanoHead depth map sequences and reference portraits?

Reviewer_ArYmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Free360Head, a framework for generating continuous 360° full-head videos from a single reference portrait. It disentangle identity generation and trajectory control, using depth sequences rendered from a generic 3D head proxy to provide dense geometric guidance for camera trajectory control. Built on the pretrained Open-Sora video diffusion Transformer, it introduces a reference-prefixed latent conditioning and an auxiliary reference reconstruction objective to disentangle identity preservation from viewpoint control. Experiments on PanoHead and RenderMe-360-derived datasets demonstrate improvements in multiview image quality, identity preservation, and trajectory accuracy compared with baseline methods.

Critical strengths

The paper disentangles identity from trajectory in a clean conditional way. The identity is acquired from arbitrary-view reference and the trajectory is represented with identity-agnostic depth. This depth sequence can be reused across identities, and provide image space geometric guidance, addressing ambiguities in large rotations. Experiments on both datasets with comprehensive metrics shows strong improvements in the resulting videos.

Critical weaknesses
  • Using a canonical head proxy to generate depth can result in incorrect identity geometry, such as shape of head/face, style of hair, etc. This could cause geometry/style inconsistencies in some cases. Robustness on the mismatching between proxy geometry and real identities is not reported.
  • The auxiliary reference reconstruction is a key design elemen. Current ablation compares the full reference-prefixed design with a spatial concatenation baseline. It is unclear how much of the improvement comes specifically from the auxiliary reconstruction.
  • 3D consistency is not directly measured beyond pose metrics. Additional tests such as reconstructing generated views into a 3D representation, such as 3D Gaussians, and measuring multi-view consistency would strengthen claims on continuous 360° full-head generation.
Questions for authors
  • How well does the proposed method generalize to truly arbitrary reference viewpoints and camera trajectories? The reference images are limited to yaw angles of ±45° and pitch angles of ±40°, while the target videos follow nine predefined rotation trajectories. Can the method handle more extreme reference viewpoints (at least two sides) and more complex, non-circular camera trajectories beyond those evaluated in the paper?
  • Can you ablate the auxiliary reference reconstruction (keeping the temporal prefix but removing supervision on that slot) to isolate its contribution to identity preservation?
  • How large is the geometric mismatch between the generic head proxy and the actual head shapes? Since the same proxy is used for different identities, could you provide a simple quantitative analysis, such as silhouette comparison between the rendered proxy and ground-truth heads, to measure how well the generic geometry matches different subjects?
Reviewer_47JSOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Free360Head, a model that turns a single portrait into a 49-frame, 360° full-head video along a requested camera path. Identity is controlled by the reference image while a depth-render of a coarse head guides the viewpoint trajectory. The model itself is based on Open-Sora 1.3 takes about a week on four A800s to train. The model is trained using using 41,940 pairs from 320 PanoHead-generated heads and 146 RenderMe-360 heads reconstructed with 3DGS. Each pairs one of 10 references (yaw ±45°, pitch ±40°) with one of 9 constant-pitch 360° orbits. Both quantitative and qualitative results show improvement with respect to prior work.

Critical strengths
  1. The paper is well written and easy to follow

  2. Both quantitatively and qualitatively the paper has better results than prior work. However, synthetic results seem much better than the real ones

Critical weaknesses
  1. The architecture is not explained very well, it is hard to see how the conditions and video tokens interact with each other.

  2. To me, the reference being concatenation to both the condition and the video tokens separately temporally does not make sense. A temporal concatenation in DiT like models is essentially a sequence concatenation at the attention level. A repeating concatenation is just concatenating the same token twice with the only difference between them being the RoPE embeddings. There should be an ablation with the reference only concatenated to the condition vs. only to the video tokens vs. to both (as the method does not) to evaluate which design choice is the best.

  3. There needs to be more evaluation of all methods on real world data.

Questions for authors

Why does DiffPortrait360 perform so much worse on PanoHead than RenderMe360 data?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.