acceptodds
Under review as a conference paper at ICLR 2027

HairFlow: Modeling 3D Hair Strands with Multi-Modal Conditioned Diffusion

Abstract

Modeling 3D hair strands is challenging due to complex hairstyle geometry, scarce high-quality training data, and incomplete or noisy input observations. Foundation models offer broader hairstyle priors, but their outputs are not directly usable for strand-based rendering and simulation. Existing approaches require costly per-instance optimization to fit hair priors to meshes from foundation models and 2D orientations from specialized predictors trained on limited hair data, restricting efficiency and generalization. We present HairFlow, a multimodal conditioned diffusion framework for efficient and controllable hair strands modeling. HairFlow conditions strand generation on two complementary representations: a coarse mesh that constrains global shape and multi-view sketches that specify local strand flow. HairFlow combines an Orientation VAE with a conditioned diffusion transformer to generate volumetric fields that are traced into explicit strands. To overcome the lack of paired sketch–strand training data, we introduce an automatic synthesis pipeline that generates geometrically aligned multi-view sketches with diverse styles representative of inference-time inputs. Experiments show that HairFlow generates plausible 3D strands that faithfully follow the input conditions across diverse hairstyles, substantially reduces inference time, and supports flexible multimodal control.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Hair Reconstruction, 3D Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_6G2HOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents HairFlow, a framework for generating 3D hair strands conditioned on a coarse hair mesh and multi-view sketches. The mesh defines the overall hairstyle shape, while the sketches provide information about local strand directions. Both conditions can be obtained from input images using pretrained models or provided by artists. The method first encodes the mesh and sketches into condition tokens using a pretrained mesh encoder and a camera-aware sketch encoder. A conditional diffusion transformer then generates latents in the space of a pretrained Orientation VAE. These latents are decoded into a volumetric orientation field, from which explicit hair strands are extracted through strand tracing. The authors also introduce a synthetic sketch generation pipeline to construct aligned mesh-sketch-strand training data from existing 3D hair assets. The paper evaluates HairFlow against existing hair reconstruction methods and demonstrates its use with different hairstyles, sketch styles, and artist-provided guidance.

Critical strengths

The method produces convincing visual results across diverse and complex hairstyles, preserving overall geometry and strand flow. The multi-view results further demonstrate its ability to handle different hairstyles and stylized inputs.

Combining coarse meshes and multi-view sketches provides complementary control over global shape and local strand directions. The framework also supports flexible inputs and artist-guided hairstyle generation.

Critical weaknesses

Limited end-to-end efficiency improvement. Although the paper claims an 87× speedup over HairLRM's latent optimization, the full pipeline from a single image still takes 4–5 minutes, making the efficiency advantage over existing methods less significant.

Limited robustness analysis of upstream generation. The method heavily relies on image generation and image-to-3D reconstruction. However, generated multi-view images may be geometrically inconsistent, and their quality can vary across inputs and generation runs. The paper lacks evaluation of the robustness of these upstream models and their impact on final reconstruction quality.

Questions for authors

How consistent and stable are the generated multi-view images, and how do their inconsistencies affect mesh reconstruction and final strand quality?

1 comment

Robustness to Upstream Generation Variations. We acknowledge that upstream image generation is stochastic and that some geometric inconsistencies may exist across generated views. However, these variations do not necessarily lead to poor final reconstructions.

First, the image-to-3D models can leverage learned geometric priors to reconcile moderate inconsistencies across views and recover plausible coarse hair meshes. Second, HairFlow further tolerates variations in these conditions through complementary constraints: the mesh provides global geometric guidance, while the learned 3D orientation prior and sketch conditioning guide local strand flow. We also introduce mesh latent perturbations, camera jitter, and diverse sketch augmentations during training to improve robustness to imperfect inputs.

To further investigate this, we have added controlled experiments in Supp. Fig. 11, comparing different upstream models. Specifically, we compare meshes generated by Tripo H3.1, Meshy 7, and Hunyuan3D while keeping the sketch fixed, and sketches generated by Nano Banana, GPT Image 2, and Qwen-Image-2.1 while keeping the mesh fixed. Although the input conditions and resulting reconstructions exhibit some variations, HairFlow consistently produces plausible strand structures across these different models, demonstrating that HairFlow is flexible with respect to upstream model choices.

Finally, as a stochastic generative model, HairFlow does not aim to recover an identical strand configuration across different sampling runs. Given the inherent ambiguity of reconstructing occluded 3D hair from images, our goal is to generate plausible and geometrically coherent strands consistent with the available conditions, rather than a unique deterministic reconstruction. Nevertheless, severe upstream errors may still degrade the final results, and improving the consistency of upstream foundation models is beyond the scope of this work.

Reviewer_QHKiOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents an image-to-3D hair asset pipeline that applies conditional flow matching from Gaussian noise to latent space, followed by decoding into a 3D asset. To do so, they create a hair orientation VAE that encodes and decodes hair assets, achieving state-of-the-art reconstruction performance compared with HairLRM. This VAE is used to decode the conditionally encoded mesh and sketch tokens that come from an in-the-wild image and a sketch/human input, in which only the image may be available, and the sketch is generated.

Critical strengths

This paper improves upon HairLRM and HairGPT for hair generation, showing critical improvement over prior baselines. It seems to have found a good architecture for generating hair.

  • The baselines show (even in their own paper-teaser figures) poor or overly simple hair reconstruction. While HairFlow is much better, it lacks a bit of generalization like HairGPT, which can do afro-like hair.
  • The paper improves on two fronts: VAE reconstruction architecture and denoising Flow matching applied to hair.
  • Their Automatic Hair Sketch proposes a new way of generating a sketch dataset.
  • The paper looks pretty and well-formatted.
Critical weaknesses
  • I found the paper sound; however, for me, the hardest part to read was the abstract. Specifically, the "Existing approaches require costly per-instance optimization to fit hair priors to meshes from foundation models and 2D orientations from specialized predictors trained on limited hair data, restricting efficiency and generalization." Please rephrase.

  • Why is the camera important? I think this should be an ablation.

  • Why 4 views? Adding more, or removing some, may improve or worsen the results; this should also be an ablation! I would like an ablation with 3 views spaced 120 degrees apart as input. If possible in such a short amount of time.

  • In Figure 10, you provide a text prompt for 3D hair; could you further extend the results on this?

  • Throughout the paper and in the abstract, the word "costly" appears. However your claim to being faster appears in the Inference Time Paragraph which I would like to see it transformed into a table, otherwise it may deviate readers into thinking you are the fastest, which you are not: HairStep takes 0.51 minutes, HairGPT takes ~1.00 minutes, DiffLocks takes ~0.81 minutes, Im2Haircut takes 6.31 minutes, HairLRM takes ~15 minutes, and Hao et al. (2026) takes ~20 minutes, and yours takes in total takes ~5.12 minutes. I understand that your model is better at the generation part, which I think is the most important, but this should be clarified in a table.

  • You should provide failure case images as you stated in the limitations paragraph. Specifically, for me, you should try short hair (like a man's military cut); three types of afro curly hair: one long, one short, and one medium.

  • Training should specify GPU type, VRAM size, and number of hours/days to train.

  • There are some minor grammar details. Line 471 Flexible Sketch Guidance is missing a FIG. Also, throughout the text, there are Fig and Fig. with a dot. Also, if using Fig, use Tab for Tables.

  • This area lacks an open-source commitment; if possible, open-source the code.

Questions for authors
  • Since the introduction of Trellis, their SLAT seems to be pretty good, maybe it is worth a try.

  • Why not use an image-to-sketch model to reconstruct your sketches?

  • How did you run HairLRM if it has no open-source code?

2 comments

We thank the reviewer for the constructive feedback!

  1. Clarity of abstract. We have revised the sentence in the abstract to improve clarity: "Recent methods have explored converting foundation-model outputs into explicit 3D hair strands. However, orientation predictors trained on limited hair data can hinder generalization, while per-instance optimization reduces efficiency."

  2. Camera ablation. Camera parameters establish the geometric correspondence between 2D sketch directions and the 3D orientation field. We have added an ablation in Fig. 10 of the supplementary material, comparing correct camera conditioning with a deliberately perturbed camera (90° rotation). Incorrect camera parameters misalign the sketch guidance with the 3D geometry, leading to degraded strand reconstruction.

  3. Input View configuration. HairFlow is not restricted to four fixed viewpoints. During training, we randomly sample sketch viewpoints over the viewing hemisphere (with a bias toward frontal views), exposing the model to diverse viewing directions rather than a predefined set of cameras. The cross attention architecture also accommodates a variable number of input views. We have additionally included results with three input views distributed at 120° intervals (Fig. 9 in the supplementary material), demonstrating the flexibility of HairFlow under different view configurations.

  4. Training details. We have added the GPU specifications, VRAM capacity, and training time for both the Orientation VAE and diffusion transformer in the Implementation Details (Sec. 4), highlighted in red in the revised manuscript.

  5. Grammer errors. Thanks for pointing out these issues! We have corrected the errors throughout the manuscript.

  6. Design choice of 3D representation. We considered SLAT during the design of HairFlow. However, TRELLIS primarily uses sparse latent features associated with object surfaces, whereas our task requires modeling a volumetric orientation field throughout the hair region, including its interior. Adapting SLAT to this task would require reconsidering its voxel activation strategy, potentially reducing its sparsity advantage. We therefore adopt a 3DShape2VecSet-based representation without explicit surface-based voxel activation. Exploring SLAT-based representations for volumetric hair orientation fields is an interesting direction for future work.

  7. Design choice of sketch model. Most general-purpose image-to-sketch methods, such as SwiftSketch, primarily focus on object contours and salient structural lines, whereas HairFlow requires sketches that explicitly describe the internal flow of hair strands. We therefore use Nano Banana, whose semantic priors and instruction-following capabilities allow it to generate coherent hair-flow sketches, including plausible directional guidance in ambiguous regions. This approach has worked well in our experiments without additional task-specific training.

  8. Comparison with HairLRM. We contacted the HairLRM authors and provided them with our input images and Tripo-reconstructed hair meshes. The authors ran HairLRM using their implementation and shared the reconstruction results with us for comparison.

The additional experiments strengthen the evaluation; Figure 12 clarifies the drawback of your method, and the discussion of limitations provides a more balanced presentation. Although some concerns regarding generalization to challenging hairstyles and reliance on upstream foundation models remain, I believe the authors have adequately addressed my major concerns. I am therefore leaning toward increasing my rating.

Reviewer_o6PnOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents HairFlow, a multimodal conditioned diffusion framework for generating 3D hair strands from coarse 3D meshes and 2D sketches. The method uses pretrained foundation models to obtain coarse hair geometry and directional sketches, then generates a volumetric 3D orientation field through a conditional diffusion transformer and extracts explicit hair strands. The main contributions include integrating existing foundation models with a strand generation framework, replacing per-instance optimization with direct diffusion-based generation, and introducing an automatic pipeline for synthesizing training sketches. Experiments demonstrate improved reconstruction quality over previous methods, particularly for complex hairstyles, faster inference, and the ability to produce strands suitable for physical simulation.

Critical strengths
  1. Significant improvement in reconstruction quality. The paper demonstrates substantial quality improvements over previous works, particularly for complex hairstyles such as ponytails and updos that existing methods struggle to handle. The results in Figure 4 are visually impressive and represent a new standard for 3D hair reconstruction quality. This is the strongest reason supporting my recommendation for acceptance.

  2. Practical integration of existing techniques into an effective pipeline. Although the individual components are not technically novel, the paper successfully combines recent advances in 2D/3D foundation models with strand-based hair generation. The resulting pipeline is efficient and produces high-quality, explicit hair strands suitable for downstream physical simulation, as demonstrated in the supplemental video.

Critical weaknesses

Below are critical weaknesses:

  1. Insufficient validation of reconstruction quality on challenging hairstyles and unseen views. While the paper demonstrates impressive visual results, I feel the validation of reconstruction quality is still insufficient, particularly for complex hairstyles. Most of the results in the main paper and supplementary material focus on frontal and side views, while backside views are rarely shown. Since a large portion of the hair geometry is not visible from the input image, it is important to evaluate whether the method can reconstruct plausible and coherent strand structures in these occluded regions, rather than just producing visually appealing results from the input viewpoint. I would encourage the authors to show backside views of the hairstyles presented in the paper and discuss any potential artifacts. Additionally, how does the method perform on challenging hairstyles such as braids, where strand connectivity and interweaving structures are particularly important? These cases would help demonstrate whether the method can handle genuinely complex 3D hair structures beyond matching the overall silhouette and local strand orientations.

  2. Any penetration handling between hair-hair and hair-body?

Below are not critical weaknesses:

  1. (light weakness) Limited technical novelty. The paper does not introduce fundamentally new techniques, as most components have been explored in previous works. I suspect that much of the quality improvement comes from recent advances in external 2D/3D foundation models, such as Tripo H3.1 and Nano Banana, rather than the proposed framework itself. Although the integration is effective, the paper does not sufficiently isolate the contributions of these components, making it difficult to assess the actual technical advances.

  2. Missing related works: Zhou, Yi, et al. "Hairnet: Single-view hair reconstruction using convolutional neural networks." European Conference on Computer Vision. Cham: Springer International Publishing, 2018.

Questions for authors

Design choice of sketch conditioning. I have some questions about the motivation for using 2D sketches as an intermediate representation. As I understand it, the sketches are used to provide directional guidance for the hair strands. However, a sketch is essentially a sparse 2D orientation map, represented by curves rather than a dense orientation field. Why do the authors choose to use sketches instead of dense 2D orientation maps, which seem to provide more complete directional information and may be a less ad hoc representation? Since the final goal is to predict a 3D orientation field, would it also be possible to directly condition the model on the input images and 3D mesh, without explicitly generating sketches? I would appreciate more discussion or experimental evidence to justify this design choice, particularly whether sketch conditioning offers advantages in reconstruction quality, robustness, or controllability compared to these alternatives.

Design choice of canonical head coordinates. As I understand it, HairFlow generates hair strands in a standardized canonical head coordinate system. To achieve this, the inference pipeline involves several steps: separately reconstructing the head and hair meshes using Tripo, registering the hair mesh to the reconstructed head using similarity ICP, and detecting facial landmarks to align the reconstructed head with a template bust through a similarity transformation. This seems like a rather complicated and potentially unnecessary preprocessing pipeline. Why is it necessary to generate hair in a canonical coordinate system rather than directly reconstructing the hair in the original head coordinate system? Is this mainly due to the training data being aligned to a canonical head, or is there a fundamental limitation of the proposed representation or architecture? I would appreciate more discussion on the necessity of this design choice and whether the method could be extended to directly support arbitrary head geometries and poses without these registration steps.

Any open source plan?

3 comments

We thank the reviewer for the valuable suggestions !

  1. Unseen views and occluded regions. We have added extensive multi-view visualizations, including back and top views (Figs. 13–15 in the supplementary material). Beyond the complex hairstyles such as ponytails and updos already demonstrated in Fig. 4, these results further show that HairFlow produces plausible strand configurations in regions not directly observed in the input. This is enabled by the complementary conditions: the input mesh constrains the global hair geometry, while the learned 3D orientation prior from ground-truth strand data guides local strand flow in occluded regions. We also report quantitative results on 326 held-out hairstyles in Table 2, achieving angular errors of 6.25° for VAE reconstruction and 22.92° for mesh-and-sketch-conditioned generation, both outperforming HairLRM.

  2. Penetration handling between hair-hair and hair-body. HairFlow currently does not explicitly enforce collision avoidance between hair strands or between hair and the head/body. Our method focuses on recovering plausible strand geometry and flow from multimodal conditions. By representing hair with a shared 3D orientation field, HairFlow encourages locally coherent strand trajectories, which helps reduce abrupt directional inconsistencies. However, this does not explicitly enforce spatial separation between individual strands, and therefore cannot guarantee collision-free geometry. For hair-body intersections, geometric constraints could be incorporated into strand tracing, for example, by detecting intersections with the head/body surface and terminating or correcting the corresponding trajectories. We acknowledge that explicit collision handling is currently absent and identify collision-aware strand tracing and geometric constraints as important directions for future work.

  3. Limited technical Novelty. We would like to clarify that HairFlow's contributions go beyond integrating existing foundation models. First, we introduce a high-capacity orientation-only VAE and a mesh-and-sketch-conditioned diffusion framework that directly generates 3D orientation fields, replacing HairLRM's costly per-instance optimization. Second, we develop an automatic pipeline for synthesizing diverse, geometrically aligned multi-view sketch–strand pairs. Importantly, the improvements are not solely attributable to stronger upstream models. In Fig. 4, HairLRM and HairFlow use the same Tripo-reconstructed meshes. In the first two examples, HairLRM is provided with reasonably accurate dense HairStep orientation maps, while HairFlow uses only sparse sketches, yet achieves better reconstruction quality. Furthermore, Table 2 shows that HairFlow achieves substantially lower angular error even with mesh-only conditioning (27.43° vs. 36.81°), demonstrating improvements beyond the sketch guidance itself.

  4. Missing References. We thank the reviewer for pointing out the missing reference. We have included the citation in the revised manuscript.

  1. Design choice of sketch conditioning. Directly conditioning on RGB images requires the model to disentangle hair flow from appearance variations, such as lighting, hair color, and accessories. Although dense orientation maps provide richer directional information, obtaining reliable pixel-level maps from in-the-wild images remains challenging: image-editing models cannot consistently produce accurate orientation maps, while specialized predictors trained on limited hair datasets often generalize poorly to diverse hairstyles. In contrast, sketches provide sparse yet informative local directional cues through stroke geometry, while offering an intuitive interface for both foundation-model-generated guidance and artist control. Their effectiveness is also supported by our ablation in Table 2, where adding sketch conditioning reduces the angular error from 27.43° to 22.92° compared with mesh-only conditioning. We have further clarified these design motivations in the newly added Discussion (Sec. D) of the supplementary material.

  2. Design choice of canonical head coordinates. Our preprocessing pipeline serves two alignment purposes. First, it aligns the input 3D mesh with the coordinate distribution of the training data. To ensure robust convergence, all training hairstyles are normalized to a common coordinate frame and spatial range, in which the Orientation VAE learns 3D orientation fields. Therefore, inference meshes need to undergo similar normalization. Since the head and hair meshes are reconstructed independently, we first use similarity ICP to bring them into a common coordinate system. This does not require precise geometric correspondence to the template head. Second, it establishes correspondence between the 2D sketches and the 3D mesh. Since sketches represent projected hair-flow directions, camera parameters are needed to relate their 2D directional cues to the 3D geometry. We also introduce camera jitter (5°) during training to improve robustness to estimation errors. However, large misalignments can degrade reconstruction, as demonstrated by the 90° camera perturbation in Fig. 10.

The authors have addressed my concerns regarding points 1, 2, and 4. However, I still believe the technical novelty is incremental. Therefore, I maintain my score of weak acceptance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their positive assessment of HairFlow's reconstruction quality and practical value, as well as their constructive feedback. We have revised the manuscript and supplementary material to address the major concerns, as summarized below. All revisions are highlighted in red in the updated manuscript.

1.Challenging Hairstyles and Failure Cases. While HairFlow demonstrates strong reconstruction quality on complex hairstyles such as ponytails and updos (Fig. 4), certain challenging structures remain difficult. We have added examples of braided hairstyles, short, medium, and long Afro-textured hair, and military-style buzz cuts (Supp. Fig. 12). Braided hairstyles remain challenging because our orientation-field representation does not explicitly model strand connectivity or interwoven topology. For Afro-textured hair, high-frequency orientation variations are difficult to capture at the current spatial resolution, resulting in overly smooth curls. Very short hair occupies a thin volume near the scalp that is difficult to resolve accurately. We have further discussed these limitations in the supplementary material and leave topology-aware representations as an important direction for future work.

2.Inference Efficiency. As suggested by Review QHKi, we have added two tables to clarify the runtime comparison: Table 3 reports the end-to-end runtime of different methods, while Table 4 provides a stage-wise comparison between HairFlow and HairLRM. While HairFlow achieves substantially improved reconstruction quality, its end-to-end runtime (5.12 min) is longer than some existing methods, mainly due to condition preparation (3 min) and strand extraction (1–2 min). Importantly, our core generation stage takes only 6.9 seconds, achieving an approximately 87× speedup over HairLRM's per-instance optimization (~10 min). This highlights the efficiency of our diffusion-based generation framework while providing higher-quality strand reconstruction.

3.Open source plan. We plan to open-source the code and model after completing the internal review and obtaining the necessary approvals.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.