acceptodds
Under review as a conference paper at ICLR 2027

Flow Matching Reinforcement for 3D Mesh Generation via Dynamic Homing Optimization

Abstract

Flow matching is central to 3D generation, yet in practice its reinforcement learning (RL) methods are largely adapted from 2D visual generation. Representative DPO-, GRPO-, and NFT-style objectives, when applied to negative trajectories, mainly steer predicted velocities away from the corresponding directions without explicitly specifying a target velocity field toward preferred samples. In 3D generation, constrained by pretrained model capabilities, rollout diversity, and reward-distribution complexity, directly applying these RL methods yields limited gains in geometric quality. We introduce a forward-process RL method Dynamic Homing Optimization (DHO), which reformulates negative-trajectory optimization as positive-sample attraction-guided dynamic homing. Specifically, Minimum-Cost Attractive Matching (MAM) assigns each negative sample a distinct positive target, and Time-Aware Dynamic Correction (TDC) then redirects its trajectory toward the target using a remaining-time-aware corrective velocity. Building on asynchronous online DHO, we develop Flow3D-Pro, an image-to-3D geometry generation framework. Experiments show that DHO outperforms representative DPO-, GRPO-, and NFT-style objectives in 3D generation, while Flow3D-Pro produces higher-quality 3D geometry than existing mesh generation methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D mesh generation, flow matching, reinforcement learning, preference optimization, image-to-3D generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_42CFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Dynamic Homing Optimization (DHO), an RL post-training method for flow-matching 3D generators. For each input image, the model generates multiple meshes and uses Gemini to score them. Minimum-Cost Attractive Matching (MAM) assigns each retained negative sample a distinct positive target. Time-Aware Dynamic Correction (TDC) trains the velocity field to move from negative-sample interpolants toward the assigned target. DHO is applied to the Hunyuan3D 2.1-based Stage I of Flow3D-Pro after SFT. The LATTICE-based Stage II remains frozen. The experiments report higher scores than GRPO-style, DPO-style, and NFT-style objectives, as well as improvements over the evaluated image-to-3D systems.

Critical strengths
  1. The main idea is simple and intuitive. DHO gives negative samples explicit positive targets instead of only suppressing their original directions. Policy updates use initial noises and rollout endpoints without storing full sampling trajectories, which simplifies implementation.
  2. Table 1 provides a controlled comparison of RL objectives with the same SFT initialization, reward model, and frozen Stage II refiner. The appendix includes ablations of the main design choices.
Critical weaknesses
  1. Diversity is not evaluated. Redirecting negative samples toward existing positive samples may reduce output diversity. The paper provides no diversity comparison with GRPO, DPO, or NFT.

  2. The comparison settings are not fully fair. Tables 2 and 9 and Figure 6 do not control for refinement. Stage II alone raises the Table 2 user study score from 4.14 to 5.64. Please compare Stage I outputs or use the same refiner for compatible baselines.

  3. Geometric accuracy metrics are missing. ULIP and Uni3D measure feature similarity, with gains of only 0.002 to 0.003 in Table 1. Direct geometric accuracy remains unevaluated.

  4. Reward reliability is not quantified. The paper reports neither Gemini–human agreement nor test-set rewards across methods. These results would help assess reward reliability and potential reward hacking.

Questions for authors
  1. How does DHO perform at G=16? What is the distribution of B during training? With G=8, B is at most 4. When B=1, MAM is equivalent to random pairing.

  2. Can Table 2 include LATTICE and SFT + Stage II in the same comparison group?

  3. How does DHO compare with standard flow matching along x~t=(1−t)x0l+txw∗\tilde{x}_t=(1-t)x_0^l+t x^{w*}, using target velocity xw∗−x0lx^{w*}-x_0^l? Table 4’s static variant retains the negative-sample interpolants and does not test this alternative path.

Reviewer_dWDaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on image-to-3D generation and proposes a forward-process RL method designed to improve the 3D geometric quality of existing image-to-3D models. Specifically, it introduces a Dynamic Homing optimization method, including core components: minimum-cost attractive matching and time-aware dynamic correction. It enables the existing Hunyuan 3D 2.1 model to generate 3D assets with more coherent structures and superior geometric quality.

Critical strengths
  • The paper introduces a new method called DHO. Unlike existing approaches such as GRPO, DPO, and NET, it can explicitly guide the trajectories of negative samples; the technique appears sound.
  • The proposed method can be applied directly to existing pre-trained models, suggesting good practical utility.
  • The method appears to yield high-quality generation results, outperforming the baseline approaches.
Critical weaknesses
  • In Section 3.2.1, the paper describes a "one-to-one" assignment strategy within a mini-batch, where each negative sample is paired with a single positive sample. Are the samples in such a mini-batch all generated from the same image, or are they a mixture of samples generated from different images? Could there be instances where a negative sample related to "Object A" is paired with a positive sample related to "Object B"?

  • Does a small Euclidean distance in the latent space intuitively imply that the generated meshes are geometrically similar?

  • The paper only improves the first component—the module responsible for generating the overall geometry in Hunyuan3D 2.1, while leaving the second stage (Stage II) fixed. However, as shown in the visualizations in Figure 6, the overall 3D shapes produced by the original Hunyuan3D 2.1 appear correct, with differences primarily manifesting in local details. While the proposed method yields richer details, it remains unclear whether these differences stem from the improvements made to the first stage or from the existing Stage II refinement process. Therefore, the authors should provide a visual comparison between "Flow3D-Pro" and "Flow3D-Pro-Base," or alternatively, a comparison between the original Hunyuan3D 2.1 and the version incorporating the improved stage I.

  • The reward mechanism for the proposed method relies entirely on Gemini 3.1 Pro. Given the question of whether this model is fully reliable, it is advisable to include visualizations of mesh results alongside their corresponding Gemini 3.1 Pro scores, or to provide an analysis of the alignment between the Gemini scores and evaluations provided by human experts.

Questions for authors
  • Detailed description of the mini-batch;
  • The relationship between Euclidean distance in latent space and the similarity of generated meshes;
  • Visual comparison with Flow3D-Pro-Base (or the first stage only);
  • The reliability of the scores provided by Gemini 3.1 Pro.
Reviewer_LeyWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Flow3D-Pro, a framework for image-conditioned 3D mesh generation that combines Flow Matching with a reinforcement learning method called Dynamic Homing Optimization (DHO). DHO consists of Minimum-Cost Attractive Matching (MAM), which matches low-reward samples to high-reward samples in latent space, and Time-Aware Dynamic Correction (TDC), which modifies the target velocity field based on the matched targets. The paper evaluates the method on 3D generation tasks and reports improvements over existing reinforcement learning baselines.

Critical strengths

1.The paper addresses an important problem: improving the quality of conditional 3D generation through reward-based optimization.

2.DHO introduces a structured approach to sample matching and velocity correction that is potentially applicable to a broader range of Flow Matching models.

3.The paper includes comparisons with several reinforcement learning baselines and ablation studies of the proposed components.

Critical weaknesses
  1. The motivation and novelty of DHO need further clarification. The paper reports improvements over existing reinforcement learning baselines, but the rationale for its key components, namely one-to-one positive-negative matching and time-aware velocity correction, remains unclear. Given prior work on reinforcement learning for Flow Matching, the authors should better distinguish DHO from existing approaches and explain why its matching strategy is preferable to simpler alternatives, such as guiding low-reward samples toward high-reward samples.

  2. The role of RL in improving geometric reconstruction is unclear. The qualitative results suggest that DHO improves the alignment of local structures, such as eyeglass lenses. However, it is unclear whether these improvements reflect a specific benefit of the proposed RL method or compensate for limitations in the base model's image-conditioned geometry generation. The paper should clarify why these geometric errors are better addressed through RL rather than improved supervised training or geometric conditioning, and demonstrate the specific contribution of DHO under a controlled comparison.

  3. The generality of the proposed optimization method remains unverified. The MAM and TDC mechanisms do not appear inherently specific to 3D generation and could potentially be applied to other Flow Matching-based generation tasks. However, the evaluation is limited to 3D generation. Experiments on another generation task would help establish whether the proposed method provides benefits beyond the particular setting considered in this paper.

I would be willing to give this paper a high score if the authors could either clearly explain why the proposed method works or demonstrate that this method, which is not inherently tied to 3D, is generally effective as a reinforcement learning approach.

Questions for authors

see weakness

1 comment

Thank you for the detailed response and additional experiments. The new text-to-image results provide evidence that DHO is effective beyond 3D generation, addressing my concern about its generality. I will therefore raise my score.

However, I am not yet convinced that a score of 4 is warranted, as the mechanism behind the improvements remains insufficiently clear. The one-to-one matching strategy has an intuitive motivation in terms of covering more positive samples and potentially preserving diversity. However, it is less clear why this construction should offer an advantage over existing reward-based optimization methods in improving properties such as fidelity to the conditioning image. A higher-reward endpoint is not necessarily better in every desired property, particularly under a multi-objective reward.

The additional experiments strengthen the empirical case for the method, but the explanation of why the proposed matching and correction mechanisms lead to these improvements remains incomplete.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.