acceptodds
Under review as a conference paper at ICLR 2027

Multi-View Identity-Consistent Video Generation under Large Facial-Angle Variations without Cross-Paired Data

Abstract

Reference-to-video generation still struggles to preserve a known identity when the generated face turns far from a single reference view. Adding multiple views supplies missing identity cues, but in-paired training introduces a complementary failure: the model can copy one reference angle for several frames and then switch abruptly to another. We call this failure view-dependent copy-paste. We present Mv²ID, a multi-view conditioned framework that improves identity consistency while retaining natural facial motion using only in-paired supervision. The method concatenates video and reference latents, applies region-masking training to prevent direct copying and encourage complementary cue aggregation, and uses a reference-decoupled rotary positional encoding (RD-RoPE) so reference images are not interpreted as future video frames. We also build a 22K-video large-angle dataset and introduce Multi-view Reference Consistency (MvRC) for identity evaluation across viewpoints. On a held-out set of 30 identities and 150 prompts, Mv²ID obtains MvRC scores of 0.544/0.507 and a NaturalScore of 4.69, achieving the best identity consistency and text–video alignment among the compared methods while remaining competitive in naturalness.

Primary area:
generative models
Keywords:
Multi-View Identity-Consistent Video Generation under Large Facial-Angle Variations without Cross-Paired Data

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_t4i4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies "view-dependent copy-paste" in multi-view reference-to-video generation trained with in-paired data, where the generated face locks to one reference angle and then switches abruptly. Mv2ID addresses it with region-masking training and a reference-decoupled RoPE, and the authors also curate a 22K-video large-angle dataset. Mv2ID achieves the best identity consistency (MvRC) among the compared methods while remaining competitive in naturalness.

Critical strengths
  • Practical problem. This work has a clear observation on a practical problem termed "view-dependent copy-paste". I think this problem is of high value.
  • Both components add no inference-time cost, and the authors openly discuss the identity–naturalness trade-off in Sec. 5.3 and Sec. 6.
  • Carefully designed experiments provide insightful results.
  • It would be a treasure for the academic community if the 22K-video dataset was released as claimed.
Critical weaknesses
  • No quantitative evaluation or metric for the main problem "view-dependent copy-paste". Actually, there is a text-image-to-image "copy-paste" score proposed in [1]. It seems that a similar analysis about the similarity-copy-paste trade-off will greatly benefit this paper. I would definitely raise my rating if this quantitative evaluation agrees with the main conclusion.
  • MvRC and NaturalScore are limited, as mentioned in the paper. It seems that MvRC encourages "view-dependent copy-paste". And a VLM-based absolute score is not quite persuasive.
  • Ablation on data. I think the curated dataset is a major issue when comparing with baselines. As a contribution mentioned in abstract, the dataset itself deserves an ablation experiment.

[1] WithAnyone: Towards Controllable and ID-Consistent Image Generation

Questions for authors
  • How is the RR in MvRC selected, and what is the view distribution in RR? (Like, how many images are in RR, and how many of them are front/side views)
  • Are the three conditioning references included in the ten references used for MvRC? How are frames with failed face detection handled?
2 comments
AuthorsOpenReview ↗

Response to Reviewer t4i4

We thank the reviewer for the constructive suggestions on copy-paste evaluation, metrics, and the dataset.

1. Quantitative evaluation of reference preference.

Thank you for recommending WithAnyone. Its reference-versus-target comparison provides a valuable perspective on reference-copying behavior. We will cite this work in the revision.

We adapt its normalized angular-distance formulation to our setting with three conditioning references and ten held-out same-identity gallery images. The gallery excludes the conditioning references. For each valid generated-frame embedding gg, we select the closest conditioning reference:

r∗(g)=arg⁡max⁡rj∈Rsim⁡(g,rj).r^*(g)=\arg\max_{r_j\in R}\operatorname{sim}(g,r_j).

We compute the score against each gallery embedding hkh_k:

mk(g)=d(g,hk)−d(g,r∗(g))max⁡{d(hk,r∗(g)),ε}.m_k(g)=\frac{d(g,h_k)-d(g,r^*(g))}{\max\{d(h_k,r^*(g)),\varepsilon\}}.

M(g)=110∑k=110mk(g).M(g)=\frac{1}{10}\sum_{k=1}^{10}m_k(g).

d(a,b)=arccos⁡(sim⁡(a,b)).d(a,b)=\arccos(\operatorname{sim}(a,b)).

Here, ε\varepsilon prevents division by zero. Higher scores indicate stronger preference for the closest conditioning reference. Scores are averaged over valid frames within each video and then across test cases. This is a gallery-based adaptation: the held-out images provide alternative same-identity appearances rather than prompt-corresponding target GT.

Masking ratioReference-preference score ↓
40%0.417
60%0.169

Increasing masking from 40% to 60% reduces reference preference under this evaluation. We report this supplementary measure alongside identity consistency and naturalness.

2. MvRC, NaturalScore, and the identity–naturalness trade-off.

MvRC measures identity similarity between generated faces and a held-out multi-view gallery. The supplementary measure above evaluates relative preference for conditioning references. NaturalScore follows the OpenS2V-Nexus protocol and provides a complementary VLM-based assessment of facial-motion naturalness.

The component ablation in Table 2 illustrates the identity–naturalness trade-off:

ConfigurationMvRC-Arc ↑MvRC-Cur ↑NaturalScore ↑
5B base0.5460.5153.70
+ Region masking + RD-RoPE0.5350.5024.52

Our objective is to improve naturalness while retaining strong identity consistency, rather than solely maximizing reference-face similarity.

3. Dataset contribution and data ablation.

Our dataset contains 22,516 video clips and is publicly available on Hugging Face. Its construction emphasizes substantial within-clip facial-pose variation for multi-view identity learning.

We agree that the dataset contribution merits separate quantification. Table 2 isolates model components using the same training data. A data-only ablation instead varies the training data while fixing the backbone, training budget, and components. We will clarify this distinction; a controlled data-only ablation is not included in the current results.

4. MvRC gallery and failed detections.

The test set contains 30 identities and 150 cases, with five prompts per identity. MvRC uses ten held-out images per identity, totaling 300 images. Test identities are disjoint from training, and the three inference-time conditioning references are excluded from the gallery.

Images are selected for quality and diversity, including sharpness, contrast, brightness, and diversity across clips, shots, and frame positions, rather than a fixed yaw quota. We measured the gallery distribution using InsightFace (buffalo_l):

ViewYaw definitionCountPercentage
Frontal∣yaw∣<30∘\lvert\mathrm{yaw}\rvert<30^\circ19163.7%
Leftyaw≤−30∘\mathrm{yaw}\leq-30^\circ5618.7%
Rightyaw≥30∘\mathrm{yaw}\geq30^\circ5317.7%

Additionally, 16.7% of images have absolute yaw of at least 45 degrees. All 30 identities have yaw ranges above 45 degrees; 29 exceed 60 degrees, and 22 include both left and right views. These statistics describe static-gallery coverage.

Generated frames without a valid face detection or identity embedding are excluded from the MvRC average. The generated-frame failure rate is not included in the current results. We will clarify gallery composition, separation from conditioning references, and invalid-frame handling in the revision.

Reviewer_t4i4OpenReview ↗

Reviewer Response

Thank you for your response and I will keep my positive rating.

Reviewer_DVQGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies identity-preserving video generation from multiple facial reference views, with references and target clips drawn from the same source video during training. It combines latent reference concatenation, random region masking of the references, and a modified RoPE scheme that places references at a shared temporal coordinate with different spatial offsets. It also introduces a 22K-video dataset and evaluates identity similarity and naturalness using face embeddings, a vision-language judge, and a small human study.

Critical strengths

The examples of reference-angle locking identify a useful failure case for multi-view conditioning. The main comparison also shows higher face-embedding similarity than the evaluated baselines, and the human ratings provide some supporting evidence. The changes are simple and do not add inference-time modules.

Critical weaknesses

1. The novelty of the reference-conditioning design is not established. Kaleido [1] is cited as a multi-subject interface, but its actual mechanism is not discussed: it already concatenates reference-image and noisy-video latents, shifts reference coordinates outside the video spatial grid, and ablates H-only, W-only, and H/W shifts. This is directly relevant to Secs. 4.2 and 4.4, rather than just background on a different application.

TASS-RoPE [2] is even closer. It assigns a facial identity reference the coordinates (T, H+h, W+w). With video indices 0,...,T-1, this is the single-reference case of Eq. (3) at j=1 and δ=1. HarmoView's Jump-RoPE [3] introduces gaps between reference groups and between references and video tokens; Aura [4] uses temporal and spatial RoPE shifts to separate reference categories and instances. FLUX.1 Kontext [5] also predates this work's use of virtual temporal coordinates to distinguish concatenated context images, although its task is image generation/editing. These are not all the same layout. The potentially distinct choice here is to give multiple views a shared temporal coordinate and separate spatial bands. The paper needs to explain and isolate the benefit of that choice against the closest layouts. A comparison only with vanilla RoPE does not establish it. The supplementary standard RoPE identity and qualitative examples at 450 training steps do not fill this gap; distinct coordinates also do not, by themselves, prevent reference copying.

The masking contribution has a similar attribution problem. Video Alchemist [6], which is already cited, identifies same-video reference copying, pose/lighting leakage, and reduced motion, and uses reference augmentation to address them. TASS-RoPE also combines spatially shifted reference conditioning with augmentation. Please compare region masking with these ordinary reference augmentations under matched conditions. Omni-ID [7] is additionally relevant to aggregating complementary facial information across poses, though it is an image representation method rather than a drop-in video baseline. The issue is the unestablished technical increment, not the number of references.

The dataset construction also does not establish a new annotation method. Sec. 4.1 and Supplement Sec. 3 describe quality filtering, RetinaFace detection, shot/track filtering, head-pose thresholding, reference-frame sampling, optional Grounding DINO/SAM masks, and Qwen2.5-VL captions. This is a straightforward combination of existing tools. A useful curated dataset could still be a contribution, but the paper needs to show its value through a quality audit, pose/identity coverage statistics, and a controlled comparison with existing data. The checkbox comparison in Supplement Table 1 does not establish this. Even the selection rule is unclear: Fig. 3 uses a 60-degree pose-range threshold, whereas the text says 45 degrees. As presented, I see a data-curation effort, but no demonstrated innovation in annotation or evidence that the data contribution compensates for the limited methodological novelty.

2. The experiments do not isolate the claimed identity–motion improvement. The main comparison uses Wan-2.1-14B, but the component ablations use Wan-2.2-5B. There is no corresponding 14B baseline trained on the same 22K videos without the proposed components. The main result therefore does not tell us how much comes from the data and fine-tuning, and how much comes from RM and RD-RoPE.

Even in the 5B ablation, the full model reduces MvRC-Arc/Cur from 0.546/0.515 to 0.535/0.502, and also scores below RoPE alone at 0.552/0.522. An identity–motion trade-off is reasonable, but it needs direct measurement. MvRC in Eq. (4) simply averages face similarities across references and frames. It does not measure viewpoint coverage or temporal continuity: reordering the generated frames leaves it unchanged. NaturalScore is a general realism judgment from 16 sampled frames, without validation that this judge reliably detects the specific angle-locking and switching failures. Figures 6 and 14 show selected trajectories, but provide no aggregate measure of locking, abrupt changes, or achieved pose range. The human study compares the final systems rather than the component variants, so it does not resolve this attribution problem either.

Important reproduction details are also absent from the main paper and supplement, including the optimizer, learning rate, batch size, training duration, masking granularity, the value of the temporal offset, and inference settings. The supplement says the dataset is publicly released, but I could not find an access link in the submission or supplied files. These omissions make the comparison difficult to assess or reproduce. I do not think the current evidence is sufficient for acceptance.

References mentioned in this review

[1] Kaleido: Open-Sourced Multi-Subject Reference Video Generation Model. Secs. 4.2/5.4, Eq. (4), Table 2. https://arxiv.org/html/2510.18573v1

[2] Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation (ST-DRC). Sec. 3.2, Eq. (4). https://arxiv.org/html/2606.02441v1

[3] HarmoView: Harmonizing Multi-View Constraints for Identity-Consistent Video Generation. Sec. 3.1, Jump-RoPE. https://arxiv.org/html/2606.10839v1

[4] Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment. Sec. 3.2, Eq. (2). https://arxiv.org/html/2607.04311v1

[5] FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space. Token sequence construction, Eq. (2). https://arxiv.org/html/2506.15742

[6] Multi-subject Open-set Personalization in Video Generation (Video Alchemist). Sec. 3.3. https://arxiv.org/html/2501.06187v2

[7] Omni-ID: Holistic Identity Representation Designed for Generative Tasks. https://arxiv.org/html/2412.09694

Questions for authors

Questions For Authors

  1. Under the same backbone, data, and training budget, how does the shared-time/per-view-spatial layout compare with Kaleido's R-RoPE, a multi-reference extension of TASS-RoPE, and reference-group gap/shift designs? Please separate the effects of shared time, temporal gap, and per-view spatial bands, and compare RM against reference augmentation.
  2. Can you provide the direct multi-view baseline and component ablations on the 14B model, with uncertainty estimates and measurements of pose coverage, angle locking, and abrupt changes? The masking-ratio study should report identity as well as NaturalScore.
  3. What are the complete training/inference settings and the dataset access link? What are the actual pose-selection thresholds, annotation error rates, and pose/identity coverage statistics? How were the identity-disjoint split and the ten additional evaluation references constructed, and how are failed face detections at large angles handled?
  4. In the human study, what are the statistical unit, sample size, and degrees of freedom, and how are repeated ratings of the same cases by the same evaluators handled? The omnibus tests reported in Sec. 5.4 do not establish a pairwise improvement over HuMo. Please also explain the stars and uncertainty in Fig. 8.
4 comments
AuthorsOpenReview ↗

Response to Reviewer DVQG — Q1: Layout Design and Reference Augmentation

We thank the reviewer for the detailed questions. We address the four questions in separate comments for readability.

Q1. Layout design and reference augmentation

Our reference-to-video interface uses Concat-ID-style latent concatenation as the conditioning backbone; concatenation itself is not claimed as a contribution.

RD-RoPE assigns all reference views a shared temporal coordinate at fl+10f_l + 10, separating the reference group from the last video latent frame by 10 latent frames. The shared temporal coordinate avoids imposing temporal ordering among reference views; the temporal gap separates the reference group from the video timeline; and per-view spatial bands distinguish individual references. Specifically, view ii occupies (hi,wi)=(hv+iH,wv+iW)(h_i, w_i) = (h_v + iH, w_v + iW).

Region masking is applied independently to each reference image before VAE encoding, using random 16×16 patches at a 60% masking ratio. Its intended role is to encourage aggregation of complementary identity cues across references, rather than reliance on a single complete view.

Kaleido's R-RoPE, multi-reference extensions of TASS-RoPE, and reference-group gap/shift designs are relevant comparisons. These work appeared after our initial public release. Our reported component ablations evaluate RD-RoPE as a combined layout rather than separately isolating all three layout factors. We therefore limit our conclusions to the comparisons actually evaluated.

AuthorsOpenReview ↗

Response to Reviewer DVQG — Q2: 14B Comparisons and Component Ablations

Q2. 14B comparisons, component ablations, and masking ratio

The main comparison in Table 1 includes Phantom-MV and MAGREF-MV, both trained at the 14B scale, alongside our Wan-2.1-T2V-14B model. These experiments assess complete multi-view systems.

For component analysis, we use Wan-2.2-TI2V-5B to improve experimental efficiency under computational constraints, as stated in Sec. 5.1. Table 2 compares the base model, RD-RoPE only, region masking only, and their combination. The 14B main comparisons and 5B ablations therefore address complementary questions: system-level performance and component contributions. The latter is not a matched 14B component ablation.

For the masking-ratio study in Appendix A.3, NaturalScore was the primary reported metric. We additionally provide identity scores:

Masking ratioMvRC-Arc ↑MvRC-Cur ↑NaturalScore ↑
40%0.5370.5014.13
60%0.5350.5024.30

Increasing the masking ratio from 40% to 60% produces small numerical changes in identity similarity while improving NaturalScore, supporting the 60% setting in this experiment. These point estimates do not establish statistical significance.

The trajectory visualizations provide qualitative evidence; aggregate generated-video measurements of pose coverage, angle locking, abrupt changes, and associated uncertainty have not been reported.

AuthorsOpenReview ↗

Response to Reviewer DVQG — Q3: Reproducibility and Dataset Protocol

Q3. Training/inference settings and dataset protocol

Training dataset: Hugging Face.

Training and inference. Main experiments use Wan-2.1-T2V-14B, while ablations use Wan-2.2-TI2V-5B. The implementation uses AdamW with a 1,000-step linear warmup from 10−510^{-5} to 2×10−52 × 10^{-5}, per-GPU batch size 8, and bfloat16. The 14B training configuration uses 40 NVIDIA A800 GPUs. Clips are processed at 16 fps, with 81 frames at 480×832 resolution and three reference views. Region masking is used only during training. Inference uses 50 flow-matching steps, scheduler shift 5, text CFG 5.0, face-conditioning CFG 5.0, and seed 42.

Pose threshold. Appendix A.1 specifies a within-clip yaw or pitch range exceeding 45°, followed by selection of three reference frames with pairwise pose differences exceeding 45°. This criterion concerns pose variation within a clip, not a requirement that every frame have a large absolute pose. The 60% figure refers to the masking ratio, not a pose threshold.

Training coverage. The released dataset contains 22,516 clips with 178,560 primary-face annotations from facedetect_v1. Defining frontal as |yaw| < 30°, left as yaw ≤ −30°, and right as yaw ≥ 30°, the proportions are 52.2%, 20.9%, and 26.9%, respectively. All clips have a yaw or pitch range exceeding 45°; 98.9% have a yaw range exceeding 45°, with a median yaw range of 69.9°.

Test set and evaluation references. The independently curated test set contains 30 identities and 150 cases, with five prompts per identity. It is identity-disjoint from training and separate from the released training dataset. MvRC uses approximately ten additional evaluation references per identity, totaling 300 images. The actual three conditioning references are separate and excluded from this gallery.

InsightFace (buffalo_l) detects 300 faces. Their frontal/left/right proportions are 63.7%/18.7%/17.7%, and 16.7% have |yaw| ≥ 45°. All 30 identities have an estimated yaw range exceeding 45°; 29 exceed 60°, and 22 contain both left and right views. These statistics describe the static reference gallery, not generated-video motion. Training and test statistics use different pose estimators.

Failed detections. Generated frames without valid face embeddings are skipped and excluded from the MvRC average.

AuthorsOpenReview ↗

Response to Reviewer DVQG — Q4: Human Study and Statistical Interpretation

Thank you for requesting clarification of the human-study protocol and statistical analysis.

We evaluate six methods on 25 sampled cases using ten university-student evaluators who were not involved in model development or data collection. For each case, anonymized videos from the six methods are presented in randomized order. Evaluators assess identity consistency, appearance naturalness, and motion/viewpoint naturalness on 1–5 Likert scales.

The statistical unit is the case-level mean. We first average the ten evaluator ratings within each case and method, yielding 25 observations per method and 150 case–method means across six methods. Thus, individual evaluator ratings are aggregated before testing rather than counted as separate independent observations.

For identity consistency, the ordinary one-way ANOVA has degrees of freedom (5, 144), and the Kruskal–Wallis test has 5 degrees of freedom. The reported statistics are F = 81.70 and H = 283.75, respectively. These analyses use the case-level means; they do not explicitly model the shared-case pairing across methods or evaluator-specific effects.

Figure 8 reports mean ratings across cases, with the overall score defined as the average of the three criteria. Our method achieves the highest observed mean scores for identity consistency and appearance naturalness, while HuMo achieves a slightly higher mean for motion/viewpoint naturalness. The reported omnibus tests assess differences across all six methods; the comparison with HuMo describes the observed mean-score differences rather than a separate pairwise significance test.

Figure 8 displays mean scores without uncertainty bars. Stars appear above the highest mean bar for each criterion: HuMo for motion/viewpoint naturalness, and our method for identity consistency, appearance naturalness, and overall quality. They should not be interpreted as pairwise significance annotations.

Reviewer_CS4bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents Mv2ID, a multi-view identity-preserving video generation framework that handles large facial-angle variations without cross-paired training data. The method combines reference-image and video latents with two components: region-masking training to discourage direct reference copying, and RD-RoPE to separate reference images from video frames. The authors also construct a 22K-video large-angle dataset and introduce MvRC for identity evaluation. Experiments on 30 held-out identities and 150 prompts show improved identity consistency over the evaluated baselines, with competitive video quality and naturalness.

Critical strengths
  1. The paper addresses a meaningful problem in identity-preserving video generation. The analysis of view-dependent copy-paste under in-paired supervision is interesting, particularly the observation that multi-view conditioning can cause facial-angle locking and abrupt transitions.
  2. The proposed method is simple and practical. Region masking and RD-RoPE can be integrated into an existing video diffusion transformer without additional inference-time modules. The 22K-video large-angle dataset is also a useful contribution.
  3. The results are promising. In Table 1, Mv2ID improves MvRC-Arc and MvRC-Cur from 0.493/0.459 for HuMo to 0.544/0.507 and achieves the highest text-video alignment among the compared methods.
Critical weaknesses
  1. The methodological novelty is not fully established relative to closely related work. Concat-ID already uses latent concatenation for reference conditioning, while Video Alchemist has investigated copy-paste behavior under same-video supervision. More importantly, MV-S2V explores spatially shifted RoPE (SS-RoPE) and temporally shifted RoPE (TS-RoPE) for multi-view conditioning. The proposed RD-RoPE appears closely related to these designs, combining spatial offsets with temporal separation of reference tokens. Region masking also resembles conventional occlusion-based augmentation. The authors should clarify the technical differences and provide controlled comparisons with these existing strategies.
  2. The ablation results do not fully support the proposed explanation for improved identity-preserving motion. In Table 2, combining RD-RoPE and region masking increases NaturalScore from 3.70 to 4.52, but reduces MvRC-Arc from 0.546 to 0.535 and MvRC-Cur from 0.515 to 0.502. NaturalScore measures general perceptual naturalness rather than view-dependent copy-paste directly, and the trajectory comparisons are mainly qualitative. Appendix A.3 also lacks quantitative comparisons between view masking and region masking. Direct measurements of pose discontinuities and identity consistency across viewing angles would make the proposed mechanism more convincing.
  3. The experimental comparisons remain somewhat limited. The evaluation covers only 30 test identities, with 25 cases in the human study. Broader comparisons, uncertainty estimates, and pose-stratified results would help establish the reliability of the improvements.
Questions for authors

The paper addresses a relevant problem and reports encouraging improvements in identity consistency. I also appreciate the attention given to motion naturalness and the identity-naturalness trade-off. However, several design choices overlap with existing techniques, and the current experiments do not clearly establish the distinctive benefits of the proposed components. The missing comparisons with closely related methods further weaken the empirical claims. I therefore lean toward rejection, although additional controlled evidence could change my assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.