acceptodds
Withdrawn from ICLR 2027

GroupRef: Towards Identity-Faithful Video Generation with Multi-Person References

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-person reference-to-video generation requires preserving every specified identity and assigning each action to the intended individual. Existing benchmarks provide limited coverage of larger groups and often allow appearance descriptions in prompts, making it unclear whether models rely on visual references or textual appearance cues to preserve the specified identities. To address these limitations, we introduce GroupRefBench for groups of three to ten people, with prompts that assign actions to individuals by image index and omit appearance descriptions. Using the reference-to-person correspondences established during identity evaluation, the benchmark further assesses whether each individual performs the assigned actions. To support training with indexed instructions, we construct GroupRef100K, which pairs videos with indexed action annotations and diversified reference images. We further propose Cross-Modal Identity Anchoring to strengthen index-to-reference binding by applying shared learnable anchor embeddings to image-index markers and their corresponding reference-image tokens. Controlled attention analysis provides evidence that the model uses these anchor correspondences for reference selection. The resulting GroupRef models demonstrate promising performance on both GroupRefBench and OpenS2V-Eval.

Primary area:
datasets and benchmarks
Keywords:
Video Generation, Multi-Person References, Identity Preservation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_XzJFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Based on my understanding, here's a summary of the paper:

  • The authors demonstrate their pipeline to obtain a new benchmark, GroupRefBench, which is a new benchmark for Multi-S2V generation, evaluating identity, quality, etc of generated videos. Importantly the text prompts don't contain any textual cues about the subjects.
  • 100K Training samples(GroupRef100K) for training a Video gen model to evaluate on GroupRefBench.
  • The paper proposes CMIA (Cross Modal Identity Anchor) method which adds a learnable embedding to both image and text tokens during training. This is shown later to help in identity binding based from text prompt.
  • Results on GroupRefBench show the trained model from GroupRef100K with CMIA improves all identity metrics.
Critical strengths

Here are the strengths of the paper from my reading:

Clarity and Presentation: The paper is very clear in presenting the idea, benchmark curation, as well as demonstrating results. Figures and tables are self-explanatory. The paper reads well. This is important for a reader as it provides the idea in simple terms.

CMIA: The results in Table 4 clearly show that CMIA is highly effective in identity binding and preservation. It is also a simple approach which helps improve identity binding for multiple references. This has the capability of making the method widely usable by the community. Additionally, the motivation of this approach is clear.

Dataset: The need for proposing this benchmark is well-motivated (extending to 10 people, no textual descriptors except the tag).

Critical weaknesses

[W1] Synthetic Faces: The dataset is made of synthetically generated faces. As the authors might be aware, ID preservation for a synthetic face and real face is different in S2I/S2V models. i.e. an S2V model might not be able to preserve a real identity but can easily preserve synthetic identity. The motivation for using synthetic identities is unclear, as the final use-case would be real faces.

[W2] Coverage: As there are no textual identifiers input to the Qwen model for generating the face, the biases from the image generator could easily have been introduced in this dataset. If the Qwen model prefers certain ethnicities, genders or ages, it could very easily affect the data distribution in this data. The authors are suggested to report a data distribution similar to <1>. As Qwen has been probed for generation, is there a possibility of repeated faces appearing in this dataset? There is also a high possibility that a certain ethnic or age group is absent.

[W3] Number of People Analysis: The paper motivates increasing the number of faces (K) to 3-10, however there is no analysis of metrics plotted against values of K. For instance, how do all models compare on K=3,4,5,6,7,8,9,10? How does CMIA affect K=3,4,5,6,7,8,9,10 compared to the baseline?

[W4] Limited Comparisons: A strong benchmark would not only report scores for all methods, but also analyze why certian methods fail at certain axes. This comparison is lacking.

[W5] Quality Drop: Observing the numbers, visual results in the paper and also the supplemental videos provided by the authors, I observed a considerable quality drop in the generated videos compared to the RefAlign baselines.

<1> https://arxiv.org/abs/2506.20879

Questions for authors

[Q1] Are the authors planning to make the data public? This commitment wasn't noted in the paper, as the dataset is part of the contributions.

[Q2] Have the authors considered adding a quality term to the overall loss for training this model? This might help alleviate W5, the quality loss relative to baseline.

[Q3] Why was CMIA considered additive and not rotational? Recent papers in multi-S2I(subject to image generation) generation<2,3> show that adding an extra axis of rope and rotating the tokens corresponding to different identities have helped improve the ID alignment. Have the authors considered ablating on the type of embedding for CMIA?

[Q4] Have the authors considered performing a correlation analysis with OpenS2V or other S2V benchmarks? If it is decorrelated, it will make a good case for the new benchmark as it will show that GroupRef can measure a different axis compared to this.

<2> https://arxiv.org/abs/2510.20820

<3> https://arxiv.org/abs/2511.22690

Reviewer_STuGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies identity preservation in multi-person reference-to-video generation. It introduces GroupRefBench, a benchmark for reference-to-video generation with three to ten people with indexed instructions, and GroupRef100K, a dataset with indexed action annotations and diversified reference images. The authors further propose Cross-Modal Identity Anchoring (CMIA), which strengthen index-to-reference binding through shared learnable anchor embeddings. Experiments demonstrate improved identity preservation across multiple model backbones and benchmarks.

Critical strengths
  1. The paper targets the challenging problem of preserving multiple identities while assigning corresponding actions.
  2. GroupRef100K provides a useful approach to constructing multi-person training data with references.
  3. The authors compare against open-source and commercial models. The evaluation are conducted across diverse backbone scales.
Critical weaknesses
  1. CMIA is conceptually similar to index-based binding methods such as AlcheMinT. Moreover, its contribution appears modest: Table 4 shows only a modest improvement over fine-tuning without CMIA. It seems that most gain is from fine-tuning rather than proposed CMIA. As such, the novelty is limited.
  2. The Action metric assigns zero contribution to unresolved identity bindings. Thus, improvements in Action scores may largely result from better identity coverage rather than improved action execution.
  3. As shown in the supplementary GroupRef case2, GroupRef-33B fails to visibly preserve the fifth referenced identity, despite the prompt explicitly assigning her an action. In contrast, Kling3-Omni and HappyHorse-1.1 successfully depict all six identities. This failure directly concerns the paper's central objective of preserving every specified identity.
Questions for authors
  1. What is the advantage of CMIA comparing with existing index-based binding methods?
  2. Can the authors explain Why GroupRef produces missing identity in supp case2? Is the identity coverage evaluation reliable?
Reviewer_g1bwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-person reference-based video generation and introduces GroupRefBench, GroupRef100K, and Cross-Modal Identity Anchoring (CMIA).

GroupRefBench evaluates identity preservation and action assignment in scenes with 3–10 people. CMIA uses shared embeddings to associate text indices with reference images. Experiments show improvements mainly in identity preservation.

Critical strengths

Well-motivated problem. The paper addresses identity preservation and action assignment in multi-person video generation.

Potentially useful benchmark. GroupRefBench covers 3–10 people and uses image indices instead of appearance descriptions.

Critical weaknesses

Limited novelty and unclear independent contribution. CMIA is very similar to the cross-modal index binding mechanism in AlcheMinT. GroupRef-33B improves the overall score over MiniMax-H3-33B by only 0.401 points, with almost no improvement in Action. Appendix G also suggests that most of the attention improvement comes from fine-tuning rather than CMIA. In addition, GroupRef100K lacks quantitative validation of its identity–action annotations.

Insufficient evaluation of the core claims. The Action metric relies on automatic identity matching, potentially mixing identity recognition with action execution. These two factors are not adequately separated or validated against human judgments. The supplementary material also contains only six video comparisons, making it difficult to assess improvements in challenging multi-person scenarios.

Questions for authors

AlcheMinT already uses shared learnable index embeddings to connect text and reference images. What is the fundamental difference between CMIA and this existing mechanism? Given the small improvement over MiniMax-H3-33B, could the authors provide a 33B ablation without CMIA under the same training settings to establish its independent contribution?

Appendix E defines the Action score using evidence coverage and conditional rating, but reports only their combined score. Could the authors report these two components separately and validate automatic identity matching against human annotations? This would help clarify whether the model actually improves action assignment or mainly benefits from better identity matching.

Reviewer_z7vzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies reference-to-video generation with multiple people: preserving each referenced identity and making the intended person perform the assigned action. It introduces GroupRefBench, with 320 samples covering three to ten people and indexed prompts that exclude appearance descriptions. It also constructs GroupRef100K for training. The proposed CMIA adds the same learnable embedding to each image-index marker and its corresponding reference-image tokens. Fine-tuning improves identity preservation on GroupRefBench and OpenS2V-Eval, with a further improvement from CMIA in the 1.3B ablation.

Critical strengths

The benchmark asks a useful question: did the correct person perform the action, rather than did the action simply appear somewhere in the video? Covering three to ten references and removing appearance descriptions from prompts makes this a useful test of reference binding.

The identity gains are also clear in the reported results. GroupRef-14B improves IDCov from 0.314 to 0.538 on GroupRefBench and FaceSim from 55.23% to 74.05% on OpenS2V-Eval. I appreciate that Appendix B compares models on shared samples and Table 4 includes fine-tuning without CMIA. These controls help separate the effects of evaluation availability and the proposed anchors.

Critical weaknesses

Critical Weaknesses

  1. The shared identity assignment is a sensible way to evaluate who performs each action, but its accuracy is not validated against human judgments in the paper. Matching errors can affect IDCov, Garment, and Action together, and the sensitivity to the 0.10 acceptance threshold is not reported. Equations (20)–(21) combine judgment coverage and conditional ratings into D, while Tables 2 and 4 report only 20D. I therefore cannot tell how much of the Action gain comes from better fulfillment versus more judgeable items. Reporting both components and agreement with human judgments would make the benchmark results easier to trust and interpret.

  2. GroupRef100K’s annotation quality remains difficult to assess. Appendix C includes face-similarity filtering and VLM checks, which are useful safeguards. However, the caption-validation steps check index format and coverage rather than independently verifying that each action belongs to the assigned person. I could not find a quantitative human audit of these associations or of identity preservation after reference editing. The original video sources and number of unique source videos are also not clearly reported. An annotation audit and clearer source statistics would help establish how reliable and diverse the 100K samples are.

  3. Table 4 supports a positive CMIA increment, but the additional evidence for action binding is limited. Total rises from 34.679 to 36.048, while Action rises from 1.351 to 1.403 and remains below the separate-anchor variant’s 1.451. No uncertainty estimates are reported. The authors acknowledge that most attention improvement comes from fine-tuning; Appendix G gives margins of 3.39 and 3.42 percentage points without and with CMIA. More importantly, Section 6.4 refers to anchor mismatch and permutation results that are not quantified in Appendix G. Please provide these results and clarify what they establish beyond the reported attention-margin comparison.

  4. The identity gains are accompanied by declines in other reported metrics that receive little discussion. In Table 3, GroupRef-14B improves FaceSim from 55.23% to 74.05%, while Amp falls from 22.48% to 13.42% and Natural from 73.63% to 65.28%; Overall increases from 60.42% to 60.94%. Table 2 also shows IQ falling from 76.139 to 69.438 relative to RefAlign-14B. These results do not negate the identity improvement, but they matter when assessing the generated videos. Please discuss this trade-off and show examples illustrating what is gained and lost after fine-tuning.

Questions for authors
  1. Could you provide a human validation of identity matching and Action scoring? Please also report Action judgment coverage and conditional ratings alongside D, so we can distinguish improved action fulfillment from an increase in judgeable items.

  2. Could you report error rates from a human audit of identity association and action attribution in GroupRef100K? Please also clarify the original video sources and how clips from the same source video or samples with different reference variants are counted toward the 100K total.

  3. Could you provide the quantitative anchor mismatch/permutation results referenced in Section 6.4, together with paired confidence intervals for the main comparisons in Table 4? Please clarify whether the intervention results concern only attention under ground-truth conditioning or also identity and action assignment in generated videos.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.