acceptodds
Withdrawn from ICLR 2027

Count to Five! Efficient Test-Time Personalization for Pose-Guided Human Video Synthesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pose-driven human video generation aims to synthesize realistic videos of a person following a specified pose sequence. Existing methods rely on static models that generalize poorly to unseen subjects, leading to appearance drift and mismatched details. We study test-time personalization for pose-guided video diffusion, where a few posed snapshots of the target subject are available at inference. Naive fine-tuning on these snapshots is slow and prone to overfitting, and initial updates can degrade the pretrained model. We therefore propose AdaptAnyone, to our knowledge the first meta-learned framework for test-time personalization of pose-guided video diffusion models. Our instance-level meta-learning strategy treats each training subject as a separate task and learns a parameter-efficient initialization for personalizing a new subject in a few gradient steps. Guided by a sensitivity analysis, we efficiently adapt the components that most influence subject appearance and temporal consistency. With three gradient steps, AdaptAnyone outperforms both the base model and naive fine-tuning, which requires orders of magnitude more steps, in subject fidelity, image quality, and temporal stability.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
test-time adaptation, few-shot personalization, pose-guided human video synthesis, parameter-efficient fine-tuning, meta-learning, video diffusion models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_A4yJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies test-time personalization for pose-guided human video synthesis: adapting a pretrained pose-conditioned video diffusion model to a new subject from a few frames, as cheaply as possible. Each subject is treated as a task in instance-level meta-learning (first-order MAML over support/query splits, backbone frozen, only LoRA trained), so the learned initialization needs just K=3 gradient steps on M=3 support frames — 3.6 s against ≈69 h of one-off meta-training; a second contribution ranks components by perturbation-based appearance and temporal sensitivity scores and adapts only the top 29 of 120.

On UniAnimate-DiT (Wan2.1 14B) and DynamiCtrl (CogVideoX 5B) it reaches FVD 402.14 → 178.86 and FID 28.47 → 19.07, best or tied-best on 11 of 12 metrics, and beats a same-budget Naive TTA control (App. A.3: identical test-time procedure from pretrained weights) at 0.06 min versus 31.1 min. Component, K and placement ablations, cross-domain transfer, a three-seed study, 20-pair cross-driving and a 28-participant preference study complete the evidence; Ethics, Reproducibility and AI Use statements are all present.

Critical strengths
  • The efficiency claim is supported by an identically-budgeted control and a monotone baseline curve, which makes it credible rather than merely impressive.

  • The placement result has internal structure that rules out the trivial explanations, and it delivers the paper's largest single gain at 24% of the parameter budget.

Critical weaknesses
  • The sensitivity analysis behind the headline placement is, by the authors' own disclosure, largely a bfloat16 quantization artifact, and neither proxy is validated against downstream quality.

  • The K-ablation cannot support the titular claim, and the identity evidence is far thinner than a paper about identity personalization requires. On the first: Table 5 reports FVD 389.26 (K=0) / 184.90 (K=1) / 218.40 (K=2) / 178.86 (K=3) / 203.90 (K=5) — non-monotone, with K=1 within 6 of the claimed optimum — while Table 11 gives σ = 26.2 for the method over three seeds (119.1 for base). Every pairwise difference in Table 5 is inside roughly one standard deviation of a single run.

Questions for authors

Is there evidence the appearance and temporal proxies predict downstream benefit? A per-component study on ~10 components (adapt only that component, measure FVD/FaceID delta) against the proxy scores would establish or refute the mechanism. Also: why rank by the unweighted sum of two proxies with different scales and units?

Reviewer_34dhOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes a test-time personalization framework for human image animation models. The framework selects a subset of parameters to fine-tune via perturbation-based sensitivity analysis and introduces a meta-personalization training process that adapts a pre-trained generative model to one or three input frames.

Critical strengths

Although I am not deeply familiar with the literature on test-time adaptation, the proposed method appears reasonable. The experimental protocol is clearly described and fair, and the improvements in the quantitative metrics support the effectiveness of the proposed method.

Critical weaknesses

Although the reported metrics show improvements, the qualitative results do not convince me that the proposed method meaningfully improves visual quality. In Fig. 7, the frames generated by the proposed method in rows 1, 3, 4, and 5 appear blurry compared to the sharp frames produced by the base model. Furthermore, the frames in rows 1, 3, and 5 also appear blurry, and the appearance of the generated images is not more consistent with the input reference images than that of the base model.

Questions for authors
  • MTVCraft outperforms DynamiCtrl and UniAnimate on almost all metrics, yet it was not adopted as a base model. Could the authors explain why MTVCraft was not chosen as the base model?
  • The authors do not report results for MTVCraft on Unseen100-Val, stating that MTVCraft requires SMPL parameters. However, these parameters can be obtained using NLF-Pose, as in the original MTVCraft paper. Could the authors provide evaluation results for MTVCraft on Unseen100-Val?
  • Could the authors include qualitative results for DynamiCtrl: Base and DynamiCtrl: Ours in Figs. 2 and 7–9, as well as in the supplementary video?
Reviewer_DXdTOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AdaptAnyone, training an initialization adaptation using offline meta-learning so that a pose-driven video model can better retain the appearance and temporal consistency of new characters with only a few reference frames and after 3 steps of gradient updates during testing. It addresses the problem of insufficient adaptation to unseen characters in fixed models, as well as the problem of slow and tend to overfitting fine-tuning.

Critical strengths

The core contribution is to change personalization from "slowly fine-tuning from a general model during testing" to "learning how to adapt quickly offline", combined with a small and selectively sensitive LoRA parameter update. The cost of each new character only needs a few seconds to adapt.

Critical weaknesses

The support set is from the same video as the query set, which may allow the model to use scene clues instead of just learning the identity of the person. The background, lighting, and camera conditions of the same video are usually similar, so even if the model remembers these clues, it may perform well on the query frame set aside. Although the support frame and the evaluation frame during the test do not overlap, they also come from the same video. The paper does not directly use "the same person, different scene or light" to verify whether the identity representation can migrate.

Questions for authors
  1. The support set and query set coming from the same person’s video, please explain whether the splitting of video support/query will leave scene clues resulting in training shortcut.

  2. Whether to consider constructing a bench using the same identity in different scenarios and lighting conditions, and using it to evaluate whether the adaptation of identity is maintained as the scene changes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.