- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper targets few-step human–object interaction video generation. It makes two contributions. First, it introduces InteractVerse, a dataset of 37,477 curated clips totaling approximately 64.1 hours, with aligned person/product references, text, audio, motion context, pose sequences, and interaction-sensitive spatial annotations. Second, it proposes FastInteract, which distills a CoInteract teacher into a four-step generator using attention perturbation: one response enhances HOI structure, a second response characterizes product preservation, and a one-sided projection coordinates the two before augmenting the DMD target.
On HOIVG-Bench, the four-step student improves VLM-QA from 70.159 for vanilla DMD to 76.957, and product-reference DINO similarity from 0.614 to 0.628. The paper reports comparisons with many-step and few-step baselines, component ablations, and qualitative coefficient probes. The evidence supports a useful interaction–appearance trade-off, while a gap to the many-step teacher remains on interaction quality.
Critical strengths
The data and modeling contributions form a coherent whole. InteractVerse provides aligned appearance, semantic, temporal, and spatial supervision at a scale beyond several controllable HOI datasets (Section 3; Appendix B). The curation pipeline combines technical filtering, face/hand/product detection, reference extraction, verification, and interaction-aware captioning. This is a reusable resource rather than a collection of captioned videos. FastInteract then addresses a corresponding modeling difficulty: retaining interaction structure and product appearance when generation is compressed to four steps.
The ablations explain the interaction–appearance trade-off. In Table 3, Full-Visibility AP improves VLM-QA from 70.159 to 75.013, but lowers product DINO similarity from 0.614 to 0.602. Restricting the intervention to target-video queries raises VLM-QA to 76.026 and DINO_obj to 0.609. Adding the one-sided projection further raises them to 76.957 and 0.628. This progression is consistent with the proposed mechanism: attention perturbation supplies most of the interaction gain, while the restricted pathway and projection recover product consistency. The coefficient probes in Appendix D.2 show the same tension qualitatively.
The empirical comparison is broad enough to be informative. Among the listed few-step baselines in Table 2, the four-step student leads on seven of eight reported metrics, including VLM-QA and DINO_obj. The authors also disclose that several baselines require a single combined reference image and construct it accordingly. This interface difference makes the within-framework ablations the stronger evidence, but the comparison remains useful.
Critical weaknesses
The VLM-QA protocol and the attribution of gains need more complete documentation. Appendix D.1 describes the 50-question bank, the four categories, the category weights, the harmonic mean, and the critical-failure caps, but it does not provide the actual questions, the judging prompt, the failure thresholds, or the frame-sampling settings. Human agreement and uncertainty estimates are also missing. This matters for the smaller margins, such as the 0.931-point VLM-QA gain from projection over TQ-AP and the 1.071-point margin over LiveAvatar. The ablations support the method, but they do not separately establish how much InteractVerse contributes to the reported performance. A matched-data ablation would separate the dataset contribution from the distillation contribution.
The residual failure modes and practical operating range are not characterized in depth. The full model remains 5.111 VLM-QA points below the 40-step CoInteract teacher, so the improvement over vanilla DMD does not eliminate the loss of interaction quality during distillation. Appendix D.2 shows that stronger HOI enhancement can introduce product drift and artifacts, but this is a coefficient probe rather than a systematic account of failures of the final model. Appendix A explicitly excludes multiple people or objects and notes that real-time generation still requires further acceleration. Representative final-model failures, category-level results explaining the remaining teacher gap, and measured latency at a stated resolution and clip length would strengthen the practical assessment. Four versus 40 steps establishes a step reduction, not a measured tenfold speedup.
The cross-method comparison is not fully controlled across conditioning interfaces. LiveAvatar, FlashTalk, and LongCat-Video-Avatar 1.5 require a single image, so the authors combine the person and product references into one image. Reference-consistency metrics still use the original separate references. The authors disclose this adaptation, but it limits how much the cross-method ranking can establish about the distillation design by itself; the component ablations remain the more direct evidence.
Questions for authors
Metric reliability: Can you release the complete VLM-QA protocol, including the question bank, judging prompt, critical-failure thresholds, and frame-sampling procedure? Can you report paired uncertainty estimates or human agreement on a representative subset, and break down the results by action, contact, motion synchronization, and structural plausibility? Such a breakdown would clarify which errors account for the gain over vanilla DMD and the remaining gap to the teacher.
Data contribution: Which comparisons keep the training data fixed? Please specify the evaluation sample size, any train/benchmark overlap checks, and how annotation quality was verified. What evidence separates gains from InteractVerse from gains from the distillation method? A matched data ablation would clarify this directly.
Failure analysis and efficiency: Which failure types persist in the full model under its supported single-person/single-object setting? Please provide representative cases and, where available, their frequency. Please also report same-hardware student/teacher latency at a stated resolution and clip length, together with the training configuration. Extending to multiple entities or real-time streaming is future work rather than a prerequisite for acceptance.
3 comments
Response to Reviewer Bwyw
W1 / Q2. evaluation setting and Dataset contribution
Evaluation size. All reported automatic metrics are evaluated on the 135 test cases in HOIVG-Bench, released by OmniShow [1]. InteractVerse was collected from real e-commerce videos on publicly accessible livestreaming platforms, whereas HOIVG-Bench uses AI-generated person and product references. The datasets were constructed independently, and HOIVG-Bench is used only for evaluation.
Annotation quality. We apply additional VLM-based verification after image editing and mesh reconstruction. For reference extraction, the VLM checks whether the extracted person and product references preserve the identity in the source video. Samples with mismatched references, missing entities, or severe visual artifacts are discarded. We also use Qwen-Omni to filter out videos with low-quality, unclear, or noisy audio. These checks reduce the propagation of preprocessing errors into training.
Dataset contribution. The main limitation of SparseHOI-5K is its restricted identity and product coverage: its 4,850 clips contain only 34 person identities and approximately 1,000 product reference images. Generating additional captions and extracting reference images do not expand the underlying set of people, products, or recorded interactions. In contrast, InteractVerse contains 37,477 clips spanning 53 product categories and diverse interaction types, with details available on our InteractVerse Dataset Page. The clips are collected from different e-commerce livestreams at different times, providing broad coverage of person identities and products. After excluding a small number of repeated products and person identities, the dataset contains over 30,000 distinct pairs of product and person reference images. Our LiveAvatar competitor uses approximately 400,000 training samples for distillation, whereas SparseHOI-5K contains fewer than 5,000 clips. Based on this distillation experience, we consider SparseHOI-5K insufficient in both size and diversity to meet the needs of our distillation task.
W2 / Q3. Representative failure cases
We provide concrete failure cases in which the model fails to generate interactions between handheld objects and body parts other than the hands. Representative examples are available at Failure Case Page. We will incorporate these examples and a dedicated failure-case analysis into the revised appendix.
W2 / Q3. Measured efficiency and training configuration
On the same NVIDIA H20, generating a one-second video at 480 × 832 resolution takes 402 seconds for the 40-step teacher and 39.5 seconds for the four-step student, corresponding to a 10.18× measured end-to-end speedup. These timings include the full encoding and decoding pipeline. During denoising, the student eliminates the additional CFG pass, reducing the number of denoiser evaluations from 80 (40 × 2) to four, a 20× reduction.
Training was conducted on 32 NVIDIA H200 GPUs with 140 GB of device memory per GPU, using BF16 precision, Fully Sharded Data Parallel (FSDP), and a learning rate of . The model was trained for 5,000 steps over approximately three days.
W3. Conditioning-interface differences
Our task is to generate plausible interactions while preserving the person identity and product appearance specified by two separate reference images. LiveAvatar, FlashTalk, and LongCat-Video-Avatar 1.5 do not natively support this conditioning interface. We therefore use Qwen-Image-Edit to construct a composite first frame for these baselines. This adaptation may alter appearance details, and it also provides an explicit person–product spatial arrangement and initial interaction configuration.
We evaluate every method against the original person and product images, because the task's target identities are the original pair rather than the composite edit. The composite exists only because three baselines cannot ingest two separate images; it is an input workaround, not a new task specification. Scoring against it would instead measure the remaining six methods against an image they never received. Scoring each method against the input it actually received would avoid penalizing those three, but it would make the reference method-dependent: the same column would then measure fidelity to two different images. We therefore fix one task-defined target across all methods. Although the conditioning-interface asymmetry for LiveAvatar, FlashTalk, and LongCat remains, our latest user study provides complementary evidence: our evaluated outputs achieve the best mean rank in Object Consistency and a near-best mean rank in Human/Background Consistency.
References
[1] Zhou et al. OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation. ICML 2026.
Thanks for your response and the additional experiments. They address my concerns and support my initial positive assessment, so I will maintain my rating of 3
Response to Reviewer Bwyw
W1 / Q1 (a). HOI user study
We agree that broader evaluation coverage and validation against human judgments would strengthen the evidence. We conducted a perceptual user study with 12 evaluators. Each evaluator was shown 10 test cases randomly sampled from HOIVG-Bench. For each case, videos generated by the compared methods were presented in a blind, randomized order. Evaluators ranked the videos separately according to three criteria: Interaction Plausibility, Object Consistency, and Human/Background Consistency. We report the mean rank for each criterion, with lower values indicating better perceived quality.
User study results. Best and second-best values across all evaluated methods are bold and underlined, respectively.
Even compared with multi-step models, ours outputs achieve the lowest mean rank in Object Consistency and the second-lowest mean rank in both Interaction Plausibility and Human/Background Consistency. To better understand the differences between the user-study rankings and the evaluation, we collected follow-up feedback from the evaluators.
For Interaction Plausibility, evaluators perceived the motion in our outputs more closely matched how people naturally present and interact with products, whereas some competing outputs exhibited exaggerated movements that appeared less realistic. For Object Consistency, evaluators noted oversaturated product appearances in competing outputs and preferred the overall consistency of our product appearance. For Human/Background Consistency, many evaluators noted that our outputs preserved person identity while maintaining good background consistency, faithfully reproducing the background in the reference image.
A randomly selected set of 20 qualitative cases will also be provided for inspection at User Study Page. These examples are intended to help reviewers directly assess the quality of the generated videos.
W1 / Q1 (b). VLM-QA protocol and category-level results
We have prepared the complete VLM-QA protocol for anonymous release at VLM-QA, including all 50 questions, applicability rules, judging prompts, exact failure conditions, category/overall critical-failure caps, and executable sampling and scoring code. A locally deployed Qwen3.8-27B judges each applicable question independently. Inputs include the full video requested at 16 fps, a uniformly sampled 17-frame overview including both endpoints, and four additional frame strips for contact and structure. Applicability is frozen before inspecting generated outputs and shared across methods; uncertain judgments receive zero credit.
All three methods are evaluated on the same 135 HOIVG-Bench cases. All scores are on a 0–100 scale, with higher values better. Motion refers to hand–object synchronization.
Our largest gains over vanilla DMD are in contact/support (+10.693) and motion synchronization (+3.367). The largest remaining deficits relative to the teacher are in structure (−3.593) and action completion (−3.479).