acceptodds
Under review as a conference paper at ICLR 2027

Membership Inference for Generative Video Models

Abstract

With growing interest in copyright, safety, and bias of generative video models, the need for robust membership inference (MI) for these models has never been greater. Yet, only one such method currently exists, designed for old generator architectures. What's more, it comes with problematic methodological design, since the members and non-members on which it was calibrated and tested came from two fundamentally distinct distributions. To allow for robust MI research in generative video models, we construct and open-source a framework and dataset of non-members complementing the VideoUFO dataset, used to train a large text-to-video (T2V) diffusion transformer. We then examine prominent MI methods from image generation and classification across the access spectrum and adapt them for T2V models. Additionally, we present our own gray-box method called The Reversal Gap (TRG) and find that an ensemble including multiple methods achieves the best performance.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Text-to-Video Generation, Membership Inference, Model Auditing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kDR3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies membership inference (MI) against text-to-video diffusion models. It makes three contributions: 1) A distribution-matched non-member set for VideoUFO. The authors re-run the VideoUFO collection pipeline on videos uploaded after June 2025, then pair the new clips with members by Hungarian matching. They report blind baselines on the result: the best blind family reaches 0.639 AUROC, compared with 0.999 on a cross-corpus split. 2) Adaptations of image and classifier MI attacks to T2V across black-box, gray-box and white-box access: loss, PIA, per-block gradient norms, representation read-out and VidLeaks. Results are scored with a “blind-adjusted AUROC,” which residualizes each attack score on the blind features. 3) The Reversal Gap (TRG), a gray-box attack. It scores the normalized loss difference between a clip and its time-reversed copy under shared noise, so per-clip difficulty cancels. The headline result is a supervised representation read-out at 0.717 AUROC (0.692 adjusted). TRG reaches 0.608 (0.630 adjusted), and a reference model trained on disjoint data reaches 0.587 adjusted. Overall, this is an interesting paper, working on membership inference attacks for video diffusion models, which proposes a simple but effective method.

Critical strengths
  • The evaluation methodology is the main contribution, and it is done carefully. The paper reports blind baselines before attacks and holds out a feature family (spectral) from the matching. It also runs cross-split transfer and two kinds of model control: a disjoint-data reference model and random-init weights. Few MI papers do this much.

  • TRG has clear intuition, and D.1/D.2 show it removes most of the per-clip difficulty component. Within-pair correlation drops from 0.77 to 0.24, and TRG gains nothing from reference calibration.

  • The ablations are solid: depth, timestep, frames, K, attack-model capacity, sample size, captions and re-encoding. According to the experimental results, the proposed method shows the effectiveness.

  • Overall, this paper is easy to follow and well written.

Critical weaknesses
  • The compute and averaging comparison is confounded. TRG uses 32 queries, averaged over K=8 noise draws and W windows. The loss baseline uses 3 queries and a single shared noise draw. D.3 shows that averaging alone lifts TRG from 0.628 to 0.675. To isolate the effect of the reversal contrast, the plain loss needs the same K×W averaging and an equal query budget.

  • Numbers are inconsistent across sections. Denoising loss: 0.514 in Table 1 versus 0.533 in D.2. TRG: Table 1 reports 0.608 under the stated K=8 protocol, but D.3 reports 0.675 at K=8, plus 0.029 more from a second window. D.3 also gives an extrapolated ceiling of 0.684 and a variance-decomposition bound of 0.728, which need reconciling with each other. Read-out: 0.717 in Tables 1 and 3, versus 0.716 with all timesteps and blocks and 0.710 at 500 pairs in D.3. These are small differences, but they suggest different runs or subsets. TRG query budget: the 32 queries are not decomposed into K, W, timesteps and directions.

  • Citations: CogVideo (Hong et al., 2022) is autoregressive, not a diffusion transformer. Kapoor et al. (2025) is a general adversarial-attacks survey and is a weak citation for “video MI is nascent.” Several works are cited under both architecture families in Section 2.1 (Wang 2025b, Chen 2023, Blattmann 2023). “Mira Ju et al. (2024)” has formatting errors.

Questions for authors
  1. Can you reconcile the TRG numbers (0.608 vs 0.675 at K=8) and the denoising-loss numbers (0.514 vs 0.533)?
  2. What does the plain loss achieve under the same K×W averaging and query budget as TRG?
  3. How do SecMI and CLiD perform on the matched split?
2 comments
AuthorsOpenReview ↗

Response to kDR3

We thank the reviewer for their thoughtful feedback and kind words, including that the evaluation methodology "is done carefully", that "few MI papers do this much", and that the paper "is easy to follow and well written".

We found the suggestions incredibly helpful and ran all requested experiments: (1) a compute-matched comparison with loss-based MIA, (2) a full reconciliation of the reported results, and (3) SecMI and CLiD on the matched split. We summarize the key findings below; the details have already been reflected in the revised manuscript.


"The plain loss needs the same K×W averaging and an equal query budget" (W1, Q2)

This is a great point. We reran the denoising loss using identical timesteps, windows, and noise draws as the reversal gap, with up to 32 forward queries. We found that the loss does not improve with averaging (0.532-0.533, 4 -> 32 queries). With equivalent query budgets, the reversal gap is better by +0.045 [0.005, 0.082] at 8 queries, +0.062 [0.022, 0.100] at 16 queries, and +0.075 [0.036, 0.111] at 32 queries (see main response, point #3). Therefore, its advantage does NOT come from the number of queries, but from the contrast with the time-reversed clip. We added the curve of AUROC against the number of queries for both to App. D.3.


"Can you reconcile the TRG numbers (0.608 vs 0.675 at K=8) and the denoising-loss numbers (0.514 vs 0.533)?" (W2, Q1)

We thank the reviewer for noticing this! We traced all results back to the experiments that produced them, using our W&B logs, and added a table to the appendix that maps experiments to utilized subsets and settings (see main response, point #4).

  • Denoising loss, 0.514 vs. 0.533: two query protocols on the same clips. The first uses three timesteps with one noise draw each; the second uses the forward losses of the reversal-gap protocol (two timesteps, two windows, and four noise draws). Tab. 1 now uses the second for both.
  • Reversal gap, 0.608 vs. 0.675 at K = 8: under the Tab. 1 protocol, the reversal gap reaches 0.608 at K = 4 and stays at 0.604-0.609 up to K = 8. The D.3 curve came from an earlier development set (198 pairs, six timesteps, one window). Re-running that configuration on 200 pairs of the final split, the reversal gap rises from 0.593 [0.538, 0.649] at K = 1 to 0.641 [0.589, 0.691] at K = 8, i.e., the same increase as on the development set (+0.048 vs. +0.047), while the denoising loss stays at chance (0.493). We replaced D.3 with this curve and removed the extrapolated ceiling (0.684) and the variance-decomposition bound (0.728) with it.
  • Read-out, 0.717 vs. 0.716 vs. 0.710: these correspond to 192 PCs, 128 PCs, and the 500-pair point of the sample-size ablation (128 PCs, but two instead of three fold assignments), respectively. All read-out results now use a nested choice of the number of PCs, which gives 0.718.
  • The 32 queries of the reversal gap break down into 2 timesteps × 2 windows × 4 noise draws × 2 directions (forward and reversed), which we now also state explicitly in the paper.

Citations (W3)

We thank the reviewer for catching these and apologize for the oversight! All of them have been fixed in the revised manuscript: CogVideo (Hong et al., 2022) is no longer grouped with the diffusion transformers, Kapoor et al. (2025) has been replaced with a citation specific to video MI, the double listings in Sec. 2.1 (Wang 2025b, Chen 2023, Blattmann 2023) have been removed, and the Ju et al. (2024) entry has been corrected.


"How do SecMI and CLiD perform on the matched split?" (Q3)

This is an excellent question and we thank the reviewer for the suggestion! We adapted both methods for T2V and, for a fair comparison, used the same preprocessing, windows, and seeded noise as for the reversal gap. SecMI reaches 0.476 [0.455, 0.496] and CLiD 0.539 [0.505, 0.574]. Both hence score below the reversal gap at 0.608 [0.576, 0.639], even though CLiD uses twice as many queries (64 vs. 32). Both baselines are now included in the revised manuscript for more comprehensive evaluation (see main response, point #6).


Ethics flag

We thank the reviewer for raising this! While we believe there are many important and ethical reasons to pursue MI (e.g., transparency and safety of training data), we agree that it could also be misused to reveal whether a specific person’s video was used to train a model. We, thus, added an ethics statement on dual use to the paper. To further mitigate this risk, we release the non-member set and model weights only under gated access (with reviewed requests), together with video identifiers, URLs, and the collection metadata.


If the reviewer’s concerns have been addressed, we would appreciate it if they considered increasing their score.

AuthorsOpenReview ↗

Polite nudge

We once again thank the reviewer for their thoughts. Since the rebuttal period is nearing its end, we would really appreciate it if the reviewer would review our responses, which we believe have addressed all of their concerns, as outlined in the rebuttal above and included in the revised PDF. Thanks so much in advance!

Reviewer_X2seOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates membership inference in text-to-video generation models, specifically determining whether a given video appears in the model's training set. The core issue addressed by the authors is that existing evaluations often show members and non-members from different datasets, potentially leading to attacks that only identify the data source. To address this, the authors constructed a non-member set of approximately 18,000 video clips for VideoUFO. By uniformly generating descriptions, matching content, and low-level statistical features, they selected 500 pairs of samples and compared black-box, gray-box, and white-box attacks on MVDiT-VideoUFO. This paper also proposes TRG, which mitigates the impact of sample difficulty by using the difference in normalized denoising loss between forward and reverse versions of the same video. Experiments show that various existing attacks are nearly randomized in this setting, TRG outperforms other compared gray-box methods, and representation reading and joint models achieve better overall results.

Critical strengths

TRG leverages the temporal structure of videos, offering a direct approach without requiring additional training of a reference model. Establishing internal references within the same segment using both forward and reverse versions is more targeted than directly comparing the absolute losses of different videos. In Table 1, TRG's original AUROC is 0.608, higher than the denoising loss's 0.514; while in Appendix C.5, it only achieves 0.502 on a reference model trained without these members. These results support its design motivation of reducing the impact of sample difficulty. The authors also incorporated the evaluation data itself into the check. The issue of inflated member inference results due to data distribution differences has been discussed in previous research; this paper specifically applies this type of check to the video generation scenario. Unified regeneration of video descriptions, sample matching, and reporting of spectral blind baselines not involved in matching are all practically meaningful processes. In Table 2, a CLIP feature alone can achieve an AUROC of 0.999 in cross-dataset settings, demonstrating the necessity of this type of check. Furthermore, the authors also reported the biases that still exist after matching, rather than only showing favorable results. This work can provide a more credible starting point for subsequent research on video member inference.

Critical weaknesses

The data conditions required for the most powerful attacks do not clearly correspond to the threat model. Section 3 states that it does not assume the attacker has access to the training set, but the representation readings in Section 6.1, the layer-by-layer gradient attacks, and the joint model in Table 1 all require training an attack classifier with member labels. The text does not specify where these known member labels are obtained or how many auxiliary samples are needed. Cross-fitting avoids direct evaluation on the fitted samples, but it does not eliminate the premise that the attacker needs known member data. Furthermore, it is not adequately explained whether choices such as the number of principal components and time steps are all limited to the training iterations. These conditions affect the applicability of the results and their comparability with attacks that do not require such labels. Currently, blind-adjusted AUROC cannot be interpreted as member signals that have excluded differences in data sources. Definition 5.1 uses ridge regression with cross-fitting to subtract the predictable portion of blind features from the attack score, but this process only excludes partial linear dependencies in the selected features and does not guarantee the removal of nonlinear relationships or unmeasured source differences. Table 3 in the paper itself also illustrates this point: the randomly initialized model and the reference model, which had not seen these members before, still achieved scores of 0.566 and 0.587 respectively after correction, with confidence intervals higher than 0.5. Achieving higher scores for the target model is meaningful, but it may also be better at recognizing the distribution characteristics of VideoUFOs, and this does not necessarily indicate that the improvement came from a specific segment that participated in the training. This is my biggest reservation regarding the main conclusions of this paper.

Questions for authors

Regarding the conditions for supervised attacks, where do the known member labels required for characterization readings and joint model training come from, and how many samples are used for each? Please clarify this additional assumption and explain whether normalization, PCA, and hyperparameter selection are performed only within the training fold, and whether different segments of the same source video remain in the same fold.

2 comments
AuthorsOpenReview ↗

Response X2se

We thank the reviewer for their thoughtful feedback and pointing out that our proposed method "is more targeted than directly comparing the absolute losses" and that our work "can provide a more credible starting point for subsequent research on video member inference".

We found both of the reviewer’s concerns to be fair, and the suggestions incredibly helpful in making our claims more precise. Based on them, we (1) extended the threat model to the supervised attacks and split Tab. 1 accordingly, (2) made the cross-fitting fully nested, and (3) now present the gap to two controls, as well as another model with known membership, as stronger evidence for membership signal. We summarize the key findings below; the details have already been reflected in the revised manuscript.


"Where do the known member labels required for characterization readings and joint model training come from, and how many samples are used for each?" (W1, Q1)

This is a great point and we thank the reviewer for bringing this up! The rep. read-out, the per-block gradient attack, and the joint model indeed need labeled members, which the threat model in Sec. 3 did not account for. We, thus, extended the threat model accordingly: these attacks now assume that the attacker has a reference set with known members and non-members (in our evaluation, the training folds of the cross-fitting, i.e., 400 of the 500 pairs). We also split Tab. 1 into two parts (label-free and supervised attacks), so that every comparison is between attacks with the same assumptions (see main response, point #1). The sample-size ablation in the appendix further shows how the read-out depends on the number of labeled pairs. To the reviewer’s protocol questions:

  • Normalization and PCA are fitted inside each training fold only.
  • The number of PCs is now chosen with an inner cross-validation inside each training fold (instead of the best of three sizes on the cross-fitted scores). The read-out reaches 0.718 under this nested protocol, compared with 0.717 before (i.e., within noise).
  • Folds are grouped by matched pair. As each of the 1K evaluation clips comes from a different source video, no two segments of one source video can fall into different folds.
  • Fitting the PCA of the blind features inside each training fold changes the blind-adjusted AUROC of the read-out from 0.693 to 0.700.

"Blind-adjusted AUROC cannot be interpreted as member signals that have excluded differences in data sources" (W2)

We agree with the reviewer’s note about Def. 5.1, and thank them for raising what they describe as their biggest reservation. Def. 5.1 indeed only removes the part of a score that is linearly predictable from the measured blind features; we have carefully narrowed its interpretation in the paper to reflect exactly this. To strengthen evidence for membership signal, we now present the gap between the target model and two controls: (1) a reference model, trained on Panda-70M, and (2) a randomly initialized model copy (see main response, point #2). As neither has seen the members, these controls measure how much of the score a model can obtain without any membership signal. With paired bootstrap intervals, the read-out exceeds them by +0.103 [0.072, 0.138] and +0.122 [0.082, 0.163]. The gap to the reference model also holds with nonlinear residualization (+0.086 [0.055, 0.120] with gradient boosting and +0.084 [0.056, 0.115] with kernel ridge regression).

The reviewer also raises the possibility that the target model recognizes the VideoUFO distribution, rather than the specific training clips. This is an excellent question! As MVDiT-VideoUFO was trained on all of VideoUFO, no held-out VideoUFO clips exist. We, thus, fine-tuned another model, in which members and non-members come from the same collection by design (see main response, point #6). Here, the reference-calibrated loss reaches 0.626 [0.590, 0.659], which can only come from the specific training clips. We scope our claims about the reversal gap accordingly and expand the Limitations section with additional possible sources of signal to consider.


We look forward to further engaging with the reviewer. If the reviewer’s concerns have been addressed, we would appreciate it if they considered increasing their score.

AuthorsOpenReview ↗

Polite nudge

We once again thank the reviewer for their thoughts. Since the rebuttal period is nearing its end, we would really appreciate it if the reviewer would review our responses, which we believe have addressed all of their concerns, as outlined in the rebuttal above and included in the revised PDF. Thanks so much in advance!

Reviewer_RQhQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors address are very important area of research currently not evaluated enough in the community, membership inference in training data for generative models. The authors mainly introduce a data processing pipeline for exclusive evaluation on text-to-video data. The authors introduce a grey-box method performing comparably to white box methods.

Critical strengths

The authors claim to introduce the first grey-box methods (GB2) that allows for member inference for text-to-video methods. They strongly motivate the need for this contribution and show the lack of related work and prior solutions fixing this problem without the need of internal model information.

Critical weaknesses
  • The evaluation sections writing needs some clarity improvements its unclear that the white box methods while outperforming your approach suffer the requirements of extensive model internal information.
  • Methodology of TRG shows resembles to the one from Vid-SME, in general the prior works are vastly under explored and moved away saying the difference in underlying architecture has a large impact. But has this been sufficiently evaluated? I hope the authors can address the aspect show clear methodologic difference to prior art (even if underlying training algorithm has differences)
  • The novelty of the work is quite low in general outside of the dataset / dataset processing even under the assumption that the method is largely inspired by prior work
  • The evaluation is relatively weak considering a single empirical study, generally lack of theoretical reasoning as to why the method even works undermines the quality of the contribution
  • Limited to a single model creating the same problem as prior sota methods introducing limited evaluation and comparisons etc.

I currently lean to an accept due to the relevance of the topic, but I am generally on the edge and require my points to be addressed.

Questions for authors

Have been asked under weakness section.

3 comments
AuthorsOpenReview ↗

Response to RQhQ

We thank the reviewer for their thoughtful feedback and pointing out that our contribution falls into a "very important area of research currently not evaluated enough in the community" and that we "strongly motivate the need for this contribution".

We understand that the reviewer leans toward acceptance but is on the edge, and we worked hard throughout the rebuttal period to address all of their points. We found the suggestions incredibly helpful: based on them, we (1) restructured the evaluation around access levels, (2) compared our method directly against Vid-SME, (3) added a formal argument for why our method works, and (4) fine-tuned another model with known membership for more comprehensive evaluation. We summarize the key findings below; the details have already been reflected in the revised manuscript.


"The evaluation sections writing needs some clarity improvements" (W1)

This is a fair point; we apologize for not making this clear in the original submission, and have since addressed this. We restructured the opening of Sec. 6 around access levels and split Tab. 1 into two parts (label-free and supervised attacks), while adding two new columns (one indicating access levels—BB, GB or WB—and another listing what exactly is queried). We now explicitly state that the reversal gap needs only loss queries, whereas baselines need intermediate activations plus a reference set with known members and non-members.


"Methodology of TRG shows resembles to the one from Vid-SME ... has this been sufficiently evaluated?" (W2, W3)

This is a great question! We agree that the difference should be tested properly. Vid-SME and the reversal gap share the idea of contrasting a video with its time-reversed copy. However, they differ in (at least) three aspects: (1) Vid-SME is intended for video-understanding language models, and accordingly measures the Sharma-Mittal entropy of the output token distribution, whereas the reversal gap is intended for T2V models, and measures the denoising loss; (2) the reversal gap evaluates both directions under the same noise draw, and uses the forward loss for normalization (so that the per-clip difficulty is accounted for; this is not present in Vid-SME); and (3) the reversal gap does not need a reference model. Beyond arguing for this, we also tested this by adapting the Vid-SME statistics for T2V. On the same queries, Vid-SME reaches 0.526-0.547, whereas the reversal gap reaches 0.608. We added this side-by-side comparison to the appendix and the Related Work section. Finally, we have updated the contributions list to more accurately reflect this as follows: (1) the distribution-matched non-member set with the blind-baseline protocol, (2) the adaptation of image and classifier MIAs across access levels, and (3) the reversal gap.


"Lack of theoretical reasoning as to why the method even works" (W4)

We agree with the reviewer that the paper should explain this better. We, thus, added a subsection with a formal argument and further intuition. Suppose the loss of a clip is the sum of (1) a difficulty term, which is approx. equivalent for a clip and its reversal, and (2) a membership term, which applies only to the direction seen at training time. Then, the normalized reversal difference cancels the first term while keeping the second. We found that the data support both assumptions: the within-pair correlation of the losses drops from 0.77 to 0.24 under the reversal contrast (App. D.1-D.2), and the reversal gap is at chance on both controls, which never saw the members (0.502 and 0.504). The reversal gap is also better than loss-based MIA at every equivalent query budget (see main response, point #3). We now state additional failure cases: nearly time-symmetric clips and models trained with temporal-flip augmentation.


"Limited to a single model" (W5)

This is a fair point and we thank the reviewer for bringing this up! A clean MI evaluation needs a model whose exact training set is public, which we found to be extremely rare for T2V models. We address this in two ways. First, the two controls measure how much of the score a model can obtain without having seen the members, and the reversal gap exceeds them by +0.128 [0.087, 0.169] and +0.128 [0.080, 0.177] (see main response, point #2). Second, we fine-tuned MVDiT-VideoUFO on 500 clips of our non-member collection (2.5K steps, ~20 epochs) and held out 500 further clips. As members and non-members come from one collection, any signal is specific to the clips; the reference-calibrated loss is 0.626 [0.590, 0.659] here (see main response, point #6). We scope our claims about the reversal gap accordingly and expand the Limitations section with additional possible sources of signal to consider.


We look forward to any additional questions or suggestions! If the reviewer’s concerns have been addressed, we would appreciate it if they considered increasing their score.

AuthorsOpenReview ↗

Polite nudge

We once again thank the reviewer for their thoughts. Since the rebuttal period is nearing its end, we would really appreciate it if the reviewer would review our responses, which we believe have addressed all of their concerns, as outlined in the rebuttal above and included in the revised PDF. Thanks so much in advance!

Reviewer_RQhQOpenReview ↗

Reponse to Authors

Dear Authors,

thank you my main concerns have been addressed I will keep my rating which I think is quite fair for the given work! I do think the work would require higher theoretical foundation and a more novel approach for a strong/clear accept.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers and the AC

We thank reviewers for their thoughtful feedback and pointing out some of the strengths of our submission; all three reviewers found that the topic is important and under-studied, the evaluation methodology carefully designed (X2se, kDR3), and the reversal gap method has a clear intuition behind it (X2se, kDR3).

We found the suggestions incredibly helpful and ran all requested experiments. Below, we summarize the main points, shared across reviews. We use these rebuttal responses to summarize the key findings of new experiments; the details have already been reflected in the revised manuscript. We look forward to engaging with the reviewers further!


1. Threat model + supervised attacks (X2se, RQhQ)

We now state which attacks need member labels, and split Tab. 1 into two parts accordingly (as label-free and supervised attacks). The rep. read-out, the per-block gradient attack, and the joint model all assume that the attacker has a set with known members and non-members. Tab. 1 now also has a column indicating access level (BB, GB or WB) and a column listing what exactly is queried.


2. Limitations of the blind-adjusted AUROC (X2se)

We agree with the note about Def. 5.1 and carefully narrowed its interpretation. To strengthen evidence for membership signal, we now present the gap between the target model and two controls: (1) a reference model, trained on Panda-70M, and (2) a randomly initialized model copy. With paired bootstrap intervals, the read-out exceeds the two control models by +0.103 [0.072, 0.138] and +0.122 [0.082, 0.163] in blind-adjusted AUROC, and the reversal gap exceeds them by +0.128 [0.087, 0.169] and +0.128 [0.080, 0.177]. The gaps of the read-out remain when we replace the linear residualization with gradient boosting (+0.086 and +0.114) or kernel ridge regression (+0.084 and +0.102).


3. Compute-matched comparison with loss-based MIA (kDR3, RQhQ)

We reran the denoising loss using identical timesteps, windows, and noise draws as the reversal gap. We found that the loss does not improve with averaging (0.532-0.533, 4 -> 32 queries). With equivalent query budgets, the reversal gap is better by +0.045 [0.005, 0.082] at 8 queries, +0.062 [0.022, 0.100] at 16 queries, and +0.075 [0.036, 0.111] at 32 queries. Therefore, its advantage does NOT come from the number of queries, but, consistent with the intuition, from the contrast with the time-reversed clip.


4. Consistency of reported results (kDR3)

We thank the reviewer for noticing this! We traced all results back to the experiments that produced them, using our W&B logs, and updated Tab. 1, ensuring all results presented in it follow the same protocol. We also added a table to the appendix that maps experiments to utilized subsets and settings for complete transparency and reproducibility (more in the response to reviewer kDR3).


5. Relation to Vid-SME + novelty (RQhQ)

We added a side-by-side comparison with Vid-SME. On the same queries, it reaches 0.53-0.55, whereas the reversal gap reaches 0.61.


6. Additional baselines + another model (kDR3, RQhQ)

On the matched split, SecMI reaches 0.476 [0.455, 0.496] and CLiD 0.539 [0.505, 0.574]. Both score below the reversal gap’s performance at 0.608 [0.576, 0.639]. We also fine-tuned MVDiT-VideoUFO on 500 clips of our non-member collection and held out 500 further clips. The fine-tuning leaves a clear membership signal: the reference-calibrated loss is 0.626 [0.590, 0.659] here. Together with the two control models, this shows that our evaluation separates clip-level membership from differences in data source. We scope our claims about the reversal gap accordingly and expand the Limitations section with additional possible sources of signal to consider.


If the reviewers’ concerns have been addressed, we would appreciate it if they considered increasing their score.

AuthorsOpenReview ↗

Polite nudge

We thank all reviewers again for their thoughtful feedback. We posted our responses to each review on [date] and uploaded a revised PDF with the changes highlighted. The revision includes the additional experiments the reviewers requested, and these results have made the paper stronger.

As the discussion period ends soon, we would be grateful if the reviewers could read our responses and, if their concerns have been addressed, consider updating their scores. We are happy to answer any remaining questions before the deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.