acceptodds
Under review as a conference paper at ICLR 2027

TRIROUTE: Routing Visual Evidence Under Domain Shift for Cross-Domain Deepfake Detection

Abstract

Multimodal models are expected to integrate complementary signals across modalities, yet in practice they often rely on spurious shortcuts from a single modality. This failure is particularly critical in cross-domain audio-visual deepfake detection, where models may ignore visual manipulation evidence and instead rely on easier but unreliable audio cues. A common approach to mitigate such failures is to enforce invariance through regularization or domain-adversarial objectives. However, it remains unclear whether these methods change how models make decisions, or merely reshape representations without altering their usage. In this work, we show that robustness in cross-domain deepfake detection is not primarily a problem of enforcing invariance, but of controlling how information is routed at inference time. We demonstrate that loss-level interventions fail to resolve representation-level entanglement in standard binary decompositions, where modality-specific signals and domain nuisance signals remain mixed, leading to persistent shortcut reliance. We introduce TRIROUTE, a simple three-way factorization that separates shared content, modality-specific task-relevant components, and residual domain-dominant signals. This structural design explicitly constrains the prediction pathway, encouraging the model to rely on visual manipulation evidence when audio cues are misleading. Across cross-domain audio-visual deepfake benchmarks, TRIROUTE yields substantial improvements in the diagnostic FV-RA setting, where correct predictions require reliance on visual evidence. Crucially, we provide multiple lines of evidence that these gains arise from changes in inference-time routing rather than representation invariance. Head ablation, input-level masking, and subspace probing show that predictions depend critically on the visual-unique component, while domain information remains present in the representation. Our results suggest a shift in perspective for multimodal deepfake detection: instead of suppressing spurious features, robustness can be achieved by explicitly structuring the pathways through which decisions are made.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
cross-domain deepfake detection, audio-visual learning, representation factorization, evidence routing, domain shift, inference-time intervention, AV-HuBERT

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_yx8mOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem in cross-domain audio-visual deepfake detection where models rely on an “audio shortcut” and ignore manipulation evidence on the visual side. Rather than enforcing invariance through a loss, the paper proposes to solve this through the structural design of “routing,” i.e., which information is passed to the prediction head at inference time. The proposed method, TRIROUTE, decomposes frozen AV-HuBERT features into three branches and structurally excludes the residual branch from the prediction head. The model is trained on AV-Deepfake1M and evaluated by transfer to FakeAVCeleb (FAVC), LAV-DF, and AVLips.

Critical strengths
  1. The question posed (whether existing methods improve robustness by learning better representations, or by changing how the model uses its representations) is interesting.
  2. The authors explicitly exclude the individual use of each building block from their novelty claim and limit the contribution to the combination and the inference-time rule, which is honest.
  3. The experiments follow a transfer design in which training is performed on AV-Deepfake1M and evaluation is conducted on disjoint target datasets (FakeAVCeleb, LAV-DF, AVLips) that are never accessed during training, validation, or model selection.
Critical weaknesses
  1. The fragility of the nuisance partition remains unresolved. The main result of 0.838 depends on hash(speaker_id) mod 2, which is effectively a random binary speaker partition, yet a “random speaker regrouping” of the same kind drops to 0.694, and 4-way and 8-way hashes drop to 0.701 and 0.667 (Table 14). The paper does not explain why partitions of the same kind differ by 0.14, so the possibility that the result is a coincidence tied to one particular hash cannot be ruled out.
  2. There is no simple baseline that directly addresses the audio shortcut. Since the FV-RA failure is a matter of “relying on audio,” comparisons with a visual-only detector and with modality dropout (audio masking) during training are needed.
  3. The effect of push-pull is not separated. The difference between “three-way, no push-pull” (0.600) and TRIROUTE (0.838) is the effect of adding L_dom and L_adv at the same time, and there are no rows for L_dom only or L_adv only.
  4. Under leading-silence trimming in Table 17, TRIROUTE’s RV-FA drops from 0.999 to 0.810 (−0.189) and its overall AUC from 0.922 to 0.866, which is far larger than the RV-FA drop of the two-way baseline (−0.038). While the flatness of FV-RA is emphasized, the fact that performance on the audio-fake categories depends heavily on leading silence, a dataset-specific artifact, is hardly discussed.
  5. Although AV1M contains more than one million clips, only 8,000 clips are used for training, with batch size 1, 100 epochs, and 3 seeds. Batch size 1 introduces training instability. It is unclear whether the advantage of the proposed method remains when the amount of training data is increased.
Questions for authors

Clear responses to the weaknesses above would be appreciated.

6 comments
AuthorsOpenReview ↗

Response to Reviewer yx8m (Part 1)

Dear Reviewer yx8m,

Thank you for the detailed review and actionable suggestions. We have conducted new experiments addressing the main concerns: six additional nuisance partitions, matched visual-only and train-time audio-masking controls, separate push–pull ablations, matched leading-segment trimming, and an expanded seed analysis. The results identify where the proposed objectives add value, where simpler baselines are competitive, and which conditions affect stability. We provide the numerical evidence and corresponding manuscript revisions below.

W1. Fragility of the source nuisance partition

We have directly tested the reviewer's concern with six additional balanced speaker partitions, each evaluated over three seeds. This experiment clarifies both the promise and the sensitivity of nuisance-guided training. We retrained with six additional balanced binary speaker partitions, three seeds each (new).

PartitionFV-RA AUC
Original md5(speaker_id) mod 20.838
Six new random speaker partitions0.799, 0.742, 0.735, 0.727, 0.725, 0.674
All sevenMean 0.749; median 0.735

The original hash is the best of the seven. The earlier random-regrouping result (0.694) and the 4-/8-way results (0.701/0.667) are consistent with substantial partition sensitivity. Both the hash and the new binary partitions split speakers without known acquisition-condition labels. We therefore cannot explain the hash's advantage by a verified nuisance factor and acknowledge that it may reflect a favorable random realization.

An exploratory analysis finds a correlation between label decodability and gain across the seven partitions (r = 0.97); a speaker-mean-channel-feature k-means partition reaches 0.852. Neither establishes a reliable source-only selection criterion: seven partitions are too few for a robust conclusion, and the k-means procedure still uses speaker IDs. Greater label decodability could simply strengthen supervision without corresponding to a causal nuisance variable.

We will present the full distribution prominently and qualify the contribution as dependent on a suitable source-side grouping. We do not claim that arbitrary binary speaker partitions reliably deliver the originally reported gain.

W2. Visual-only and train-time audio-masking controls

We agree that the strongest test of TRIROUTE is against simpler ways to suppress audio shortcuts. We therefore added both a visual-only detector and a train-time audio-masking baseline under matched features and training settings. We trained both baselines using the same frozen features, seeds 40–42, and schedule (new).

TargetVisual-onlyTRIROUTEPaired TRIROUTE − visual-only differences
FAVC0.803 ± 0.0310.838 ± 0.024+0.043, +0.011, +0.050
AVLips0.771 ± 0.0220.789 ± 0.021−0.026, +0.023, +0.057
DF-TIMIT0.583 ± 0.1110.598 ± 0.031+0.028, −0.112, +0.129
Controlled LAV-DF0.781 ± 0.0190.873 ± 0.012+0.061, +0.100, +0.115

The comparison establishes an important trade-off: visual-only is competitive on FV-RA, while TRIROUTE retains an audio-visual inference path needed for audio-manipulation cases. With only three seeds, the FAVC, AVLips, and DF-TIMIT differences do not justify a general superiority claim. The clearest paired improvement is on controlled LAV-DF, a list constructed post hoc. Across seven nuisance partitions, TRIROUTE's mean FAVC score (0.749) is below visual-only (0.803).

We also applied train-time audio masking to the two-way model with probability 0.6 (distinct from the original test-time masking):

FAVC FV-RASeed 40Seed 41Seed 42Mean ± SD
Two-way0.5130.4850.4510.483 ± 0.031
+ Train-time audio masking0.6010.6520.6170.623 ± 0.026

Masking improves all three seeds, but remains below visual-only and the original-partition TRIROUTE result. This does not establish that TRIROUTE is superior to all modality-dropout strategies: symmetric dropout has not been evaluated.

A visual-only detector also cannot detect audio-only manipulations in the tested VidTIMIT RV-FA setting (0.472 versus TRIROUTE 0.731). However, FCD scores 0.763 there, and the leading-silence analysis in W4 reveals an audio-side weakness of TRIROUTE. We will report these trade-offs without claiming universal advantage.

AuthorsOpenReview ↗

Response to Reviewer yx8m (Part 2)

W3. Isolating the two push–pull objectives

We appreciate this request because it allows us to identify which training signal contributes to the gain. We have now isolated L_dom and L_adv in a controlled ablation. We now evaluate each term separately with matched training and checkpoint selection (three seeds; new).

Training objectiveFAVC FV-RAAVLips
Neither (FCD-style)0.671 ± 0.0330.597 ± 0.030
L_dom only0.742 ± 0.0270.694 ± 0.024
L_adv only0.703 ± 0.0360.641 ± 0.028
Both (TRIROUTE)0.838 ± 0.0240.789 ± 0.021

Each objective is associated with an improvement over the no-push-pull setting under the original partition (+0.071 for L_dom, +0.032 for L_adv on FAVC); their combination improves by +0.167. We have not established a statistically supported interaction or synergy, and the estimates are sensitive to the choice of nuisance partition.

We also identified an untraceable original value: Table 1 reports 0.600 for 'three-way, no push-pull', whereas the reproducible re-run yields 0.671 ± 0.033 (seed values 0.687, 0.633, 0.693). We will correct Table 1 and revise the previously stated +0.238 gain to +0.167. The existing 0.3×/3× weight sweep assesses weight sensitivity, not whether either term is necessary.


W4. Leading-silence sensitivity and audio-fake performance

The reviewer correctly identifies an asymmetry between FV-RA and audio-fake robustness under test-only trimming. To investigate whether this is a train–test mismatch, we added a matched train-and-test trimming experiment. With the submitted checkpoints, test-time trimming changes FV-RA only from 0.838 to 0.833, but RV-FA falls from 0.999 to 0.810, FV-FA from 0.994 to 0.898, and overall AUC from 0.922 to 0.866. The RV-FA drop (−0.189) exceeds that of the two-way baseline (−0.038) and no-push-pull model (−0.050); after trimming, the two-way baseline scores 0.962 versus TRIROUTE's 0.810.

To test whether this reflects a train–test cue mismatch, we removed the first 13 frames (0.52 s) from both feature streams during source training and target evaluation, retraining seeds 40–42 (new).

Change relative to original settingSeed 40Seed 41Seed 42Mean
FV-RA−0.091−0.011−0.047−0.050 (0.838 → 0.788)
RV-FA−0.006−0.005−0.022−0.011
FV-FA−0.032−0.041−0.069−0.047

Removing the leading segment during both training and testing largely prevents the RV-FA collapse seen under test-only trimming. However, FV-RA now also decreases by 0.050 on average, indicating that the original FV-RA result is not wholly insensitive to the leading segment. We cannot isolate the precise cue within that fixed 0.52-second interval.

We will report the RV-FA, FV-FA, and overall changes beside FV-RA in the main text, describe the implemented trimming procedure precisely, and withdraw the claim that stable FV-RA under test-time trimming rules out audio-shortcut reliance. The two-way baseline has not been retrained with this cut, so the retrained-setting comparison remains incomplete.

AuthorsOpenReview ↗

Response to Reviewer yx8m (Part 3)

W5. Training scale, batch size, and reproducibility

Thank you for raising reproducibility and training stability. We have extended the seed analysis beyond the submitted runs and will report the broader distribution, alongside the fixed training protocol, so that the improvement and its variability can be assessed together. We use 8,000 AV1M source-training clips and 2,000 source-validation clips, batch size 1 for variable-length features and within-clip alignment, and up to 100 epochs with source-validation early stopping. All internal controls share these choices, but that does not establish robustness to batch size or dataset scale.

Additional runs reveal substantially greater variability than the submitted three-seed summary:

  • Re-running the three submitted seed IDs yields 0.824 ± 0.022, compared with the submitted 0.838 ± 0.024.
  • Six further seeds (46–51) yield 0.741, 0.706, 0.786, 0.821, 0.690, 0.613 (mean 0.726, SD 0.074).
  • Across all nine runs, the reported mean is 0.763, SD 0.082. The expanded runs reveal a broader training-seed distribution than the initial three-seed estimate.

Five of the six added seeds are below the lowest originally submitted seed (0.813). Seed 51 is especially weak on FV-RA (0.613), with checkpoint selection at epoch 16 rather than epochs 88–97 for the other reported runs. This is consistent with an unstable training/selection process, but the results alone do not establish the cause.

We will use the nine-seed result as the main summary, list all seeds with source-validation AUC and selected epoch, and explicitly discuss the uncertainty. Baseline comparisons currently use only seeds 40–42; their observed differences (+0.035 over visual-only and +0.167 over no push–pull on those seeds) cannot be extrapolated to all nine runs without additional matched-seed evaluations.

We have not tested padded batches, gradient accumulation, or larger AV1M training subsets. The existing unsupervised scaling experiment concerns a different model and does not answer this question. We will state these gaps rather than imply training-scale robustness.

Reviewer_YJLwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies cross-domain audio-visual deepfake detection, focusing on a common failure mode: a multimodal detector may rely on easy audio shortcuts and ignore visual manipulation evidence when the test domain changes. The authors propose TRIROUTE, which splits frozen AV-HuBERT features into shared content, modality-specific task features, and a residual branch intended to collect domain-sensitive signals. Only the shared and task-specific features are passed to the prediction head, while the residual branch is excluded at inference time. The method is evaluated mainly on the FakeVideo–RealAudio setting, where detecting the fake requires visual evidence because the audio is real. Results show that TRIROUTE improves cross-domain detection in this setting, and masking or ablating the visual-unique branch substantially reduces performance, supporting the claim that the model uses visual evidence more than the two-way baselines.

Critical strengths
  1. The paper focuses on a clear and important failure mode. Rather than relying only on overall accuracy, it examines FakeVideo–RealAudio cases, where audio shortcuts cannot solve the task. This makes it easier to assess whether the model uses visual evidence or relies on shortcuts.
  2. The authors use head ablation, input masking, residual-only readout, and subspace probes to examine the model’s behavior. These analyses support the claim that the gains come from greater use of visual features, not just an increase in overall scores. The cross-domain protocol is also clearly described. Target data are kept out of training, model selection, and hyperparameter tuning. Additional target datasets are evaluated with the same source-selected checkpoints, which makes the transfer results more convincing.
Critical weaknesses
  1. The link to earlier audio-visual detection work could be clearer. Prior studies have explored modality-specific features, cross-modal representations, regularization, and audio-visual fusion. They may not address TRIROUTE’s specific focus on controlling which features reach the classifier under domain shift, but they are still relevant.
  2. The evaluation centers on one specific setting. TRIROUTE’s strongest gains appear in FakeVideo–RealAudio cases, where the model must rely on visual evidence. This is a useful test, but it does not establish that the method works equally well for other manipulation types or videos with little speech or lip movement.
  3. The routing tests do not fully rule out dataset-specific visual cues. Ablation and masking show that predictions depend on the visual-unique branch. However, they do not establish whether it captures general manipulation evidence or visual patterns tied to the datasets.
Questions for authors
  1. Why does TRIROUTE need three branches? Could you compare different numbers of branches and include stronger baselines based on shared/private features or domain separation?
  2. How much does performance depend on the nuisance partition? If reliable speaker IDs are unavailable, what other source-side information could define the residual branch?
  3. Does the method work beyond speech-related face manipulation? Could you test videos without speech, clips with little lip movement, or other types of visual manipulation?
3 comments
AuthorsOpenReview ↗

Response to Reviewer YJLw (Part 1)

Dear Reviewer YJLw,

Thank you for the constructive review. We have strengthened the paper in three ways: clearer attribution of established architectural ingredients, additional controlled cross-dataset evaluations, and more targeted analyses of visual dependence. These additions sharpen the central empirical question, when nuisance-guided routing provides useful visual evidence under cross-domain shift, without conflating visual dependence with proof of a particular causal cue. We address each point below.

W1. Positioning relative to prior audio-visual detection work

We appreciate the opportunity to sharpen the novelty claim: TRIROUTE is not presented as inventing shared/private factorization, but as testing a specific way to train the residual and task representations against source nuisance information. We will revise Related Work to explicitly connect: (i) audio-visual fusion and synchronization to Chung & Zisserman, Zhou & Lim, AVAD, SpeechForensics, and AVH-Align; (ii) shared/private representations to DSN; (iii) three-way factorization to FCD and supervised domain representations to DIVA; and (iv) adversarial nuisance suppression to DANN and Kim et al.

These ingredients are not individually novel. Relative to an FCD-style factorization, TRIROUTE combines source speaker-group supervision of the residual with adversarial suppression of that signal in the task representation, and evaluates this combination through targeted visual-dependence diagnostics. We will replace wording that presents residual exclusion or a general 'routing principle' as unique to our method.

We will also state the limits of the empirical gain alongside this contribution: FAVC FV-RA AUC is 0.838 for the original partition but 0.749 on average over seven partitions, while a visual-only baseline achieves 0.803. The evidence therefore supports a partition-sensitive training improvement over FCD, not a general superiority claim over simpler modality-specific detectors.


W2. Evaluation scope and controlled target construction

Our primary target is the challenging FV-RA case, where a detector can appear successful by exploiting the real audio rather than the manipulated video. The additional evaluations clarify both where the method transfers and where its scope remains limited. We now distinguish native annotations from constructed directional evaluations:

TargetEvaluation labelN
FAVCNative FV-RA6,667
LAV-DF (original)Native FV-RA13,358
LAV-DF (duration-matched, re-encoded)Native FV-RA9,846
AVLipsConstructed FV-RA-type from binary labels/audio audit7,602
DF-TIMITNative face-swap / original-audio cases960
VidTIMITConstructed RV-FA with cloned speech640

Under the original speaker partition, the additional AV1M-trained comparisons are:

TargetTRIROUTEVisual-onlyFCD
Controlled LAV-DF0.873 ± 0.0120.781 ± 0.0190.702
AVLips0.789 ± 0.0210.771 ± 0.0220.597
DF-TIMIT0.598 ± 0.0310.583 ± 0.1110.545
VidTIMIT RV-FA0.7310.4720.763

The original LAV-DF evaluation contains an important confound. Duration alone yields 0.641 AUC, video-zeroed TRIROUTE 0.702, and the audio-reliant two-way baseline 0.942 (versus TRIROUTE 0.914). We therefore will not use that result as evidence of visual routing. In the new controlled list, real/fake duration is matched and all clips are re-encoded identically; duration-only falls to 0.509 and two-way to 0.561. The controlled TRIROUTE result (0.873) is more informative, but this control was constructed after inspecting the original target results and will be explicitly identified as post hoc.

The controlled LAV-DF gain is the clearest additional finding. Differences from visual-only on AVLips and DF-TIMIT are modest or noisy, and TRIROUTE trails FCD on VidTIMIT RV-FA. We will narrow claims accordingly and identify speech-free, low-lip-motion, and generator-stratified evaluation as untested.

AuthorsOpenReview ↗

Response to Reviewer YJLw (Part 2)

W3. What the visual-dependence tests establish

The reviewer raises an important distinction between visual dependence and manipulation-specific visual evidence. Our ablations directly establish the former; we now supplement them with controlled tests targeting concrete non-visual confounds, while keeping the stronger causal interpretation appropriately bounded. We will revise Sections 6.4–6.6 to use 'dependence on the visual-unique factor' rather than claiming definitive mechanistic proof of shortcut elimination.

The dependence is measurable: masking the visual-unique factor reduces FAVC FV-RA AUC from 0.838 to 0.491. On AVLips, full visual masking reduces AUC from 0.789 to 0.475, whereas full audio masking changes it by only −0.003.

We added a post-hoc controlled LAV-DF evaluation to test specific confounds. With duration matching and uniform re-encoding, TRIROUTE achieves 0.873 ± 0.012; zeroing video reduces this to 0.534 ± 0.016, while audio-only and duration-only classifiers achieve 0.527 and 0.509. These controls argue against duration and audio-only cues as explanations on this constructed list, but they do not rule out transferable or dataset-specific visual artifacts.

Because the controlled list was designed after inspecting the original LAV-DF results, we will report it as an exploratory robustness control, not as independently prespecified confirmation. Generator- and identity-disjoint visual-cue analyses remain open.

AuthorsOpenReview ↗

Response to Reviewer YJLw (Part 3)

Q1. Why three branches?

The three-branch design is motivated by a testable division of labour: shared evidence, modality-unique evidence, and a residual that absorbs supervised source nuisance information. We have added comparisons that directly test this design choice rather than relying on architectural intuition alone. Three branches correspond to three intended roles: shared cross-modal information, modality-unique information, and a nuisance-supervised residual. This is a useful design choice, not a proven minimal or optimal factorization. We will remove the statement that three roles constitute 'the minimal partition'.

ConfigurationFAVC FV-RA AUC
Two-way baseline0.483
Two-way + push-pull0.531 ± 0.030
Three-way without push-pull (FCD re-implementation)0.671 ± 0.033
Three-way + push-pull, residual included in head0.762 ± 0.030
TRIROUTE, residual excluded0.838 ± 0.024

These results suggest that the nuisance objectives are more effective with a dedicated residual factor than in the tested two-way design. Excluding the residual improves AUC by 0.076 relative to including it under the original partition, but the included-residual model also improves over the no-push-pull setting (0.762 versus 0.671). Thus exclusion contributes; the data do not establish that it is necessary.

We will correct the original untraceable no-push-pull value (0.600) to the reproducible 0.671 ± 0.033, add parameter counts (TRIROUTE: 2.98M training / 2.01M inference), and state explicitly that four-or-more-branch and DSN-style matched controls have not been tested. These results also remain subject to the partition sensitivity discussed in Q2.


Q2. Nuisance partitions and unavailable speaker IDs

The expanded partition audit identifies an important design condition: the quality of source-side grouping matters, and the method's gain should be interpreted in that context. Across the original hash (0.838) and six new balanced random speaker partitions (0.799, 0.742, 0.735, 0.727, 0.725, 0.674), FAVC FV-RA averages 0.749 (median 0.735). This is above FCD without push-pull (0.671) but below visual-only (0.803). The original hash is the most favorable tested draw, and we will no longer present it as representative.

Random clip-level labels yield a smaller average improvement over FCD (+0.025, positive in three of six draws). This suggests grouping quality matters, but does not identify a reliable nuisance definition or source-only selection rule.

If speaker IDs are unavailable, recording metadata or clip-level channel-feature clustering are possible alternatives, but neither speaker-ID-free approach has been evaluated. Our exploratory k-means variant uses speaker-mean features and still requires speaker IDs. We will state this dependency explicitly and avoid claiming annotation-free robustness when speaker identifiers are absent.


Q3. Applicability beyond speech-related face manipulation

TRIROUTE is designed and evaluated for speech-conditioned face manipulation, where cross-modal shortcuts are especially relevant. We appreciate the suggestion to test how far this mechanism extends beyond that setting. We have not evaluated speech-free videos, low-lip-motion strata, or predominantly non-speech visual manipulations. Because the front end uses mouth-crop AV-HuBERT features, our current evidence concerns speech-conditioned face manipulation.

Leading-silence trimming and audio masking are not substitutes for speech-free evaluation: they perturb audio while retaining footage of a speaking face. Likewise, DF-TIMIT FV-RA AUC (0.598 versus visual-only 0.583, with visual-only SD 0.111) is too uncertain to support a broader transfer claim.

We will narrow the title/abstract/conclusion claims to the evaluated speech-conditioned settings and identify speech-free and lip-motion-stratified tests as future work.

Reviewer_uuSWOpenReview ↗
Rating
2
Confidence
1
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies audio shortcut reliance in cross-domain audio-visual deepfake detection. TRIROUTE is proposed, which decomposes frozen AV-HuBERT features into three components. The residual components receive source-side nuisance supervision and are excluded from the prediction head, while alignment, adversarial, and orthogonality objectives encourage separation between the components.

Critical strengths

This paper explores a failure mode of practical significance. By focusing on the detection of "False Associations" (FV-RA), this study reveals how models may rely on audio "shortcuts"—a dependency often obscured by aggregated performance metrics. This category-based analytical approach offers a valuable perspective for research into cross-domain multimodal detection.

The evaluation protocol employed in this study is rigorously controlled. Both during the training phase and the selection of model checkpoints, only data from the source domain were used. The evaluation was conducted across multiple target datasets, incorporating various random seeds and independent experimental repetitions.

Critical weaknesses

The authors identify FCD as the most closely related existing work. However, there seems no direct experimental comparison between them. What’s the inherent difference between the two? And what’s the advantages of the proposed TRIROUTE?

The substantial dependence on the nuisance partition remains insufficiently explained. The authors explicitly acknowledge this limitation. However, FV-RA AUC decreases from 0.838 with the fixed binary speaker hash to 0.694 with random speaker regrouping, while finer speaker hashes also perform considerably worse. The distinction between the fixed hash and random speaker regrouping is particularly unclear because both partition speakers without explicit supervision about the underlying recording or acquisition conditions. The reported variation across training seeds does not capture uncertainty across different partition realizations. Evaluating several independently generated binary speaker partitions, together with a source-only criterion for assessing their suitability, would clarify whether the headline performance is representative.

The survey of related work is insufficient. The paper didn’t compare the proposed approach against existing methods from the recent period (2025–2026). In particular, there is a lack of comparison with recent techniques addressing issues such as audio shortcuts, modality imbalance, and cross-dataset generalization. The authors should either include these methods in their comparisons or provide a justification for why a fair comparison is not feasible. Failure to address these omissions limits the ability to assess the true competitiveness of the proposed method. A preliminary review suggests that the following works may be highly relevant to this paper:

[1] Boldisor D A, Smeu S, Oneata D, et al. Investigating self-supervised representations for audio-visual deepfake detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 43018-43029.

[2] Shahzad S A, Hashmi A, Yamagishi J, et al. SAVe: Self-Supervised Audio-visual Deepfake Detection Exploiting Visual Artifacts and Audio-visual Misalignment[J]. arXiv preprint arXiv:2603.25140, 2026.

[3] Anshul A, Gopal S, Rajan D, et al. Intra-modal and cross-modal synchronization for audio-visual deepfake detection and temporal localization[C]//2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025: 13826-13836.

Questions for authors

Already included in the weakness section.

5 comments
AuthorsOpenReview ↗

Response to Reviewer uuSW (Part 1)

Dear Reviewer uuSW,

Thank you for the review. Your comments prompted three concrete improvements: a matched FCD implementation, a seven-partition sensitivity audit, and a clearer comparison with recent work. These additions let us state more precisely what TRIROUTE contributes beyond prior factorization methods, where the gain is reproducible, and what remains conditional on the source grouping. We respond point by point below.

W1. Direct comparison with FCD and scope of the contribution

We appreciate this request: a matched FCD comparison is the most direct way to isolate what TRIROUTE adds beyond an established factorization. We re-implemented FCD under our AV1M-only training and source-validation selection protocol (new).

TargetFCDTRIROUTE
FAVC0.671 ± 0.0330.838 ± 0.024
AVLips0.5970.789
DF-TIMIT0.5450.598
LAV-DF (duration-matched, re-encoded)0.7020.873
VidTIMIT (RV-FA)0.7630.731
LAV-DF (original list)0.9210.914

The matched comparison shows that TRIROUTE adds value beyond FCD on four of six target evaluations, with the largest gains on FAVC and controlled LAV-DF. FCD remains stronger on two settings, which we report to delineate the method's scope.

We matched frozen AV-HuBERT features, AV1M data manifests, seeds, optimization schedule (up to 100 epochs, patience 15), source-validation checkpoint selection, and the four shared factorization losses and weights. Relative to this FCD implementation, TRIROUTE adds source speaker-group supervision of the residual and adversarial suppression of that group signal in the task representation. These terms use two additional training-only domain classifiers (2.98M versus 2.55M trainable parameters); both methods retain the same 2.01M-parameter inference path.

We agree that three-way factorization and residual exclusion are not novel to TRIROUTE. FCD already uses the same inference components, while DSN, DIVA, and DANN establish further precedents. We will correct the Related Work and contribution statement accordingly. Our narrower contribution is the specific nuisance-supervised/adversarial training combination and its evaluation under audio-shortcut-sensitive cross-domain tests.

Two corrections matter for interpretation. First, the reported 0.838 uses the most favorable of seven tested speaker partitions; the mean across partitions is 0.749 (W2), reducing the mean FAVC advantage over FCD to 0.078. Second, the original 'three-way, no push-pull' value of 0.600 could not be traced to a run; its reproducible re-run is 0.671 ± 0.033, which we will use in the revised ablation. These qualifications will accompany the main comparison.

AuthorsOpenReview ↗

Response to Reviewer uuSW (Part 2)

W2. Nuisance-partition sensitivity and representativeness

We agree that partition sensitivity is important to characterise, and have now conducted a substantially broader audit rather than relying on the original hash. We evaluated six additional independent, balanced binary speaker partitions (three training seeds per partition; new), alongside the originally reported hash. Each random partition assigns the same 1,657 speakers into groups of 829 and 828.

Source partitionFAVC FV-RA AUC
Original speaker hash0.838
Random partition 10.799
Random partition 20.742
Random partition 30.735
Random partition 40.727
Random partition 50.725
Random partition 60.674
Across all sevenMean 0.749; median 0.735

The partition audit shows that the strongest configuration (0.838) is not the across-partition average; we therefore report both the original configuration and the full partition distribution. We agree that this substantially qualifies the original headline. FCD, which does not use these labels, achieves 0.671, while visual-only achieves 0.803; thus the mean partition improves on FCD but does not outperform visual-only. The earlier 'random speaker regrouping' result of 0.694 was a separate draw and falls within the observed range.

Importantly, md5(speaker_id) mod 2 and random balanced speaker regrouping do not encode verified recording-condition labels. The original hash was fixed before target evaluation, but we have no evidence that it is intrinsically better aligned with acquisition nuisance factors. We therefore do not attribute its advantage to a known nuisance mechanism.

Across seven partitions, improvement correlates with source-label decodability (r = 0.97), and clustering speaker-mean channel features gives 0.852. These are exploratory observations, not a validated source-only partition-selection criterion; the clustering variant also still requires speaker IDs. Random clip-level labels provide a smaller average improvement over FCD (+0.025; positive in three of six draws).

We will report the full partition distribution in the main paper, distinguish partition variation from training-seed variation, and make our claim explicitly conditional on an informative source-side grouping. We cannot yet guarantee strong transfer from an arbitrary partition.


W3. Recent related work and comparison fairness

Thank you for these relevant references. We have expanded the positioning to make the relationship between representation learning, real-only training, and our supervised source-to-target protocol explicit. We reviewed all three cited works and will discuss their assumptions and evaluation protocols explicitly. We have not run them under a matched protocol and therefore do not claim to outperform them.

  • Boldisor et al. (CVPR 2026) study transfer of self-supervised audio, visual, and audio-visual representations using linear probes and real-data anomaly detection across several datasets. This is particularly relevant because TRIROUTE also uses frozen AV-HuBERT features. A source-only linear probe on those same features would be a useful matched control, but we have not completed it.
  • SAVe (Shahzad et al., 2026) uses real-video-only training, self-blended pseudo-manipulations, and synchronization cues. Its training supervision differs from our AV1M fake-label setting; a released-model comparison would be informative but would not be training-matched.
  • Anshul et al. (ICCV 2025) combine real-video temporal synchronization pretraining with downstream detection/localization. A training-matched comparison would require reproducing that pretraining and retraining the detection stage on our AV1M subset; this has not been completed.

We will also correct our baseline terminology. SpeechForensics and AVAD are released-checkpoint references evaluated on common target splits, not AV1M-training-matched baselines. SpeechForensics is substantially stronger on FAVC FV-RA (0.975 versus 0.838). The AV1M-trained internal controls are two-way factorization, DANN, CORAL, our FCD implementation, visual-only, and train-time audio masking.

The revised Related Work will position TRIROUTE relative to recent audio-shortcut, modality-imbalance, and cross-dataset methods, and the limitations will explicitly identify the outstanding matched comparisons.

Dear Reviewer uuSW,

We would like to gently follow up on our response. In particular, we added a matched FCD comparison, a seven-partition sensitivity audit that explicitly characterizes the dependence on the source grouping, and a substantially expanded discussion of the recent work and the limits of unmatched comparisons. We also revised our claims to reflect the observed partition variability rather than presenting the strongest partition as representative.

We would be grateful if you could reconsider the rating in light of these additions and clarifications. Thank you again for your careful review and valuable feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal summary for the Area Chair

Dear Area Chair,

Thank you for handling our submission. Below we list what was added during the rebuttal and how the claims changed.

1. Additional results

  • Matched FCD comparison (uuSW). Under our AV1M-only protocol, TRIROUTE improves on a re-implemented FCD on four of six target evaluations (FAVC FV-RA 0.838 under the original partition vs. 0.671; duration-matched LAV-DF 0.873 vs. 0.702).
  • Partition audit (uuSW, YJLw, yx8m). Six additional random speaker partitions, three seeds each. Across the seven partitions FV-RA averages 0.749 (median 0.735), above FCD (0.671). We now report the full distribution instead of the single original partition.
  • Simple baselines (yx8m). A visual-only detector reaches 0.803, and train-time audio masking moves the two-way baseline from 0.483 to 0.623, under matched features and schedule. On duration-matched LAV-DF, TRIROUTE reaches 0.873 against 0.781 for visual-only, and visual-only does not detect audio fakes (VidTIMIT RV-FA 0.472 vs. 0.731).
  • The two objectives ablated separately (yx8m). Each helps on its own (L_dom 0.742, L_adv 0.703); together they reach 0.838.
  • Leading-segment control (yx8m). Removing the leading segment in both training and testing largely prevents the RV-FA collapse seen under test-only trimming (−0.189 → −0.011).
  • Seeds (yx8m). Nine seeds in total give 0.763 ± 0.082, which we adopt as the main summary.
  • Recent work (uuSW). The released AV-HuBERT probe of Boldisor et al. reaches FV-RA 0.715 on our FAVC split (RV-FA 0.999, FV-FA 0.996).

2. Changes to the claims

  • The contribution is stated as the combination of nuisance supervision and adversarial training on top of an FCD-style factorization, and each ingredient is attributed to prior work.
  • Claims are limited to speech-conditioned face manipulation, and the gain is stated as conditional on an informative source-side grouping.
  • One value in Table 1 (0.600) could not be traced to a run and is replaced by the reproducible 0.671. We also say where simpler baselines are competitive (visual-only on FV-RA).

TRIROUTE targets one failure mode, fake video with real audio. It improves on the closest prior factorization in the matched comparisons above, and the paper now states the conditions under which that gain holds. Thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.