acceptodds
Under review as a conference paper at ICLR 2027

SyndromeFlow: Joint Temporal Alignment and Recomposition Inference for Audiovisual Watermark Authentication

Abstract

Replacing or replaying watermarked segments can alter a video while leaving the embedded watermarks intact. Benign audio delays and clock drift make these edits harder to localize: tampering can bias alignment, and alignment errors can in turn create false tamper evidence. We propose SyndromeFlow, an audiovisual watermarking framework that jointly infers alignment and sparse tampering under a shared Bayesian posterior. Our code links segments through parity checks, so copying a segment with its watermark can break consistency with surrounding content. The receiver records failed checks as a syndrome and scores how candidate alignments and edited regions jointly explain it. Local tamper evidence thus refines alignment, while posterior averaging produces separate audio and video localization maps. Compared with a matched align-then-verify receiver using the same watermark, SyndromeFlow increases audio AUPRC from 0.566 to 0.637, reduces mean absolute alignment error from 5.6 ms to 0.9 ms, and maintains video AUPRC above 0.99. A controlled comparison of the inference steps attributes most of the improvement to tamper evidence revising alignment; averaging contributes further localization gains when alignment remains uncertain. In a complete-system comparison on the same speaking-person benchmark, each method embeds its own watermark before local copying. SyndromeFlow achieves video AUPRC of 0.994, compared with 0.086 for the strongest of five image-watermark baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audiovisual watermarking, Tamper localization, Bayesian joint inference, Temporal alignment, Structured prediction

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_og1zOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work propose SyndromeFlow, an an audio-visual joint watermarking method. The contributions include authentication via temporal alignment and sparse tampering, AV parity representation for content-binding evidence, and a syndrome-based receiver for audio and video localization.

Critical strengths
  • This method has novel design on the watermarking and localization mechanism. By proposing the GraphCode Encoder, SyndromeFlow Receiver, Shared factor scoring, and Joint posterior readout, this method is able to learn and localize the manipulation part correctly.
  • The results are convincing. Compared to current methods on joint watermarking, SyndromeFlow has better performance on multiple task and metrics.
Critical weaknesses
  • The writing and presentation could be improved. The discussion of related work is limited, and an additional section describing the difference between SyndromeFlow and other methods could be added if possible.
  • The motivation could be clarified. Although there are some new approaches proposed in this work, I am still interested in the actual motivation, or how these technologies are came out and why they are adapted.
Questions for authors

Please see Critical Weaknesses part above. If these concerns could be solved, I will consider raise my score.

Reviewer_gfz5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies audiovisual watermark authentication when watermarked content is copied along with its watermark (replacement from another marked source or replay) while benign audio offsets and clock drift are also present. It proposes SyndromeFlow: a keyed parity code binds distant video and audio states, and the receiver jointly scores candidate alignment paths and sparse tamper supports under one posterior, so local tamper hypotheses can revise alignment. On 140 AV-Deepfake1M++ sources it reports video AUPRC 0.994 vs ≤0.086 for image-watermark baselines, audio AUPRC 0.637 vs 0.072 for AudioSeal, and a readout decomposition showing most of the gain comes from tamper evidence correcting the alignment path.

Critical strengths
  1. This paper is clearly written and easy to follow.
  2. The threat model of copying content together with its watermark of is realistic.
  3. The joint path–support formulation is clean and well motivated.
Critical weaknesses
  1. The evaluation seems narrow and partly tailored to the attacks. Audio AUPRC (0.64) lags video (0.99) with no explanation; the support hypotheses (23 fixed 500 ms audio intervals, quadrant TopK masks) closely match the four test geometries; and robustness covers one codec setting, 6 s clips, an 80 ms deformation bound, and no adaptive attacker or hyperparameter sensitivity.
  2. The headline comparison is against baselines that cannot detect marked copying by construction: the five image watermarks have no temporal binding and AudioSeal’s presence score is not a localization score. No temporally bound AV watermark (V2A-Mark, LAVA, Alattar-style indexing) is compared, so the 0.994 vs 0.086 gap reflects the threat model, not the method. The matched gain (pilot+prior → C: 0.586 → 0.637 audio AUPRC) is modest.
Questions for authors
  1. Why is no temporally bound AV watermark included as a baseline? How much of the gain comes from joint inference versus simply having a time-bound code?
  2. How often is the true path outside the top-20 pilot proposals, and what happens then?
  3. What limits audio AUPRC to ~0.64, and how does the method behave for replay segments that are not ~1 s long or not aligned to 250 ms bins?
Reviewer_dLLgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SyndromeFlow, an audiovisual watermarking method that detects manipulated audio and video while accounting for timing changes. The authors use a parity graph to link information from different audio and video regions, embed this information using QIM, and jointly estimate audio alignment and manipulated regions.

Critical strengths
  • I think the problem is interesting, especially when an attacker copies content that already contains a watermark. Detecting a watermark does not necessarily mean that the content is authentic.

  • Linking information across different audio and video regions is an interesting approach.

Critical weaknesses
  • I am not convinced by the comparison with existing watermarking methods. The paper reports a large improvement in video localization. However, most of the baselines are image watermarking methods that are not designed to detect copied content that already contains a watermark. Also, looking at Table 3, video AUPRC is already around 0.994 before introducing the joint inference. So where exactly is the improvement coming from?

  • My main concern is that the experimental setup is very limited. The authors use six-second videos, timing changes within 80 ms, and alignment paths that match the grid searched by the receiver. More importantly, the audio edits are always 500 ms long, and the receiver is designed to search for edits of exactly this duration. What happens if the audio edit is shorter or longer? What if the timing change does not match the search grid? The attacks are also limited to copying or replaying watermarked content. I would like to see how the method behaves under more realistic manipulations, including generated or modified audio and video.

  • What happens if the content changes but the bits remain the same? How often does this happen?

  • The authors show that audio AUPRC improves from 0.586 to 0.637. This is a useful improvement, but most of it comes from using tamper information to correct the alignment. Averaging over different alignment paths adds only around 0.005. From what I understand, the method reduces the effect of parity checks that might belong to manipulated regions. Could a simpler method that ignores or gives less weight to these checks achieve similar results? I think this comparison is important to show why the proposed joint inference is needed.

Questions for authors
  • Can the authors test different audio-edit durations and timing changes that do not match the receiver's search grid?

  • How often can content change without changing the quantized bits used by the watermark?

  • Can the authors compare against a simpler alignment method using the same watermark information?

  • Can the authors compare against more relevant audiovisual watermarking methods, such as LAVA, V2AMark, and SepVAMark? These seem more appropriate for the proposed task than relying mainly on image watermarking baselines.

2 comments

Thank you for the additional experiments. I appreciate the effort. However, the new results show that most of the video localization performance comes from temporal binding and local decoding, rather than joint inference. Also, the improvement over the simpler soft reweighting method is quite small, so I am still not sure why the additional complexity is needed. Overall, I find the idea interesting, but I think the results do not fully support the claimed contribution. I will maintain my original score.

AuthorsOpenReview ↗

some clarification for your remaining concern

Thank you for the careful follow-up. We would like to clarify three points for your remaining concern.

First, the proposed contribution includes the AV authentication framework itself (as we reported in Sec.1). Its content-binding parity representation, temporal decoding, and localization model jointly support marked-copy detection. Temporal binding and local decoding explain its strong video performance. The soft-reweighting control uses this same framework, so that comparison measures the additional benefit of joint inference (Table 5).

Second, this additional benefit is concentrated in difficult cases. Among the 246 attacks with alignment error above 10 ms under soft reweighting, joint inference reduces mean error from 39.87 to 31.17 ms and brings 49 cases from 13 sources below 10 ms. In the donor example discussed in Section 4.3, correcting a 60.6-ms alignment error changes audio localization IoU from 0 to 1. These cases show how a small overall mean improvement can include substantial corrections to individual authentication failures.

Third, the joint reference accelerated the search process greatly, which is very important in actual application. On 20 received conditions from ten held-out sources, it reduces inference time from 252 to 97 ms at the default budget. Across the tested budgets, it is 2.59–3.19× faster than a matched forward calculation. This makes the joint receiver substantially cheaper to compute while preserving its localization decisions.

We hope this would address your concern.

Reviewer_x7DtOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

SyndromeFlow is a audio-video watermarking framework to detect and localize edited segments by linking distant parts of the clip with parity checks. The key idea is to estimate audio-video timing drift and the edited region together. On 140 video clips dataset (person speaking), it finds edited video regions almost perfectly (AUPRC 0.994 versus 0.086 wrt the best image watermarking baseline) and improves audio localization.

Critical strengths
  1. Well defined and motivated problem: the timing shift and a local edit can look the same and they should be estimated together.
  2. The solution is principled. One probabilistic model explains both timing and edits (instead of relying on heuristics applied one after another)
  3. Reproducible. Frozen settings, split hashes and a planned code release.
Critical weaknesses
  1. The baseline comparison is unfair. The five image watermarks do not bind content to time or position. It is expected that they score near random (0.06–0.09). Frame-index watermarking methods or LAVA (although cited) are not compared
  2. The contribution of the core idea is somewhat small. The joint inference path adds 0.05 audio AUPRC, averaging over timing paths adds 0.005.
  3. Limited dataset and experimentation. Only 140 person speaking clips. Each 224×224 and 6 second duration.
  4. The attack scheme is quite limited. Only copy and replay of brightness channels. No generative edits, cropping, rescaling, frame-rate changes, repeated re-encoding etc.
  5. Audio localization is somewhat weak. An audio AUPRC of 0.64 implies that many edited audio intervals are missed or mislabeled.
  6. The content quality impact is unclear. Video is tuned, but the audio quality is not reported
  7. Most importantly, the paper is AI-written and very hard to read. Dense notation. Key details pushed to the appendix.
Questions for authors

As mentioned in the weaknesses section, there are various gaps, including but not limited to, only one small dataset, unfair baseline comparisons, limited adversarial attack scheme. There is limited or no evidence for the generalizability of the proposed framework for other types of videos. The paper is also very dense and hard to read. The paper does not seem to be ready for publication in this current form. It is not clear whether the weakness may be addressed timely during the short author response period. Would love to hear from authors to reevaluate the rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.