- Rating
- 3
- Confidence
- 4(+1)
- Informativeness
- 3(+1)
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents STELA for detecting and localizing partially manipulated speech. It combines segment-oriented acoustic augmentation and boundary-free interior sampling with Paired State Modeling (PSM), an utterance-specific prototype objective that encourages consistent representations for frames with the same authenticity label. Structured Manipulation Localization (SML) then expands thresholded frame predictions into alternative interval hypotheses and estimates their quality using interior, boundary, and contextual evidence. Experiments evaluate frame detection and temporal localization on PartialSpoof, HAD, PartialEdit, and LAV-DF, with additional cross-domain frame-level evaluation including ADD2023.
Critical strengths
-
The method delivers meaningful improvements in precise temporal localization, which supports the practical value of evaluating complete interval hypotheses rather than relying entirely on thresholded frame predictions.
-
The ablations provide useful evidence beyond the full-system comparison. The appendix further distinguishes candidate coverage from ranked-output recall and documents failure cases, making the contribution and its limitations easier to assess.
Critical weaknesses
-
Cross-domain generalization is demonstrated for frame classification, but not for the complete localization pipeline. Table 2 evaluates frame accuracy and F1, whereas the interval-level results in Table 3 use in-domain training and evaluation. The learned SML scorer additionally consumes duration, component count, and posterior statistics, whose relationship to proposal quality may change across manipulation methods and datasets. Consequently, improved frame-level transfer does not establish that the final interval ranking remains reliable under the same shift. Freezing both stages after source-domain training and reporting target-domain localization would directly test an important part of the claimed robustness.
-
The evidence for recovering difficult fragmentation is more qualified than the overall localization gains suggest. In Table 5, removing proposal merging reduces PartialSpoof mAP only from 83.473% to 83.170%, while boundary adjustment and learned quality estimation account for substantially larger changes. The appendix appropriately explains that difficult fragmentation is uncommon and reports a meaningful Top-5 recall gain on Long-split events. However, for Endpoint-only events, Full SML achieves 81.19% recall when all output proposals are considered but only 0.85% among the Top-5 (Table 9). Thus, generating a correctly aligned interval often does not translate into selecting it. This limits the demonstrated effectiveness on a central motivating failure pattern and warrants further analysis of the scorer's behavior under sparse interior evidence.
Questions for authors
-
Can the authors report cross-domain localization AP/mAP with both the frame encoder and SML frozen, using the existing source/target setup and selecting all settings on source-domain development data?
-
What causes the large all-output versus Top-5 recall gap on Endpoint-only events? Does recall improve at moderate output budgets, and are accurately aligned merged candidates systematically penalized for weak interior responses?
4 comments
R2: Proposal Ranking under Sparse Interior Evidence (W2, Q2)
Thank you for highlighting this important limitation. We agree that generating accurately aligned proposals does not necessarily ensure that they receive sufficiently high quality scores. To further investigate this issue, we conducted additional ranking and scoring analyses on the difficult fragmentation subsets defined in Appendix A.3, using the original SML scorers.
(1) Recall under different proposal budgets. As shown in Table 1, recall increases substantially as the output budget expands. For Endpoint-only events, [email protected] increases from 0.85% at Top-5 to 40.96% at Top-20 and 75.61% at Top-50, approaching the all-output recall of 81.19%. Among correctly localized Endpoint-only events, the median rank of the first accurate proposal is 20. These results indicate that accurate intervals are often successfully generated but ranked relatively low.
Table 1. [email protected] under different output budgets (%).
| Subset | Top-5 | Top-10 | Top-20 | Top-50 | All |
|---|---|---|---|---|---|
| Long-split | 19.41 | 28.52 | 43.67 | 57.06 | 58.96 |
| Endpoint-only | 0.85 | 4.25 | 40.96 | 75.61 | 81.19 |
| Interior-pulse | 4.07 | 7.68 | 17.43 | 33.95 | 36.41 |
(2) Analysis of proposal quality estimation. We further compared accurately aligned Merge proposals (IoU > 0.95) with higher-ranked, poorly localized proposals from the same utterances. For Endpoint-only events, accurate Merge proposals receive an average score of only 0.070, compared with 0.774 for higher-ranked Top-5 incorrect proposals.
To reduce the influence of proposal duration, we additionally matched candidates within the same utterance with similar lengths (ratio within [0.5, 2]). As shown in Table 2, accurately aligned Merge proposals still receive lower scores despite having substantially higher true IoU.
Table 2. Length-matched proposal comparison on Endpoint-only events.
| Proposal | True IoU | Mean posterior | SML score | Duration (s) |
|---|---|---|---|---|
| Accurate Merge | 0.997 | 0.345 | 0.052 | 0.927 |
| Higher-ranked incorrect | 0.171 | 0.382 | 0.148 | 0.900 |
Furthermore, the mean interior posterior and predicted quality score exhibit strong positive Spearman correlations among accurately aligned Merge proposals across all three subsets: 0.921 for Long-split, 0.710 for Endpoint-only, and 0.875 for Interior-pulse, averaged over two scorer seeds. For Long-split and Endpoint-only, this positive association persists in within-utterance analyses accounting for proposal duration, component count, and true IoU. Together with the length-matched comparisons, these findings suggest that the quality estimator may rely heavily on interior spoof responses, potentially underestimating well-aligned proposals when interior evidence is sparse.
Overall, these findings explain the substantial gap between All-output and Top-5 recall on Endpoint-only events: accurately aligned proposals are often generated but ranked lower, with their underestimated quality scores closely associated with sparse interior evidence. The substantial recall recovery at moderate proposal budgets further supports this interpretation. We will clarify this distinction between proposal generation and ranking effectiveness in the revised manuscript.
Dear Reviewer MFdt,
We thank you for the valuable feedback, which has encouraged us to further examine the cross-domain behavior of our localization framework and the limitations of proposal quality estimation. We have updated the manuscript accordingly, with all changes highlighted in blue for ease of reference.
Specifically, we added a new subsection, “Cross-Domain Temporal Localization,” to the appendix and referenced it in Section 4.2. This subsection reports AP, mAP, and AR on PartialSpoof, HAD, PE-E2, and ADD2023, with the frame encoder and SML trained on PE-E1 and kept frozen during target-domain evaluation. We also extended the existing “Diagnostic Analysis of Proposal Merging” subsection with additional proposal-budget recall results, a duration-matched comparison of proposal scores, and an analysis of the relationship between interior spoof responses and predicted quality. These analyses provide further insight into why accurately aligned proposals for Endpoint-only events may receive relatively low quality scores and consequently fall outside the top five ranked predictions.
We hope these additional experiments and analyses provide a clearer understanding of our method's effectiveness and limitations. We appreciate your time and consideration and look forward to your further feedback.
Thank you!
Thanks for the rebuttal, which addressed my major concerns. Therefore, I will maintain my inital rating toward weak acceptance.
R1: Cross-Domain Generalization of the Complete Localization Pipeline(W1, Q1)
We thank the reviewer for identifying this important gap in our evaluation. To directly assess cross-domain localization, we conduct additional experiments using PE-E1 as the source domain and PS, HAD, PE-E2, and ADD2023 as unseen target domains. Both the frame-level model and SML are kept frozen after source-domain training, with no target-domain adaptation.
Table 1. Cross-domain temporal localization mAP (%) with models trained on PE-E1.
*CC-Unit converts contiguous spoof-positive frames into proposals with unit confidence. *
Table 2. Cross-domain localization performance of the complete STELA + SML pipeline (%).
The results show that SML improves cross-domain localization mAP over direct frame-based localization on all four target datasets, with particularly substantial gains on HAD (15.70% → 47.91%) and PE-E2 (17.24% → 40.48%). The complete pipeline also achieves the highest mAP among the compared methods on three of the four target domains, indicating that the source-trained proposal generation and quality estimation mechanisms retain useful predictive capability under domain shift.
Nevertheless, cross-domain localization remains challenging. Absolute mAP is still low on PS and ADD2023, and CFPRF-PRN outperforms our method on PS. Moreover, the decline in AP at stricter IoU thresholds indicates that precise boundary alignment remains sensitive to domain changes. Thus, while the new experiments support the transferability of the complete SML pipeline, they do not imply that its proposal-quality relationships are fully domain-invariant. We will include these results and clarify this remaining limitation in the revised manuscript.