- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
SoundSense studies sound-effect placement in comics and webtoons. The manga study removes artist-written onomatopoeia and uses its original presence as a panel label. The webtoon benchmark uses regions, descriptions, and audio clips curated by an author. VLMs propose sound regions and descriptions, but the main evaluation scores region placement. The paper also examines presentation settings, coordinate formats, and matching rules.
Critical strengths
- I like the application. Comics contain plausible sound moments without visible lettering. Regions, descriptions, and reference clips could be useful for sound design in visual narratives.
- The paper reports coordinate-format failures, matching sensitivity, clustered uncertainty, and content-free baselines. It also discloses AI-assisted annotation and detectable inpainting. These details help readers understand the limits of the results.
Critical weaknesses
- I find it very hard to understand the paper itself. The wording used is a bit weird to me, which makes me feel like I'm reading an AI-generated paper.
- The manga labels include non-auditory mimetics, while absent lettering does not mean silence. I need a semantic audit across works before treating agreement with these labels as evidence of sound understanding.
- There is no independent agreement study, so missed events and alternative valid placements can become model errors. Reviewing selected model predictions with the same annotator does not establish reference completeness.
- The curator reads continuous chapters, while models receive individual segments. Some reference placements may rely on information the model never receives. This needs checking with annotators given the same input as the models.
- No independent listeners check whether the clips and descriptions fit the depicted events. The clips are also unused by the localization scorer, so they do not resolve the model evaluation's semantic gap.
- The main scorer ignores description meaning. A bird chirp could match a region annotated for a door impact. Unlike category-aware audio-visual event localization (https://arxiv.org/abs/2411.11278), this leaves sound identity unchecked.
Questions for authors
See Weaknesses.