- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces AURALLOOP, a language-guided encoder re-entry mechanism for audio question answering. At two intermediate language-model layers, a shared low-rank cell uses the evolving language state to modulate a cached acoustic representation. The method reruns the frozen tail of the audio encoder and injects incremental embedding changes into the original audio-token positions. The Qwen2.5-Omni-7B implementation has about 3.20M trainable parameters. The authors evaluate AURALLOOP on ADQA, VoxParadox, MMSU, and MUGEN. Comparisons include direct audio-and-language adaptation, feature-access methods, single-call re-entry, and a joint LoRA model trained under a matched training-work budget. AURALLOOP achieves the highest mean accuracy among the shared-protocol methods on all four benchmarks. The paper also analyzes task-specific gains, paired answer corrections and regressions, transcription quality, and inference latency.
Critical strengths
-
The comparisons address meaningful alternatives to the proposed mechanism. The paper compares AURALLOOP with direct audio-and-language adaptation, static feature access, one-call re-entry, and a joint LoRA baseline given additional training work. The authors also include interventions that examine the role of updated conditioning. These controls make the evidence more informative than a comparison against language-only adaptation alone.
-
The analysis describes where the method helps and what it costs. The paper reports per-category performance, paired correct-to-wrong and wrong-to-correct changes, transcription results, and time-to-first-token overhead. The gains are strongest for some paralinguistic judgments, while speaker counting and transcription can regress. This accounting helps readers assess when the accuracy–latency tradeoff may be useful.
Critical weaknesses
- The uncertainty of the incremental gains on the evaluation populations is not quantified. The reported gains over the late single-call model are positive on all four benchmarks: 0.57 percentage points on ADQA, 1.10 on VoxParadox, 0.41 on MMSU, and 0.42 on MUGEN. The paper reports three-seed variation and paired correctness counts, but Appendix O explicitly does not provide confidence intervals or significance estimates for resampling the evaluation population. The authors also note that questions may share recordings, overlapping crops, or duplicate sources. Thus, the consistent direction across benchmarks and seeds is encouraging, but seed variation does not establish how stable these margins are across underlying audio sources. This matters particularly for the smaller gains. A paired, source-grouped bootstrap or similar uncertainty analysis would help establish their robustness.
Questions for authors
Q1: Could you report paired uncertainty intervals for AURALLOOP’s main comparisons, resampling at the original recording or source-group level where possible? In particular, how uncertain are the gains over the late single-call and work-controlled joint-LoRA baselines?
3 comments
Thank you for the detailed response. I will keep my positive rating.
Thank you for taking the time to review our response and for your positive assessment. We appreciate your constructive feedback.
Response to Reviewer ar69: Paired Uncertainty Across Audio Sources
Thank you for the positive assessment and for identifying the missing uncertainty analysis. We have added paired source-group bootstrap intervals for all eight comparisons against Late and work-controlled joint LoRA, alongside the three-seed variation.
Weakness and Q1. Paired uncertainty across audio sources
Predictions are paired by question identity and training seed. For question i, we first average the correctness difference across the three matched checkpoint pairs as di=31∑s=13(zs,iTwo−zs,iR), where z denotes binary correctness and R the reference method.
Source components connect questions that share an original recording or a verified duplicate/crop link. For multi-audio questions, the reference and all candidate recordings enter this connectivity. The same mapping is used across methods and seeds.
Each bootstrap replicate samples G complete source components with replacement and retains all their questions. We compute 100 times the sampled sum of di divided by the sampled question count. This denominator preserves question-weighted accuracy when component sizes differ. Each question contributes one seed-averaged paired difference.
We use 10,000 replicates and take the 2.5th and 97.5th percentiles for each pointwise 95% interval. Invalid predictions and runtime failures remain incorrect in the full benchmark denominator.
Results for both requested baselines
C is training-work-controlled joint LoRA. Late is the independently trained single-call model after layer 18, and Two is AuralLoop. N counts questions and G counts source components.
The source CI is obtained by resampling source components while keeping the selected checkpoint pairs fixed. Seed SD measures variation among the three paired training-seed margins. We report both side by side:
Both comparator intervals exclude zero on ADQA, VoxParadox and MMSU. Both MUGEN intervals include zero, so its positive mean margins remain inconclusive at the source level.
Changes in the revised manuscript
New and revised passages are highlighted in dark blue.
The revised manuscript replaces the earlier uncertainty paragraph with the grouping rule, estimator and complete results above.