acceptodds
Under review as a conference paper at ICLR 2027

AuralLoop: Language-Guided Encoder Re-entry for Audio Question Answering

Abstract

One-pass audio-language models complete acoustic encoding before language-side inference, so intermediate language states cannot guide further encoder computation. We introduce AuralLoop, a parameter-efficient architecture that makes acoustic encoding responsive to evolving language context. A shared low-rank cell conditions a cached acoustic anchor on an intermediate language state and reruns the original frozen encoder tail. Incremental embedding differences are injected into existing audio-token positions at later language layers. Two re-entry steps share the anchor and encoder weights, with the second conditioned on a language state that has incorporated the first revision. Instantiated on Qwen2.5-Omni-7B, AuralLoop uses 3.20M trainable parameters while keeping pretrained backbone weights frozen. It requires no additional encoder, generated control tokens, or chain-of-thought supervision during adaptation. Evaluations on ADQA, VoxParadox, MMSU, and MUGEN compare re-entry with direct acoustic adaptation, static feature access, and additional training computation. On VoxParadox, AuralLoop improves mean accuracy by 1.10 percentage points over a single re-entry at a later language layer and by 0.97 points over joint audio-language LoRA under a controlled training-work budget. Same-depth interventions suggest task-dependent benefits from updated language conditioning, while capability analyses show the largest gains in paralinguistic perception. Paired error and latency analyses characterize the accompanying answer regressions and prefill cost. These results support language-guided encoder re-entry as a targeted approach to acoustic question answering.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
audio-language models, audio question answering, language-guided encoder re-entry, parameter-efficient fine-tuning, acoustic perception

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ar69OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AURALLOOP, a language-guided encoder re-entry mechanism for audio question answering. At two intermediate language-model layers, a shared low-rank cell uses the evolving language state to modulate a cached acoustic representation. The method reruns the frozen tail of the audio encoder and injects incremental embedding changes into the original audio-token positions. The Qwen2.5-Omni-7B implementation has about 3.20M trainable parameters. The authors evaluate AURALLOOP on ADQA, VoxParadox, MMSU, and MUGEN. Comparisons include direct audio-and-language adaptation, feature-access methods, single-call re-entry, and a joint LoRA model trained under a matched training-work budget. AURALLOOP achieves the highest mean accuracy among the shared-protocol methods on all four benchmarks. The paper also analyzes task-specific gains, paired answer corrections and regressions, transcription quality, and inference latency.

Critical strengths
  1. The comparisons address meaningful alternatives to the proposed mechanism. The paper compares AURALLOOP with direct audio-and-language adaptation, static feature access, one-call re-entry, and a joint LoRA baseline given additional training work. The authors also include interventions that examine the role of updated conditioning. These controls make the evidence more informative than a comparison against language-only adaptation alone.

  2. The analysis describes where the method helps and what it costs. The paper reports per-category performance, paired correct-to-wrong and wrong-to-correct changes, transcription results, and time-to-first-token overhead. The gains are strongest for some paralinguistic judgments, while speaker counting and transcription can regress. This accounting helps readers assess when the accuracy–latency tradeoff may be useful.

Critical weaknesses
  1. The uncertainty of the incremental gains on the evaluation populations is not quantified. The reported gains over the late single-call model are positive on all four benchmarks: 0.57 percentage points on ADQA, 1.10 on VoxParadox, 0.41 on MMSU, and 0.42 on MUGEN. The paper reports three-seed variation and paired correctness counts, but Appendix O explicitly does not provide confidence intervals or significance estimates for resampling the evaluation population. The authors also note that questions may share recordings, overlapping crops, or duplicate sources. Thus, the consistent direction across benchmarks and seeds is encouraging, but seed variation does not establish how stable these margins are across underlying audio sources. This matters particularly for the smaller gains. A paired, source-grouped bootstrap or similar uncertainty analysis would help establish their robustness.
Questions for authors

Q1: Could you report paired uncertainty intervals for AURALLOOP’s main comparisons, resampling at the original recording or source-group level where possible? In particular, how uncertain are the gains over the late single-call and work-controlled joint-LoRA baselines?

3 comments
AuthorsOpenReview ↗

Response to Reviewer ar69: Paired Uncertainty Across Audio Sources

Thank you for the positive assessment and for identifying the missing uncertainty analysis. We have added paired source-group bootstrap intervals for all eight comparisons against Late and work-controlled joint LoRA, alongside the three-seed variation.

Weakness and Q1. Paired uncertainty across audio sources

Predictions are paired by question identity and training seed. For question ii, we first average the correctness difference across the three matched checkpoint pairs as di=13∑s=13(zs,iTwo−zs,iR)d_i=\frac{1}{3}\sum_{s=1}^{3}(z^{\mathrm{Two}}_{s,i}-z^R_{s,i}), where zz denotes binary correctness and RR the reference method.

Source components connect questions that share an original recording or a verified duplicate/crop link. For multi-audio questions, the reference and all candidate recordings enter this connectivity. The same mapping is used across methods and seeds.

Each bootstrap replicate samples GG complete source components with replacement and retains all their questions. We compute 100 times the sampled sum of did_i divided by the sampled question count. This denominator preserves question-weighted accuracy when component sizes differ. Each question contributes one seed-averaged paired difference.

We use 10,000 replicates and take the 2.5th and 97.5th percentiles for each pointwise 95% interval. Invalid predictions and runtime failures remain incorrect in the full benchmark denominator.

Results for both requested baselines

C is training-work-controlled joint LoRA. Late is the independently trained single-call model after layer 18, and Two is AuralLoop. NN counts questions and GG counts source components.

The source CI is obtained by resampling source components while keeping the selected checkpoint pairs fixed. Seed SD measures variation among the three paired training-seed margins. We report both side by side:

DatasetN / GReferenceTwo−reference (pp)95% CI (pp)Seed SD (pp)
ADQA3000 / 600Late+0.57[0.16, 0.98]0.19
ADQA3000 / 600C+0.94[0.48, 1.42]0.11
VoxParadox2000 / 400Late+1.10[0.54, 1.65]0.10
VoxParadox2000 / 400C+0.97[0.24, 1.71]0.51
MMSU5000 / 1000Late+0.41[0.09, 0.73]0.13
MMSU5000 / 1000C+0.62[0.19, 1.06]0.06
MUGEN1750 / 350Late+0.42[-0.11, 0.96]0.49
MUGEN1750 / 350C+0.29[-0.27, 0.83]0.06

Both comparator intervals exclude zero on ADQA, VoxParadox and MMSU. Both MUGEN intervals include zero, so its positive mean margins remain inconclusive at the source level.

Changes in the revised manuscript

New and revised passages are highlighted in dark blue.

The revised manuscript replaces the earlier uncertainty paragraph with the grouping rule, estimator and complete results above.

  • Appendix O.1 (pp. 43–44) now defines the paired estimator and source-group resampling.
  • Table 28 (p. 44) adds all eight comparisons with source intervals and paired-seed SDs.
  • Section 5.5 states the updated conclusion on p. 10, distinguishing the positive contrasts on ADQA, VoxParadox and MMSU from the inconclusive MUGEN margins.

Thank you for the detailed response. I will keep my positive rating.

Thank you for taking the time to review our response and for your positive assessment. We appreciate your constructive feedback.

Reviewer_auafOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work presents AuralLoop, a parameter efficient way to enable language guided acoustic re-entry. When an audio input is processed through an encoder and projector and passed through a language model, the audio token representation continues to evolve within the language model and the intermediate language states cannot guide further update in the acoustic encoder. AuralLoop uses a caches acoustic anchor and a shared low-rank cell to modulate the acoustic representation using intermediate language representation. Every call modulates the anchor, reruns the frozen encoder tail and injects incremental difference onto the existing audio token predictions. The second call uses a language state incorporating the first revision. The paper presents three contributions.First, it introduces a shared encoder re-entry mechanism that couples intermediate language states with renewed acoustic computation using approximately 3.2M trainable parameters. Both re-entry steps reuse the cached acoustic anchor and frozen encoder tail, while the second step benefits from the language state updated by the first revision. Second, the authors conduct extensive evaluations against direct acoustic adaptation, static feature-access methods, and additional training computation across four audio-question-answering benchmarks. Third, they investigate the resulting performance and computational trade-offs through capability-specific evaluations, paired error analyses, and inference latency measurements.

Critical strengths
  1. Well-motivated and positioned contribution: The paper clearly motivates the limitations of conventional audio-language models and positions the proposed approach relative to existing methods. The related work discusses both audio-based approaches and related visual perception methods that incorporate language-guided re-encoding. Table 5 provides a useful comparison of different approaches addressing similar challenges and clarifies how the proposed method differs from existing paradigms.
  2. Clear architectural visualization: Figure 2 effectively illustrates the proposed encoder re-entry mechanism, including the shared acoustic anchor, frozen encoder tail, and successive language-conditioned revisions. The figure makes it easier to understand the interaction between acoustic and language representations.
  3. Extensive architectural evaluations: The paper presents detailed experiments investigating different architectural decisions within AuralLoop and comparisons against alternative approaches such as PCLM and DeepStack. Particularly useful are the experiments investigating which acoustic encoder layers should constitute the stem and tail, along with the placement and number of re-entry steps. These evaluations provide useful evidence for understanding the design choices behind the proposed architecture.
  4. Consistent improvements across benchmarks: AuralLoop outperforms joint audio-language LoRA under the same two-epoch training exposure across all four evaluated benchmarks: ADQA, VoxParadox, MMSU, and MUGEN. The experiments investigate performance across different acoustic understanding capabilities, while Figure 3 provides an informative visualization of the improvements on VoxParadox, particularly for paralinguistic attributes.
  5. Comprehensive supplementary evaluations: The appendix contains extensive additional experiments and analyses that help explain the advantages and limitations of AuralLoop relative to alternative approaches. These supplementary results provide useful context for understanding the architectural decisions, evaluating the contribution of individual components, and interpreting the reported improvements.
  6. Clarification of architectural edge cases: Appendix C establishes useful properties of the proposed computation graph, including the conditions under which AuralLoop reduces to the standard language-LoRA pipeline. These properties help clarify the relationship between the proposed architecture and conventional adaptation approaches.
Critical weaknesses

Dense methodological presentation: Section 3 introduces several architectural components, mathematical formulations, and notations in a relatively compact manner, making the proposed mechanism difficult to follow initially. Although Appendix B (Table 6) provides a detailed overview of the notation and tensor dimensions, the main methodology section would benefit from a more gradual explanation of how the different components interact. In particular, the relationship between the cached acoustic anchor, shared low-rank cell, incremental embedding revisions, and language-layer conditioning could be explained more intuitively before introducing the mathematical formulations. This would make the overall computation graph easier to understand without repeatedly referring to the appendix.

Questions for authors

Clarity of the computation graph: Section 3 introduces the acoustic anchor, shared low-rank cell, step-specific gates, incremental embedding differences, and language-layer conditioning in a relatively compact formulation. While Figure 2 and Appendix B help explain these components, could the authors provide a more intuitive explanation of how the different operations interact, particularly how the second re-entry incorporates information from the first revision while both calls restart from the same acoustic anchor?

2 comments
AuthorsOpenReview ↗

Response to Reviewer auaf: Computation Graph and Updated Conditioning

Thank you for the positive assessment and the suggestion to make the computation graph more intuitive. Both calls start from the same acoustic anchor, while the continuing language sequence carries the first revision into the second condition. We now introduce this distinction before the equations.

Q1. How does the second call use the first revision?

For Qwen2.5-Omni-7B, the frozen stem comprises the audio frontend and encoder layers 1–24. It runs once and caches Ci0C_i^0. The frozen tail, comprising layers 25–32 and native output processing, initially produces Ei0E_i^0, which is packed with the observed prompt.

  1. Read r1r^1. Language layers 1–8 process the complete sequence. The final prompt-position state supplies the first condition.
  2. Revise the acoustic representation. The shared cell combines Ci0C_i^0 with r1r^1 to produce Ci1C_i^1. The shared frozen tail yields Ei1E_i^1.
  3. Continue the language computation. Inject the scaled, channel-gated difference Ei1−Ei0E_i^1-E_i^0 at existing audio-token positions. Layers 9–18 process this modified sequence and yield r2r^2.
  4. Revisit the original anchor with r2r^2. The second cell takes Ci0C_i^0 and r2r^2 to produce Ci2C_i^2. The shared frozen tail then produces Ei2E_i^2, and the second injection uses Ei2−Ei1E_i^2-E_i^1.
  5. Complete prefill and generate. Layers 19–28 complete prefill. Answer tokens subsequently use the full decoder and layer-wise KV caches, without repeating re-entry for each token.
QuantityFirst re-entrySecond re-entry
Acoustic anchorCi0C_i^0The same Ci0C_i^0
Language conditionr1r^1, after layer 8r2r^2, after layers 9–18 process the first revised sequence
Embedding differenceEi1−Ei0E_i^1-E_i^0Ei2−Ei1E_i^2-E_i^1
RMS referenceDetached, regularized per-token RMS of Ei0E_i^0The same initial-embedding reference
ParametersShared cell and tail, with α1,γ1\alpha_1,\gamma_1Shared cell and tail, with α2,γ2\alpha_2,\gamma_2

Thus, Ci0C_i^0 supplies the fixed acoustic starting point, r2r^2 carries the updated language context, and Ei1E_i^1 supplies the second subtraction reference. These three roles explain how the second call incorporates the first revision while revisiting the original anchor. The intervening nonlinear language layers process the two injections at different depths.

The cell's position-dependent sigmoid gate and scalar αk\alpha_k control the acoustic modification, while the channel vector γk\gamma_k controls embedding injection. Cell weights and affine LayerNorms are shared, with step-specific αk,γk\alpha_k,\gamma_k. Initial Ci0C_i^0 and Ei0E_i^0 are cached without autograd, and the RMS reference is detached. Gradients pass through r1,r2r^1,r^2, the frozen tail's inputs, and the −Ei1-E_i^1 subtraction. Both conditions are available during prefill.

Existing evidence for the conditioning dependency

The submitted same-depth shadow intervention obtains an alternative layer-18 condition by running layers 9–18 without the first injection. The receiving path retains its intact first revision and subtraction reference Ei1E_i^1. The norm-matched variant matches the whole-clip norm of the second injected residual, after reference scaling and channel gating, to that of the intact residual.

The original three-seed means are:

DatasetIntact feedback (%)Norm-matched shadow (%)Difference (pp)
ADQA54.6354.20+0.43
VoxParadox33.2032.53+0.67
MMSU68.7268.48+0.24
MUGEN35.2035.16+0.04

Differences use unrounded means. This intervention probes the effect of updated conditioning at the same language depth and residual norm. The observed sensitivity is strongest on VoxParadox and small on MUGEN, consistent with the task-dependent contribution of the second condition.

Changes in the revised manuscript

New and revised passages are highlighted in dark blue.

  • Revised presentation: Sections 3.1–3.2, pp. 3–4, explain the forward pass and distinguish the fixed anchor from the updated condition. Figure 2, p. 4, highlights this dependency and labels both subtraction references.
  • Implementation clarification: Appendix E, pp. 18–19, clarifies the fixed anchor, updated condition and gradient paths. Figure 5 on p. 19 is retained.
  • Existing supporting analysis: Section 5.3 and Table 3, p. 8, Appendix N.3, pp. 41–42, and the absolute scores in Table 25, p. 40. These results were already reported in the submission.
AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer auaf,

Thank you again for your time and thoughtful feedback. As the discussion period approaches its end, we would be grateful if you could take a look at our response when you have a chance.

We have clarified how the first injection influences the language condition for the second call while both calls use the same cached anchor. The revised explanation appears in Section 3, Figure 2, and Appendix E. We would appreciate hearing whether this clarification addresses your concern and would be happy to explain further if needed.

Thank you for your time and consideration.

Best regards,
The Authors

Reviewer_Ww3uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AuralLoop, which uses intermediate language states to guide additional audio-encoder computation. It caches audio features after encoder layer 24, modifies them using the prompt-end hidden state, and reruns the frozen encoder tail. Changes in audio embeddings are added to audio-token positions after language layers 8 and 18. The method trains 3.20M parameters, including language LoRA, using answer-only supervision. Experiments on four audio-QA benchmarks show improvements mainly in acoustic perception.

Critical strengths
  1. Language-guided audio re-encoding is a useful architectural direction. The method reuses the original encoder and requires relatively few trainable parameters.
  2. The experiments include several useful controls, including joint audio-language LoRA, additional training work, single-call variants, and interventions on the second conditioning state. Three-seed results and latency measurements strengthen the evaluation.
Critical weaknesses
  1. The added architectural complexity brings only small gains. Compared with Joint LoRA (C), accuracy improves by just 0.29-0.97 percentage points, while time to first token increases by about 19-23%. Such modest gains seem to offer limited value given the many complicated changes to the model.
  2. The evaluation leaves broader audio reasoning insufficiently tested. Commonly used and standard audio reasoning benchmarks like MMAU and MMAR are not included.
  3. Generality is not established. Performance evidence is limited to Qwen2.5-Omni-7B and multiple-choice evaluation. Results on another backbone and open-ended audio QA would help show whether the approach transfers beyond the tested setting.
  4. The presentation is very difficult to follow. For example, Eq. (1) introduces the stem and tail without immediately explaining what they denote. Section 3 uses many symbols before giving a clear overview of the forward pass. The baselines are also only briefly described in the main text, with important details left to the appendix. Figure 2 is too dense to help clarify the method.
Questions for authors

See weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Ww3u (1/3): Accuracy and Computational Cost

Response to Reviewer Ww3u (1/3)

W1. Accuracy gains and computational cost

Thank you for raising the accuracy–cost tradeoff. Against training-work-controlled joint audio-language LoRA (C), AuralLoop improves mean accuracy by 0.29–0.97 percentage points at 19.3–23.4% higher mean time to first token (TTFT). The revised analysis explains these aggregate margins through complete capability comparisons with both strong controls.

Two denotes AuralLoop, and Late is the independently trained layer-18 single-call model. We report both comparisons to distinguish the cost of re-entry from the incremental cost of a second call:

DatasetTwo−C (pp)TTFT vs CTwo−Late (pp)TTFT vs Late
ADQA+0.94+20.4%+0.57+8.9%
VoxParadox+0.97+19.3%+1.10+8.4%
MMSU+0.62+19.9%+0.41+8.7%
MUGEN+0.29+23.4%+0.42+10.0%

Where the gains arise

We expanded the original MMSU six-category and VoxParadox ten-attribute breakdowns to include C and Late, retaining every category. Accuracies are percentages. Differences and pointwise 95% source-group intervals are in pp. N counts questions, means average three seeds, and differences use unrounded values.

MMSU (all six categories)

CapabilityNCLateTwoTwo−C [95% CI]Two−Late [95% CI]
Semantic perception63562.3660.7360.47-1.89 [-3.23, -0.53]-0.26 [-1.21, 0.68]
Phonological perception93550.3750.2751.16+0.78 [-0.32, 1.92]+0.89 [0.07, 1.75]
Paralinguistic perception101054.9257.5659.74+4.82 [3.70, 5.97]+2.18 [1.41, 2.97]
Semantic reasoning110889.8089.2088.33-1.47 [-2.15, -0.81]-0.87 [-1.40, -0.35]
Phonological reasoning97782.8482.9482.40-0.44 [-1.24, 0.31]-0.55 [-1.09, 0.00]
Paralinguistic reasoning33553.4353.7355.72+2.29 [0.49, 4.12]+1.99 [0.69, 3.36]

VoxParadox (all ten attributes)

CapabilityNCLateTwoTwo−C [95% CI]Two−Late [95% CI]
Age20030.3334.0034.50+4.17 [1.58, 6.77]+0.50 [-1.51, 2.56]
Gender20043.8343.6744.33+0.50 [-1.50, 2.48]+0.67 [-0.67, 1.96]
Emotion20041.6742.6741.67+0.00 [-2.50, 2.64]-1.00 [-2.83, 0.83]
Pitch20023.8324.1723.83+0.00 [-2.25, 2.15]-0.33 [-1.82, 1.15]
Volume20022.6720.3321.00-1.67 [-3.65, 0.17]+0.67 [-0.67, 1.94]
Speed20023.6721.8321.83-1.83 [-3.96, 0.33]+0.00 [-1.76, 1.81]
Range20021.6720.5022.17+0.50 [-1.65, 2.64]+1.67 [0.16, 3.37]
Intonation20043.5045.1748.17+4.67 [2.03, 7.33]+3.00 [1.17, 4.91]
Speaker identity20030.1733.6739.50+9.33 [6.49, 12.40]+5.83 [3.54, 8.33]
Speaker count20041.0035.0035.00-6.00 [-8.37, -3.69]+0.00 [-1.67, 1.82]

On MMSU, paralinguistic perception improves by 4.82 pp over C and 2.18 pp over Late, while semantic perception and semantic reasoning have lower mean scores. On VoxParadox, speaker identity improves by 9.33/5.83 pp and intonation by 4.67/3.00 pp against C/Late. Speaker counting falls by 6.00 pp against C and has the same observed three-seed mean as Late.

Weighting the unrounded category results by N recovers C/Late/Two overall accuracies of 68.10/68.31/68.72 on MMSU and 32.23/32.10/33.20 on VoxParadox. MMSU paralinguistic perception contributes +0.9733 pp against C, offset by −0.3533 pp elsewhere, yielding +0.62 pp overall. The small benchmark-wide margin therefore combines larger gains on selected acoustic judgments with regressions on others.

These results identify the acoustic judgments that benefit from re-entry and make the cost tradeoff explicit. For workloads that prioritize paralinguistic perception, speaker identity or intonation, the gains may justify additional prefill time. Late remains a lower-cost option. Individual cross-zero category intervals remain inconclusive.

Changes in the revised manuscript

  • Section 5.4 (pp. 8–9) summarizes the stronger-control capability findings.
  • Section 5.5 (pp. 9–10) makes both cost comparisons explicit on p. 10. Appendix P, Table 30 (p. 45) retains the original cost values.
  • Appendix S.1 (pp. 50–51) adds the complete C/Late capability comparisons in Tables 36–37.
  • Appendix O.1 (pp. 43–44) defines the source estimator and reports all eight main C/Late comparisons in Table 28 (p. 44).
AuthorsOpenReview ↗

Response to Reviewer Ww3u (2/3): Reasoning, Transfer, and Open-Ended QA

Response to Reviewer Ww3u (2/3)

Thank you for these suggestions. We address W2 and both parts of W3 with MMAU/MMAR evaluation, a second audio-language system, and free-form audio question answering.

W2. Evaluation on MMAU and MMAR

We evaluate the 1,000-question MMAU-v05.15.25 test-mini split and the original 1,000-question MMAR task. For Qwen, C denotes work-controlled joint LoRA, Late denotes late single-call re-entry, and Two denotes AuralLoop. We reuse the originally selected Qwen checkpoints from three matched training seeds without benchmark-specific tuning. All methods receive the same audio, option order and label-only prompt, with greedy decoding and a 16-token cap. Invalid outputs and runtime failures count as incorrect.

All MCQ results below report three-seed mean accuracy in percent and paired differences in percentage points. Differences use unrounded means. The pointwise 95% intervals use 10,000 source-component bootstrap replicates of seed-averaged paired differences.

EvaluationNCLateTwoTwo−C [95% CI]Two−Late [95% CI]
MMAU-mini100059.2759.9761.17+1.90 [1.22, 2.64]+1.20 [0.50, 1.92]
MMAR100043.1043.7745.37+2.27 [1.47, 3.07]+1.60 [0.88, 2.35]

AuralLoop improves over both controls on both benchmarks, with positive source intervals for all four contrasts. These intervals quantify source uncertainty for the selected checkpoints. Across training seeds, the MMAU Two−C margins are −0.60, +4.20 and +2.10 pp (sample SD 2.41 pp).

W3a. Transfer to Audio Flamingo 3

We evaluate C, Late and Two within Audio Flamingo 3 using a common adaptation split, answer-only objective and development-selection protocol. Here, C is the joint-adapter control with an extended training allowance. Re-entry uses an anchor after audio layer 24, language boundaries 8/18 and a width-256 cell. Each selected method–seed checkpoint is evaluated on both tasks. Appendix S.3 gives the training horizons, learning-rate candidates and validation schedule.

EvaluationNCLateTwoTwo−C [95% CI]Two−Late [95% CI]
AF3/VoxParadox200034.8735.6836.03+1.17 [0.69, 1.65]+0.35 [-0.18, 0.87]
AF3/MMAU-mini100059.0059.8761.17+2.17 [1.34, 3.01]+1.30 [0.57, 2.07]

The gains over C extend to both AF3 tasks. Against Late, the MMAU interval is positive and the VoxParadox interval includes zero. AF3 changes the audio encoder while retaining the Qwen2.5 language lineage, so the results support transfer between these two audio-encoding systems. The measured training-work and TTFT comparisons remain specific to Qwen.

W3b. Open-ended audio question answering

We evaluate the 625-question open subset of MMAU-Pro. The original selected Qwen checkpoints receive audio and the question without choices or reference answers. Answers are generated greedily with a common free-form prompt and a 256-token cap.

Qwen2.5-7B-Instruct scores the answers using the official rubric, with method identities hidden. Judge generation uses temperature 0.1 and a 512-token cap. Appendix S.4 specifies the retry and strict-parsing protocol. Paired score differences are averaged over matched seeds within each question before source resampling.

After the permitted retries, every selected method–seed checkpoint had a valid overall score for all 625 questions. All means and both paired intervals use this same question set and all three seeds.

EvaluationNCLateTwoTwo−C [95% CI]Two−Late [95% CI]
MMAU-Pro open6253.2383.2883.291+0.052 [0.034, 0.071]+0.003 [-0.014, 0.020]

The mean overall rubric score, on the raw 1–5 scale, shows a small gain over C. The comparison with Late remains inconclusive because its interval includes zero. These intervals quantify source-level uncertainty conditional on the recorded judge scores.

Changes in the revised manuscript

  • Section 5.6 and Table 5 (p. 10) summarize the added evaluations.
  • Appendix S.2 (p. 52, Table 38) gives the MMAU/MMAR protocol and results.
  • Appendix S.3 (pp. 52–53, Table 39 on p. 53) describes the AF3 configuration and within-system comparisons.
  • Appendix S.4 (p. 53, Table 40) specifies open-answer generation, judging and the scope of the conclusion.
AuthorsOpenReview ↗

Response to Reviewer Ww3u (3/3): Method Clarity and Presentation

Response to Reviewer Ww3u (3/3)

Thank you for the specific presentation suggestions. We have reorganized the method around its forward pass and expanded the main-text baseline descriptions. Below we address each of the four points. All layer indices refer to Qwen2.5-Omni-7B.

W4.1. Stem and tail definitions before Eq. (1)

The stem comprises the audio frontend and encoder layers 1–24. It runs once per clip and caches the acoustic anchor Ci0C_i^0. The tail comprises layers 25–32 and native output processing. Section 3.1 defines this division before Eq. (1), then explains reconstruction, pooling, post-normalization and projection. The same frozen tail produces the initial Ei0E_i^0 and is reused at both re-entries.

W4.2. Forward pass before the equations

Section 3.1 now introduces the execution order in words:

StageComputation and information flow
Initial encodingCache Ci0C_i^0, obtain Ei0E_i^0 from the tail, and pack the audio embeddings with the question and options.
First re-entryLanguage layers 1–8 yield the prompt-end condition r1r^1. The shared cell modifies Ci0C_i^0 using r1r^1, and the frozen tail produces Ei1E_i^1.
First injectionAdd the scaled, channel-gated difference Ei1−Ei0E_i^1-E_i^0 at existing audio-token positions. Layers 9–18 process the complete revised sequence.
Second re-entryThe resulting condition r2r^2 drives another call from Ci0C_i^0. Its tail output Ei2E_i^2 supplies the second difference Ei2−Ei1E_i^2-E_i^1.
Answer generationLayers 19–28 complete prefill. Subsequent answer tokens traverse the full decoder with its layer-wise KV caches.

Section 3.2 makes the key dependency explicit: the anchor stays fixed while the continuing language sequence carries the first revision into r2r^2. Separately, Ei1E_i^1 is the second subtraction reference. Sections 3.2–3.3 distinguish the cell's position-dependent gate and step-specific scale from the injection's channel gate. Both injections use the detached, regularized per-token RMS of initial Ei0E_i^0. Both conditions come from the final prompt position during prefill.

W4.3. Main-text baseline descriptions

Section 4 explains the feature-access and inference-time mechanisms alongside the adaptation controls:

ControlMechanism and purpose
Language-only LoRARank-8 language q/o adapters with the native frozen audio path.
Joint audio-language LoRARank-20 acoustic-tail q/k/v/o and rank-4 language q/o adapters for direct acoustic adaptation.
Joint LoRA (C)Additional updates on the same data match synchronized training work. Inference remains single-pass.
PCLM portQuestion/choice-conditioned mixing of frozen features from audio layers 5/15/25/30/32 through trained interfaces.
DeepStack portFixed features from audio layers 8/16/24/32 are injected after language layers 2/4/6/8 without an intermediate language condition.
AADContrasts real-audio and same-length zero-audio logits during decoding, using the selected language-LoRA checkpoint.
IAANUses a noise reference to select encoder-neuron activations for amplification, using the selected language-LoRA checkpoint.
Early/late single-callIndependently trained one-call models at language layer 8/18, retaining the full language stack.

AAD and IAAN use the selected language-LoRA checkpoint without further training. PCLM and DeepStack are mechanism ports under the shared protocol. Appendix I gives implementation, budget and selection details. External systems and their training histories are reported separately in Appendix Q.

W4.4. A clearer Figure 2

Panel (a) shows initial encoding and anchor caching. Panel (b) shows the continuing language sequence and two re-entry branches. The teal path follows the first injection through layers 9–18 to r2r^2. The labels “Subtract Ei0E_i^0” and “Subtract Ei1E_i^1” identify the two subtraction references, and both cells visibly receive the same anchor.

The caption identifies the shared frozen tail and RMS reference, then lists the trainable components. Figure 5 retains the detailed cell and gradient operations. The revised overview and detailed diagram therefore serve complementary purposes.

Changes in the revised manuscript

New and revised passages are highlighted in dark blue.

  • Method overview and figure: Sections 3.1–3.2, pp. 3–4, and Figure 2, p. 4.
  • Main-text baseline descriptions: Section 4, pp. 5–6.
  • Supporting method details: Appendix B, Table 7, p. 15, Appendix E, pp. 18–19, with Figure 5 on p. 19, and Appendix I, Table 14, p. 26.
AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer Ww3u,

Thank you again for your time and constructive feedback. As the discussion period approaches its end, we would be grateful if you could take a look at our responses when you have a chance.

Following your suggestions, we have expanded the accuracy–cost and capability analyses, added the reasoning, AF3, and open-ended QA evaluations, and clarified the re-entry mechanism in the revised manuscript. We would appreciate hearing whether these additions address your main concerns and whether any points would benefit from further clarification.

Thank you for your time and consideration.

Best regards,
The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of our responses and manuscript revisions

Dear Area Chair and Reviewers,

Thank you for your thoughtful feedback. We have addressed the reviewers' comments point by point and incorporated the corresponding clarifications and additional analyses into the revised manuscript. Below, we highlight the main contribution and findings, followed by the revisions for each reviewer.

Core contribution and findings

AuralLoop introduces language-guided encoder re-entry, in which intermediate language states condition a new acoustic read from the same cached anchor. The first injection shapes the condition for the second call as language processing proceeds. Our complete comparisons with the joint-adapter control C and the late single-call control Late show where this design helps. On MMSU, paralinguistic perception improves by 4.82 pp over C and 2.18 pp over Late. On VoxParadox, speaker identity and intonation improve against both controls. Other categories show regressions, and the latency analysis quantifies the additional cost. These findings identify acoustic judgments that benefit from re-entry and clarify the accuracy–cost tradeoff.

Reviewer Ww3u

  • W1. Accuracy and computational cost. Sections 5.4–5.5 and Appendix S.1 provide complete MMSU and VoxParadox capability breakdowns against the joint-adapter control C and the late single-call control Late. We report both gains and regressions alongside the latency comparisons against both controls.
  • W2. Reasoning evaluation. Section 5.6 and Appendix S.2 add the MMAU and MMAR evaluations.
  • W3. Evaluation scope. Appendix S.3 adds an AF3 evaluation using a joint-adapter control with an extended training allowance. Appendix S.4 adds open-ended QA, showing a small gain in the mean overall rubric score over C, while the comparison with Late remains inconclusive.
  • W4. Presentation and clarity. Sections 3–4, Figure 2, and Appendices E and I clarify the re-entry computation and baseline definitions.

Reviewer auaf

Section 3, Figure 2, and Appendix E clarify how the two calls share the same cached anchor while the language condition changes between calls. The first injection influences the language states that produce the second condition through the intervening language layers. The first re-entry embedding serves as the reference for the second embedding difference.

Reviewer ar69

Appendix O.1 reports source-group paired confidence intervals for all eight main comparisons with C and Late, with variation across training seeds reported separately. The interpretation distinguishes gains supported by these intervals from comparisons that remain inconclusive.

We would be grateful if you could take these clarifications and additional results into account in your final assessment. Thank you again to all three reviewers and the Area Chair for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.