acceptodds
Under review as a conference paper at ICLR 2027

Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models

Abstract

Long-context reasoning remains a fundamental challenge for large language models, as excessively long inputs often lead to the forgetting of salient information. This issue is even more pronounced in the speech domain, where audio, as a low-compression modality, requires significantly more embeddings than text to preserve both semantic content and acoustic cues. To address this, we introduce Vox-Infinity, the first benchmark specifically designed to evaluate long-context understanding in spoken language models. Vox-Infinity systematically extends audio history along two dimensions: turns and duration. It covers a diverse range of representative scenarios, including dialogues with varying structural depth and semantic complexity. Crucially, it provides explicit answer provenance annotations and organizes samples based on the context length required to resolve each query, enabling precise and length-aware evaluation of model performance. Furthermore, we present the first comprehensive study of history modeling strategies in this setting, analyzing how models balance the trade-off between preserving long-range semantics and retaining recent acoustic signals. Cases and datasets are available at https://vox-infinity.github.io.

Primary area:
datasets and benchmarks
Keywords:
Long Audio Bench

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BLvdOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Vox-Infinity to evaluate how spoken language models retain and use information in extended histories. It contains three subsets: Ultra Multi-Turn Dialogues based on SpokenWOZ, Personal Monologues reconstructed from GigaSpeech, and Beyond-Semantic Dialogues based on IEMOCAP and emotion-controlled TTS. These target fine-grained information retrieval, long-form speech understanding, and memory for emotional cues, respectively.

Critical strengths
  1. The paper brings multi-turn precise retrieval, long single-speaker speech, and emotional memory into one evaluation framework, covering needs beyond short-clip audio QA.

  2. Beyond-Semantic contains 2,758 natural samples and 2,622 semantically controlled samples. Synthesizing different emotions from neutral text is a meaningful design for testing whether acoustic cues provide additional information.

Critical weaknesses
  1. The paper repeatedly claims to be the first benchmark for long-context understanding in spoken language models and treats the absence of answer-provenance annotations in prior work as a central distinction. However, marking ContextDialog as “x” under Prov. in Table 1 requires correction or explanation. Its §3.1 explicitly requests a supporting utterance for each QA pair and validates the evidence; its appendix describes this as metadata for locating information in dialogue history.

  2. BLAB [1] also already studies emotion and localization in long audio. The more defensible additions here are therefore the combination of three task types, stratification by evidence distance, and the controlled emotional-memory design, rather than a broad claim of being the first long-audio understanding evaluation.

  3. Different bins contain different questions and sessions. There is no paired control that keeps the question and evidence content fixed while varying only evidence position or distractor-history length, and no oracle-audio condition supplying only the supporting segments.

[1] Ahia, Orevaoghene, et al. "Blab: Brutally long audio bench." arXiv preprint arXiv:2505.03054 (2025).

Questions for authors
  1. Why is ContextDialog marked “x” under Prov. in Table 1? How do the annotations here differ from its supporting utterance in timestamps, multiple evidence spans, and evidence necessity?

  2. Could the authors report no-history, supporting-segments-only oracle audio, and full-history conditions? For a fixed set of QA pairs, does the trend persist when evidence content is held constant and only position or distractor duration changes?

  3. Which models had max_position_embeddings modified, and what were their native limits and positional settings? How many audio or text tokens did each model actually receive, and how were chunking, speakers, and turn boundaries handled? Please provide checkpoint/API versions, evaluation dates, prompts, and generation settings, and verify the trend within natively supported ranges.

4 comments
AuthorsOpenReview ↗

Response to Reviewer BLvd (1/2): Contributions and evaluation setup

Thank you for your constructive feedback. Part 1 addresses W1/Q1, W2, and Q3; Part 2 reports W3/Q2 controls.

W1/Q1: ContextDialog and supporting-evidence annotations.

Thank you for pointing out ContextDialog's supporting-utterance annotations. We will update its annotation entry in Table 1.

Vox-Infinity extends the evaluation scope and capability coverage. As summarized in Table 1, ContextDialog averages 68.9 seconds and 10.60 turns per dialogue, whereas our Ultra Multi-Turn Dialogues (UMT) and Beyond-Semantic Dialogues (BSD) average 35.89 and 55.23 turns, respectively. Personal Monologues (PM) additionally covers long continuous speech. Together, these settings assess precise retrieval across many turns, long-form understanding and reasoning, and memory for emotional cues.

Our annotations link answers to utterances or segments with temporal boundaries and can associate multiple supporting segments with a question. QA pairs and their evidence are manually checked for consistency. We use the interval from the earliest supporting evidence to the query to organize evaluation, allowing questions from the same history to have different evidence distances. These annotations support fine-grained analysis within the broader benchmark design.

W2: Relation to BLAB and contribution.

BLAB studies emotion and localization in long audio. Vox-Infinity establishes a systematic evaluation framework for long dialogues and long-form speech while examining how history representations affect model capabilities. Its design has three connected aspects:

  1. Complementary long-history capabilities. Ultra multi-turn dialogues, personal monologues, and emotional dialogues assess fine-grained retrieval, cross-segment understanding and reasoning, and paralinguistic emotional memory, covering both frequent turn exchanges and long continuous speech.

  2. Systematic comparison of text and audio histories. We compare the two representations on the same tasks, examining their differences in preserving precise semantic information and acoustic emotional cues, and how these differences vary with evidence distance. This provides empirical guidance for history representation in spoken dialogue systems.

  3. Semantically controlled acoustic emotional memory. Alongside natural emotional dialogues, we synthesize different emotional expressions from neutral text to reduce direct lexical cues to the answer. Combined with text/audio history comparisons, this more specifically tests whether models retain and use acoustic emotional cues from past utterances.

Supporting-evidence annotations and distance-based grouping enable fine-grained analysis of these capabilities. We will emphasize this overall positioning in the abstract and related work and clarify the relationship to prior benchmarks.

Q3: Model versions, positional limits, and input organization.

  1. Open-source checkpoints. The evaluated weights were downloaded from these Hugging Face repositories:
ModelRepository
Qwen3-OmniQwen/Qwen3-Omni-30B-A3B-Instruct
GLM-4-VoiceTHUDM/glm-4-voice-9b
MiMo-AudioXiaomiMiMo/MiMo-Audio-7B-Instruct
Audio Flamingo 3nvidia/audio-flamingo-3
Audio Flamingo Nextnvidia/audio-flamingo-next-hf
  1. Positional configurations.
ModelOriginal limitUMT / BSDPM
Qwen3-Omni65,53665,53665,536
GLM-4-Voice8,19232,76865,536
MiMo-Audio8,19232,76865,536
Audio Flamingo 332,76832,76865,536
Audio Flamingo Next131,072131,072131,072

Limits below 32,768 were raised to 32,768 for UMT/BSD. PM contains longer continuous speech histories, producing longer encoded input sequences, so limits below 65,536 were raised to 65,536 for PM. Higher original limits were retained. Thus, GLM and MiMo were extended for all three subsets, and Audio Flamingo 3 only for PM. We will add this per-model, per-subset table to Section 4.1.1.

  1. Utterances, roles, and turn boundaries. The dataset explicitly annotates turn boundaries. We input history as successive messages following these boundaries, preserving chronological order and dialogue roles. Each message contains its turn's audio in the audio-history condition or its transcript in the text-history condition. Turn segmentation and role assignments are supplied by the data, rather than inferred by the model.

  2. Questions and prompts. After the history, a final human/user message contains the original question, without additional task instructions, reasoning guidance, or answer hints. Messages follow each model's corresponding chat template.

AuthorsOpenReview ↗

Response to Reviewer BLvd (2/2): Supporting-evidence controls

W3/Q2: No-history, supporting-turn-only, and full-history controls.

We randomly sampled 100 QA examples from each of four subsets: Ultra Multi-Turn Dialogues (UMT), Personal Monologues (PM), natural emotion, and semantically controlled synthetic emotion. We compared three conditions on the same QA examples and reference answers:

  1. No history: the current question without historical audio.
  2. Supporting turns only: only the annotated utterance audio needed to answer the question, without transcripts or reference answers.
  3. Full history: the complete audio history associated with the question.

Semantic tasks.

SubsetModelNo historySupporting turns onlyFull history
PMQwen3-Omni12%84%72%
PMAudio Flamingo Next8%82%38%
PMGLM-4-Voice6%76%24%
UMTQwen3-Omni2%86%78%
UMTAudio Flamingo Next0%78%40%
UMTGLM-4-Voice0%72%38%

In all six comparisons, supplying the supporting turns substantially improves accuracy over no history. Relative to supporting turns only, full-history accuracy falls by 12, 44, and 52 percentage points on PM and by 8, 38, and 34 points on UMT.

Because the questions and reference answers are held fixed, these differences cannot be attributed to changing the question composition between conditions. They show an additional challenge in locating and using the same evidence within a complete audio history, compared with receiving the supporting turns directly. This provides within-question validation complementary to the distance curves.

Supporting-turn input still requires accurate speech perception and answer production. For example, some UMT errors involve reproducing exact digit sequences such as telephone numbers. A small number of no-history answers can also draw on prior knowledge: a question about neural networks' poor performance in the 1980s can be answered in terms of limited computation and data without hearing the particular monologue.

Emotion tasks.

SubsetModelNo historySupporting turns onlyFull history
Natural emotionQwen3-Omni48%56%66%
Natural emotionAudio Flamingo 324%28%34%
Natural emotionAudio Flamingo Next24%30%32%
Synthetic emotionQwen3-Omni28%48%40%
Synthetic emotionAudio Flamingo 310%30%18%
Synthetic emotionAudio Flamingo Next18%28%22%

For natural emotion, full history improves accuracy over supporting turns alone by 10, 6, and 2 percentage points. This is consistent with surrounding dialogue providing situational and semantic information that helps interpret the target utterance's emotion.

Semantically controlled synthetic emotion shows a different pattern. Supporting turns improve accuracy over no history by 20, 20, and 10 points, whereas full history reduces accuracy relative to supporting turns by 8, 12, and 6 points. Here, the target emotion is conveyed through the acoustic expression of neutral text, so the surrounding semantics need not provide the same emotional cues as in natural dialogue.

These results reveal the task-dependent role of history: semantic tasks and controlled emotion show an additional challenge in using evidence within the full history, while natural emotion can also benefit from the interpretive information provided by surrounding dialogue.

Thank you for the detailed response. I suggest that the authors characterize the evidence-distance curves as descriptive trends and include the context extension settings for each model in the revised manuscript. The authors’ rebuttal has adequately addressed my main concerns, so I am raising my score from 2 to 3.

AuthorsOpenReview ↗

Follow-up to Reviewer BLvd

Thank you for your follow-up and for reassessing our submission. In the revision, we will describe the evidence-distance curves as descriptive trends and report the context extension settings for each model. We appreciate your constructive feedback.

Reviewer_1CGAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Vox-Infinity, the first benchmark specifically designed to evaluate long-context understanding in spoken language models. It analyzes the models using three representative dialogue scenarios: Ultra Multi-Turn Dialogue, Personal Monologue, and Beyond-Semantic Dialogue. The results show that the evaluated models reveal an overall recency effect, with model performance generally declining as answer-supporting evidence appears farther from the query, and that fine-grained semantic evidence is difficult to retrieve from long audio histories, while text histories preserve lexical information more effectively but lose speech-specific paralinguistic cues.

Critical strengths
  • The benchmark has substantial scale.
  • The required context control helps evaluate long-context abilities.
Critical weaknesses
  • The benchmark does not seem to separate long-context failure from task difficulty.
  • Multi-hop reasoning in Personal Monologues requires further verification.
Questions for authors
  • See Critical Weaknesses.
    • The benchmark currently classifies the QA examples into required-context-duration bins. However, those may also differ in reasoning demands, such as direct retrieval, inference from the dialogues, or integration of multiple pieces of evidence. Can the authors provide a breakdown and examine whether the performance trends across duration bins persist within comparable question types?
    • The previous question is also related to the multi-hop reasoning claim in Personal Monologues. Can the authors provide examples and analyses of both the questions and model responses, showing what information must be combined across the monologue and whether models successfully perform the required reasoning?
  • For the local models, can the authors also provide the original max_position_embeddings for each of them so it is clear whether the models are potentially affected by the number set?
  • Can the authors explain why Ultra Multi-Turn Dialogue does not have 8mins+ coverage?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 1CGA

Thank you for your constructive feedback.

W1/Q1: Task composition and evidence-distance trends. The two fine-grained recency curves analyze Ultra Multi-Turn Dialogue (UMT) and Beyond-Semantic Dialogue (BSD) separately. UMT questions retrieve explicitly stated semantic details, such as slot values; BSD questions recall emotional cues associated with the relevant utterances. Within each of these subsets, questions follow a consistent construction procedure and target the same core operation across distance bins. Thus, these curves do not mix nearby direct-retrieval questions with distant multi-hop questions from Personal Monologues. They examine distance-related performance within semantic-detail retrieval and emotional recall, respectively. We will make this task-specific interpretation clearer and distinguish it from the broader scenario-level results for Personal Monologues.

W2/Q2: Evidence and model-response analysis in Personal Monologues. We checked both complete transcripts and existing outputs. Numbers identify transcript segments.

Example 1: Quantitative causal-chain comprehension (YOU1000000155_2). Question: “Why doesn't the seventy percent rule work well for properties with a very low after-repair value?”

Evidence:

[64] Such as fifty thousand dollars as mentioned earlier the thirty percent deducted from the a r v includes the holding cost and the closing cost as well as a profit. All of the fees cost and profit would only be fifteen grand. If the fees and holding costs were to total like ten grand that would leave just five thousand dollar in profit for the house flipper. [68] And i don't know any house flipper who's gonna take on the risk of flipping a house for just five grand.

Qwen3-Omni's response:

The 70% rule doesn't work well for properties with a very low after-repair value because the 30% of the value that is supposed to cover all fees, costs, and profit becomes too small. For instance, with a $50,000 property, the 30% is only $15,000. If the fees and holding costs are $10,000, that leaves only $5,000 for profit, which is not enough for a house flipper to take on the risk.

The response accurately reconstructs the speaker's multi-step argument: the 30% allowance covers both costs and profit; a low valuation leaves only $15,000; deducting $10,000 leaves $5,000; this profit does not justify the risk in the speaker's assessment. It connects the numerical relationships to the decision being explained.

Example 2: Cross-segment evidence integration (YOU1000000016_7). Question: “What are the advantages of using a multicast platform over pushing multiple streams directly from your computer?”

Evidence:

[25] But the option that i'm about to show you now, option number two, will require no additional bandwidth, will put your computer under no additional load, and you'll be able to use a larger arrange of livestreaming software. [35] They also all have integrated comments or integrated chat meaning that no matter which services you're broadcasting to, you can pull in the chat to one central location.

Qwen3-Omni's response:

Using a multicast platform requires no additional bandwidth, puts no additional load on your computer, and allows you to use a larger range of livestreaming software.

The reference answer combines resource efficiency from segment 25 with centralized chat/comment monitoring from segment 35. The model correctly covers bandwidth and computer load, and adds a supported software-compatibility benefit, but omits centralized chat. This illustrates incomplete coverage of complementary evidence across segments, despite accurate extraction of local information.

Q3: Original positional limits and evaluation settings.

ModelOriginal limitUMTPersonal MonologuesNatural emotionSemantically controlled emotion
Qwen3-Omni65,53665,53665,53665,53665,536
GLM-4-Voice8,19265,536131,07265,53665,536
MiMo-Audio8,19265,53665,53665,53665,536
Audio Flamingo 332,76865,53665,53665,53665,536
Audio Flamingo Next131,072131,072131,072131,072131,072

GLM-4-Voice uses seq_length; the other models use max_position_embeddings in the language-model config, nested in text_config where applicable. We will report these settings in Section 4.1.1.

Q4: Why UMT has no 8min+ bin. UMT emphasizes frequent turn transitions and fine-grained retrieval, whereas Personal Monologues and other scenarios complement it with longer temporal spans. UMT contains 991 sessions and 7,086 questions, averaging 35.89 turns per session. Its longest session is approximately 7.17 minutes, and its longest evidence-to-query distance is approximately 6.91 minutes. Consequently, its required-context-duration bins cover 0–2, 2–4, and 4–8 minutes, with no examples in 8min+. The subsets thus cover complementary turn-count and duration dimensions.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal and Revised Manuscript

Thank you again for your time and thoughtful feedback. As the discussion period is nearing its end, we would appreciate your thoughts on our rebuttal and the additional results in the revised manuscript. Please let us know if you have any remaining questions or concerns; we would be happy to discuss them further.

Thank you for your response.

W1/Q1. I would still like to see an analysis of this to support the interpretation provided.

Q3. Why is the limit set to 131,072 for GLM-4-Voice but 65,536 for MiMo-Audio, while both have native 8K context windows? Furthermore, extending both models well beyond their native context lengths (8x or even 16x) raises concerns about potential capability degradation. Those extended configurations should be verified to determine whether they preserve baseline performance on shorter, in-distribution inputs.

Therefore, some of my concerns remain unresolved, and I will maintain my score.

AuthorsOpenReview ↗

Follow-up to Reviewer 1CGA: Evidence-Distance Controls and Positional Settings

Thank you for the follow-up. We summarize the two controls added to the revised manuscript and clarify why GLM-4-Voice used a larger positional limit for Personal Monologues.

W1/Q1. Task composition and evidence-distance trends.

1. Same questions at four evidence distances (Appendix E.1, Table 8). For each subset, we use the same 100 questions across all four conditions, keeping the supporting audio and reference answers fixed. We add complete turns from other sessions and trim non-evidence turns after the supporting audio as needed to keep each history approximately eight minutes long. The columns show the distance from the evidence to the query.

SubsetModel0–2 min2–4 min4–6 min6–8 min
Ultra Multi-TurnQwen3-Omni64%61%56%41%
Ultra Multi-TurnAudio Flamingo Next69%65%61%49%
Natural emotionQwen3-Omni72%68%57%37%
Natural emotionAudio Flamingo Next60%57%54%31%
Controlled emotionQwen3-Omni40%35%29%26%
Controlled emotionAudio Flamingo Next37%34%31%22%

Accuracy declines at every successive interval for both models on all three subsets, with drops of 14–35 percentage points from the nearest to the farthest interval. Because the same questions are used at every distance, changes in the mix of question types or reasoning demands cannot explain the decline in this experiment.

2. Direct evidence access versus the original full history (Appendix E.2, Table 9). Each row compares the same 50 questions under two conditions: directly providing the annotated audio segments containing the answer cues, or providing the original full audio history. The reference answers and scoring procedure are unchanged.

SubsetDistanceQwen3-Omni: EvidenceQwen3-Omni: FullAudio Flamingo Next: EvidenceAudio Flamingo Next: Full
Ultra Multi-Turn2–4 min86%54%80%58%
Ultra Multi-Turn4–8 min84%50%82%50%
Personal Monologues4–8 min88%78%86%52%
Personal Monologues8 min and above90%66%84%42%
Natural emotion4–8 min50%52%52%54%
Controlled emotion4–8 min50%40%40%36%

On Ultra Multi-Turn Dialogues and Personal Monologues, evidence-only accuracy differs by just two percentage points between the two intervals for each model, while full-history accuracy drops in the farther interval. Together, the two controls show that question composition alone does not explain the distance trend and point to difficulties in finding and using distant evidence within the full history.

Q3. Why extend the limits, and why 128K for GLM-4-Voice on Personal Monologues?

We extended the positional limits to evaluate complete audio histories without truncation. In preliminary runs with the original settings, some inputs that exceeded the configured limit produced repeated tokens or incoherent answers.

Although GLM-4-Voice and MiMo-Audio both have an original limit of 8,192 positions, the same recording takes up different amounts of context. GLM-4-Voice can interleave text and speech tokens during generation, but the relevant difference for the input budget is how densely each model represents the audio history. GLM-4-Voice uses about 12.5 speech tokens per second, whereas MiMo-Audio groups four consecutive audio frames into one patch, yielding 6.25 language-model positions per second. The same audio thus occupies roughly twice as many positions in GLM-4-Voice.

In our preliminary runs, 65,536 positions did not accommodate a small number of the longest Personal Monologue inputs for GLM-4-Voice. We therefore used 131,072 for its audio-history evaluation on that subset, while 65,536 accommodated the MiMo-Audio inputs. Appendix C, Table 6 lists the settings used for each model and subset.

Reviewer_tj5MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Vox-Infinity is a benchmark for long-context spoken language understanding across multi-turn retrieval, personal-monologue reasoning, and emotion-dependent recall. It annotates the evidence supporting each answer and groups questions by the temporal span from the earliest annotated evidence to the query. Evaluations of seven spoken language models compare text and audio histories and examine performance across evidence distances. A semantically controlled emotion subset and human checks of both QA construction and automated grading provide additional validation.

Critical strengths
  1. Answer-provenance annotations make the resource more informative than grouping examples only by total recording length. Together with the three spoken scenarios and text/audio comparisons, they support a useful examination of retrieval, reasoning, and paralinguistic recall. This is the main contribution supporting acceptance.

  2. The paper reports manual verification of QA pairs and their evidence, as well as 297/300 agreement between human and automated grading on sampled outputs. These checks provide meaningful support for benchmark and scoring quality. The controlled-emotion subset further strengthens the effort to distinguish acoustic cues from semantic cues.

Critical weaknesses
  1. The distance curves compare different questions, so difficulty and evidence composition can vary with distance. The paper acknowledges this for Personal Monologues and appropriately describes the broader pattern as a tendency. Nevertheless, the curves do not isolate memory decay. A small evidence-only control would clarify whether errors arise from recognizing the relevant content or recovering it within a long history. This limitation reduces the strength of the diagnostic conclusions while leaving the benchmark's resource value intact.

  2. Section 4.1.1 increases the configured positional limit of some local models to 32,768 without identifying the affected checkpoints and the number of inputs beyond their original limits. Separating supported-range and extended-range results is important for reproducibility and interpretation. This is a request to clarify the evaluation conditions; it is not evidence that the runs were incorrect or truncated.

Questions for authors
  1. Clarification: Which checkpoints required positional extension, and how many evaluated inputs exceeded each checkpoint's original configured limit? Please distinguish supported-range and extended-range results where relevant.
  2. Focused validation of the distance interpretation: Can the authors report evidence-only accuracy on a small subset, together with sample and session counts for the longest duration bins? This would help determine how much of the observed pattern is attributable to recovering distant evidence rather than question composition.
5 comments
AuthorsOpenReview ↗

Response to Reviewer tj5M (1/2): W1/Q2 — Supporting-evidence controls

Thank you for recognizing the value of the provenance annotations and benchmark validation. We address W1/Q2 with additional supporting-evidence controls and the coverage of the longest distance bins.

W1/Q2: Separating access to supporting evidence from performance with full history.

We randomly sampled 100 QA examples from each of four subsets: Ultra Multi-Turn Dialogues (UMT), Personal Monologues (PM), natural-emotion dialogues, and semantically controlled synthetic-emotion dialogues. For the same QA examples, we compared three conditions: no history, providing the question without historical audio; supporting turns only, providing only the audio utterances annotated as necessary to answer that question; and full history, providing the complete history used for that question. The supporting-turn condition supplies audio, not a transcript or the reference answer. All results below are accuracy (%).

1. Semantic retrieval and monologue understanding.

SubsetModelNo historySupporting turns onlyFull history
PMQwen3-Omni12%84%72%
PMAudio Flamingo Next8%82%38%
PMGLM-4-Voice6%76%24%
UMTQwen3-Omni2%86%78%
UMTAudio Flamingo Next0%78%40%
UMTGLM-4-Voice0%72%38%

Across all six comparisons, providing the supporting turns substantially improves accuracy over no history. Full-history accuracy is lower than supporting-turn accuracy by 12, 44, and 52 percentage points on PM, and by 8, 38, and 34 points on UMT. Because each comparison uses the same questions and reference answers, these differences cannot be attributed to a change in question composition between the two conditions. They show an additional challenge in accessing and using relevant content within the complete audio history, beyond answering from directly supplied evidence.

Direct evidence access still requires accurate speech perception and answer production. Inspection of UMT errors identified cases involving exact digit sequences, such as telephone numbers, which can be reproduced incorrectly even when the relevant turn is supplied. The small number of correct no-history responses can also arise from prior knowledge or cues in the question. For example, a PM question about why neural networks performed poorly in the 1980s can be answered in terms of limited computation and data without hearing the particular monologue.

2. Natural and semantically controlled emotion recall.

SubsetModelNo historySupporting turns onlyFull history
Natural emotionQwen3-Omni48%56%66%
Natural emotionAudio Flamingo 324%28%34%
Natural emotionAudio Flamingo Next24%30%32%
Synthetic emotionQwen3-Omni28%48%40%
Synthetic emotionAudio Flamingo 310%30%18%
Synthetic emotionAudio Flamingo Next18%28%22%

Natural emotion shows a different pattern: full history improves accuracy over supporting turns alone by 10, 6, and 2 points. This is consistent with surrounding dialogue helping interpret the emotional meaning of the target utterance through its conversational situation and semantic context. Thus, for natural emotional speech, the surrounding history can provide useful interpretive information.

For semantically controlled synthetic emotion, supporting turns improve accuracy over no history by 20, 20, and 10 points, whereas full history lowers accuracy relative to supporting turns by 8, 12, and 6 points. Here, the target emotion is manipulated using neutral text, so surrounding semantics need not provide the same emotional support as in natural conversations. The observed pattern is consistent with a greater burden of locating and interpreting the relevant acoustic cue within the longer history.

Together, these controls clarify the task-dependent role of history: semantic tasks and controlled emotional recall show a cost of using evidence within full histories, while natural emotion can benefit from the surrounding conversational context. They complement the distance curves with within-question comparisons of direct evidence access and full-history performance.

3. Coverage of the longest evidence-distance bins.

SubsetLongest binQA examplesDistinct session IDs
UMT4–8 min230123
PM8 min+653121
Beyond-Semantic Dialogues8 min+5611

These counts describe the full benchmark, separately from the 100-example samples above. Bins measure the temporal span from the earliest annotated supporting evidence to the query, rather than total recording duration. Sessions are counted by dataset session ID; natural and synthetic dialogues retain their respective IDs.

AuthorsOpenReview ↗

Response to Reviewer tj5M (2/2): W2/Q1 — Positional configurations

W2/Q1: Original positional limits and evaluation settings.

Thank you for requesting clarification of the evaluation configurations. The table below lists the original positional limits of the language-model components and the settings used for each subset.

ModelOriginal positional limitUltra Multi-Turn / Beyond-SemanticPersonal Monologues
Qwen3-Omni65,53665,536 (unchanged)65,536 (unchanged)
GLM-4-Voice8,19232,76865,536
MiMo-Audio8,19232,76865,536
Audio Flamingo 332,76832,768 (unchanged)65,536
Audio Flamingo Next131,072131,072 (unchanged)131,072 (unchanged)

We selected the positional limits according to the input-length requirements of the subsets. For Ultra Multi-Turn Dialogues and Beyond-Semantic Dialogues, models with original limits below 32,768 were extended to 32,768, while the others retained their original limits. Personal Monologues contains longer continuous speech histories, which produce longer input sequences after audio encoding. For this subset, models with original limits below 65,536 were therefore extended to 65,536; models already configured for at least this length retained their original settings.

Specifically, GLM-4-Voice and MiMo-Audio received positional extensions in all three subsets. Audio Flamingo 3 was extended from 32,768 to 65,536 only for Personal Monologues. Qwen3-Omni and Audio Flamingo Next retained their original positional limits.

The values in the table refer to the language-model positional configurations, not the audio encoder's input-duration limits. The corresponding field is seq_length for GLM-4-Voice and max_position_embeddings in the language-model configuration for the other models, including the nested text_config where applicable.

The 32,768 setting mentioned in Section 4.1.1 corresponds to the extensions used for Ultra Multi-Turn Dialogues and Beyond-Semantic Dialogues; Personal Monologues uses the separate 65,536 extension setting described above. We will include this per-model, per-subset table in the revision to make the original positional limits and actual evaluation settings explicit.

Thank you for the response. The additional experiments are helpful, but they do not fully isolate the effect of evidence distance. The number of inputs exceeding the models' native context limits also remains unclear.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Title: Summary of Rebuttal and Manuscript Revisions

We thank the reviewers for their careful assessment and constructive feedback. We have provided point-by-point responses and added experiments and implementation details. Below we summarize the main updates; all section and table numbers refer to the revised manuscript. Changes relative to the original submission are marked in blue in the annotated version.

1) Comparing three input conditions to distinguish evidence understanding from retrieval within long histories.

Section 4.1.4 and Table 4 compare question-only, evidence-only, and full-audio-history inputs on the same 100 questions per subset. Evidence-only inputs provide the annotated audio segments containing the information needed to answer, rather than transcripts or answers. On Ultra Multi-Turn Dialogues and Personal Monologues, directly providing these segments improves accuracy over full history by 8–12 percentage points for Qwen3-Omni and 38–44 points for Audio Flamingo Next. Semantically controlled emotion follows the same ordering. Natural emotion shows that surrounding dialogue can help interpret the target utterance’s emotion. Appendix D, Table 7 provides additional results for GLM-4-Voice and Audio Flamingo 3.

2) Adding two evidence-distance experiments to examine whether question composition explains the recency trend.

Appendix E.1, Table 8 holds each question, supporting audio, and reference answer fixed while keeping total history duration at approximately eight minutes. It compares four evidence-to-query distance intervals using the same 100 questions per subset. Both models show declining accuracy with increasing distance across Ultra Multi-Turn Dialogues, natural emotion, and semantically controlled emotion, with drops of 14–35 percentage points from the nearest to the farthest interval.

Appendix E.2, Table 9 compares direct evidence access with the original full history on distant questions, using the same 50 questions per listed interval. For Personal Monologues at eight minutes and above, evidence-only versus full-history accuracy is 90% versus 66% for Qwen3-Omni and 84% versus 42% for Audio Flamingo Next. Together, these experiments show that the decline on distant evidence is not explained solely by differences in question composition across bins. The Figure 3 caption now explicitly characterizes the original curves as descriptive recency trends.

3) Documenting model- and subset-specific positional settings and input handling.

Section 4.1.1 now points to Appendix C, Table 6, which lists the original checkpoint limits and audio-history evaluation settings for each model and subset. The accompanying text explains differences in text-history settings and the relevant positional-encoding parameters. Appendix C also describes chronological turn-by-turn input, the use of each model’s chat template, and the original question as the final user message. Counts of inputs exceeding the original positional limits were reported separately in our response to Reviewer tj5M.

4) Providing monologue examples and correcting the comparison with prior work.

We provided Personal Monologue examples in our response to Reviewer 1CGA. We also corrected Table 1 to reflect that ContextDialog provides supporting-utterance annotations.

Our rebuttal further clarifies the distinction from ContextDialog and BLAB. Vox-Infinity systematically covers fine-grained retrieval in ultra-multi-turn dialogues, long-form monologue understanding and reasoning, and emotional recall. It compares text and audio histories and uses emotional speech synthesized from neutral text to reduce lexical cues, enabling targeted evaluation of the retention and use of acoustic emotional information.

We hope these additional experiments, implementation details, and revisions address the reviewers’ main concerns. We thank the reviewers for their continued consideration and discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.