- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
this paper proposes a probabilistic-symbolic intent disambiguation framework, where a probabilistic measure quantifies how speech ambiguity affects action decisions, symbolic reasoning localizes and resolves the critical intent differences underlying action disagreement through embodied interaction. This paper also introduce a benchmark SPEAR-Bench. which systematically varies perceptual, paralinguistic, and content ambiguity to evaluate task understanding and embodied control under ambiguous speech.
Critical strengths
1.the paper is focusing on ambiguity at task intent level rather than improving speech recognition. It's practical in embodied device use cases.
2. This work recognize the action uncertainty from intent uncertainty.
3. this work take into consideration the environmental feedback with language understanding.
Critical weaknesses
The environmental evidence does not always reflect user intent. exploring the environment may not help with problems like if a persons is looking for an object but both objects are visible in the scene. Also, a certain object not visible in the seen doesn't mean it doesn't exist. she the constrains in figure 2 might not help with the disambiguation.
Questions for authors
- how are different levels of ambiguity defined and classified? does the proposed method perform consistently across different levels of ambiguity?
- Does the evaluation across different languages and Accents? Since the input is speech if I understand correctly and it's not text. the accents and pronunciation may affect the final accuracy. How robust is the proposed method?
- The proposed methods show it will use environment feedback, I am wondering if complexity of the environment affects the final results.
- How many different environments and scenarios have been tested? how does the method perform as environmental complexity increases.
4 comments
Response to Reviewer rH6F (Part 2 of 2)
Q2: Robustness Across Languages
- Does the evaluation across different languages and Accents? Since the input is speech if I understand correctly and it's not text. the accents and pronunciation may affect the final accuracy. How robust is the proposed method?
We thank the reviewer for highlighting cross-language robustness. To further evaluate our method across languages, we conduct a supplementary French speech experiment on SPEAR-Bench under the P0–T0–C0 condition. As shown in Table R3, our method achieves 83.33% and 77.78% task success with English and French speech, respectively, a decrease of only 5.55 percentage points. These results provide preliminary evidence of our method's robustness across spoken languages.
Table R3: Task Success Across Languages on SPEAR-Bench (P0–T0–C0).
| Language | Success (%) |
|---|---|
| English | 83.33 ± 2.78 |
| French | 77.78 |
Q3/Q4:Environmental Coverage and Complexity Analysis
- How many different environments and scenarios have been tested? how does the method perform as environmental complexity increases.
- The proposed methods show it will use environment feedback, I am wondering if complexity of the environment affects the final results.
Environment and Scenario Coverage. We thank the reviewer for highlighting environmental coverage and complexity. The 36 source tasks in SPEAR-Bench cover all six ALFWorld task types, all four scene types, and 30 distinct FloorPlans, spanning diverse task goals, environment layouts, and interaction requirements. Each task is further instantiated under 27 speech ambiguity conditions, yielding 972 evaluation instances. Detailed task and scene distributions are provided in Appendix C.1 (Table 7).
Performance Across Environmental Complexity. To analyze how environmental complexity affects performance, we divide the environments into Low, Medium, and High complexity groups based on the number of searchable locations. As shown in Table R4, task success decreases and interaction steps increase with environmental complexity. This trend is consistent with the greater exploration effort required to acquire discriminative evidence in larger search spaces, indicating that environmental complexity is an important factor affecting task success and interaction cost.
Table R4: Performance across environmental complexity levels on SPEAR-Bench (Qwen3.8-Omni-Flash).
| Complexity | All Search Loc. | # Tasks | Success (%) ↑ | Steps ↓ |
|---|---|---|---|---|
| Low | ≤17 | 13 | 84.52 | 19.13 |
| Medium | 18–50 | 19 | 77.71 | 25.01 |
| High | ≥51 | 4 | 69.44 | 29.92 |
feedback
Thank you for the detailed responses and additional experiments. The clarification of ambiguity levels and the analysis of performance across different ambiguity conditions address my main concerns. While the evaluation of different accents and more diverse languages remains limited. I will maintain my original score.
Thank you for your positive feedback. We are pleased that our clarifications and additional experiments have addressed your main concerns. We sincerely appreciate the time and effort you have devoted to reviewing our work.
We agree that broader evaluation across diverse languages and accents would further strengthen our work. We will acknowledge this limitation and consider extending SPEAR-Bench for systematic cross-language and cross-accent evaluation in future work.
Response to Reviewer rH6F (Part 1 of 2)
Dear Reviewer rH6F,
We sincerely thank you for your thoughtful review and constructive feedback. We greatly appreciate your recognition of the practical importance of task-level speech ambiguity, our modeling of how intent uncertainty affects action decisions, and the integration of environmental feedback into speech understanding. We address your questions and clarify our contributions below.
Q1: Ambiguity Definitions and Performance Across Levels
Ambiguity Definition and Classification. We thank the reviewer for the question regarding ambiguity definitions and levels. SPEAR-Bench defines three ambiguity dimensions with three levels each: (1) Perceptual (P): P0–P2 correspond to zero, one, and at least two corrupted task-relevant speech segments; (2) Paralinguistic (T): T0–T2 contain zero, one, and at least two cue types, such as tone, fillers, and pauses; (3) Content (C): C0 uses direct instructions, C1 introduces interruptions or self-corrections, and C2 involves context-dependent references or omissions. All variants preserve the original task goals and environments and undergo automated checks and human verification.
Performance Across Ambiguity Levels. As shown in Table R2, our method achieves the highest success across most ambiguity levels and shows greater robustness as perceptual and content ambiguity increase, while baselines degrade more substantially. Some methods perform better at C1 than C0 with Qwen, whereas DeepSeek shows the opposite trend. This is because speech ambiguity introduces not only interference but also additional information that Qwen can exploit through speech cues, whereas DeepSeek relies on ASR transcripts and cannot fully utilize such information.
Table R2: Performance Across Ambiguity Levels on SPEAR-Bench (Success, %)
(a) DeepSeek-V4.1-Flash
(b) Qwen3.8-Omni-Flash