acceptodds
Under review as a conference paper at ICLR 2027

SONATINE: Spatial Audio Understanding via Tool-Integrated Reasoning in Audio LLMs

Abstract

Most general-purpose Audio Large Language Models (Audio LLMs) accept only single-channel audio and therefore cannot use inter-channel spatial cues. Existing approaches to spatial audio understanding internalize these cues by feeding dense features extracted from multichannel First-Order Ambisonics (FOA) recordings into the model. However, such adaptation improves spatial understanding at a potential cost to general audio understanding. Spatial measurement can instead be externalized: signal-processing estimators compute quantities such as sound direction directly from FOA recordings, and their outputs are provided to an unchanged Audio LLM. But to reliably measure the queried target, such an estimator must operate on a target-informative window in which the queried sound is clear and overlap is limited. Because the estimator measures all active sounds in its input window, spatial information from other sources can otherwise be mixed into the measurement. To address this problem, we introduce SONATINE, a tool-integrated reasoning framework in which spatial estimators act as tools and the Audio LLM specifies a time window as an argument to each tool call. The Audio LLM, with no added spatial module, receives the question and only the mono channel, while the selected tool accesses all FOA channels within the specified window and returns a spatial observation. Within SONATINE, obtaining reliable spatial measurements requires adaptive window selection, in which the Audio LLM uses the question and mono audio to choose a target-informative window for each tool call. To teach this behavior and general spatial tool use, we construct SONATINE-15k, a dataset of 14,990 tool-use trajectories, and use it for Supervised Fine-Tuning (SFT), followed by Group Relative Policy Optimization (GRPO). On SO-Bench, our 7B model surpasses all evaluated internalized spatial baselines while using far fewer spatial QA examples, with its largest advantage under heavy source overlap. By externalizing spatial measurement, SONATINE also enables spatial audio understanding in closed-source models through prompting alone, helps retain general audio understanding, and supports transfer from FOA to a six-channel format without further training by changing only the tool implementations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio-Language Models, Tool-Integrated Reasoning, Spatial Audio Understanding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cTy5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SONATINE separates monaural semantic understanding from multichannel spatial measurement. The Audio LLM receives the FOA W channel and question through its existing interface, then selects a spatial tool and time window. Tools access the complete multichannel recording and return direction, trajectory, distance, or spatially cropped audio. The central design is to select windows in which the queried sound is clear and interference from other sources is limited.

Critical strengths
  1. In the duration-matched ablation, target-informative supervision exceeds high-energy-window supervision by 5.47 percentage points under heavy overlap.

  2. The interface supports closed-source tool use and six-channel transfer through backend replacement while retaining much of the model’s general-audio capability.

Critical weaknesses
  1. Window-supervision ablations are provided, but a single-round or fixed-pipeline control with the same tools and target temporal grounding is missing, leaving the additional value of iterative tool reasoning unclear.

  2. Human agreement rates for the answer extractor are not reported, leaving the stability of the GRPO and overall gains uncertain.

Questions for authors
  1. Could you include a pipeline baseline using the same spatial tools and target temporal grounding without multiple interaction rounds?

  2. What is the agreement rate between the free-form answer extractor and human annotations?

  3. What are the complete adaptation costs, including distance-tool training, trajectory synthesis, and GRPO, and the end-to-end inference latency?

  4. What spatial error does the six-channel-to-FOA conversion introduce, and how does a simplified policy perform using the same six-channel backend?

5 comments
AuthorsOpenReview ↗

Response to Reviewer cTy5 (1/3)

We thank the reviewer for the specific questions on the contribution of iterative tool reasoning, the reliability of the evaluation, the adaptation cost, and format transfer. We report below the additional experiments and analyses we conducted to answer each question.

W1/Q1. The additional value of iterative tool reasoning

To isolate the effect of iterative tool reasoning, we ran a single-round experiment. With the same SONATINE 7B policy and the same tools, we kept the first tool call and its window unchanged and forced the model to answer immediately after receiving its result, and evaluated on the full SO-Bench.

Result 1: the paper model with the second tool call blocked.

TaskSONATINE 7B (paper)One call
estimate_azimuth70.270.7
estimate_elevation73.574.2
estimate_distance82.382.3
spatial_temporal60.659.8
identify_source_by_doa / by_location55.2 / 52.055.0 / 51.5
classify_motion46.851.6
compare_distance61.344.1
compare_elevation73.952.9
relative_left_right64.955.6
multi_hop24.317.1
onset_from_location / speech_content79.1 / 39.979.5 / 40.3
count / detect_source / detect_time58.8 / 50.2 / 77.458.8 / 50.4 / 77.3
16-task overall60.6557.57

On the direct-estimation tasks, a single round already achieved performance similar to iterative calling. In contrast, on the compare tasks and the multi_hop task, which deal with relations between sources, the single-round performance was much lower.

This difference shows that two kinds of abilities are separated. The measurement itself, such as direction or distance, is handled by adaptive window selection. Because the first call of the model already uses a window in which the target is clearly heard, a single measurement is as accurate as iterative calling. That is, the window-selection ability resides in the learned policy, and using it does not require multiple rounds.

Where iterative tool reasoning contributes, in contrast, is spatial understanding that requires combining measurements to produce an answer. The single-round setting lowers the accuracy of compare_distance, compare_elevation, relative_left_right, and multi_hop by 7 to 21 points. The first three require measuring two sources separately and comparing them, and multi_hop requires gathering the directions of several sources and selecting the one that satisfies a condition. These tasks do not end with measuring a single source. The spatial information of several sources must be gathered and the relations among them inferred. Such reasoning cannot be replaced by a single measurement and is possible only with iterative tool reasoning that interleaves measurement and reasoning. Moreover, SONATINE outperforms Spatial-Omni-7B on three of these tasks (relative_left_right, compare_elevation, and compare_distance), and this advantage disappears or shrinks considerably in the single-round setting. Because a measurement in SONATINE is a tool call, the model can explicitly choose a target-informative window for each source the question refers to and obtain a separate measurement for each. The results show that this provides a real benefit and that iterative tool reasoning is what makes it possible.

In summary, in the SONATINE framework, adaptive window selection is responsible for the accuracy of the measurements, and iterative tool reasoning is responsible for the reasoning that combines the measurements. We will add this experiment and its interpretation to the paper.

AuthorsOpenReview ↗

Response to Reviewer cTy5 (2/3)

W2/Q2. Agreement between the extractor and human annotation

We would first like to describe the role of the extractor in more detail. The extractor in SONATINE does not decide whether an answer is correct. Without seeing the gold answer, the extractor only extracts what the free-form answer claims, such as a source name, left or right, a time interval, or a transcript. The comparison with the gold answer is performed by deterministic code, and numeric tasks such as angle, distance, and onset are scored by rules without an LLM. The extractor therefore only reads the answer and does not score it. Because it does not know the correct answer, it has little room for bias toward particular answers.

Nevertheless, as the reviewer points out, the paper did not report how accurate the extraction is, so we measured it by direct comparison with human-made labels. On the evaluation side, for the 11 tasks scored with the extractor, we compared 200 SO-Bench answers from three systems with different answer styles (Spatial-Omni-7B, SONATINE 7B, and Gemini with prompting only). On the GRPO reward side, we compared 200 completions from the 7 tasks for which the extractor gave the reward during the GRPO training. One annotator wrote down what each answer claims without seeing the gold answer, the extractor output, the system that produced the answer, or whether the item came from evaluation or training. The human labels were converted into scores by the same scoring code as the extractor outputs.

Result 2: extractor accuracy against human labels.

SO-Bench answers (evaluation, Qwen3.8-27B)GRPO completions (reward, Qwen3-4B)
Items200200
Agreement with human label97.0%98.0%
Pass/fail decision accuracy99.0%99.5%
Cohen's kappa (pass/fail)0.9800.990
Mean score (extractor / human)0.463 / 0.4600.430 / 0.435

The extractor output agreed with the human label on 97.0% and 98.0% of the items, and the pass/fail decision differed for only 3 of the 400 items. In two of them the extractor misread the answer, once giving credit to a wrong answer and once giving zero to a correct one. In the remaining one, the human and the extractor read almost identical time intervals, which fell on either side of the pass threshold. The difference in mean score was 0.003 to 0.005, which is too small to change the results.

We also checked, using the metrics logged during training, whether the signal from the extractor was stable during GRPO training. If the policy had drifted toward answers that the extractor cannot read or answers that fool the extractor, the fraction of answers whose content the extractor could not read (unclear) would have risen, or the reward would have approached 1.

Result 3: the extractor's reward signal during GRPO (extractor-scored tasks).

StepsRewardExtractor unclear
1-500.4871.87%
51-1000.4951.42%
101-1500.5111.22%
151-2000.5130.66%

As training progressed, the unclear rate decreased from 1.9% to 0.7%, the reward rose slowly without saturating, and the answer length remained constant. Together with the GRPO column of Result 2, this supports that the gain from GRPO does not come from extractor errors or exploits. We will add the role of the extractor, the blind annotation procedure, and the two results above to App. E.3.

Q3. Adaptation cost and end-to-end latency

We summarize, stage by stage, the cost of building SONATINE 7B from Qwen2.5-Omni-7B, including the training of the distance tool.

Result 4: adaptation cost.

StageDataCompute
Trajectory construction14,990 trajectoriesCPU, a few minutes per task
Rationale synthesis (Qwen3.6-27B)14,990~4 GPU-h, Pro 6000
Distance regressor199K windows, 38.7M params~18-20 GPU-h, Pro 6000
SFT 7B14,990, 2 epochs5 h 52 min, Pro 6000 ×2
GRPO 7B200 steps~24 h, H100 ×4

We also measured the end-to-end inference latency. On 315 questions randomly sampled from SO-Bench, we measured the time and the number of generated tokens per question, running sequential inference with batch size 1 on a single RTX Pro 6000.

Result 5: per-question inference cost.

Per questionSONATINE 7B
Seconds (mean / median / p90)4.75 / 4.18 / 8.54
Tool execution (mean)0.33
Tokens (mean / median / p90)297 / 253 / 548
Tool calls / model replies1.88 / 2.88

Most of the time is spent on generating tokens over multiple tool calls, and tool execution takes only a small part of it. We will add both tables to App. D.

AuthorsOpenReview ↗

Response to Reviewer cTy5 (3/3)

Q4. Error of the six-channel-to-FOA conversion and simplified policies

In short, the six-channel-to-FOA conversion loses both directional information and the spectral characteristics of the signal, and even with the same six-channel backend, an untrained policy did not reach the performance of SONATINE. First, to examine what the conversion preserves independently of any model, we passed a single plane wave without noise or reverberation through the six-channel array from every direction of the measured array transfer functions, converted it to FOA, and analyzed it per frequency band. We examined two conversions. One is the full-band complex least-squares Ambisonics signal matching (ASM) that we used in the main text to produce the inputs of Spatial-Omni-7B and JAEGER. The other is a variant that is identical to ASM below 1.5 kHz and replaces the objective with magnitude least squares (MagLS) only above it. Since this is the case of a single source without noise, these numbers correspond to an upper bound on the information the conversion can preserve.

Result 6: per-band analysis of the six-channel-to-FOA conversion.

BandAzimuth error, median (ASM / MagLS variant)Accuracy within 20° (ASM / MagLS variant)W level (ASM / MagLS variant)Dipole-to-W ratio (MagLS variant, ideal 0 dB)
200-500 Hz4.2° / 4.2°100% / 100%-0.4 / -0.4 dB-0.6 dB
0.5-1 kHz12.1° / 12.1°75% / 75%-1.7 / -1.7 dB0.3 dB
1-1.5 kHz21.2° / 21.2°47% / 47%-3.5 / -3.5 dB0.7 dB
1.5-2 kHz24.2° / 24.1°44% / 43%-6.3 / -2.2 dB-3.4 dB
2-3 kHz38.7° / 19.6°30% / 51%-10.9 / -1.4 dB-4.3 dB
3-4 kHz55.1° / 26.3°20% / 39%-13.6 / -1.6 dB-5.3 dB
4-6 kHz48.3° / 27.7°25% / 38%-13.7 / -1.2 dB-4.7 dB
6-8 kHz45.6° / 38.0°27% / 27%-14.9 / -2.2 dB-4.7 dB

Direction is preserved only below 1 kHz. Above it, even for a single noise-free source, a large azimuth error of around 20° or more occurs.

The conversion also changes the spectral distribution of the signal. ASM attenuates the W channel by 11 to 15 dB above 2 kHz, producing a signal with its high frequencies cut, whereas the MagLS variant keeps the level but discards the high-frequency phase, so that the summed energy of the dipoles (X, Y, Z channels) becomes 3 to 5 dB smaller than W. In real FOA, this ratio is exactly 0 dB for a single plane wave. That is, even for a single clean source, the converted FOA systematically has a dipole deficit that does not exist in real FOA. An FOA-trained encoder therefore receives inputs outside its training distribution whichever conversion is used, and the failures of the baselines can be regarded as stemming from this.

We believe this is a limitation of the internalized design. To be used on a new array, an FOA-trained model must either be retrained or receive input converted to FOA, and as shown above, converting the signals of a real six-channel array to FOA incurs a loss. SONATINE, on the other hand, keeps the policy as is and replaces only the tool backend with one for six channels, so no conversion is involved.

Next, we compared simplified policies on the same six-channel backend, with the same 350 questions and the same scoring. Base 7B is Qwen2.5-Omni 7B without any additional training, and Base 7B + tools is the same model given the same tools as SONATINE by prompting. Lower is better for MAE and WER.

Result 7: simplified policies on the six-channel backend.

PolicySingle-speaker azimuth MAE (°)Single-speaker azimuth acc@20Left-right acc.Order-conditioned azimuth acc@20Location-conditioned WERDistance MAE (m)
Base 7B, no tools21.70.6140.5000.3710.6371.177
Base 7B + tools27.70.4710.4860.2290.6721.034
SONATINE 7B10.10.8570.6000.4710.6160.329

When the untrained base model is given the same tools by prompting, it falls below even the base model without tools on every task except distance. The performance on EasyCom can therefore be attributed to the learned policy rather than to the six-channel backend. We will add the per-band conversion analysis and the policy comparison on the same backend to the paper.

We thank the reviewer once again for the helpful questions.

Reviewer_pMBrOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces SONATINE, a tool-integrated reasoning framework in which spatial estimators act as tools and the Audio LLM specifies a time window as an argument to each tool call. The Audio LLM, with no added spatial module, receives the question and only the mono channel, while the selected tool accesses all FOA channels within the specified window and returns a spatial observation. Within SONATINE, obtaining reliable spatial measurements requires adaptive window selection, in which the Audio LLM uses the question and mono audio to choose a target-informative window for each tool call. To teach this behavior and general spatial tool use, the paper constructs SONATINE-15k, a dataset of 14,990 tool-use trajectories, and uses it for Supervised Fine-Tuning, followed by Group Relative Policy Optimization. On SO-Bench, the 7B model surpasses all evaluated internalized spatial baselines while using far fewer spatial QA examples, with its largest advantage under heavy source overlap. By externalizing spatial measurement, SONATINE also enables spatial audio understanding in closed-source models through prompting alone, helps retain general audio understanding, and supports transfer from FOA to a six-channel format without further training by changing only the tool implementations.

Critical strengths
  • The paper introduces SONATINE, a tool-integrated framework in which an Audio LLM retains its native input interface while selecting spatial tools and time windows to obtain spatial information.
  • It constructs SONATINE-15k, a dataset of 14,990 tool-use trajectories designed to teach adaptive window selection, and shows that the resulting model outperforms the released Spatial-Omni-7B checkpoint, with its largest advantage under heavy source overlap.
  • It shows that SONATINE's externalized design enables prompting-based spatial understanding in closed-source models, data-efficient adaptation, retention of general audio understanding, and transfer to a six-channel recording format without further training.
Critical weaknesses
  1. The paper highlights using 52 times fewer training examples, with 14,990 trajectories compared with 787k examples for Spatial-Omni-7B, but the data accounting is incomplete: Appendix C shows that the distance estimation tool get_distance among the four tools is actually a Conformer neural network trained separately on 199,250 labeled single-source audio windows. Counting only the policy trajectories leaves out nearly 200,000 supervised training examples.
  2. Multi-step reasoning and tool calls also add inference latency and token overhead, but the paper reports no latency measurements at all.
  3. On the Direct Spatial Estimation category, which directly tests spatial measurement, SONATINE 7B scores 74.14 with SFT and 75.36 with GRPO, which is lower than 78.01 for Spatial-Omni-7B. In Table 2b on clean scenes without source overlap, SONATINE is more than 10 points behind Spatial-Omni-7B at 80.98 versus 92.16. Its overall score advantage mainly comes from non-spatial perception and complex reasoning, while its raw localization accuracy on non-overlapping sources is worse.
Questions for authors
  1. On SO-Bench, how many tool calls, generated tokens, and seconds does each question take on average, and how much slower is it than single-pass inference with Spatial-Omni-7B?
  2. How much does the neural distance estimator trained on 199,250 windows contribute to the overall score, and why is SONATINE more than 10 points behind Spatial-Omni-7B on single-source direct spatial estimation without overlap in Table 2b?
3 comments
AuthorsOpenReview ↗

Response to Reviewer pMBr (1/3)

We thank the reviewer for the careful examination of the scope of our training-data accounting, the inference cost, and the per-task performance differences.

W1/Q2. Training-data accounting and the contribution of the distance tool

We mentioned in Sections 3 and 5 that about 200K examples were used to train the distance tool, and the introduction also states that we used 52 times fewer "QA training examples". We had no intention at all of hiding this, but we believe that we did not convey the scope of the comparison clearly enough, so that it could be read as the reviewer points out. We apologize for the confusion.

We reported the comparison in terms of QA data because QA data are far more expensive to obtain. Training the distance tool requires only audio and distance labels, whereas QA data additionally require questions and answers in natural language. Such questions and answers are difficult to obtain and costly to create, so QA data are far more expensive to collect than the training data of the distance tool.

In addition, in our framework the tools can be replaced while the trained LLM policy is kept as is. It is therefore also possible to take an already-trained spatial estimator and use it as a tool. Considering such cases, we thought it more natural to count the data used to train the LLM policy than to add the training data of the tools to the training cost of the framework, and this is why we reported in the paper that we used 52 times less QA data.

We agree, however, that the data used to train the distance tool should also be visible when the whole system is evaluated. Counting all the training data, SONATINE 7B uses 14,990 trajectories to train the policy and 199K windows with scalar distance labels to train the distance regressor (38.7M parameters). For a fair comparison, we counted the baseline by the same standard. Spatial-Omni-7B uses 787K QA examples and, in addition, 328K clips with frame-level labels to pretrain its SO-Encoder separately.

Both models thus have a separately trained spatial component. That of SONATINE is a small regressor trained with scalar distance labels, whereas the spatial encoder of Spatial-Omni-7B is pretrained on more clips with richer frame-level labels. We will make clear in Sec. 5 that the scope of the comparison is QA data, and add this accounting to App. E.1, which describes the amount of training data of Spatial-Omni-7B.

Next, to answer the reviewer's question of how much the distance tool trained on 199,250 windows contributes to the overall score, we measured its contribution in two ways. First, we recomputed the benchmark average without the two tasks that use the distance tool (estimate_distance and compare_distance).

Result 1: overall score without the two distance tasks.

Model16-task14-task (no distance)
Spatial-Omni-7B57.4755.31
SONATINE 7B (SFT)60.3658.70
SONATINE 7B (SFT+GRPO)60.6559.06

Without the distance tasks, the margin of SONATINE over Spatial-Omni-7B actually widens from +3.18 to +3.75. That is, the advantage of SONATINE over the baseline does not come from the trained distance tool.

Next, we retrained the distance regressor with the same recipe using only a subset of the training windows, and measured the performance of the tool alone on the 486 windows of estimate_distance. Acc@1m is the fraction of estimates whose error is within 1 m.

Result 2: distance regressor trained on fewer windows.

Train windowsAcc@1mMAE (m)
199,250 (paper)83.50.579
49,809 (25%)80.90.659
19,922 (10%)76.70.709

The performance of the tool alone is maintained to a reasonable degree even with about 20K labels. We then re-evaluated SONATINE on the two tasks that require distance, without retraining the policy and replacing only the regressor behind get_distance with each of the two smaller ones. The overall score in the table is the 16-task average that reflects the re-evaluation of these two tasks.

Result 3: SONATINE with the smaller distance regressors.

Regressor train windowsestimate_distancecompare_distance16-task overall
199,250 (paper)82.361.360.65
49,809 (25%)79.857.160.23
19,922 (10%)74.559.860.07
Spatial-Omni-7B85.459.857.47

With the 10% regressor, estimate_distance drops by 8 points, and compare_distance stays between 57.1% and 61.3% depending on the amount of data. The drop in estimate_distance agrees with the drop in the accuracy of the tool alone within 1 to 2 points. That is, the policy passes on the accuracy of the tool almost unchanged, and distance performance can be regarded as a property of a replaceable tool rather than something tied to the policy. We will add all three results to the paper.

AuthorsOpenReview ↗

Response to Reviewer pMBr (2/3)

W2/Q1. Inference latency and number of tokens

We agree that the inference cost increases with the use of tools and that this should have been reported in the paper. On 315 questions randomly sampled from SO-Bench, we measured the latency, the number of generated tokens, and the number of tool calls, running sequential inference with batch size 1 on a single RTX Pro 6000. All values below are per question, and the tool execution time is included in the total time.

Result 4: per-question inference cost.

Per questionSONATINE 7BSpatial-Omni-7B
Seconds (mean / median / p90)4.75 / 4.18 / 8.540.40 / 0.36 / 0.66
Tool execution (mean)0.33-
Tokens (mean / median / p90)297 / 253 / 54814 / 13 / 23
Tool calls / model replies1.88 / 2.880 / 1

SONATINE is about 11.9 times slower per question than Spatial-Omni-7B. Most of the time, however, is spent on token generation by the LLM rather than on tool execution (0.33 s of the 4.75 s is tool execution). The large number of generated tokens arises because SONATINE generates more tokens as it calls the tools repeatedly.

This repetition is possible because we model spatial understanding as tool-integrated reasoning. An internalized model such as Spatial-Omni-7B answers in a single pass from a fixed representation that encodes the entire audio once, independently of the question, and has no step in which a new measurement is obtained while answering. In SONATINE, a measurement is a tool call, so the model can choose a target-informative window for each source the question refers to, obtain a separate measurement for each, and repeat this as many times as needed.

To verify that this repetition actually contributes to performance, we also ran a single-round experiment. With the same SONATINE 7B policy and the same tools, we kept the first tool call and its window unchanged and forced the model to answer immediately after receiving its result, so that it had to answer from a single measurement.

Result 5: performance when tool calls are limited to one (SO-Bench).

TaskSONATINE 7B (paper)One call
estimate_azimuth70.270.7
estimate_elevation73.574.2
estimate_distance82.382.3
compare_distance61.344.1
compare_elevation73.952.9
relative_left_right64.955.6
multi_hop24.317.1
16-task overall60.6557.57

The direct estimation tasks, which can be answered with a single measurement, perform the same with one call. In contrast, the spatial reasoning tasks that require gathering and comparing measurements of several sources (compare_distance, compare_elevation, relative_left_right, multi_hop) drop by 7 to 21 points. For the remaining nine tasks not shown in the table, the difference between the two settings was within 1 point, except for classify_motion (46.8 vs. 51.6). Among the four tasks that drop, relative_left_right, compare_elevation, and compare_distance are tasks on which SONATINE outperforms Spatial-Omni-7B. When tool calls are limited to one, this advantage disappears or shrinks considerably. This shows that choosing a target-informative window for each source the question refers to and measuring each separately provides a real benefit, and that repeated tool calls are what make this possible. The additional latency of SONATINE comes from these repeated calls, so it can be seen as the cost of obtaining gains on these spatial reasoning tasks. We will add both tables and a discussion of this trade-off to the paper.

AuthorsOpenReview ↗

Response to Reviewer pMBr (3/3)

W3/Q2. Direct estimation performance and the source of the gain

As the reviewer points out, SONATINE is below Spatial-Omni-7B on direct estimation of sources without overlap. On the isolated sources without overlap in Table 2(b) (1,548 questions), Spatial-Omni-7B scores 92.2, while SONATINE scores 81.0 with the SFT model and 82.3 with the SFT+GRPO model, a gap of about 10 points. To examine the cause of this gap, we applied the tool alone, without the policy, to the same questions. In this case the accuracy is 83.6 when the tool is applied to the whole recording and 82.5 when it is applied to the least-overlap window chosen with test annotations. That is, the policy already extracts almost all of the precision that the tool can provide. The 10-point gap therefore comes from the precision of the tools themselves, not from how the policy uses them.

As the reviewer says, the overall gain of SONATINE is not uniform across categories. Most of it, however, comes from complex spatial-semantic reasoning. Of the 3.18-point difference in the 16-task average, 2.58 points come from the Complex Spatial-Semantic category (32.58 vs. 42.90), and the contribution of Non-Spatial Perception is 0.81 points. Even when the three Non-Spatial Perception tasks are excluded and only the remaining 13 spatial tasks are averaged, SONATINE is higher, at 60.30 versus 57.38 for Spatial-Omni-7B.

Complex spatial-semantic reasoning consists of tasks that require a spatial measurement matching the condition of the question, combined with other information. For example, in spatial_temporal, which asks for the direction of a sound during a specified time interval, SONATINE places the measurement window on that interval (60.6 versus 32.2 for Spatial-Omni-7B). In speech_content, which asks what a speaker at a specific location said, it applies spatial_crop toward that direction and transcribes the returned audio (39.9 versus 24.8). An internalized encoder, on the other hand, compresses the whole clip into fixed spatial tokens, so it cannot adapt the measurement to the question explicitly.

For the same reason, SONATINE is also ahead on direct estimation when there is overlap. The lead of Spatial-Omni-7B on direct estimation is due to the precision gap on isolated sources described above. Under heavy overlap, where the target overlaps substantially with other sources, SONATINE outperforms Spatial-Omni-7B, as shown in Table 2 of the paper. To look at an even harder case, we separately collected and evaluated the 427 direct-estimation questions in which the target overlaps with another source for its entire active time.

Result 6: accuracy on questions whose target is always overlapped.

Always-overlapped targetSpatial-Omni-7BSONATINE 7B (SFT+GRPO)
Azimuth (acc. within 20°)11.742.7
Elevation (acc. within 10°)32.644.0
Distance (acc. within 1 m)63.074.1
Mean of the three35.753.6

In this case the gap between the two models widens further.

We believe that handling overlap and reasoning tasks, where SONATINE is strong, is important for the spatial understanding of Audio LLMs. In real-world audio, it is far more natural for the target source to be heard overlapped with other environmental or background sounds than for a single source to be heard alone. As for reasoning tasks, they evaluate whether an Audio LLM has acquired spatial understanding beyond spatial measurement. Spatial understanding does not end with measuring where a sound is. Answering a question also requires connecting the measured location with other information such as the type of the sound, time, and the content of speech. In real situations, it is more natural to ask about relations between sources, or relations to time and semantics, such as where another sound was while a certain sound was heard, than to ask directly for the direction or distance of a sound as a number.

SONATINE is far ahead of the internalized baseline on such tasks, which shows the benefit of the externalized design, in which the model iteratively decides how to measure depending on the question and uses the results in its reasoning. We will state the precision limitation on isolated sources explicitly in the main text, and add the tool-alone comparison, the per-category breakdown of the gain, and the always-overlapped evaluation to the paper.

We thank the reviewer once again for the valuable comments.

Reviewer_E3sAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SONATINE, a tool-calling audio LLM framework for spatial audio understanding. SONATINE employs external spatial estimation tools while the Audio LLM processes monaural audio (W channel in FOA) and call signal processing tools on a specific temporal range and channel. The key design is adaptive window selection, which aims to identify segments where the target sound source is clearly audible with minimal interference. The authors construct SONATINE-15k dataset to train tool-use behaviors through SFT followed by GRPO. Experiments on SO-Bench demonstrate competitive performance against existing spatial audio models. The paper also evaluates general audio understanding retention, prompting-based tool use with closed-source models, and cross-format transfer to six-channel recordings.

Critical strengths
  1. The motivation and methodology in this paper is reasonable, coherent, and easy to understand. The proposed framework provides a straightforward and practical approach to spatial audio understanding by integrating Audio LLMs with external spatial estimation tools. Similar approaches have proved effective in other audio understanding or broader multimodal areas. The overall methodology is clearly presented and logically consistent with the identified challenges.

  2. The experiments are relatively thorough, covering comparisons with existing spatial audio models, ablations on adaptive window selection, retention of general audio understanding, and cross-format transfer. The results demonstrate competitive overall performance, with particularly notable improvements under heavy source overlap.

Critical weaknesses

I think there are two major weaknesses which leads to weak rejection:

  1. Limited technical novelty. The proposed framework is largely a straightforward integration of existing spatial audio estimation tools with standard LLM tool-calling capabilities. The central contribution, adaptive window selection, essentially amounts to having the Audio LLM identify a temporal segment where the target source is most clearly audible and invoke the corresponding tool on that segment. This is a rather natural extension of existing tool-use paradigms. Although the framework supports iterative tool calls, it does not introduce sufficiently new mechanisms for planning, execution, or verification beyond conventional tool-use agents. Consequently, the methodological contribution appears rather limited.

  2. Limited applicability under persistent source overlap. The effectiveness of adaptive window selection relies on the existence of temporal segments where the target source can be relatively clearly observed with limited interference. However, such segments may not exist when multiple sound sources overlap continuously. In these scenarios, selecting a different time window may not resolve the ambiguity of target-specific spatial estimation. Moreover, the paper's "heavy overlap" setting does not necessarily represent such cases, leaving the robustness of the proposed approach under persistent source overlap insufficiently explored.

Questions for authors

The "heavy overlap" setting in Section 6.1 does not necessarily imply that no relatively clean target-informative window exists. How does SONATINE perform where the target source always overlaps with other sources? Would the tool return the results for another source?

3 comments
AuthorsOpenReview ↗

Response to Reviewer E3sA (1/2)

We sincerely thank the reviewer for the careful reading of our paper and for giving us the opportunity to clarify the limitation under persistent source overlap and the contributions of our work. We ran additional experiments to examine the points raised, and we report the results below.

W1. Technical novelty

We agree that SONATINE is a framework built on existing tool-integrated reasoning, and we do not claim a new planning or verification algorithm. We believe the novelty and contributions of our work lie in the following three points.

First, to the best of our knowledge, SONATINE is the first framework that addresses spatial audio understanding through tool-integrated reasoning. It is designed so that the Audio LLM decides which source to measure in which window based on the question and the content of the audio, and answers using the spatial measurements returned by the tools.

Second, we built SONATINE-15k, a tool-calling trajectory dataset that teaches adaptive window selection. Adaptive window selection is not an ability that follows naturally from simply providing tools to an Audio LLM. To verify this, we evaluated two open-source Audio LLMs without any training. Both models were given the same tools as SONATINE, together with a prompt instructing them to "choose a window in which the target is active and overlaps least with other sources", so that they would perform adaptive window selection.

Result 1: open-source Audio LLMs given the tools by prompting only (SO-Bench).

ModelSettingNon-Spatial PerceptionDirect Spatial EstimationGrounding & RelationsComplex Spatial-SemanticOverallContainment (%)
Qwen2.5-Omni 7BWithout tools20.4733.7336.6413.2027.20-
+ tools (prompt)20.5231.1822.7111.1320.9949.40
Qwen3-Omni 30BWithout tools20.7532.8436.4914.9527.47-
+ tools (prompt)29.1869.8333.0813.8434.4362.82
SONATINE 7B (SFT+GRPO)Trained on SONATINE-15k62.1775.3664.3642.9060.6580.62

All scores are percentages. Containment is the fraction of the first tool-call window that falls inside the minimum-overlap interval of the queried source.

Qwen2.5-Omni 7B, the backbone of SONATINE 7B, does not benefit from the tools. Its containment is close to that of a randomly placed window (about 46%), and its overall score even drops. Qwen3-Omni 30B does better, but even when instructed to choose a window, its containment still falls short of that of SONATINE 7B. With the tools, Qwen3-Omni 30B improves greatly on Direct Spatial Estimation, where a single measurement can be given directly as the answer. However, it does not improve over the setting without tools on Grounding & Relations and Complex Spatial-Semantic Reasoning, where measurements must be combined with relations between sources, time, or semantics. Thus, the gain obtained from the tools does not carry over to spatial reasoning tasks.

In other words, the behavior of calling a spatial tool on the right interval and using its measurement in reasoning is not obtained from a pre-trained Audio LLM by prompting alone, and requires training. Since no data existed to teach this behavior, we built a new training dataset, SONATINE-15k, consisting of 14,990 trajectories verified by tool execution. We will add this result to the paper.

Third, we showed that externalizing spatial understanding through tool-integrated reasoning has four advantages over encoder-based internalized approaches. The first is data efficiency. SONATINE 7B outperforms Spatial-Omni-7B on SO-Bench with about 15K tool-calling trajectories, while Spatial-Omni-7B uses 787K QA examples. The second is retention of general audio understanding. On MMAU and MMAU-Pro, SONATINE 7B stays close to its base model, whereas Spatial-Omni-7B drops substantially. The third is transfer across spatial audio formats. By replacing only the tool backend, the FOA-trained policy works on the six-channel array of EasyCom without retraining, with a single-speaker azimuth error of 10.1° against 85.9° for Spatial-Omni-7B. The last is applicability to closed-source models, which cannot be equipped with a spatial encoder. With the same tools and prompting alone, both GPT-Audio 1.5 and Gemini 3.8 Flash improve substantially on SO-Bench. These results show that tool-integrated reasoning can be an alternative to internalized approaches for spatial audio understanding. We will revise the contributions in the introduction to make these points clearer.

AuthorsOpenReview ↗

Response to Reviewer E3sA (2/2)

W2/Q1. Performance under persistent source overlap

As the reviewer points out, the heavy-overlap questions in Section 6.1 also include cases in which the target source is heard alone for part of the time. Those results alone therefore did not sufficiently show the performance when the target always overlaps with other sources. To examine this, we separately collected and evaluated the 427 direct-estimation questions (171 azimuth, 175 elevation, 81 distance) in which the queried source overlaps with another source for its entire active time.

Result 2: accuracy on questions whose target is always overlapped.

Always-overlapped targetSpatial-Omni-7BSONATINE 7B (SFT+GRPO)
Azimuth (acc. within 20°)11.742.7
Elevation (acc. within 10°)32.644.0
Distance (acc. within 1 m)63.074.1
Mean of the three35.753.6

Both models degrade under persistent overlap. Even in this case, however, SONATINE is more accurate than the internalized baseline, Spatial-Omni-7B, on all three tasks. Together with Table 2 of the paper, this shows that the gain of SONATINE over Spatial-Omni-7B grows as the overlap becomes more severe.

As the reviewer points out, the tools can return results for a source other than the target. The signal-processing FOA tools of SONATINE return the most dominant direction within the given window, so when the target always overlaps with other sources, the top candidate may not be the target. This is a limitation of the tools themselves. Within this limitation, however, the result still depends on the window in which the tool is called. To verify this, on the 346 direction-estimation questions among those above, we applied the same tool to three kinds of windows and compared which source the top candidate points to. The three windows are the whole recording, a random window placed on the target's active time, and the window that SONATINE 7B actually selected.

Result 3: the source that the tool's top candidate points to (%).

WindowTop-1 = targetTop-1 = co-active competitor
Whole recording8.760.1
Random window on target28.046.5
SONATINE window36.737.0

Here, a co-active competitor is another source that sounds simultaneously with the target. With the windows selected by SONATINE, the top candidate is the target most often and a co-active competitor least often. In summary, although cases in which the tool returns another source still remain under persistent overlap, the adaptive window selection of SONATINE reduces their rate from 60.1% for the whole recording to 37.0%, which is also lower than the 46.5% of a random window placed on the target. This contributes to the higher accuracy than the internalized baseline in Result 2. We will add both results to Sec. 6.1 and also state explicitly the limitation that, under persistent overlap, the tool can return the direction of another source.

We thank the reviewer once again for the valuable comments.

Thank you for the detailed rebuttal and the substantial additional experiments.

I particularly appreciate the new evaluation under persistent source overlap and the tool-level analysis. These results substantially alleviate my concern W2, demonstrating that adaptive window selection can still be beneficial even when the target always overlaps with other sources. I therefore acknowledge that my original concern should be narrowed to cases where no sufficiently target-informative window exists. However, I also want to note that the remaining source-confusion rate indicates that this limitation is not entirely resolved.

My primary concern remains W1 regarding technical novelty. The additional experiments demonstrate the importance of task-specific tool-use training, but do not sufficiently establish a methodological contribution beyond existing tool-calling and SFT/RL paradigms. As also noted by Reviewer cTy5, the current ablations do not clearly distinguish the benefits of feedback-driven adaptation from those of multiple tool calls. A comparison with a pre-planned workflow using the same tools and number of calls would help clarify this contribution.

Overall, I appreciate the authors' considerable efforts and acknowledge that the rebuttal strengthens the experimental validation. However, given the remaining novelty concern, I will maintain my original rating of 2.

Comments on the paper

1 comment

We thank all reviewers for their constructive feedback. We are encouraged that the reviewers found SONATINE well motivated and practical (E3sA), effective when many sounds overlap (E3sA, cTy5, pMBr), and supported by thorough experiments (E3sA). They also valued our training dataset (pMBr) and the flexibility of keeping spatial measurement outside the model. This lets it work with closed-source models and new microphone arrays while preserving general audio ability (cTy5, pMBr).

During the response period, we added the following experiments and analyses.

  1. We show that open-source Audio LLMs given the same tools, but not trained on SONATINE-15k, choose target-informative windows less reliably and gain little on reasoning tasks. This indicates that our dataset plays a key role in learning this behavior (E3sA).
  2. We evaluate questions whose target persistently overlaps with other sources, where SONATINE still outperforms Spatial-Omni-7B by a wide margin and its chosen windows make the tool far less likely to point to a competing source (E3sA).
  3. We restrict the model to a single tool call and find that single-source estimation is unchanged while relation-based tasks drop substantially. This suggests that measurement accuracy comes mainly from adaptive window selection, while additional tool calls support reasoning that combines several measurements (cTy5, pMBr).
  4. We analyze the conversion of six-channel recordings to FOA for encoder-based baselines and find that it loses directional information above 1 kHz and distorts the spectrum, even for a single clean source. SONATINE avoids this loss by replacing only the tool backend (cTy5).
  5. We verify that the answer extractor agrees with blind human labels on 97–98% of items and that its reward signal stays stable throughout GRPO training (cTy5).
  6. We report the cost of each training stage and the per-question latency, most of which comes from token generation rather than tool execution (cTy5, pMBr).
  7. We show that the gain over Spatial-Omni-7B remains when the distance tasks are excluded, and that it holds even with a distance regressor trained on only 10% of the data (pMBr).

We hope these results help address the reviewers’ concerns and further support our framework as an effective approach to spatial audio understanding.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.