- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how spatial audio encoders should allocate shared computation among sound event detection (SED), distance prediction (DIS), and direction-of-arrival estimation (DOA). It starts from DSpAST, whose three tasks share transformer parameters but execute transformer computation separately.
Critical strengths
-
At approximately 47.8% of the three-branch computational budget, 1.5BM_DOA obtains a mean relative improvement of 7.89% ± 1.0%. Appendix Table 7 also supports the spatial-prediction gains in absolute terms: MAE decreases from 14.67 to 13.31 and MDE from 0.628 m to 0.530 m, although mAP decreases from 54.72 to 54.14. This is a meaningful encoder trade-off rather than an efficiency claim inferred solely from parameter count.
-
The authors compare all three two-branch grouping configurations, three different depth combinations, and corresponding versions with and without a mixer, using four random seeds. The configuration with an independent DOA branch consistently achieves the highest average aggregation improvement across all test budgets, providing empirical support for its grouping design.
Critical weaknesses
-
Comparisons focus on original Spatial-AST/DSpAST and ESpAST variants. They do not establish whether early branching followed by late merging outperforms conventional early sharing followed by late branching, or existing affinity-driven sharing principles, under the same input features, pretraining, FAMO, and comparable MACs.
-
MTSL (ICML 2023) already merges structures using local task similarity. DTME-MTL (ICCV 2025) and TGLoRA (AISTATS 2026) study transformer token-space adaptation and task-related allocation of adapters, respectively. The manuscript cites these works without experimental comparisons.
-
Comparisons mainly cover Spatial-AST/DSpAST and internal variants. Existing sharing architectures are not directly compared using the same backbone and similar MAC budgets, leaving the incremental contribution unclear.
Questions for authors
-
Could you compare with existing sharing architectures using the same backbone, training settings, and similar MAC budgets?
-
What is the performance difference between original DSpAST and the modified 3B_base under a common evaluation protocol?
-
How does affinity determine task grouping and merging depth, and has this rationale been validated on independent samples or seeds?
-
- What are the performance, inference latency, and peak memory usage of a single checkpoint selected using a validation set?
4 comments
Question 3
-
Our analysis of task-wise affinity using intermediate representations and our affinity-guided task grouping were inspired by MTSL (L96). We observed consistent task-affinity trends across all four random seeds on the stage-3 training dataset and therefore reported the results averaged across seeds. We also observed the same qualitative trends on the evaluation split.
-
Task-wise affinity does not directly determine the merging depth. We evaluated different merging depths as part of our ablation study to compare the effects of:
- Early merging: Less independent processing of task-specific representations and more shared computation.
- Late merging: More independent processing of task-specific representations and less shared computation.
-
Our results suggest that maintaining a balance between task-specific and shared processing is beneficial, which aligns with a central motivation of multi-task learning. Specifically, the of DOA-separated configurations varies non-monotonically with merging depth (), both with and without mixers.
Question 4
- SpatialSoundQA does not provide an explicit validation split for checkpoint selection. We systematically evaluated all tested combinations of task groupings, merging depths, and mixer settings, reporting both favorable and unfavorable results.
- We report per-sample inference latency and peak memory usage measured on an Apple M4 GPU. Models without mixers or merging show similar peak memory usage regardless of the number of branches due to sequential branch-wise computation: all 12 transformer blocks are executed for one branch before processing the next branch.
| Metric | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MAC (G) | 147.4 | 146.5 | 98.4 | 98.1 | 82.2 | 82.0 | 70.1 | 69.9 | 57.9 | 57.8 | 49.7 |
| Latency (ms) | 332 | 311 | 213 | 208 | 180 | 176 | 153 | 151 | 125 | 124 | 105 |
| Peak memory (MB) | 379 | 358 | 360 | 358 | 364 | 360 | 364 | 360 | 363 | 360 | 354 |
Revision to the Manuscript
- We revised Section 2.1 (Related Work: Multi-Task Learning) to clarify how task-specific representations can be obtained.
List of references used in the response to Reviewer 8P9o
[1] Gurulingan, N. K., Zonooz, B., & Arani, E. (2023, July). Multi-task structural learning using local task similarity induced neuron creation and removal. In International Conference on Machine Learning (pp. 12205-12223). PMLR.
[2] Lian, D., Zhou, D., Feng, J., & Wang, X. (2022). Scaling & shifting your features: A new baseline for efficient model tuning. Advances in Neural Information Processing Systems, 35, 109-123.
[3] Jeong, W., & Yoon, K. J. (2025, October). Resolving token-space gradient conflicts: Token space manipulation for transformer-based multi-task learning. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 2887-2897). IEEE.
[4] Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2022). LoRA: Low-rank adaptation of large language models. International Conference on Learning Representations (ICLR).
[5] Gangwar, N., Rangi, A., Deshmukh, R., Rahmanian, H., Dattatreya, Y. & Kani, N.. (2026). Parameter-Efficient Multi-Task Learning via Progressive Task-Specific Adaptation . Proceedings of Machine Learning Research, 300:3916-3924
[6] Wilkinghoff, K., & Tan, Z. H. (2026, May). Dspast: Disentangled representations for spatial audio reasoning with large language models. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 14747-14751). IEEE.
[7] Vandenhende, S., Georgoulis, S., De Brabandere, B., & Van Gool, L. (2019). Branched multi-task networks: deciding what layers to share. arXiv preprint arXiv:1904.02920, (BMVC 2020)
[8] Zamir, Amir R., et al. "Taskonomy: Disentangling task transfer learning." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018
Thank you for the detailed response. I am maintaining my score of 3.
Thank you for the time and effort you devoted to reviewing our manuscript. Below, we provide our responses to the issues raised in the review.
Comparison with existing multi-task learning architectures
(Weaknesses 1, 3 and also Question 1)
Existing multi-task sharing architectures typically share early layers and introduce task-specific branches at intermediate layers, using layers with task-specific weights (MTSL [1]) or parameter-efficient fine-tuning PEFT modules such as SSF [2] (DTME-MTL [3]) and LoRA [4] (TGLoRA [5]), thereby forming a tree-like structure. However, such existing architectures are not directly applicable to our baseline (DSpAST [6]) for two reasons:
These properties make ESpAST structurally closer to an inverted-tree architecture, unlike prior MTL architectures that begin with shared layers and branch later.
Adapting these existing architectures to DSpAST [4] would require substantial architectural modifications and method-specific tuning to establish comparable training settings, constituting a separate research direction beyond the scope of our study.
Lack of comparison with cited prior multi-task learning works (Weakness 2)
Novelty beyond local task similarity (Weakness 2)
DSpAST under a common evaluation protocol (Question 2)