- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an approach to predicting task affinity for multi-task learning on tabular data. It uses dataset statistics and distributional differences to train a model on a subset of observed pairwise gains. The predicted affinities then guide task grouping. Experiments evaluate prediction accuracy, computational cost, and downstream performance.
Critical strengths
- The paper addresses a practical problem: identifying beneficial task groups without training models on all task pairs.
- The proposed approach is simple and provides interpretable predictions that can be integrated into downstream tasks.
Critical weaknesses
-
The manuscript contains many errors. For example, “conflicting gradients updates” (page 5, line 235), “to large number of tasks” (page 8, line 415), “all benchmarks datasets” (page 9, line 444), “21th” (page 11, line547), “Summery” (page 13, line 678), and “IEEE, IEEE” (page 11, line 580). The authors should revise the manuscript. Figure 1 is poorly presented, several y-axis labels are overlap. This making this figure difficult to read.
-
The main experiments cover only three datasets and two baselines. More diverse datasets and competitive baselines are need to prove the effectiveness of the proposed method.
Questions for authors
-
The hypothesis in Section 3 needs stronger justification. The same inputs can support different prediction tasks. For example, the same housing records could be used to predict rent and electricity consumption. But these tasks may depend on different features, which may not benefit from joint learning. Why should similar input distributions indicate positive transfer? Please clarify the reason of this.
-
Tables 1 and 8 appear to report the same experimental setting, but their correlation values differ. The proposed method’s Chemical result is $0.50\pm0.03$ versus , while GRAD-TAE’s Landmine result is versus . Table 4 reports Landmine’s within-task distance as , whereas the text gives . Could the authors explain these difference?
Response to Reviewer 5Gpd
W1 & Q2: Presentation & Discrepancies
We thank the reviewer for the careful reading and for catching these typos. We have corrected all the listed errors and proofread the full manuscript. We have also redesigned Figure 1 so the y-axis labels no longer overlap.
We apologize for the inconsistencies. The values in Table 8 (Chemical correlation and Landmine GRAD-TAE) were typos. Table 1 and the main text report the correct values (Chemical: +0.50 correlation; GRAD-TAE on Landmine: −0.26). We have updated Table 8 and the corresponding text in Appendix A.4. We have also corrected the Landmine within-task distance in the Appendix text (1.13 ± 0.09) so that it matches Table 4.
W2: Datasets and Baselines
We agree that a broader evaluation would strengthen the paper. However, conducting additional experiments is not feasible within the rebuttal period. We note the following regarding the scope of our evaluation: School, Chemical, and Landmine are standard tabular MTL benchmarks (Zhang & Yang, 2021). They cover both regression and classification and span different levels of inter-task divergence (Table 4). Our comparison includes three baselines: TAG and GRAD-TAE in the main text, and the Linear Surrogate model (Li et al., 2023) in Appendix A.5. Other task-grouping methods, such as Higher-Order Approximation (HOA; Standley et al., 2020) and the meta-learning-based predictor MTGNet (Song et al., 2022), require exhaustive or extensive MTL training. HOA is equivalent to the exhaustive pairwise evaluation that we use for ground truth.
Q1: Justification of the Hypothesis
Our setting assumes that all tasks share a common input schema, with each task having its own dataset (e.g., different schools, molecules, or landmine fields). We agree that similar input distributions alone do not guarantee positive transfer. For example, two tasks could have similar input distributions but substantially different input-to-label relationships, potentially leading to negative transfer. Our hypothesis is therefore not that input similarity guarantees positive transfer, but that input-distribution similarity can provide a useful signal of task relatedness.
In our setting, the hypothesis is motivated by two considerations: Theory: domain-adaptation theory (Ben-David & Borbely, 2008) bounds cross-task generalization error by the divergence between input distributions. Energy Distance and Feature-Mean Gap serve as efficient proxies for this divergence. Empirical support: these features correlate negatively with MTL gain (Appendix A.2.2), consistently in Landmine and more weakly in School, where all correlations are below 0.1 in magnitude. We therefore treat the hypothesis as a useful prior, not a universal rule. So, we learn a regression model on observed gains rather than using a single similarity score.