- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the severe imbalance problem in continuous regression, using financial sentiment regression as a representative testbed, and identifies two coupled challenges: at the data level, labels are concentrated in the neutral region, causing predictions to collapse toward neutrality during MSE fine-tuning (neutral collapse); at the task level, the auxiliary classification task interferes with regression, and the two tasks converge at asymmetric rates, making it impossible to optimize both simultaneously with fixed task weights.
The authors propose the SAS framework: DSP parameterizes partition thresholds in the continuous label space as trainable variables, jointly learning the auxiliary classification partitions through soft assignment and three regularization terms; TCB dynamically adjusts task weights based on windowed convergence speed derived from EMA loss trajectories, and reweights auxiliary classes according to class-specific losses. Both components are jointly optimized within a single loss function and one backward pass, with the auxiliary head discarded after training, incurring no additional inference cost.
Experiments cover 3 private financial, 3 public financial, and 4 cross-domain datasets, involving 7 backbones and 12 baselines, including main experiments, ablation studies, per-region MSE analysis, imbalance sweep, boundary condition evaluation, and cost analysis.
Critical strengths
- The problem motivation is clear, and both challenges are directly supported by empirical evidence.
- The method design is reasonable and the engineering is complete.
- The experimental coverage is comprehensive.
- The ablation and boundary analysis are honest.
- The cross-domain validation design is appropriate.
Critical weaknesses
Scope misalignment and missing downstream evaluation.
The abstract and introduction heavily emphasize financial sentiment analysis as the core motivation and application scenario. Because financial sentiment analysis typically serves as an upstream component of quantitative trading systems in practice, readers naturally expect the subsequent experiments to include downstream evaluations—such as trading simulations, backtests across stock markets, or the translation of MSE reductions into trading win rates and profitability. However, the paper only reports standard ML regression metrics (MSE, MAE, Pearson r) and auxiliary classification metrics, without any downstream trading or economic evaluation. Furthermore, the Limitations section fails to explicitly clarify this boundary.
The claimed MSE improvement is marginal, statistically questionable, and comes at a high cost.
The authors highlight a "10.5% MSE reduction," but this number is measured against a weak baseline (Fixed-MTL). If you compare it to the Single-task model, the gain shrinks to only about 6.3%. More critically, when compared to the strongest competing method (DWA), the difference is just 0.0002 (0.0193 vs. 0.0195). The authors mention in §5.1 that the multi-seed standard deviation is up to 0.0008 MSE. This means the gap between SAS and DWA is 4 times smaller than the seed noise itself, making it highly likely to be statistically insignificant. Although the authors state they use Holm-Bonferroni tests "where reported," the main tables contain no significance markers (e.g., p-values or asterisks) for these comparisons. Thus, readers cannot verify if SAS truly outperforms DWA on regression, or if the gain is just random seed variance.
- On top of that, Table 27 shows SAS requires 35.7% more training time than DWA (38h vs. 28h). Spending nearly 1.5 times the training cost for an MSE drop that cannot be statistically distinguished from seed noise is a very poor trade-off. The paper needs to provide a clearer cost-accuracy Pareto analysis or explicitly justify why this extra overhead is worth it.
Questions for authors
Same to Weaknesses.
3 comments
(continued)
Table R4: Paired t-test of SAS against the closest of the twelve baselines.
| Dataset | Mean difference | Cohen's d | Holm-corrected p |
|---|---|---|---|
| NEU | 0.0003 | 3.00 | 0.0051 |
| FXE | 0.0003 | 1.90 | 0.0132 |
| INV | 0.0008 | 5.06 | 0.0010 |
Following your suggestions, we have added these p-values to Appendix A.12.4 (Lines 1995–1997, 2035–2037 and 2107–2122).
Summarized W3: Cost-accuracy trade-off against DWA
A3: We respectfully clarify that the extra training time of SAS is a one-time cost, and SAS keeps the inference cost of the original backbone, since the auxiliary head is discarded after training. To show the cost-accuracy trade-off, we combine the training time and memory in Table 33 with the MSE on NEU in Table 29. Table R5 shows that SAS lies on the cost-accuracy Pareto front with the lowest MSE, together with Single-task and DWA, where the 10 extra hours over DWA are spent once in training, and the gain over DWA on NEU is significant on the test split as shown in Table R4.
Table R5: Training time (one seed, summed over NEU, FXE, and INV), memory, and validation MSE on NEU.
| Method | Training time (h) ↓ | Memory (GB) ↓ | MSE ↓ | Pareto front |
|---|---|---|---|---|
| Single-task | 24.0 | 22.1 | 0.0206 | ✅ |
| Fixed-MTL | 27.0 | 22.4 | 0.0216 | ❌ |
| DWA | 28.0 | 22.5 | 0.0195 | ✅ |
| LDS | 29.0 | 22.6 | 0.0201 | ❌ |
| FAMO | 32.0 | 23.0 | 0.0198 | ❌ |
| SAS (Ours) | 38.0 | 23.8 | 0.0193 | ✅ |
Following your suggestions, we have revised Appendix A.13 (Lines 2128–2130 and 2174–2181) to present this Pareto analysis.
Best Wishes,
The Authors of Paper 53203
Dear Reviewer dqmS,
Thank you for your time in reviewing our work, and for recognizing its clear motivation and comprehensive experiments.
We believe that our rebuttal has addressed your concerns, with the downstream trading simulation, the per-seed significance tests, and the cost-accuracy comparison. As suggested, we will keep these experiments in the final version.
Thank you again for your constructive feedback.
Best Wishes,
The Authors of Paper 53203
Response to Reviewer dqmS
Dear Reviewer dqmS:
Thank you for recognizing the clear motivation, the method design, and the comprehensive experiments. As your questions are the same as the weaknesses, we respond to the weaknesses W1 to W3 below. We have responded point by point to your concerns regarding downstream evaluation, significance tests, and cost report. In the marked copy appended to the revised PDF (pages 47–94), the revisions made in response to your comments are tagged with @Reviewer dqmS. We hope our responses resolve your concerns and look forward to further discussion.
Summarized W1: Add downstream trading evaluation and clarify the Limitations
A1: Following your suggestion, we further evaluate the downstream financial decisions with a daily trading simulation on EUR/USD, where the average predicted sentiment of each day's texts in the held-out test split of NEU sets a long, short, or flat position for the next day, on metrics (including hit rate, annualized return, and Sharpe ratio). Table R1 shows that SAS reaches the highest hit rate, annualized return, and Sharpe ratio of the four methods in this simulation, as its Sharpe ratio reaches 0.66 against 0.47 for DWA, the strongest of the three baselines.
Table R1: Downstream trading simulation on EUR/USD.
Following your suggestions, we have added these results to Appendix A.22 (Lines 2415–2425 and 2474–2479), and updated the Limitations in Appendix A.0 (Line 809) to state that backtests across markets remain future work.
Summarized W2 (1): Headline gain against Single-task and the strongest baseline
A2 (1): We respectfully clarify that Fixed-MTL is the headline reference because it shares the backbone and the auxiliary head of SAS and differs in DSP and TCB alone, which isolates the contribution of the two components. To report the gain against the other references as well, Table R2 lists the MSE reduction of SAS over Fixed-MTL, Single-task, and the strongest baseline (i.e., DWA on NEU and FXE, and FAMO on INV) on the original validation split of NEU, FXE, and INV in Table 29 (Lines 2053–2075).
Table R2: MSE reduction of SAS over three references.
Following your suggestions, we have revised Appendix A.12.4 (Lines 2039–2042) to report these validation reductions, the conclusion (Lines 478–485) to report the corresponding test-split reductions over Single-task and the strongest of all compared methods, and Table 3 (Lines 383–392) to add the change against the best other baseline on the other financial and cross-domain datasets.
Summarized W2 (2): Significance of the gain over DWA within seed noise
A2 (2): To check whether SAS truly outperforms DWA beyond seed variance, we report the per-seed MSE of SAS and the closest of the twelve baselines (i.e., DWA on NEU, and FAMO on FXE and INV) on a held-out test split of NEU, FXE, and INV (i.e., selecting on 10% of the training data and reporting the original validation split once as the test split), together with a two-sided paired t-test with Holm correction over the three datasets. The methods share the split and initialization of each seed, and the paired test compares their MSE within each seed. Table R3 shows that SAS has a lower MSE than the closest of the twelve baselines in each of the five seeds on the three datasets.
Table R3: Per-seed test MSE of SAS and the closest of the twelve baselines.
Table R4 shows that the gain over the closest of the twelve baselines is significant after Holm correction on the three datasets, as the corrected p-value is 0.0051 on NEU.