acceptodds
Under review as a conference paper at ICLR 2027

Learning to Rank with Weak Return Signals: Shortcut-Orthogonal Gradient Allocation for Cross-Sectional Stock Ranking

Abstract

Cross-sectional stock ranking predicts relative future returns, so learning to rank fits the task. Weighted ranking objectives emphasize comparisons based on return differences, rank positions, or ranking importance. These methods decide which comparisons matter; whether the placement of the weights can also encourage a competing ordering when the directional signal is weak has received little attention. We show that weight allocation itself can induce an objective-level shortcut preference and that, in stock ranking, return magnitude is such a competing ordering. If weighting creates this preference, it should be visible before the model learns anything. At equal scores, the loss response toward a candidate shortcut depends only on training labels and comparison weights. We propose Shortcut-Orthogonal Gradient Allocation (SOGA), a diagnose-then-correct framework that measures the excess response over uniform weighting and minimally corrects risky allocations while retaining the original comparison emphasis as far as the constraints allow. Empirically, every tested base allocation flagged by the diagnostic degrades ranking relative to uniform weighting. Across the four primary equity panels, SOGA consistently improves flagged ranking objectives, with a corrected objective surpassing every tested uncorrected published or reference objective. The correction also transfers across backbones and outperforms uniform shrinkage in matched comparisons.

Primary area:
learning on time series and dynamical systems
Keywords:
learning to rank, pairwise loss weighting, shortcut learning, cross-sectional stock ranking, financial time series, pre-training diagnostic, ensemble selection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7xUtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates how weighting pairwise comparisons in learning-to-rank models can encourage unintended ranking behaviour, particularly learning return magnitude rather than direction. The authors propose SOGA, which identifies potentially harmful weighting schemes through a pre-training gradient diagnostic and adjusts them using constrained KL-divergence optimization. The approach is evaluated on Chinese and US equities and cryptocurrency, across several ranking objectives and architectures. The authors report improvements in cross-sectional RankIC, particularly against weighted objectives that otherwise perform poorly.

Critical strengths

The paper raises an interesting question about how pairwise weighting can affect what a ranking model ultimately learns. The proposed diagnostic is relatively simple, interpretable and applicable before training. The authors also provide a reasonable number of comparisons across ranking objectives and architectures, including uniform weighting and alternative correction methods. The experiments linking ranking deterioration to return magnitude are particularly useful.

Critical weaknesses

Novelty and positioning within the literature:

I am not entirely convinced by the novelty of the proposed approach. SOGA combines a gradient-based diagnostic with constrained adjustment of pairwise weights, but the paper does not sufficiently explain how this differs from existing approaches to ranking debiasing, importance weighting and constrained learning-to-rank. The literature review is also rather limited, particularly given the financial focus. Poh et al. (2021) is cited but barely discussed, despite its relevance to cross-sectional learning-to-rank and its more comprehensive financial evaluation and being one of the first to use LTR for cross-sectional strategies (see also follow up works by the same authors). Similarly, the relationship with Kwiatkowski and Chudziak (2025), from which several objectives are taken, deserves more attention. The authors should clarify what SOGA contributes beyond these existing methodologies and how the reported findings relate to previous results.

Empirical validation and performance reporting:

I find the empirical evidence insufficient given the exclusively financial application. The three Chinese equity datasets share the same approximately 2.5-year out-of-sample period, starting in June 2020 and therefore excluding the initial COVID-19 market crash, while cryptocurrency is evaluated only from July 2024 to December 2025, excluding the 2022 crypto collapse and FTX failure. These choices raise concerns about potential cherry-picking of evaluation periods, particularly given that the proposed mechanism concerns return magnitude and may behave differently during extreme market conditions. The absence of rolling or expanding walk-forward validation, despite the availability of much longer historical datasets, makes it difficult to rule out period-selection effects. Annual walk-forward testing would be a natural robustness exercise, particularly given the sensitivity of financial ranking to changing market regimes. The paper also does not systematically report RankIC standard deviations, ICIR or multiple-testing-adjusted p-values for improvements over uniform weighting, which are relatively modest compared with gains over poorly performing weighted objectives. Finally, performance is assessed primarily through average daily RankIC against four-day or ten-day forward returns. Although supplementary top-minus-bottom spreads are provided, there is no evaluation of risk-adjusted performance, turnover or transaction costs, as commonly reported in financial learning-to-rank research. Overall, the evidence suggests that SOGA can improve certain harmful weighting schemes, but does not convincingly establish robust improvements over competitive alternatives. The empirical evaluation is also confined entirely to financial return ranking, with no evidence that the proposed diagnostic generalizes to other ranking tasks or shortcut mechanisms. Given that SOGA is presented as a methodological contribution to learning-to-rank, this limits the broader significance of the findings.

Questions for authors

Novelty and Related Work: Could the authors clarify the methodological novelty of SOGA relative to existing ranking-debiasing and constrained reweighting approaches? In particular, how do the findings relate to Poh et al. (2021) (and their follow up papers) and Kwiatkowski and Chudziak (2025), which are cited but not discussed in sufficient depth? Generalisation: Could the authors demonstrate whether SOGA generalizes beyond financial return ranking, for example to information retrieval or recommendation systems? All current experiments concern financial assets and the same return-magnitude shortcut. Testing a different ranking problem and shortcut mechanism would help establish the broader applicability of the method. Walk-forward validation and potential period selection: Why was no rolling or expanding walk-forward validation conducted? The Chinese equity test periods begin after the initial COVID-19 crash, while the cryptocurrency test excludes the 2022 market collapse and FTX failure. These choices raise concerns about potential cherry-picking of evaluation periods. Given the available historical data, annual out-of-sample testing would seem feasible. How stable are the results across different market regimes and prediction horizons? Statistical Significance: Could the authors provide RankIC standard deviations, ICIR, paired confidence intervals and Holm-adjusted p-values for the main comparisons against uniform weighting? In particular, which improvements remain statistically significant after accounting for serial dependence and multiple testing? Financial Performance: Do the reported RankIC improvements translate into better risk-adjusted financial performance? A consistent long-short portfolio comparison reporting Sharpe ratio, realized volatility, maximum drawdown and turnover, ideally net of transaction costs, would help establish the economic significance of the results.

3 comments

We thank the reviewer for the detailed feedback from a financial learning-to-rank perspective. We have expanded the evaluation with temporal validation, multiple-testing control, cost-aware portfolios and non-financial ranking tests. We also clarify the methodological distinction and related literature below.

R1. Methodological distinction from debiasing and reweighting

SOGA differs from existing approaches primarily in the source of bias it addresses. Label debiasing concerns supervision quality, importance weighting modifies the sampling or estimation distribution, and constrained ranking generally regulates the learned outputs. SOGA instead examines whether the pair-weight allocation within a specified objective rewards an unintended competing order before the learner is trained.

The diagnostic measures this preference from labels and pair weights at equal scores. Its correction removes the excess relative to a uniform reference while retaining the original objective’s intended emphasis. This is the substantive methodological contribution, supported in the submission by controlled relocation of identical weights, analysis of learned magnitude dependence and the derived objective-level correction. We will emphasise this distinction more prominently in the revised introduction and related work. The KL projection is the chosen optimisation mechanism rather than the basis of the novelty claim.

The distinction also has empirical consequences. Compared with a held-out-tuned penalty on the shortcut dependence of learned scores, SOGA improves frozen LambdaRank by 2.6–6.2 detection widths on all four markets, with Holm-adjusted significance. Correcting the objective before training therefore provides a different and more effective intervention than controlling the shortcut only in model predictions.

R2. Connection to financial learning-to-rank research

We appreciate the suggested references. They address complementary aspects of financial ranking, and we will clarify their relationship to the objective-level problem studied here.

Poh et al. (2021) compare ranking algorithms for cross-sectional investment and identify LambdaMART as a strong method on monthly NYSE data. Our work investigates a different stage of the pipeline: whether the pair weights within a ranking objective induce a competing order, regardless of the selected learner. To connect this analysis to established practice, we tested LightGBM’s built-in lambdarank. Its uncorrected objective underperforms uniform weighting by 4.0–7.8 widths across the four equity markets. Applying SOGA to the frozen LambdaRank table on the same learner recovers 3.6–7.5 widths, significant after Holm correction. This establishes the relevance of the diagnostic to a widely used ranking implementation.

Kwiatkowski and Chudziak (2025) compare ranking losses for stock selection, including formulations that emphasise high-ranked observations. Our weighted hinge v1 and v2 instantiate this type of emphasis, and the diagnostic flags both. In the walk-forward, SOGA raises weighted hinge v1 RankIC from 0.0254 to 0.0880 on CSI300 and from −0.0413 to 0.0609 on US. Their work establishes the practical importance of loss selection, and ours identifies a specific failure mechanism in pair-weight placement and provides a targeted repair. We will incorporate this relationship into the related-work discussion.

R3. Walk-forward evaluation and period robustness

To address temporal robustness, we added an annual expanding-window evaluation across 25 equity market-years: CSI300 and CSI500 (2015–2022), CSI1000 (2017–2022), and US (2018–2020). Each test year uses earlier data only, separated by a (k+1)-day gap to avoid overlapping label horizons. The diagnostic, response tolerance and correction are recomputed from each training fold, so no test-period labels enter these steps.

SOGA improves weighted hinge v1 RankIC in every one of the 25 market-years, including periods of pronounced market disruption. Frozen LambdaRank is likewise repaired in every evaluated year. The pooled comparisons below quantify the improvement and place it alongside the uniform-weighting reference.

Walk-forward testUniform RankICHinge v1 RankICHinge v1 + SOGASOGA vs uniform, widths (Holm p)
CSI300, 2015–20220.07970.02540.0880+2.82 (0.004)
CSI500, 2015–20220.09010.01000.0934+1.35 (0.18)
CSI1000, 2017–20220.0915−0.00770.0947+1.28 (0.18)
US, 2018–20200.0366−0.04130.0609+5.41 (< 10⁻⁶)
CSI panels pooled———+1.99 (0.004)

The consistent improvements over uncorrected weighted objectives support robustness across market conditions and both label horizons (four days for CSI, ten days for US). Against uniform weighting, SOGA has positive RankIC differences on all four markets, with Holm-adjusted significance on CSI300, US and pooled CSI. We will report yearly and pooled results to make both aspects of the evidence clear.

R4. Statistical reliability and uniform-weighting comparisons

We have strengthened statistical reporting through paired daily analyses, within-period variability and seed-level variation. For every objective-market combination, we will report mean RankIC, daily standard deviation, ICIR and across-seed standard deviation, together with paired block-bootstrap confidence intervals, Newey–West tests and Holm adjustment within prespecified comparison families.

All 16 original single-split repairs relative to uncorrected weighted objectives are Holm-significant. Against uniform weighting, weighted hinge v1 + SOGA has higher mean single-split RankIC on all four markets, although these smaller differences are not Holm-significant. IC and winsorised IC exceed uniform significantly on all four single-split markets, and longer walk-forward RankIC comparisons are significant on CSI300, US and pooled CSI. These results substantiate the primary objective-repair claim while accurately distinguishing it from the comparison against a strong uniform baseline.

As a reference for variability, uniform weighting on CSI300 has mean daily RankIC 0.0632, daily standard deviation 0.141 and ICIR 0.45. Reporting this context alongside confidence intervals allows the magnitude and reliability of the improvements to be assessed directly.

R5. Cost-aware, risk-adjusted portfolio evaluation

We added a matched walk-forward long–short backtest for all objectives. Predictions made at the close of day t are traded at the close of t+1, with top-decile longs, bottom-decile shorts and k staggered holding sub-portfolios. We charge 10 basis points per side. Suspended A-share entries and those near the price limit (±9.5%) are excluded uniformly across methods.

Under this protocol, weighted hinge v1 + SOGA produces higher post-cost Sharpe ratios than uniform weighting on all four markets, with the CSI300 confidence interval including zero:

MarketUniform SharpeSOGA SharpeDifference [95% CI]
CSI3002.492.79+0.30 [−0.01, +0.60]
CSI5003.193.78+0.59 [+0.24, +0.94]
CSI10003.424.04+0.62 [+0.22, +1.03]
US0.581.05+0.47 [+0.06, +0.89]

The additional risk metrics are also informative. Maximum drawdown improves on CSI300 (−15.6% versus −21.2%) and US (−25.0% versus −28.4%), and daily turnover is less than three percentage points above uniform weighting on each market. Much of the gain comes from the short leg, making short-sale availability and realised costs relevant to implementation. We will include these considerations while retaining the clear improvement under the reported evaluation protocol.

R6. Generalisation beyond financial ranking

SOGA’s diagnostic is defined by the response of a pair-weighted objective to a specified competing ordering, not by financial return magnitude alone. Its use in another ranking task requires a suitable candidate ordering, while the diagnostic and correction remain unchanged.

On KuaiRec recommendation ranking, we define magnitude as distance from the mean relevance label within a list. Three of four flagged tables exhibit harmful learned behaviour, and SOGA repairs these cases by 2.5–4.1 detection widths. This provides direct evidence that the same objective-level issue and correction can arise beyond financial ranking.

The competing ordering must oppose the task target: standard graded relevance in MSLR is aligned with that target and therefore is not itself a shortcut under our definition. We will clarify this criterion, distinguishing the general diagnostic formulation from the task-specific selection of a candidate shortcut.

R7. Cryptocurrency stress-period evaluation

We extended the cryptocurrency experiment to annual walk-forward test years 2022–2025, including the Terra/Luna and FTX disruptions. The equal-weight panel declined 75.7% in 2022. To handle disrupted trading histories consistently, returns spanning gaps longer than five days or documented unit changes are marked missing according to rules fixed before training.

Across these years, SOGA raises weighted hinge v1 pooled RankIC from −0.0144 to 0.0575, approximately 10 detection widths (Holm-significant), close to uniform weighting at 0.0585. The improvement therefore persists through severe stress periods and restores performance to approximately the uniform baseline. The central finding is robust repair of a harmful weighted objective, rather than an assertion of superiority to uniform weighting in every subperiod.

In 2022 alone the two are level, 0.3 detection widths apart, and in the Terra/Luna and FTX windows SOGA is slightly above uniform weighting (by 0.0002 and 0.0016 RankIC). The extended study provides an additional test of the proposed mechanism in a distinct, highly volatile market. We will include the cryptocurrency walk-forward and the stress-window results in the revision.

Reviewer_fE6iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is focused on how comparison weighting in learning-to-rank can lead to less than desirable return magnitude shortcuts in stock rankings. The authors propose a way to diagnose these preferences before training and corrects the weights using constrained optimization. Experiments across several datasets and ranking models show improvements over problematic weighted objectives, although gains over uniform weighting are generally modest.

Critical strengths

S1) The authors uncover an interesting issue where comparison weights in ranking objectives can unintentionally encourage learning magnitude rather than the intended ordering.

S2) The proposed diagnostic can identify this issue before training, without needing to fit a model, which is important.

S3) They provide a principled correction while trying to preserve the original weighting structure; generally easy to use and seems mostly model-agnostic.

S4) Generally, the empirical analysis is sufficient with considering across ranking objectives, datasets, and architectures, with additional comparisons against alternative corrections.

Critical weaknesses

W1) The method novelty is there, but limited for ICLR. Besides the diagnostic, the correction is largely standard constrained reweighting, and it is unclear how general the approach is beyond this specific shortcut.

W2)The diagnostic relies on the loss response at equal prediction scores; however, this may not reflect what happens throughout training. It is unclear how reliably this initial preference predicts the final learned ranking.

W3) Much of the improvement comes from fixing weighted objectives that perform poorly compared to uniform weighting. The gains over uniform weighting are generally small... this makes the practical benefit less impactful.

W4) The main equity datasets share similar market conditions and results primarily reported using RankIC. See question below.

Questions for authors

Q1) Is there a way to include varied market conditions expanding on W4?

Q2) Can this framework generalize to other types of ranking shortcuts, or does each require a separately designed diagnostic and correction?

Q3) How does the pre-training diagnostic correlate with shortcut behavior after training? Are there cases where this initial diagnostic is misleading?

Q4) Given the relatively small gains over uniform weighting, do your improvements translate to meaningful practical benefits, such as downstream portfolio returns after transaction costs, etc.?

[Note that my score is flexible based on the discussion and your response to these questions along with the comments from other reviewers.]

3 comments

We thank the reviewer for the detailed assessment of novelty, diagnostic reliability and empirical support. We have undertaken additional controlled comparisons, temporal tests and cross-domain evaluations. Unless noted otherwise, new model comparisons use five paired seeds, and detection widths are twice the standard error of paired daily RankIC differences, as defined in the submission.

W1. Methodological novelty and contribution

We would clarify that SOGA’s central methodological contribution is the diagnosis and correction of a previously overlooked property of weighted ranking objectives: the placement of pair weights can favour an undesirable competing ordering, even with unchanged target labels. Rather than only determining which comparisons deserve emphasis, SOGA quantifies this unintended preference from labels and weights before training and corrects its excess while preserving the intended emphasis. The KL projection implements this targeted correction. The novel contribution lies in the objective-level diagnosis and resulting formulation.

For example, with mean-centred returns, the pairs (3, 1) and (−1, −3) contribute +2 and −2 toward magnitude ordering. Equal weights cancel this response, whereas unequal weights can make magnitude ordering preferable without changing either label. This example isolates the failure mechanism and motivates why inspecting pair-weight placement before training provides information unavailable from the weighting rule alone.

The submitted paper already establishes this mechanism through the controlled studies, mathematical diagnostic and correction. We will strengthen its presentation in the introduction and related-work discussion so that the specific novelty is immediately apparent, rather than being obscured by the standard optimisation machinery.

The added experiments further demonstrate the value of correcting the objective itself. Against a shortcut-alignment penalty applied to learned scores and tuned on held-out data, SOGA improves frozen LambdaRank by 2.6–6.2 detection widths across all four equity markets, with Holm-adjusted significance and consistent gains over five seeds. Table 3 shows a corresponding benefit for the top-only objective. This supports the methodological distinction between preventing a shortcut incentive in the loss and penalising its manifestation after training.

W2 / Q3. Reliability of the pre-training diagnostic

The diagnostic is designed to identify uncorrected pair-weight tables whose training objective favours magnitude ordering. We evaluated whether this pre-training signal anticipates the undesirable behaviour of the resulting trained models, using flags computed solely from training labels.

Among 57 uncorrected weight tables, all 41 flagged cases exhibit a greater worst-decile share after training than uniform weighting, compared with 1 of 16 unflagged cases. Across 152 trained configurations, pre-training displacement correlates with post-training worst-decile share at +0.82 (95% cluster-bootstrap CI [0.75, 0.91]), with within-market correlations of +0.75 to +0.82. These results extend the main paper’s 22-of-22 flagged cases and show that the diagnostic is predictive of learned behaviour, rather than merely describing the initial weights.

For corrected tables, interpretation of the binary flag requires care: SOGA constrains aggregate displacement, while the screen evaluates two response components separately. Consequently, a corrected table may satisfy the correction tolerance yet remain flagged at the boundary. We will clarify this distinction in Section 4.2 and report the bottom-only KuaiRec case as a limitation of the screening criterion, without conflating it with the effectiveness of the proposed correction.

W3. Comparison with uniform weighting

Uniform weighting is an important reference because it avoids the problematic emphasis by construction. SOGA addresses a different objective: retaining useful non-uniform emphasis while removing the part that rewards a competing ordering. Its advantage over uniform weighting is therefore complementary evidence to the primary claim of repairing harmful weighted objectives.

On the original approximately 2.5-year test splits, weighted hinge v1 + SOGA improves mean RankIC over uniform weighting on all four markets, although the smaller differences are not Holm-significant. We therefore added annual expanding-window walk-forward evaluation, recalculating the diagnostic and correction from each fold’s training data. Against uniform weighting, weighted hinge v1 + SOGA improves pooled RankIC from 0.0797 to 0.0880 on CSI300 (+2.82 widths, Holm p = 0.004) and from 0.0366 to 0.0609 on US (+5.41 widths, p < 10⁻⁶). The pooled CSI result is also significant (+1.99 widths, p = 0.004). CSI500 and CSI1000 show positive but individually non-significant differences. The longer evaluation thus provides stronger support for the benefit of preserving well-directed weighting emphasis.

Walk-forward testUniform RankICHinge v1 + SOGA RankICSOGA vs uniform, widths (Holm p)
CSI300, 2015 to 20220.07970.0880+2.82 (0.004)
CSI500, 2015 to 20220.09010.0934+1.35 (0.18)
CSI1000, 2017 to 20220.09150.0947+1.28 (0.18)
US, 2018 to 20200.03660.0609+5.41 (< 10⁻⁶)
CSI panels pooled+1.99 (0.004)

The repair effect is substantial even for a widely used objective. LightGBM’s built-in lambdarank is 4.0–7.8 detection widths below uniform weighting on the four markets, and SOGA recovers 3.6–7.5 widths over the uncorrected frozen LambdaRank table on the same learner, with Holm-adjusted significance. These results demonstrate both the practical relevance of objective-induced shortcuts and the effectiveness of SOGA’s targeted repair. We will include the walk-forward table and the LightGBM comparison in the revision.

W4 / Q1. Robustness across market regimes

To assess whether the observed repair depends on the original test window, we added annual expanding-window evaluation over 25 equity market-years: CSI300 and CSI500 (2015–2022), CSI1000 (2017–2022), and US (2018–2020). The periods include the 2015 A-share crash, 2018, the 2020 US shock and 2022, providing substantially different market conditions.

SOGA improves RankIC over uncorrected weighted hinge v1 in all 25 market-years, and also repairs frozen LambdaRank in every evaluated year. Pooled over the years, SOGA lifts RankIC from 0.0254 to 0.0880 on CSI300, from 0.0100 to 0.0934 on CSI500, from −0.0077 to 0.0947 on CSI1000 and from −0.0413 to 0.0609 on US, and it is above uniform weighting in 7 of 8 years on CSI300, 5 of 8 on CSI500, 5 of 6 on CSI1000 and 3 of 3 on US. The table below shows the US years, which include the 2020 shock. This consistency indicates that the weight-induced failure and its correction are not confined to one market regime or evaluation period.

US test yearUniform RankICHinge v1 RankICHinge v1 + SOGA RankIC
20180.0381−0.04100.0576
20190.0480−0.04340.0711
2020, including the shock0.0210−0.03920.0524

We also extended cryptocurrency walk-forward testing to 2022–2025, covering the Terra/Luna and FTX disruptions and the 75.7% decline of the equal-weight panel in 2022. SOGA improves the uncorrected weighted hinge v1 by approximately 10 detection widths in pooled RankIC (Holm-significant) and reaches a result close to uniform weighting. These additional evaluations support the robustness of the objective repair under severe market stress. We will include the yearly results and the cryptocurrency stress test in the revision.

Q2. Extension to other shortcuts and ranking tasks

The diagnostic formulation extends beyond return magnitude: the response can be computed for any specified candidate ordering that may conflict with the ranking target. Applying it to another task therefore requires identifying a meaningful competing ordering, but not redesigning SOGA. Automatically discovering every possible shortcut is a separate research direction beyond the present scope.

For equity ranking, replacing magnitude with 20-day volatility flags the same weight tables across all four markets. Magnitude correction alone brings 11 of 16 corrected tables within one response width for volatility. Adding a volatility constraint changes individual weight cells by at most 0.075% and yields no significant RankIC change. The result suggests that SOGA is not tied to a single measurement of the undesirable preference.

On KuaiRec recommendation ranking, with magnitude defined as distance from a list’s mean label, three of four flagged tables exhibit adverse learned behaviour, and SOGA repairs them by 2.5–4.1 widths. Importantly, the candidate order must genuinely compete with the target. Item popularity in this KuaiRec setting and standard graded relevance in MSLR do not meet that criterion. Together, these tests demonstrate transfer to another ranking domain while establishing the necessary task-specific definition of a shortcut. We will state this criterion, with the volatility and KuaiRec results, in the revision.

Q4. Portfolio performance after transaction costs

We appreciate the reviewer’s emphasis on economic relevance. We therefore tested whether improvements in ranking quality carry through to risk-adjusted portfolio performance under a consistent walk-forward trading protocol and explicit transaction costs.

All objectives use the same long–short rule: long the top predicted decile, short the bottom decile, hold for k days in staggered sub-portfolios, and trade at the close of t+1 using scores from the close of t. We charge 10 basis points per side and apply identical A-share eligibility restrictions. Post-cost Sharpe ratios for weighted hinge v1 + SOGA exceed uniform weighting on all four markets: 2.79 versus 2.49 (CSI300), 3.78 versus 3.19 (CSI500), 4.04 versus 3.42 (CSI1000), and 1.05 versus 0.58 (US). The 95% confidence intervals exclude zero for three of the four differences.

MarketUniform SharpeSOGA SharpeDifference [95% CI]
CSI3002.492.79+0.30 [−0.01, +0.60]
CSI5003.193.78+0.59 [+0.24, +0.94]
CSI10003.424.04+0.62 [+0.22, +1.03]
US0.581.05+0.47 [+0.06, +0.89]

Thus, the improved rankings also yield higher long–short risk-adjusted returns under the evaluated cost assumption. The gains arise predominantly from the short leg. We will therefore report the effects of short-sale constraints and trading costs when discussing practical implementation. This evidence complements RankIC with a directly relevant investment measure.

Reviewer_cXTNOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how loss weights affect stock ranking. Some weights may encourage a model to rank stocks by return size, not by return direction. SOGA checks this problem before training, when all scores are equal. It then adjusts the weights while keeping them close to the original ones. The authors test the method on four equity panels, eight architectures on CSI1000, and a new cryptocurrency panel. The results show that SOGA can repair several harmful weighting schemes.

Critical strengths

The controlled experiments help explain how weight placement can hurt ranking. Figure 1 keeps the inputs, model and budget fixed and rearranges the same weight values. Figure 2 adds a magnitude-trained model and comparisons restricted to similar magnitudes. Together, these give useful evidence for the proposed failure mechanism. This is my main reason to support acceptance.

Critical weaknesses

The comparison with simple shrinkage is incomplete. Appendix H (p. 22) says the trained shrinkage control in Table 7 stops at SOGA's coverage centroid, while its shortcut response remains 28–43 detection widths outside the allowed band. This shows an advantage over that coverage-matched version, but not over shrinkage meeting the same response tolerance. Figure 4 shows a same-tolerance weight table for top-only, which is a different base from the frozen LambdaRank used in Table 7. I therefore cannot yet tell whether SOGA gives better trained results than this simpler response-matched alternative.

Questions for authors

For frozen LambdaRank, do you have trained results for uniform shrinkage that meets the same shortcut-response tolerance as SOGA? If available, please report RankIC, response, and coverage for both methods. This would clarify whether the more complex correction is needed.

1 comment

We thank the reviewer for the encouraging assessment and for recognising the controlled evidence in Figures 1 and 2. The suggested response-matched shrinkage comparison directly tests whether SOGA’s gains arise simply from moving pair weights toward uniform. We have therefore added this experiment on frozen LambdaRank, using five paired seeds.

Q1. Response-matched shrinkage and preservation of objective emphasis

We appreciate the importance of comparing both approaches under the same response criterion. The new results show that SOGA meets the prescribed tolerance while retaining substantially more of the original pair-weight emphasis than shrinkage. This difference also improves the learned ranking.

Both methods start from the same frozen LambdaRank weight table. We move the shrinkage baseline toward uniform weighting until it first satisfies the one-width magnitude-response tolerance imposed on SOGA, and separately check coverage. Shrinkage meets the coverage floor except for shortfalls of at most 0.001 on CSI300 and CSI500. Unlike the earlier Table 3 comparison, which stopped the shrinkage path at the coverage centroid of the SOGA solution, this experiment explicitly matches the response criterion. We will clarify the distinction in the revision.

Under this common tolerance, shrinkage traverses more than 99.99% of the KL path toward uniform, leaving every weight within 1.3% of its uniform value. SOGA reaches the same tolerance after only 24–31% of that path. Thus, shrinkage largely eliminates the original weighting structure, whereas SOGA selectively corrects the shortcut preference while preserving the objective’s useful pair emphasis.

We assess the trained models using RankIC and worst-decile share. The latter measures the fraction of the 10% largest losers placed among a model’s top 10% predictions. Lower values indicate less of the undesirable selection behaviour illustrated in Figure 2.

CSI300CSI500CSI1000US
Shortcut response (widths), SOGA / shrinkage1.00 / ≤ 1.001.00 / ≤ 1.001.00 / ≤ 1.001.00 / ≤ 1.00
Coverage floor, SOGA / shrinkagemet / short by 0.001met / short by 0.0005met / metmet / met
KL distance travelled, SOGA / shrinkage30.5% / > 99.99%31.2% / > 99.99%30.1% / > 99.99%23.9% / > 99.99%
Worst-decile share, SOGA5.9%6.9%4.8%5.5%
Worst-decile share, shrinkage9.2%7.7%5.0%11.9%
Worst-decile share, uniform weighting8.8%7.8%5.2%7.4%
RankIC, SOGA minus shrinkage (widths)+1.74+1.73+0.78+1.55

SOGA achieves higher RankIC than response-matched shrinkage on all four markets, with a lower worst-decile share than both shrinkage and uniform weighting in every case. On CSI300 and CSI500, the RankIC improvements exceed one detection width and hold for all five paired seeds. These results support the benefit of retaining objective-specific emphasis while correcting its harmful response.

The result also persists in an eight-year CSI300 walk-forward (2015–2022), with shrinkage recalibrated from each fold’s training data. SOGA achieves pooled RankIC 0.0858 versus 0.0783, a gain of 1.64 detection widths across all five seeds. Accordingly, under the same base objective and response tolerance, SOGA provides a more selective and empirically stronger correction than global shrinkage. We will include this comparison in the revised paper.

Comments on the paper

1 comment

Dear Reviewers and AC,

Thank you for taking the time to review our paper. As the discussion is coming to a close, we would like to briefly summarise the main contribution and the additional evidence from our responses.

Our work shows that the placement of pairwise weights alone can make a ranking loss favour ranking stocks by the magnitude of their returns rather than their direction, even when the labels remain unchanged. Importantly, this can be checked directly from the labels and weights, before any model is trained. SOGA performs this check and makes the smallest necessary adjustment to the weights to remove the problem. In our responses, we have addressed the reviewers' concerns from five aspects: how SOGA differs from simpler corrections, whether the diagnostic predicts post-training behaviour, its consistency across time and markets, its generality beyond financial ranking, and its practical value.

  • SOGA goes beyond simply shrinking weights towards uniform. Reaching the same tolerance through uniform shrinkage requires every weight to be within 1.3% of uniform. SOGA achieves the same tolerance with only 24% to 31% as much movement, while obtaining higher RankIC on all four markets. Correcting the loss before training also outperforms a penalty on the model's scores, tuned on held-out data, by 2.6 to 6.2 detection widths on every market. Unlike label debiasing, importance weighting and output constraints, SOGA directly addresses what the weighting scheme itself rewards.
  • The pre-training diagnostic predicts post-training behaviour. All 41 flagged tables placed more of the worst-performing stocks in their top 10% than uniform weighting did, compared with only 1 of the 16 unflagged tables.
  • The results are consistent across time and markets. Across 25 market-years, including the 2015 crash, the 2020 COVID shock and 2022, SOGA improves the flagged losses in every year. We observe the same pattern in crypto, including the Terra/Luna and FTX periods.
  • The approach extends beyond return magnitude and financial ranking. The diagnostic and correction remain unchanged. Only the competing ordering needs to be specified. Using volatility instead of return magnitude identifies the same problematic tables. On KuaiRec, SOGA corrects all three flagged tables that show degraded performance, improving them by 2.5 to 4.1 widths. In MSLR, graded relevance does not create the same ordering shortcut because it does not compete with the target. The correction also works across eight backbones and with LightGBM.
  • The improvements translate into practical gains. LightGBM's built-in LambdaRank, based on the LambdaMART-style approach identified as strongest by Poh et al., performs 4.0 to 7.8 widths worse than uniform weighting, and SOGA recovers most of this gap. SOGA also outperforms uniform weighting itself on all four markets in the walk-forward evaluation, with statistically significant improvements on CSI300, the US market and the pooled CSI panels. Its post-cost long-short Sharpe ratio is also higher on all four markets.

Further details are provided in our individual responses. We will incorporate these results into the revised paper and hope they clarify both the contribution and the practical value of SOGA.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.