- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies whether changing from a prediction loss to a decision-focused loss gives useful, additional directions for updating a predictor.
Its main application is sparse index tracking, which setting is not my expertise. From my understanding, a covariance predictor produces , a subset of assets is selected and a quadratic program determines the portfolio weights. The paper distinguishes two stages of gradient propagation: covariance-space geometry and parameter-space geometry.
The experiment compares multiple shrinkage settings, factor models, and neural covariance models. Paper reports small DFL gains on scalar shrinkage equity configurations, and larger gains in higher-capacity settings, and the transfer experiments involve spatial data and shortest paths.
My understanding of this paper’s overall contribution is a combination of local gradient analysis, a empirical diagnostic, and a DFL objective. However, given the many obvious numerical inconsistency, theoretical overclaim and abnormal narratives that seem to be generated directly from an LLM, I lean to a rejection.
Critical strengths
1. The question is relevant and worthwhile
Determining when DFL provides enough benefit over its training cost is a useful research problem. This paper studies the predictor as well as the downstream optimizer, specifically, an informative decision gradient can have limited influence after passing through a restricted predictor.
To me, this problem is not far-fetched and it also seems to be highly relevant for DFL researchers.
2. Main theoretical claims are sound, the figures are informative
I have checked the central chain-rule identity, rank-one result and the conditional near-rank-one bound, all appear correct under its stated assumptions. Fig. 2 and 3 also illustrate the two objectives, one predictor and misalignment very well. Moreover, the empirical study part covers multiple markets, universe sizes, sparsity levels, and predictor families.
Critical weaknesses
Despite the strengths above, the paper shows severe obvious numerical inconsistencies, vacuous theoretical guarantees, and abnormal narratives that seem to be generated directly from an LLM. Due to the 1-2 critical weakness limit of this block, I moved few weaknesses to the Question section.
1. Rank-one collinearity overclaim
This is my biggest concern regarding the theoretical and methodological part. Proposition 4 establishes a local geometric fact, but it cannot establish anything like:
MSE and task loss have the same stationary points their training trajectories converge to the same predictor
The paper acknowledges this in Appendix, but it still repeatedly provides stronger claims:
“MSE suffices” and “all paths simply converge.” the family “collapses onto two-stage training.” the theory predicts ‘the same as two-stage’. etc.
These statements do not follow the theorem. An important case is an MSE stationary point: DFL can move away from that point even when has rank one. The proposition’s restriction to two nonzero gradients does not address this situation.
2. Many numerical inconsistencies across the draft
There are major, obvious numerical contradictions between different sections describing the exact same experiments. For instance:
Structured shrinkage at K=20 is described as gaining 1.6%, p=0.002, 9/9 wins. Table 8 reports p=0.010, 8/9 wins, Figure 9 reports 1.3%, Appendix AB gives 1.26%.
For the neural model at K=20, nine folds, Table 14 gives TE 0.0391 → 0.0363, worst-fold TE 0.0652 → 0.0652. Tables 4, 13 give mean TE 0.0331 → 0.0276, worst-fold TE 0.0564 → 0.0360.
The text claims 14.5% CVaR improvement and 20.5% drawdown improvement, referring to Appendix R. But Table 14 reports 5.9% and 6.9%. Full-capacity negativeIdentity gains are 2.24% in the table and 4.13% in the appendix text.
Questions for authors
1. Abnormal text, mainly in the appendix
There are many abnormal texts, mainly focused in the appendix, which seems to be generated by an LLM without human checking. For instance:
“A reader who suspects our DFL simply fails to train has to explain why it trains here.”
“That is the weakest position a negative result can occupy: it can be dismissed without reading the proof, either as a broken DFL implementation or as a task too easy to show a gap."
“We had previously written 120× from an estimate rather than a measurement. “
“The measured gap is wider than an earlier draft of this paper claimed. “
I am wondering if these weird sentences are the draft revision history or chain-of-though from an LLM.
2. Computational cost are overstated
The approximately 3,300-fold runtime ratio compares a particular shrinkage-DFL implementation with POET. It is not a matched measurement of MSE versus DFL training cost for the same architecture.
3. The shortest-path bottleneck experiments measure the wrong Jacobian
The theory concerns whereas Appendix Y explicitly measures
A bottleneck of width bounds the rank of the input Jacobian. It does not generally bound the parameter Jacobian when the decoder is trainable.
-
Why softmax does not satisfy the simplex constraint? Appendix L says the QP enforces budget and nonnegativity exactly whereas softmax does so approximately. But mathematically, softmax satisfies both constraints exactly. The QP may provide a useful inductive bias, but this explanation is incorrect.
-
Higher rank alone does not guarantee a distinct task-gradient direction. The summary of Corollary 2 drops the essential noncollinearity assumption. I think the appendix’s formal statement is more careful/proper than the interpretation in main text.