- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the gap between market forecasting and portfolio construction by directly generating joint signed allocations from raw multi-asset histories. It combines a frozen Kronos encoder with a cross-asset flow Transformer and uses imitation learning to initialize plausible portfolio proposals. It then refines these proposals with portfolio-level rewards to improve trading performance.
Critical strengths
- The paper reports substantial performance gains after RL refinement across several evaluation metrics.
- The paper provides relatively detailed descriptions of the training, inference, and evaluation procedures.
- The proposed method is clear and easy to understand.
Critical weaknesses
- Unclear motivation for the use of flow matching. Some statements in this manuscript are overstated.
- Insufficiently justified method design, especially the RL formulation and optimization objective.
- Insufficient empirical validation, including weak controls and limited market coverage.
Questions for authors
- The motivation for using flow matching is unclear. The paper does not establish why flow matching is needed for portfolio allocation. Continuous joint actions and stochastic candidate generation can also be achieved with simpler stochastic Transformer policies.
- The Stage 2 formulation is closer to a contextual bandit than dynamic portfolio RL. The action does not affect future market states, and current portfolio holdings are not included in the policy input.
- The RL training objective is misaligned with deployment. Training optimizes the Sharpe ratio of a fixed 24-hour portfolio with normalized gross exposure, while deployment generates hourly portfolios with variable exposure and transaction costs.
- The empirical baselines and controls are insufficient. The current comparisons do not isolate the contribution of flow matching. Stronger matched stochastic-policy baselines under the same RL objective and model capacity are needed.
- The use of policy-gradient-based Flow-GRPO is insufficiently justified. Since both the financial objective and the flow generation process are largely differentiable, the paper should justify why policy-gradient optimization is preferable to direct differentiable optimization.
- The experiments only consider cryptocurrency markets. Evaluation on broader equity universes, such as the Nasdaq-100 or S&P 500, is needed to support generalization.
6 comments
Q2. Contextual-bandit formulation of stage 2
Stage 2 is deliberately formulated as a contextual bandit over target books. As in standard portfolio RL for liquid markets (e.g., Jiang et al., 2017), the policy is a price taker and its trades do not move future prices. Holdings therefore affect outcomes only through trading costs, which the ledger charges on every change of the target book. Under this formulation the reward measures the quality of a complete book without the credit-assignment noise of long, path-dependent rollouts. We state this formulation explicitly in Section 3.3 of the revised manuscript.
AllocFlow also extends directly to the closed-loop setting: with the current holdings as input, the closed-loop policy reaches a Calmar of 4.59 with a turnover of 1.38, compared with 4.043 and 1.976 for AllocFlow (table under Q3). We start from one of the Flow-GRPO policies (seed 42) and give its action head the current holdings as an additional input. For each asset, six holdings features (the position, its signed logarithm, whether the asset is held, and the gross and net exposure of the book) are embedded by a two-layer MLP and added to the action token of that asset. The last layer of this MLP is zero-initialized, so training starts exactly from the trained policy. Training runs for 2000 steps with a learning rate of 1e-5 and a batch size of 16.
Q3. The RL training objective and deployment
Putting deployment terms into the reward lowers the deployed performance. With gross exposure in the reward, the Calmar falls from 4.043 to 2.76, and with fees in the reward, it falls to 1.98.
The reward is a controlled comparison (Section 3.3): the eight candidate books of the same market state are held over the same 24-hour path at the same gross exposure, so their reward differences come only from their composition, not from their leverage. Every reported result is measured on the deployed ledger, where the policy generates a new book with its own gross exposure every hour and pays the fee on every rebalance. We compare our reward with the two aligned alternatives described by the reviewer and with the closed loop of Q2. The alternatives are single runs with the same seed as the policy of Q2, and AllocFlow is reported as in Table 1.
| Training objective | MDD % ↓ | Sharpe ↑ | Calmar ↑ | TO ↓ |
|---|---|---|---|---|
| AllocFlow (Table 1) | 12.81 ±1.51 | 2.192 ±0.086 | 4.043 ±0.379 | 1.976 |
| Sharpe scaled by gross exposure | 18.1 | 1.76 | 2.76 | 2.07 |
| Sharpe after fees | 26.0 | 1.37 | 1.98 | 1.30 |
| Closed loop of Q2 with holdings input, fine-tuned from AllocFlow | 11.9 | 2.15 | 4.59 | 1.38 |
Putting gross exposure into the reward raises the MDD to 18.1%. Charging fees in the reward lowers the turnover to 1.30, but the MDD rises to 26.0%. Combined with a no-trade band at execution, AllocFlow reaches a Sharpe of 2.46 and a Calmar of 5.75. We add this comparison to Section 5.5 of the revised manuscript.
Q5. Why policy-gradient Flow-GRPO rather than direct differentiable optimization
Policy-gradient optimization performs substantially better. From the same IL checkpoint and with the same reward, Flow-GRPO reaches a Calmar of 4.043 ±0.379, while direct differentiable optimization reaches at most 1.74.
For the differentiable alternative, we follow DRaFT and back-propagate the same gross-normalized 24-bar Sharpe through the deterministic 4-step ODE for 2000 steps. The loss combines the Sharpe reward with an L1 anchor to the IL book and a penalty on gross exposure, and the table lists the learning rate and the weights of the reward and the anchor in each setting.
| Setting | Learning rate | Reward weight | Anchor weight | Calmar ↑ | Sharpe ↑ | TO ↓ |
|---|---|---|---|---|---|---|
| IL start (Table 1) | - | - | - | 0.955 | 0.954 | 1.143 |
| DRaFT 1 | 1e-5 | 0.1 | 1.0 | 1.14 | 1.07 | 1.17 |
| DRaFT 2 | 1e-5 | 1.0 | 1.0 | 1.04 | 0.87 | 3.59 |
| DRaFT 3 | 5e-5 | 0.1 | 1.0 | 1.74 | 1.41 | 1.19 |
| DRaFT 4 | 1e-5 | 1.0 | 0.1 | 0.95 | 0.80 | 4.50 |
| Flow-GRPO (ours, Table 1) | - | - | - | 4.043 ±0.379 | 2.192 ±0.086 | 1.976 |
Policy gradients are preferable for three reasons. First, Flow-GRPO only needs reward values, so it also applies to rewards computed on books rounded to the execution grid, such as the deployed after-fee P&L, which provide no useful pathwise gradient. Second, each pathwise gradient follows a single realized 24-hour price path and pushes the book toward whatever that path rewarded. A larger reward weight therefore mainly increases turnover, to 3.59 and 4.50 per day in DRaFT 2 and 4. Group-relative scoring instead standardizes rewards within each group of candidates and clips the advantage, which makes it robust to heavy-tailed returns. Third, the stochastic sampler that generates diverse candidate books also provides the closed-form likelihoods that Flow-GRPO optimizes. We have added this comparison to Section 5 of the revised manuscript.
Q6. Evaluation on equity universes
AllocFlow targets crypto perpetual futures. The market trades around the clock, allows long and short positions at the same fee, and has a universe of the 20 most-traded contracts that rotates every week. The hourly bars, the 24-bar horizon, the rotating universe and the cost model of AllocFlow are built for this structure. We thank the reviewer for this suggestion, and the revision states this scope explicitly in Section 1.
Wording of claims
We thank the reviewer for this comment. In the revision, every claim cites its supporting table or figure.
We hope these responses address the reviewer's concerns, and we would be grateful if the reviewer would consider raising the score. Please feel free to let us know if there are any further questions.
We thank the reviewer for the careful review and the valuable suggestions. We address the reviewer's concerns below.
Q1 and Q4. Motivation for flow matching and matched stochastic-policy baselines
The new controls show that flow matching is central to effective RL refinement in AllocFlow. Under the identical GRPO objective and matched model capacity, Flow-GRPO reaches a Calmar of 4.043 ±0.379, while the best stochastic Transformer policy reaches 1.92.
The one-pass Transformer in Table 2 is exactly the stochastic Transformer policy described by the reviewer. It is a bidirectional Transformer with about 1.0B parameters, matched to the flow head and reading the same tokens. It outputs a mean book μ and samples actions from N(μ, σ²I). Its stage 2 uses the same GRPO code as the flow head, with eight candidates per state, the same 24-bar gross-normalized Sharpe reward, the same clipping, KL and distillation terms, and 2000 steps, and differs from the flow head only in the policy kernel, a Gaussian instead of the flow SDE, which is what Section 5.1 of the submission refers to as different RL training kernels. We report it for exploration scales σ from 0.05 to 0.8.
The Gaussian policy is also sensitive to its exploration scale: the other four values of σ (0.05, 0.1, 0.2 and 0.8) all stay below a Calmar of 1 (−0.55, −0.62, 0.96 and −0.12; the σ = 0.2 run is the one in Table 2) and trade about three times the book per day. Starting from imitation policies of similar quality (Calmar 0.878 and 0.955), RL adds 3.1 to the Calmar of the flow policy, three times its gain for the Gaussian policy.
Flow matching fits portfolio allocation for three reasons. First, the action is a joint distribution over books of 20 signed weights, for example when choosing which of several correlated assets to short. Second, group-relative RL compares candidate books, so the candidates should be coherent alternatives. The flow samples complete alternative books from the learned distribution, while a diagonal Gaussian perturbs a single mean book coordinate by coordinate, which in the sweep above raises turnover to about three times the book per day. Third, the stochastic sampler of the flow provides the closed-form transition likelihoods that Flow-GRPO optimizes.
We have added the σ sweep to Table 2 and state this motivation directly in Section 3 of the revised manuscript.