- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
DIHAS estimates the cost saving from replacing one execution option while keeping the continuation policy fixed. It combines shared-innovation replay, a financial surrogate, a bounded residual, an inventory control variate, and alternative price-mean scenarios. The estimates bias RL action selection. Experiments report execution costs and estimator diagnostics on a 52-stock Shenzhen panel.
Critical strengths
- The paired-saving target is clearly defined. Holding the reference continuation and market innovations fixed gives the action comparison a precise interpretation within the simulator. The PICV derivation states its assumptions, and the separate variance guard provides an empirical check before using fitted coefficients.
- The full systems show consistent cost improvements over the local references across years. The date-block analysis in Table A1 is useful because repeated windows and training seeds are not independent market observations. The paper also distinguishes these system-level results from its narrower mechanism diagnostics.
Critical weaknesses
- The main comparison does not establish the contribution of the evidence layer. DIHAS uses 12 options and a paired terminal reward, while the local DQN/PPO baselines use 99 primitive actions; additional replay work also differs. The closest evidence-off comparison is limited to three seeds in 2020 and has a schedule mismatch. Its mean result favors evidence-off for DDQN and PPO (Table A6). A fully matched comparison is essential to determine whether the proposed evidence feedback improves execution.
- The benefit of the structured estimator is not directly established on a common prediction task. In RQ2, independently retrained variants are evaluated on their own states and reference policies, so lower pinball loss need not reflect a better estimator of the same target. The financial surrogate has no separately reported held-out saving-prediction comparison. The PICV guard supports variance reduction, but does not resolve whether the surrogate and residual improve conditional-mean estimation or action ranking.
Questions for authors
- What happens when evidence is disabled across all three years while retaining the same 12 options, terminal reward, training schedule, and seeds? Please report paired test-cost differences and the additional replay cost.
- On fixed held-out states with a common frozen reference and replay draws, how do the full estimator, surrogate-only model, and an equally sized unstructured predictor compare in mean-saving error and action-ranking regret?
3 comments
Response to Reviewer BLHN
W2 & Q2 | How do the predictors compare on common states?
W2 | Quantile loss and a common prediction target
Figure 5 measures pinball loss after independent retraining, with each variant using its own states and continuation. We agree that a lower loss there leaves open which model predicts a common target better. In Appendix H, we fit all three predictors on the same 512 states per year. Each training label averages 16 PICV-adjusted paired savings for one state. The structured and unstructured refits have 100,398 and 100,395 trainable parameters, respectively.
On each year's 256 held-out states, the predictors share the legal options, a frozen DDQN continuation for that year, and 4,096 replay draws per state. We evaluate mean saving in the center scenario. Regret measures the gap in mean saving between the best legal option and the one the predictor selects.
In the revised paper. RQ2 in Section 5.2 (pp. 7–8), Figure 5 (p. 8), and Appendix C.1 (p. 20) for the original diagnostic. Appendix H describes the common task, fitting and test states, and reference policy (pp. 32–33).
Q2 | Standalone surrogate, residual, and the PICV guard
The financial surrogate alone has the lowest mean absolute error (MAE) among the three refits in every year. Adding the bounded residual raises the MAE in this comparison. For the surrogate and unstructured refits, all paired regret intervals against the structured refit include zero (Table S13). These comparisons leave the ranking benefit of the bounded residual unresolved. The PICV guard in Table 2 tests variance reduction in the saving estimate. The common-state results below address prediction error and option ranking.
Table R3. Mean absolute error (MAE) of center-mean savings and option regret (bp). S is the surrogate plus residual, F the financial surrogate alone, and U the unstructured refit. The asterisk marks the original 2019 probe. Data from Table S12.
| Year | Saving MAE S | Saving MAE F | Saving MAE U | Option regret S | Option regret F | Option regret U |
|---|---|---|---|---|---|---|
| 2019* | 0.158 | 0.146 | 0.155 | 0.144 | 0.130 | 0.128 |
| 2020 | 0.168 | 0.133 | 0.173 | 0.111 | 0.096 | 0.110 |
| 2021 | 0.124 | 0.111 | 0.126 | 0.096 | 0.106 | 0.098 |
| 2022 | 0.110 | 0.095 | 0.114 | 0.081 | 0.083 | 0.085 |
| 2023 | 0.095 | 0.088 | 0.099 | 0.065 | 0.066 | 0.063 |
The deployed model also uses quantiles, positive-saving probabilities, and scenario aggregation. Table S15 evaluates its frozen center-scenario mean output on the same states. This output has higher mean regret than each refit in all five years. In 2023, its MAE and regret are 0.084 and 0.076 bp, compared with 0.088 and 0.066 bp for the financial surrogate alone.
In the revised paper. Appendix H, Tables S12 and S13, for common-task results and paired intervals (p. 33), Table S15 for the frozen deployed model (p. 34), and Table 2 (p. 8) and Appendix C.2 (p. 20) for the PICV variance guard.
Response to Reviewer BLHN
Thank you for asking us to isolate what evidence feedback contributes. Appendix G (Table S11, p. 33) separates the effect of turning feedback on at execution time from differences between the policies learned with and without it. On a fixed sample of 2,048 windows for 2020 DDQN, turning feedback on for the same Full checkpoint saved 0.311 bp. With feedback disabled at execution, the policy trained with feedback cost 0.299 bp more than the policy trained without it. This left a net saving of 0.012 bp on that sample. The full test set gave 0.067 bp (Table 3).
The new frozen-policy replays also evaluate feedback using inputs available at decision time. On 225 common trading dates in 2024, the same Shenzhen-trained DDQN checkpoint pairs saved 0.0376 bp in Shenzhen and 0.0533 bp in Shanghai. Both paired 95% date-block intervals were above zero. For PPO and GRPO, the intervals included zero at both venues (Tables 4 and S39). These results concern execution cost. The common-state predictor comparison in Appendix H found the lowest mean absolute error for the standalone surrogate in all five years, with no consistent ranking advantage from adding the residual.
Response to Reviewer BLHN
Thank you for asking us to isolate the effect of feedback and evaluate the predictors on the same task. We apologize that the original system comparison varied the options, terminal reward, and replay work together, leaving the feedback's contribution unclear. The original pinball losses also came from different states and continuations, leaving the structured estimator's performance on a shared target unclear. We address two questions. How much does turning feedback off change paired test costs and replay work when the 12 options, terminal reward, training schedule, and seeds match? How do the structured estimator, standalone surrogate, and capacity-matched unstructured predictor compare in mean-saving error and option regret on the same held-out states, frozen reference, and replay draws? We report the matched retraining in RQ4 (Table 3) and the common-state prediction test in Appendix H.
W1 & Q1 | What does matched evidence feedback change?
W1 | What the main comparison measures
The main table compares complete execution systems. DIHAS uses 12 options to express decisions that can adjust price and size for several minutes. Its paired terminal reward uses the cost difference between two complete executions of the same parent order on the same replay path. This ties the training signal to the cost of completing the order. We retain the local DQN and PPO implementations' 99 primitive actions to evaluate those baselines in their configured form. The complete-system comparison therefore includes differences in the action interface, reward, and additional replay. To measure the contribution of evidence feedback, Full and Off share the 12 options and the paired terminal reward. We enable feedback only in Full and report the incremental replay work separately.
In the revised paper. Figure 1 and Section 3 for the paired terminal reward (pp. 2–3), Section 5.1 and Table 1 for the system comparison and compute accounting (pp. 5–6), Appendix F.2–F.3 for the action interfaces (pp. 28–29), and Appendix G for the matched design (p. 31).
Q1 | Matched training and the cost of replay
The 2020 comparison used three seed pairs per backend and mixed planned training horizons. Off minus Full was −0.268 bp for DDQN and −0.185 bp for PPO, favoring Off in both cases (original Table A6). We retrained both arms with the same 12 options, terminal reward, planned 120K-step horizon, and five matched seeds per backend and year. The training-window draws and optimizer random streams follow the same seed rules. Both arms use identical test windows and weights (Appendix G).
Full has lower mean TC in 5 of 9 cells in the original three years (45 pairs) and 9 of 15 through 2023 (75 pairs). Five of the 15 pointwise intervals favor Full, five favor Off, and five include zero. The largest estimated saving is 0.671 bp for 2021 PPO (Table 3).
Table R2. Off minus Full Test TC (bp). Positive values favor Full. Brackets give pointwise 95% date-block intervals for five fixed seed pairs. Data from Table 3.
By 120K steps, each Full run uses 658,944 additional evidence queries, covering 10.46 to 11.09 million simulated minutes. Annual mean refresh times range from 66.0 to 77.8 seconds. Both arms retain the terminal verifier. Tables S9 and S10 report costs for both arms, paired seed differences, and replay work. Section 5.1 describes the input timing limitation.
In the revised paper. Appendix B.5 for the archived schedule mismatch (p. 18), RQ4 in Section 5.2 and Table 3 for the new comparison (p. 10), and Appendix G, Tables S9 and S10, for seed-paired costs and replay work (p. 31).