- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TREFIN is a probabilistic financial forecasting framework in which measurable asset concepts, a signed dependence graph, and market states form the sole interface to the return decoder. These states evolve through typed dynamics and are supervised using future measurements during training. The decoder produces asymmetric, heavy-tailed return distributions. The theoretical analysis establishes state feasibility, distributional validity, finite-horizon error propagation, and a conditional downside-scale response. Experiments report competitive forecasting performance on a 32-asset task, supplemented by state interventions, a graph-free retraining pilot, additional temporal splits, and a separately fitted cryptocurrency application.
Critical strengths
The strongest aspect is the commitment to a measurable intermediate representation without a raw-history bypass. The intermediate states have defined financial meanings and observable training targets, rather than being explanations attached after prediction. Maintaining competitive accuracy under this restriction is a worthwhile contribution, even without a large improvement over unrestricted predictors.
I also appreciate the work put into the appendix. The authors distinguish structural guarantees from empirical diagnostics and disclose several unfavorable or inconclusive results. The validity arguments make the proposed system easier to inspect, while the discussion appropriately avoids equating conditional response constraints with causal identification.
Critical weaknesses
The main remaining concern is empirical support for the learned dynamics. The reviewer package is helpful: the core mechanism is implemented, and the NoGraph variant removes graph dependence throughout the rollout rather than merely zeroing the initial input. However, this does not resolve the inconclusive evidence in Table 10 and Appendix E.3. The graph-free retraining pilot improves MSE from 6.9169 to 6.8495, while the energy-score difference has a confidence interval containing zero. One seed cannot establish that the graph is harmful, but it also cannot establish its benefit. A persistence comparison is particularly important for the concept and graph forecasts, since several measured quantities change slowly and graph estimates are carried forward between refits. I would like clearer evidence of what the learned dynamics add beyond providing an inspectable state interface.
The core training and calibration procedures are much clearer after inspecting the code, but the main comparative result remains difficult to assess. The implementation fits mean shrinkage on validation predictions after checkpoint selection, which is appropriate. In Table 4, however, TREFIN's MSE changes from 6.8542 before calibration to 6.8171 afterward, compared with 6.8415 for TEDM. It remains unclear whether competing methods receive the same calibration opportunity. The supplied package explicitly excludes baseline implementations and paper-scale experiment outputs, so it does not resolve the missing baseline adapter and training details noted in Appendix E.2. These details matter for both the small point-error advantage and the energy-score comparison.
Some passages read like internal reporting instructions or AI generated sentences rather than finished exposition. For example, " It must not be described as interval-coverage calibration." on p 37, "Results for unexecuted ... retraining ablations are not filled in by interpolation" on p 42. These words can be regarded as traces of the editing process or a strong tone of internal audit explanation. I won't let these factors affect my rating but they are a little bit strange.
Questions for authors
-
Were the competing methods given the same validation-only mean-shrinkage calibration as TREFIN? If not, could you provide a matched comparison for the strongest baseline? My concern is the attribution of the small accuracy advantage, rather than the validity of TREFIN's calibration procedure.
-
Do the concept and graph forecasts improve over carrying the current measured state forward? A persistence comparison, together with any additional matched graph-free retraining results, would help establish the empirical value of the dynamics. I do not expect every component to improve every metric.
-
I appreciate the compact reviewer package and its explicit scope. Could you provide the resolved paper-scale configurations and the baseline adapter specifications needed to check Table 4, including their probabilistic heads, objectives, and calibration settings? For Table 10, the corresponding full and graph-free run configurations would also help connect the supplied implementation to the reported experiment.
6 comments
Response (2/4) to Weakness 1 and Question 2: persistence and matched graph retraining
Thank you for identifying persistence and matched retraining as complementary tests of the learned dynamics. We appreciate your observation that different components need not improve every metric; the completed experiments distinguish concept prediction, the complete graph pathway and the incremental effect of graph updates.
Concept evolution improves aggregate prediction beyond persistence. We compare and against the same measured future targets, test origins and five original seeds in each fold. Lead zero is excluded; concept MAE uses robust-standardized units.
| Fold | Overall concept MAE reduction | Improved coordinates /26 | Lead-5 graph IoU: learned / persistence |
|---|---|---|---|
| 1 | 4.64% | 12 | 0.5851 / 0.6791 |
| 2 | 6.82% | 15 | 0.6394 / 0.7359 |
| 3 | 9.12% | 17 | 0.6252 / 0.6999 |
Overall concept MAE averages equally over coordinates, future leads, assets, origins and seeds; each coordinate count uses its MAE averaged over leads 1–5. Learned concepts improve aggregate error in every period, with heterogeneous gains across the named quantities.
Graph persistence has higher IoU at every evaluated lead and lower continuous graph MAE in all three folds. Both forecasts use the same target, refitted every five observations and held between refits; lead 5 crosses a refit. IoU uses strict off diagonal, with empty unions zero.
We summarize these results in of the revised paper and report every future lead and all 26 coordinates, including continuous graph errors, in .
The explicit graph pathway contributes after the remaining parameters adapt. TREFIN uses signed, continuous graph weights in typed concept propagation, decoder features and joint-return precision. The additional Full/NoGraph comparisons retrain from initialization on fold 1 with seeds 1/7/19/37/73, identical prepared data, AdamW, batch 64, a 20-epoch cap and patience 6. Validation ES selects checkpoints; all receive the same validation-only mean calibration .
NoGraph zeros at initialization, every transition and decoding and removes graph-target loss, retaining the market state's residual-dependence summary. The selected protocol gives each variant four nominal validation candidates; Full selects , and NoGraph selects with . Choices are frozen before five-seed evaluation.
Entries are mean ± sample SD; concept MAE averages future leads 1–5.
| Protocol | Variant | MSE () | ES () | Concept MAE |
|---|---|---|---|---|
| Fixed | Full | 6.8174 ± 0.0065 | 8.9752 ± 0.0180 | 0.4923 ± 0.0037 |
| Fixed | NoGraph | 6.8268 ± 0.0092 | 9.0569 ± 0.0297 | 0.5140 ± 0.0205 |
| Selected | Full | 6.8170 ± 0.0055 | 8.9754 ± 0.0191 | 0.4925 ± 0.0047 |
| Selected | NoGraph | 6.8268 ± 0.0093 | 9.0569 ± 0.0296 | 0.5141 ± 0.0205 |
Full has lower mean MSE, ES and concept MAE under both protocols, including when each variant selects its own configuration. The complete five-variant study is reported in of the revised paper. The original single-seed pilot remains separate in .
Graph updating has period-dependent predictive effects. The additional three-fold Full/FrozenGraph study retains in every FrozenGraph transition and decoder while retraining other active parameters. Each fold/variant uses calibrated seed-1 validation ES to select among the same four nominal candidates; the inactive slow-refresh candidate duplicates FrozenGraph's base and is evaluated once. All six choices are frozen before testing; each selected model receives the same validation-only positive mean calibration before five-seed evaluation.
Full has lower mean MSE in folds 1 and 3 and lower mean ES in fold 3. FrozenGraph has lower concept and continuous graph MAE in all three folds. Mean continuous graph MAE for Full/FrozenGraph is 0.0083/0.0056, 0.0069/0.0038 and 0.0071/0.0042 in folds 1–3. These results test updating the signed weights while both models retain the initial relational information; Full/NoGraph separately tests the complete graph pathway.
We provide all prediction/state metrics, selected runs and paired results in and of the revised paper. This study retains its own selected checkpoints, separately from the original persistence cohort.
Response (3/4) to Weakness 2 and Question 3: resolved configurations and baseline adapters
Thank you for asking for the configurations and adapter specifications that connect the implementation to each reported experiment. We have organized the resolved settings by experiment family so that the original comparison, the graph-free pilot and the matched retraining studies can each be checked against their own protocol.
Resolved TREFIN configuration. The original fold-1 comparison uses seeds 1/7/19/37/73, training 2009–2017, validation 2018–2019, testing 2020–2021, horizon 5 and embargo 5. The common settings are:
| Component | Resolved setting |
|---|---|
| Measurements | Concepts at 5/21/63/126 rows; graph window 126, refit stride 5; market window 21 |
| Widths | Hidden 64; lead embedding 12; graph embedding 24 |
| Transition | Step .20; damping [.12,.90]; diffusion cap .15; forcing cap .40; graph refresh .10; residual scale .01 |
| Decoder | ; ; mean-bound multiplier 5; scale floor .05; risk-strength cap 1 |
| Auxiliary losses | , ; monotonicity .01; sensitivity .10; radius target .995 |
| Optimization | AdamW; learning rate .001; weight decay .0001; cosine period 20; norm clip 1; batch 64 |
| Selection | Maximum 20 epochs; patience 6; validation-ES improvement |
| Scenarios | Train/validation/test 16/64/256; evaluation RNG seed 271828 |
| Calibration | Positive mean scalar [.05,1]; validation ES; tolerance ; frozen dynamics and shape |
Graph-penalty selection uses rolling training windows and candidates [.01,.03,.1,.3]; the realized selection is .10 in all resolved paper-scale runs. Prepared metadata retains the selected value, fitted scales and projection bounds alongside input hashes and exact split indices.
We document the TREFIN splits and common configuration in of the revised paper. In , panels A-B record the selected backbone training runs, panel C records calibration scalars, and panel D records the original pilot.
The original Full/NoGraph pilot is documented in its own setting. Both runs use fold 1, seed 1, the same prepared data and the uncalibrated configuration. Full completes 11 epochs and selects epoch 5 (validation ES .06359918); NoGraph completes 10 and selects epoch 4 (validation ES .06402534). NoGraph adds ablation_variant: NoGraph, zeroing the graph at initialization, every transition and decoding. Its pilot configuration retains , a parameter-independent graph-target term. The uncalibrated Full/NoGraph MSE is 6.9169/6.8495 (), and ES is 9.0526/9.0557 ().
The revised paper records this single-seed pilot separately from the five-seed structural studies in and .
The five-variant and three-fold Full/FrozenGraph studies each retain their own resolved configurations. Full/FrozenGraph freezes all six validation choices before testing: 21 unique search fits, 24 additional selected fits and 30 final evaluations. FrozenGraph omits its parameter-constant graph-target loss and uses the augmented gain matrix with graph row ; gain coefficient .1 and target .995 remain unchanged. The protocol and selected runs are documented in of the revised paper.
Baseline results are tied to their actual adapters and selection procedures. The specifications identify inputs, predictive heads, objectives, candidate budgets, scenario construction and calibration for every comparator in the matched studies. TEDM uses its native stochastic sampler; Timer-XL uses centered joint training residuals. Native implementations are distinguished from pinned backbone adapters, and each comparison uses the common validation-only calibration described in our preceding response.
We provide these specifications and selected-run records in and of the revised paper. We have prepared the original resolved TREFIN and pilot configurations, preparation hashes, pinned baseline sources, launch commands, selection records and per-origin losses for the planned public code release.
Response (1/4) to Weakness 2 and Question 1: equal calibration opportunity
Thank you for distinguishing the validity of mean calibration from the attribution of the accuracy gains. We agree that each competing predictor should receive the same opportunity, and have completed the matched comparison across three chronological folds with five seeds per model and fold. The complete code repository and experimental records can be found at the following Anonymous GitHub link: https://anonymous.4open.science/r/TREFIN-F7AE/.
The same validation-only calibration is applied to every listed predictor. The submitted protocol explicitly specifies mean calibration for TREFIN. The comparison below evaluates TEDM, Timer-XL and TREFIN both before and after the same procedure. Test periods are 2020–2021, 2022–2023 and 2024–March 2025, with identical test origins and five-step targets within each fold and separate training/validation periods.
After checkpoint/candidate selection, each predictor is frozen and validation ES selects ϱ∈[0.05,1]. The transformation Xϱ(s)=X(s)+(ϱ−1)μ preserves the predictor's innovations. Its before/after comparison therefore isolates calibration within that fixed model.
Entries are mean ± sample SD over five seeds. TEDM uses native stochastic scenarios; Timer-XL uses a centered joint training-residual adapter; testing uses 256 scenarios. The table identifies the task adapters used for these comparisons, with their inputs, objectives, candidate budgets and pinned source revisions specified in the revised paper.
TREFIN retains strongly competitive prediction under equal calibration opportunity. It has lower MSE than both comparators in folds 1 and 3 and lower ES in folds 1 and 2. Its MSE is 0.049% above Timer-XL in fold 2, and its ES is 0.0024% above Timer-XL in fold 3. The same three-fold, five-seed protocol is complete for TimeXer and TimeMixer++, with all six main predictive metrics reported.
An additional native-implementation comparison applies identical calibration opportunity to nine further implementations on fold 1. TREFIN's calibrated MSE is 6.8171±0.0066, versus 6.8260±0.0069 for the closest native point comparator, MASTER-style, (MSE ×104).
We provide the complete cross-period comparisons, native implementations, before/after calibration results and adapter specifications in Table 2, Appendices G.1-G.3, Tables 15 and 17 and Fig. 10 of the revised paper. These results support the central contribution: a measured, inspectable state interface together with strongly competitive predictive performance.