- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how evaluation protocols affect temporal cascade prediction. It compares Random Cascade, Full Temporal, and Inductive User protocols, introduces Taoke with promotion and purchase records, and presents CasTemp, a lightweight reference method. Experiments compare popularity prediction on four datasets and conversion prediction on Taoke.
Critical strengths
-
Examining the time semantics of cascade prediction is useful. The protocol comparisons show that model rankings and errors can depend substantially on the evaluation setup, which matters when interpreting benchmark progress.
-
Taoke is a new useful task and provides a new benchmark beyond predicting cascade size alone.
Critical weaknesses
-
The comparisons do not isolate the effect of temporal leakage. E.1 defines Random Casdcade using fixed local windows relative to each casdcade's creation time, whereas Full temporal uses global calendar windows and includes cascades active in the observation window. Thus, the protocols also change cascade age, prediction targets, and sample selection. The error gaps described as leakage-driven inflation in Sect. 4.4 could partly reflect these task and population differences. A chronological split retaining the same local observation definition, with comparable test populations, is needed to support that attribution.
-
The information available to CasTemp and its baselines needs clearer specification. The paper constructs the competition graph from promoter sets and samples cross-cascade temporal sequences, without making their prediction-time cutoffs explicit. For conversion prediction, CasTemp also uses historical features and predicted popularity. It is unclear whether they get the same inputs.
Questions for authors
Please see the two concerns raised in the weaknesses.
5 comments
Thanks for the response.
How much does cascade information improve sales forecasting over a baseline using only sales history and product features?
Do these gains remain consistent across different forecast dates?
Where is the updated pdf?
Thank you for the follow-up. We ran a controlled comparison to measure how much cascade embeddings add to sales forecasts that already use sales history and product features, and to see whether the gains recur across forecast dates.
The shared baseline is a simple MLP that uses seven days of sales history and historical changes in price, discount rate, and commission rate. We concatenate each model’s cascade embedding with those inputs and train the MLP, keeping the cascade encoder fixed. Within each period, the variants use the same held-out products and training dates. For each forecast, we train the model on the previous day’s targets and construct all inputs using information available before the corresponding cutoff.
We evaluated August 10–12, 2025 on Taoke and March 17–19, 2025 on a second dataset collected in the same Taobao promotion scenario. The March data have the same structure as Taoke, with the same types of forwarding records, sales labels, and similar product features. Applying the same evaluation protocol in both periods lets us test whether the gains recur several months apart with temporal generalization. The table reports sales MSLE.
| Forecast date | Shared MLP | + CasTemp | + CasCN | + VNOIP | + CTCP |
|---|---|---|---|---|---|
| 2025-08-10 | 9.831 | 6.319 | 6.932 | 9.553 | 8.935 |
| 2025-08-11 | 9.694 | 6.057 | 6.891 | 9.602 | 8.883 |
| 2025-08-12 | 9.547 | 6.207 | 6.663 | 9.739 | 8.459 |
| 2025-03-17 | 8.763 | 5.801 | 6.537 | 8.881 | 8.568 |
| 2025-03-18 | 8.289 | 5.923 | 6.225 | 8.304 | 7.985 |
| 2025-03-19 | 8.580 | 5.877 | 6.309 | 8.471 | 8.137 |
| Mean MSLE | 9.117 | 6.031 | 6.593 | 9.092 | 8.495 |
CasTemp’s embedding reduced mean MSLE from 9.117 to 6.031 (33.85%), with daily reductions ranging from 28.54% to 37.52%. The summary below gives each forecast date equal weight within each period and overall. Relative reduction compares the mean MSLE with the corresponding shared MLP mean.
| Model supplying the embedding | Mean MSLE ↓ | August relative reduction | March relative reduction | Overall relative reduction | Dates improved |
|---|---|---|---|---|---|
| CasTemp | 6.031 | +36.08% | +31.33% | +33.85% | 6 / 6 |
| CasCN | 6.593 | +29.53% | +25.60% | +27.69% | 6 / 6 |
| VNOIP | 9.092 | +0.61% | −0.09% | +0.28% | 3 / 6 |
| CTCP | 8.495 | +9.61% | +3.68% | +6.83% | 6 / 6 |
VNOIP’s embedding increased MSLE on three of six dates and yielded a net reduction of only 0.28%. This is consistent with the limited benefit observed for VNOIP in Table 22. With each model’s cascade representation and a shared predicted-popularity signal already present, adding forwarding and sales histories and historical changes in price, discount rate, and commission rate increased VNOIP’s MSLE slightly, from 8.925 to 8.939. CasTemp, CasCN, and CTCP improved on all six dates in the embedding comparison and also improved in Table 22, from 2.315 to 1.998, 8.639 to 6.347, and 9.578 to 7.961, respectively.
The shared MLP serves as an ablation to isolate the contribution of each fixed cascade embedding. The full configurations in Table 22 show how the models perform when their cascade representations are combined with predicted popularity and, in the expanded setting, historical inputs. The two comparisons show that sales forecasting performance depends on both the cascade representation and how the model uses it with the other inputs.
We have also included this new experiment in the revised manuscript in Appendix M.6 (Tables 23–24). The earlier revision PDF upload may not have completed due to an issue with OpenReview. We have verified that the revised PDF was uploaded successfully and is now accessible from the submission page on OpenReview. Thanks again for your reminder, and we welcome more follow-up questions.
Thank you for raising these concerns. We address them with a fixed-task APS comparison and a matched-input Taoke sales experiment.
W1: Full Temporal and a chronological comparison with identical local windows
Full Temporal is designed for rolling forecasts of active cascades. At each calendar cutoff, eligible cascades share the same observation window and forecast interval. As the cutoff advances, the benchmark retains ongoing cascades and admits newly active ones based on information visible at that time. This reflects how the forecast population changes in deployment.
The revised Section 4.4, Table 11, and APS panel of Figure 1 describe the original results as cross-protocol MSLE gaps. The new controlled APS comparison appears in Appendix M.1 and Table 17. The held-out observation period begins on January 1, 2005 and ends at the first test forecast on January 1, 2010. Local windows are anchored to cascade creation. The target is new citations in the five years after observation ends, and labels mature at the end of that horizon.
All conditions share the same test cascades, local windows, targets, test inputs, validation rule, model configurations, and training-set size. Eligibility uses information visible at the observation cutoff.
On the identical test set, A versus B shows that admitting later-maturing training examples lowers MSLE for all five models. CasFlow improves from 2.40 to 1.61 and moves from fourth to first, while CasTemp improves from 1.90 to 1.69. Relaxing only the feature rule also lowers MSLE, and CasTemp remains first.
W2: Prediction-time inputs and a matched-input sales comparison
Appendix H now specifies that CasTemp uses information available at each forecast cutoff. The competition graph and cross-cascade walks use forwarding records from the current observation window and earlier visible history. Each cascade’s promoter set contains the deduplicated senders and receivers of these records. Graph edges use Jaccard overlap, and the walks use the same cutoff-filtered records. Historical forwarding, sales, price, discount-rate, and commission-rate features follow the same cutoff.
In the original comparison (Section 6.3 and Table 4), CasTemp fuses historical sales and product covariates with its cascade representation. Removing predicted popularity from CasTemp raises MSLE from 1.955 to 3.506. The oracle-popularity variant achieves 1.863.
Appendix M.5 and Table 22 now report the matched-input sales comparison. The baseline encoders were designed for popularity forecasting. We retain each encoder and reuse CasTemp’s feature-fusion design at the sales head to incorporate Taoke’s additional inputs. With Base inputs, each head receives its model’s cascade representation and the same predicted-popularity signal. With Expanded inputs, each head also receives the same cutoff-safe forwarding and sales histories and the same price, discount-rate, and commission-rate features. Within each setting, all heads have the same architecture and capacity. Both settings share the test products and forecast cutoff.
The expanded inputs improve CasCN, CTCP, and CasDo, have little effect on VNOIP, and increase errors for CasFlow and DeepHawkes under the shared fusion design. The CasTemp–CasCN gap narrows from 6.324 to 4.349 MSLE. With common covariates, a shared predicted-popularity signal, and matched sales-head capacity, CasTemp remains first at 1.998 MSLE, compared with 6.347 for CasCN, the strongest baseline in this setting. CasTemp’s fusion builds on informative cascade embeddings, and the mixed results suggest that fusion gains depend on the encoder.
We appreciate your feedback. The Revision Summary lists the changes, and we’d be glad to discuss any remaining questions.