acceptodds
Under review as a conference paper at ICLR 2027

From Leakage to Fidelity: Reliable Benchmarking for Temporal Cascade Prediction

Abstract

Temporal cascade prediction is widely studied, yet its empirical foundations remain fragile. Most existing works report results under random cascade splits that mix past and future signals, rely on datasets with limited features and no downstream conversion labels, and compare increasingly complex models without systematically examining whether benchmark conclusions are protocol-dependent. This paper argues that the field should move from leakage-prone evaluation toward fidelity-aware benchmarking. We introduce a protocol suite and renewed evaluation standard for temporal cascade prediction, centered on the Full Temporal protocol, overlap-based leakage diagnostics, and analyses of performance inflation and temporal drift. To broaden the scope of benchmark tasks, we also present Taoke, a real-world e-commerce cascade dataset with rich promoter/product features and observed purchase conversions, enabling both first-stage popularity forecasting and second-stage conversion forecasting under a shared benchmark asset. Finally, we include CasTemp as a lightweight reference method and additionally probe a larger same-task internal extension to verify that this pipeline remains operational at substantially greater scale. Together, these components turn cascade prediction from a protocol-sensitive leaderboard exercise into a more reliable analysis and benchmarking problem, while still providing a practical reference pipeline for large-scale evaluation and conversion-aware modeling.

Primary area:
datasets and benchmarks
Keywords:
Temporal Cascade Prediction, Information Cascade Graph, Benchmark

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_tFhAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how evaluation protocols affect temporal cascade prediction. It compares Random Cascade, Full Temporal, and Inductive User protocols, introduces Taoke with promotion and purchase records, and presents CasTemp, a lightweight reference method. Experiments compare popularity prediction on four datasets and conversion prediction on Taoke.

Critical strengths
  1. Examining the time semantics of cascade prediction is useful. The protocol comparisons show that model rankings and errors can depend substantially on the evaluation setup, which matters when interpreting benchmark progress.

  2. Taoke is a new useful task and provides a new benchmark beyond predicting cascade size alone.

Critical weaknesses
  1. The comparisons do not isolate the effect of temporal leakage. E.1 defines Random Casdcade using fixed local windows relative to each casdcade's creation time, whereas Full temporal uses global calendar windows and includes cascades active in the observation window. Thus, the protocols also change cascade age, prediction targets, and sample selection. The error gaps described as leakage-driven inflation in Sect. 4.4 could partly reflect these task and population differences. A chronological split retaining the same local observation definition, with comparable test populations, is needed to support that attribution.

  2. The information available to CasTemp and its baselines needs clearer specification. The paper constructs the competition graph from promoter sets and samples cross-cascade temporal sequences, without making their prediction-time cutoffs explicit. For conversion prediction, CasTemp also uses historical features and predicted popularity. It is unclear whether they get the same inputs.

Questions for authors

Please see the two concerns raised in the weaknesses.

5 comments

Thank you for raising these concerns. We address them with a fixed-task APS comparison and a matched-input Taoke sales experiment.


W1: Full Temporal and a chronological comparison with identical local windows

Full Temporal is designed for rolling forecasts of active cascades. At each calendar cutoff, eligible cascades share the same observation window and forecast interval. As the cutoff advances, the benchmark retains ongoing cascades and admits newly active ones based on information visible at that time. This reflects how the forecast population changes in deployment.

The revised Section 4.4, Table 11, and APS panel of Figure 1 describe the original results as cross-protocol MSLE gaps. The new controlled APS comparison appears in Appendix M.1 and Table 17. The held-out observation period begins on January 1, 2005 and ends at the first test forecast on January 1, 2010. Local windows are anchored to cascade creation. The target is new citations in the five years after observation ends, and labels mature at the end of that horizon.

All conditions share the same test cascades, local windows, targets, test inputs, validation rule, model configurations, and training-set size. Eligibility uses information visible at the observation cutoff.

  • A: Both constrained. Training labels must mature by the first test forecast on January 1, 2010. Features use snapshots available at each example’s observation cutoff.
  • B: Labels relaxed. We replace some of A’s examples with equally many non-test examples whose labels mature after that forecast. A’s feature rule still applies.
  • C: Features relaxed. We retain A’s training examples and labels and use snapshots from January 1, 2010 to January 1, 2015 only for the designated training-side cross-cascade statistics.
  • D: Both relaxed. We apply both changes.
APS · Fixed test set · MSLE ↓ABCD
CasFlow2.401.612.131.54
CasTemp1.901.691.821.64
CasCN2.351.982.061.85
VNOIP2.221.721.951.68
CTCP2.572.282.362.04

On the identical test set, A versus B shows that admitting later-maturing training examples lowers MSLE for all five models. CasFlow improves from 2.40 to 1.61 and moves from fourth to first, while CasTemp improves from 1.90 to 1.69. Relaxing only the feature rule also lowers MSLE, and CasTemp remains first.


W2: Prediction-time inputs and a matched-input sales comparison

Appendix H now specifies that CasTemp uses information available at each forecast cutoff. The competition graph and cross-cascade walks use forwarding records from the current observation window and earlier visible history. Each cascade’s promoter set contains the deduplicated senders and receivers of these records. Graph edges use Jaccard overlap, and the walks use the same cutoff-filtered records. Historical forwarding, sales, price, discount-rate, and commission-rate features follow the same cutoff.

In the original comparison (Section 6.3 and Table 4), CasTemp fuses historical sales and product covariates with its cascade representation. Removing predicted popularity from CasTemp raises MSLE from 1.955 to 3.506. The oracle-popularity variant achieves 1.863.

Appendix M.5 and Table 22 now report the matched-input sales comparison. The baseline encoders were designed for popularity forecasting. We retain each encoder and reuse CasTemp’s feature-fusion design at the sales head to incorporate Taoke’s additional inputs. With Base inputs, each head receives its model’s cascade representation and the same predicted-popularity signal. With Expanded inputs, each head also receives the same cutoff-safe forwarding and sales histories and the same price, discount-rate, and commission-rate features. Within each setting, all heads have the same architecture and capacity. Both settings share the test products and forecast cutoff.

Taoke · Sales MSLE ↓Original Table 4Base inputsExpanded inputsΔ MSLE (expanded − base)
CasTemp1.9552.3151.998−0.317
CasCN8.3808.6396.347−2.292
VNOIP9.3398.9258.939+0.014
CTCP10.4949.5787.961−1.617
CasFlow18.99220.01024.586+4.576
CasDo35.58830.93524.343−6.592
DeepHawkes42.51536.13847.932+11.794

The expanded inputs improve CasCN, CTCP, and CasDo, have little effect on VNOIP, and increase errors for CasFlow and DeepHawkes under the shared fusion design. The CasTemp–CasCN gap narrows from 6.324 to 4.349 MSLE. With common covariates, a shared predicted-popularity signal, and matched sales-head capacity, CasTemp remains first at 1.998 MSLE, compared with 6.347 for CasCN, the strongest baseline in this setting. CasTemp’s fusion builds on informative cascade embeddings, and the mixed results suggest that fusion gains depend on the encoder.


We appreciate your feedback. The Revision Summary lists the changes, and we’d be glad to discuss any remaining questions.

Thanks for the response.

How much does cascade information improve sales forecasting over a baseline using only sales history and product features?

Do these gains remain consistent across different forecast dates?

Where is the updated pdf?

Thank you for the follow-up. We ran a controlled comparison to measure how much cascade embeddings add to sales forecasts that already use sales history and product features, and to see whether the gains recur across forecast dates.

The shared baseline is a simple MLP that uses seven days of sales history and historical changes in price, discount rate, and commission rate. We concatenate each model’s cascade embedding with those inputs and train the MLP, keeping the cascade encoder fixed. Within each period, the variants use the same held-out products and training dates. For each forecast, we train the model on the previous day’s targets and construct all inputs using information available before the corresponding cutoff.

We evaluated August 10–12, 2025 on Taoke and March 17–19, 2025 on a second dataset collected in the same Taobao promotion scenario. The March data have the same structure as Taoke, with the same types of forwarding records, sales labels, and similar product features. Applying the same evaluation protocol in both periods lets us test whether the gains recur several months apart with temporal generalization. The table reports sales MSLE.

Forecast dateShared MLP+ CasTemp+ CasCN+ VNOIP+ CTCP
2025-08-109.8316.3196.9329.5538.935
2025-08-119.6946.0576.8919.6028.883
2025-08-129.5476.2076.6639.7398.459
2025-03-178.7635.8016.5378.8818.568
2025-03-188.2895.9236.2258.3047.985
2025-03-198.5805.8776.3098.4718.137
Mean MSLE9.1176.0316.5939.0928.495

CasTemp’s embedding reduced mean MSLE from 9.117 to 6.031 (33.85%), with daily reductions ranging from 28.54% to 37.52%. The summary below gives each forecast date equal weight within each period and overall. Relative reduction compares the mean MSLE with the corresponding shared MLP mean.

Model supplying the embeddingMean MSLE ↓August relative reductionMarch relative reductionOverall relative reductionDates improved
CasTemp6.031+36.08%+31.33%+33.85%6 / 6
CasCN6.593+29.53%+25.60%+27.69%6 / 6
VNOIP9.092+0.61%−0.09%+0.28%3 / 6
CTCP8.495+9.61%+3.68%+6.83%6 / 6

VNOIP’s embedding increased MSLE on three of six dates and yielded a net reduction of only 0.28%. This is consistent with the limited benefit observed for VNOIP in Table 22. With each model’s cascade representation and a shared predicted-popularity signal already present, adding forwarding and sales histories and historical changes in price, discount rate, and commission rate increased VNOIP’s MSLE slightly, from 8.925 to 8.939. CasTemp, CasCN, and CTCP improved on all six dates in the embedding comparison and also improved in Table 22, from 2.315 to 1.998, 8.639 to 6.347, and 9.578 to 7.961, respectively.

The shared MLP serves as an ablation to isolate the contribution of each fixed cascade embedding. The full configurations in Table 22 show how the models perform when their cascade representations are combined with predicted popularity and, in the expanded setting, historical inputs. The two comparisons show that sales forecasting performance depends on both the cascade representation and how the model uses it with the other inputs.

We have also included this new experiment in the revised manuscript in Appendix M.6 (Tables 23–24). The earlier revision PDF upload may not have completed due to an issue with OpenReview. We have verified that the revised PDF was uploaded successfully and is now accessible from the submission page on OpenReview. Thanks again for your reminder, and we welcome more follow-up questions.

Reviewer_qRALOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a new evaluation protocol and benchmark (Taoke) for temporal cascade prediction, along with a reference method CasTemp for it. It examines and evaluates the validity of three protocols: the traditional Random Cascade protocol that randomly splits at cascade level, a Full Temporal protocol that splits along the temporal axis, and an Inductive User protocol that splits at both temporal and user levels. Through evaluation, the paper observes that random Cascade introduces leakage and leads to performance inflation, and proposes Full temporal as the main evaluation protocol with Inductive User as a stress testing. The proposed benchmark Taoke provides richer features and downstream conversion labels. The proposed reference method CasTemp models cascade via competition graph, recency-aware temporal encoder and shallow prediction heads, and shows superior performance on both Taoke and existing benchmarks.

Critical strengths
  • The problem is well-motivated and the proposed Full Temporal regime makes sense.
  • The contributed dataset is richer than existing datasets in features and downstream outcomes, and the proposed reference method is both efficient and attains strong performance across datasets and settings.
Critical weaknesses
  • The connection between protocol validity, the core argument of the paper, and the proposed benchmark and method is not super strong, especially that it seems Taoke mostly contributes in downstream conversion and feature richness while the temporal drift and popularity ranking instability tied to protocol validity seems most prominent in prior dataset like APS.
  • The documentation of the proposed is a bit vague, with mostly size/depth info and temporal drift analysis while curation and distribution info are limited.
Questions for authors

See Weaknesses -

  • It would be helpful to provide more discussions or clarifications on the connections between Taoke and protocol validity. e.g. more clarification or analysis on how Taoke's properties tie back to the protocol validity observations, and if there is interaction effects between feature richness/downstream conversion and the protocol choice that's similar to first-stage popularity tasks, etc.
  • It would be helpful to provide more detailed documentation of the proposed dataset, e.g. the curation process, and the distribution of features and labels, etc.
2 comments

Thank you for asking how Taoke connects to the protocol analysis and for suggesting more detail on dataset curation and distributions.


W1,Q1 (Part 1): Taoke’s connection to temporal evaluation

Taoke captures a deployment setting in which products and promoters persist across periods while their activity changes. Table 12 reports an adjacent-window cascade Jaccard similarity of 0.916 and the highest burstiness JSD among the four datasets (0.0223). Table 10 shows that Full Temporal retains 63.3% train–test promoter overlap. This combination motivates evaluating ongoing products at successive calendar cutoffs, where the same entities remain active as their propagation patterns change.

Table 16 shows how protocol choice affects forwarding prediction on Taoke. CasTemp ranks first across all three protocols, while third place shifts from CasCN to DeepHawkes to CasFlow. From Random Cascade to Full Temporal, CasTemp’s MSLE decreases from 1.0113 to 0.7984, while CasFlow’s increases from 2.1328 to 2.2877. Taoke extends this analysis by allowing us to study both forwarding and downstream sales on the same products.


W1,Q1 (Part 2): Training-time constraints and side information

The newly revised Appendix M.4 and Table 21 report how the benefit of side information varies with the training regime. We evaluated CasTemp on the same 569 held-out products, with August 11, 2025 as the target day. All conditions share the same test cutoff and seven-day observation window. Past-only training uses 2,272 examples with August 10 targets. Future-exposed training replaces 454 of those examples with August 12 examples from the same training products, keeping the training-set size fixed. Both feature settings retain seven days of forwarding and sales history. The side-information setting adds historical price, discount-rate, and commission-rate changes available before each example’s forecast cutoff.

Feature gain is the reduction in MSLE after adding side information.

CasTemp · MSLE ↓Forwarding history onlyForwarding with side infoForwarding gainSales history onlySales with side infoSales gain
Past-only0.8490.792+0.0572.7422.448+0.294
Future-exposed0.7930.767+0.0262.5062.351+0.155

Side information improves CasTemp on both tasks under both training regimes. The largest gain is for sales forecasting under Past-only training, where MSLE drops from 2.742 to 2.448, a 10.7% reduction. Relative to Past-only training, the observed side-information gain under Future-exposed training is smaller by 0.031 MSLE for forwarding and 0.139 for sales. This fixed-cohort comparison connects Taoke to the protocol study by showing how the training-time constraint changes the measured benefit of side information for propagation and downstream sales forecasts on the same products.


W2,Q2: Dataset curation and documentation

Taoke covers a specific Taobao business scenario. We selected products whose average daily sales quantity over the preceding 30 days was at least five units in that scenario. Table 2 and Appendix A summarize the benchmark’s scale, cascade structure, and temporal characteristics. The revision adds the selection rule and release fields in Appendix A.1, and daily label distributions in Appendix M.3 and Table 20.

The release contains timestamped forwarding snapshots, daily product-level forwarding and sales aggregates, historical price, discount-rate, and commission-rate changes, and feature matrices indexed by anonymized product and promoter IDs. The supervised forwarding and sales tables cover the same 2,862 products. Section 6.1 defines the sales forecasting target as daily aggregate transaction value. Across the August 10–12 target days, median daily forwarding ranges from 40 to 48, and median daily sales value ranges from 2,570.90 to 3,143.95. The provided anonymous repository documents the release layout and the files used by the benchmark loader.


Thanks for your suggestions. The Revision Summary lists the updates, and we welcome any follow-up questions.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer qRAL, thank you again for your thoughtful feedback and for recognizing the value of the Full Temporal protocol and the Taoke benchmark. We hope our rebuttal and the additional experiments have addressed your questions about Taoke. We’d be glad to hear your thoughts and welcome any further discussion.

Reviewer_ZmZqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies evaluation reliability in temporal cascade prediction. It compares three protocols, Random Cascade, Full Temporal, and Inductive User, and analyzes model rankings, performance gaps, and temporal drift. The authors also introduce Taoke, a dataset containing promotion cascades, purchase conversions, and auxiliary features, and propose CasTemp as a lightweight reference model. The experiments cover four datasets and three protocols, along with conversion prediction, ablations, and efficiency analysis. The non-public Taoke-large extension is used only for large-scale stress testing.

Critical strengths

1.The paper addresses a practically important evaluation problem. The multi-protocol comparisons show that performance estimates and model rankings may not remain stable under a different split. Full Temporal preserves continuity for long-lived entities, while Inductive User provides a stricter diagnostic of user transfer.

2.Taoke connects propagation with observed purchase conversions and includes product and promoter features, extending the scope beyond popularity prediction.

Critical weaknesses
  1. The cross-protocol comparison does not adequately isolate the effect of leakage. Random Cascade uses local observation and target windows relative to cascade creation, whereas Full Temporal uses global windows and allows the same cascade to contribute examples in multiple periods (Appendix E.1). This changes cascade age at observation, input history, target intervals, and sample composition at the same time. The results support protocol sensitivity, but do not establish the APS error gap as a quantitative estimate of performance inflation caused specifically by temporal leakage. Section 4.4 attributes the APS gap to temporal leakage and explains it as models memorizing future cascade trajectories, which is stronger than the current cross-protocol comparison supports; this does not negate its observations on protocol sensitivity and temporal drift.

  2. The toy experiments lack metric definitions or aggregation details that explain the reported relationship between MSLE and MALE. If both metrics use the same examples, the same normalization weights, and the same log-space residuals, they should satisfy MSLE≥MALE2\mathrm{MSLE} \geq \mathrm{MALE}^{2}. However, 17 of the 32 metric pairs in Tables 8–9 do not satisfy it, and this cannot be explained by rounding at the four decimal places reported; all 9 pairs in Table 4 do satisfy it. The manuscript provides no metric definition or aggregation rule that would explain the difference. This leaves the interpretability and reliability of the toy diagnostic evidence in question, and formulas, the computation procedure, or corrected results are needed to judge whether its mechanistic conclusion holds.

Questions for authors

1.Do you have a comparison that keeps the observation windows, prediction targets, and evaluation examples fixed, while changing only the time constraints on the availability of training labels and feature information? If not, would you agree to describe the reported inflation values as overall cross-protocol gaps and qualify their interpretation as estimates of a leakage effect size?

2.Please provide the exact MSLE and MALE formulas and aggregation procedures used in the toy experiments, and explain the discrepancies reported in Tables 8–9. Would the clarification or any correction change the mechanistic conclusions drawn from the toy experiments?

3.How are the promoter sets used in CasTemp's competition graph, its cross-cascade walks, and its historical features constructed or updated relative to each prediction cutoff? Please specify the permitted event and statistical information so that their availability within the corresponding observation window can be verified.

3 comments

Thank you for the detailed review. We address the role of temporal constraints, clarify the toy metrics, and specify the information available to CasTemp at prediction time.


W1,Q1: Full Temporal and the controlled APS comparison

Full Temporal is designed for rolling forecasts of active cascades. At each calendar cutoff, all eligible cascades share the same observation window and forecast interval. As the cutoff advances, the benchmark refreshes the cohort, retaining ongoing cascades and admitting newly active ones as they become observable. This reflects how the population being forecast changes in deployment. Inductive User preserves this temporal structure and adds user disjointness to evaluate transfer to unseen users.

The revised Section 4.4, Table 11, and APS panel of Figure 1 describe the original results as cross-protocol MSLE gaps. The added controlled APS comparison appears in Appendix M.1 and Table 17. The held-out observation period begins on January 1, 2005 and ends at the first test forecast on January 1, 2010. Local windows are anchored to cascade creation. The target is new citations in the five years after observation ends, and labels mature at the end of that horizon. All four conditions share the test cascades, local windows, targets, test inputs, validation rule, model configurations, and training-set size. Eligibility uses information visible at the observation cutoff.

  • A: Both constrained. Training labels must mature by the first test forecast on January 1, 2010. Features use snapshots available at each example’s observation cutoff.
  • B: Labels relaxed. We replace some of A’s examples with equally many non-test examples whose labels mature after that forecast. A’s feature rule still applies.
  • C: Features relaxed. We retain A’s training examples and labels and use snapshots from January 1, 2010 to January 1, 2015 only for the designated training-side cross-cascade statistics.
  • D: Both relaxed. We apply both changes.
APS · Fixed test set · MSLE ↓ABCD
CasFlow2.401.612.131.54
CasTemp1.901.691.821.64
CasCN2.351.982.061.85
VNOIP2.221.721.951.68
CTCP2.572.282.362.04

On the fixed test set, admitting later-maturing training examples under the same feature rule lowers MSLE for all five models. CasFlow improves from 2.40 to 1.61 and moves from fourth to first, while CasTemp improves from 1.90 to 1.69. Relaxing the feature rule alone also lowers MSLE, with CasTemp remaining first. These results show that the availability of training information can change both errors and rankings on a fixed forecasting task.


W2,Q2: Toy metric computation

In the original Tables 8–9, we excluded examples with squared log-space residuals above 10 from MSLE to limit the influence of occasional extreme errors by several baseline models. Each residual is the prediction minus the target on the log scale. The retained set and filtered MSLE are

\mathrm{MSLE}_{\mathrm{filtered}}=\frac{1}{|K|}\sum_{i\in K}r_i^2.$$ MALE averaged absolute residuals over the full test set. The different sample sets explain why the usual inequality can fail. We now label the statistic as *filtered MSLE* in Tables 8–9 and document the rule in Appendix E. The main benchmark results, including Table 4, use the standard full-set definitions. Appendix M.2 adds Tables 18–19, the standard-metric counterparts to Tables 8–9. Both metrics use all N test examples. $$\mathrm{MSLE}=\frac{1}{N}\sum_{i=1}^{N}r_i^2,\qquad \mathrm{MALE}=\frac{1}{N}\sum_{i=1}^{N}|r_i|.$$ The main observation in Scenario 2 still holds. The MLP ranks first by MSLE under Random Cascade and fifth under Full Temporal, where CasTemp ranks first. We also updated Table 1 in Section 4.1 to compare both protocols within Scenario 2. The corrected toy results show protocol sensitivity, while the controlled APS experiment separately examines the role of future information. --- **Q3: Information available to CasTemp at each cutoff** Appendix H specifies that CasTemp builds its inputs from historical information available when each forecast is issued. We apply the forecast cutoff before constructing the competition graph and cross-cascade walks. Each cascade’s promoter set consists of the deduplicated senders and receivers of eligible forwarding events from the current observation window and earlier visible history. The graph uses Jaccard overlap between these sets, and the walks use the same cutoff-filtered records. Historical forwarding, sales, price, discount-rate, and commission-rate features follow the same time constraint. The sales head uses predicted popularity computed from these historical inputs. At the next forecast cutoff, newly available records can update the promoter sets and graph edges. --- Thanks for the feedback. See the Revision Summary for the changes. We look forward to continuing the discussion.
AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer ZmZq, thank you again for your feedback. We hope our rebuttal and the additional experiments have addressed your concerns. We would appreciate it if you could revisit your assessment in light of the new results. We welcome your thoughts and would be happy to discuss any remaining questions.

AuthorsOpenReview ↗

Further Follow-up on Our Response

Dear Reviewer ZmZq, we would welcome your thoughts on the controlled APS comparison and the clarified toy metrics before the discussion deadline. We hope these additions address your concerns and would be happy to clarify any remaining questions. Thank you again for your time and consideration.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revision Summary

  • APS protocol comparison. Relabeled the original results as cross-protocol MSLE gaps (Figure 1, Section 4.4, and Table 11). Relaxing either training-time constraint separately lowered MSLE for all five models on the fixed test set. With only the label rule relaxed, CasFlow moved from fourth to first as its MSLE dropped from 2.40 to 1.61 (Appendix M.1, Table 17).
  • Toy evaluation metrics. Documented the original MSLE filter (Appendix E, Tables 8–9) and added full-test-set MSLE and MALE. In Scenario 2, standard MSLE still places the MLP first under Random Cascade and fifth under Full Temporal, where CasTemp ranks first (Table 1 and Appendix M.2, Tables 18–19).
  • CasTemp inputs. Specified how the competition graph, cross-cascade walks, and historical features all use information available by the same forecast cutoff (Appendix H).
  • Taoke documentation and experiments.
    • Documented product selection and release fields for 2,862 products with paired forwarding and sales labels (Appendix A.1), defined the sales target (Section 6.1), and reported daily label distributions (Appendix M.3, Table 20).
    • Side information improved both tasks under both training regimes. The largest gain was a 10.7% reduction in sales MSLE under past-only training (Appendix M.4, Table 21).
    • With common inputs and matched sales-head capacity, CasTemp retained the lowest sales MSLE at 1.998, compared with 6.347 for CasCN, the strongest baseline (Appendix M.5, Table 22).
AuthorsOpenReview ↗

Rebuttal Summary

Dear Reviewers,

Thank you for your feedback and discussion. We summarize our rebuttal responses below, with references to the revised manuscript. The additional experiments and clarifications are marked in blue.

Reviewer ZmZq

  • W1 and Q1. We describe the original APS differences as cross-protocol MSLE gaps and added a controlled comparison with identical test cascades, local windows, targets, and test inputs. Relaxing training-label timing alone lowers MSLE for all five models and moves CasFlow from fourth to first. The four conditions also examine feature timing (Section 4.4, Figure 1, Table 11, and Appendix M.1, Table 17).
  • W2 and Q2. We documented that the original toy MSLE filtered squared log residuals above 10 while MALE used the full test set. We added standard full-set results. In Scenario 2, the MLP still ranks first under Random Cascade and fifth under Full Temporal, where CasTemp ranks first. Table 1 now reports these standard values (Appendix E and M.2, Tables 18–19).
  • Q3. We specified how the competition graph, promoter sets, cross-cascade walks, and historical features use information available before each forecast cutoff (Appendix H).

Reviewer qRAL

  • W1 and Q1. We connected Taoke’s persistent entities and changing activity to temporal evaluation using the existing protocol and drift statistics. We also added a fixed-cohort comparison of training-time constraints and side information. Side information improves both forwarding and sales forecasting, with a 10.7% sales MSLE reduction under past-only training (Tables 10, 12, and 16, Appendix M.4, Table 21).
  • W2 and Q2. We added the product-selection rule, release-field documentation, the sales-target definition, and daily label distributions (Appendix A.1, Section 6.1, and Appendix M.3, Table 20).

Reviewer tFhA

  • W1. The fixed-task APS comparison addresses the requested chronological comparison while preserving the local task and test population (Appendix M.1, Table 17).
  • W2. We clarified prediction-time cutoffs and added a comparison with common covariates, a shared predicted-popularity signal, and matched sales-head capacity. CasTemp retains the lowest sales MSLE, at 1.998 versus 6.347 for CasCN (Appendix H and M.5, Table 22).
  • Follow-up on cascade information and forecast dates. We compared a sales-history-and-product-feature MLP with versions augmented by frozen cascade embeddings. CasTemp reduces mean MSLE from 9.117 to 6.031 (33.85%), improving on all six dates across August and March datasets from the same promotion scenario. CasCN and CTCP also improve on all six dates. VNOIP improves on three dates, with a 0.28% overall reduction (Appendix M.6, Tables 23–24).

We hope these revisions address the concerns raised and would welcome your assessment of the new evidence. We would be glad to discuss any remaining questions and would appreciate reconsideration where the additional results address the basis of the original assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.