- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper focuses on extending the neural marked temporal point process (MTPP) in two aspects: (i) it also models observed auxiliary variables (referred to as "action" in the paper) in addition to continuous-time events, and (ii) it extends the categorical mark space (i.e., each event has a single "target") to "multi-target" events (i.e., each event is a set of multiple targets, so duplicates are not allowed). The high-level ideas are summarized below:
For (i), the marked intensity (for each "target") is computed by summing up (1) the base target-specific intensity computed from observed event history and (2) a function modeling the historical effects from the auxiliary actions; see Eq. 2 and Eq. 6.
For (ii), the probability of each multi-target set is factorized into the probability of set cardinality and the multi-target set probability conditioned on its cardinality (Eq. 10), with follow-up refinement (Eq. 11-14).
They performed experiments on three datasets: one with single-target events and two with multi-target events. Evaluation metrics include sequence negative log-likelihood (Eq. 8), target/set negative log-likelihood, and NDCG@10 (often used in recommendation systems).
Critical strengths
- This paper aims to address useful extensions of classical MTPPs by incorporating informative observed auxiliary action variables in real-world applications. For example, in e-commerce, the purchase event is related to other consumer behaviors, such as page views or adding to carts, that have not been explicitly modelled by prior MTPP work.
- While prior work looked into joint probabilistic modeling of event time and set-valued events (Chang et al., 2024), they only captured negative correlations between targets but not positive correlations; this work extends to modeling both positive and negative correlations between targets.
Critical weaknesses
-
While the extensions are novel and well-motivated in their ideas, the methodology is not clearly explained, the terminology/notation is imprecise, and the modeling choices are not well-justified. For instance, and are mixed and and are mixed in Sec 2 and Sec 3. It is unclear how the parameterization is optimized with training data, which architecture the paper uses to compute the logits , why some design choices were made, or which component is conditioned on history and which is not; see "Questions For Authors" for more details.
-
Experiments lack soundness or reproducibility with lots of essential details missing and no accompanying code for assessment. What hyperparameters did the experiments choose for each architecture (in addition to learning rate, batch sizes, and optimizer in Table 5 in Appendix)? Is Target NLL/Set NLL (or mark NLL) computed conditioned on the true event time or sampled event time (Eq. 14 that defines the final target-set distribution does not condition on time or history)? It is unclear how the standard neural MTPPs (NHP/THP/S2P2) are extended to multi-target events. Prior work on set-valued events (Chang et al., 2024) is more closely related to target-set modeling than NHP/THP/S2P2 but is not included in the experiments. How is NDCG@10 computed, since all prior work does not consider any ordering in targets? See "Questions For Authors" for more questions.
Questions for authors
Although I wish to support the novelty of this paper, the current manuscript needs some work to clarify technical details and justify its modeling choices. A lot more detail would be needed to assess the correctness and soundness of the experimental results.
Concrete questions that would be good to clarify in the next version of this manuscript and would impact the assessment of correctness and soundness:
Methodology:
- Is the base intensity only a function of time, or is it also conditioned on event history?
- As a core component, how does the binary lifecycle state get updated between 0 and 1 when it sees more observations?
- What is in Eq. 2, is it a function or a parameter? What does this imply about the effect of a previous action on future events over time?
- How does an observed distribution look like (Sec 3.1)? In other words, how does such a distribution present in the training data?
- What are the parameters in and how to use the observations to optimize them?
- What architecture does each component use?
- Is the target-set distribution (Eq. 9) conditioned on event time or event history?
- How did the classifier for cardinality distribution trained on observations?
- How does sampling a target-set event work? First sample then conditioned on ?
- What does the sequence NLL (Eq. 8 for single-target events) for multi-target events look like if the set probability is normalized to non-empty sets (Eq. 9)?
- How does the cross-target interaction modelled via Eq. 12 with ?
- The inner product in Eq. 12 also depends on the magnitude of the vectors ; how does this impact that defines the interaction score? Why is there a factor of 2 when computing ?
- Is Eq.15 the training objective for target-set events? Why would we want to refine to be close to (which assumes conditional independence)?
- If the interaction parameters are optimized using in training, is only used in prediction, and is defined in Eq. 14 hyperparameters or learned from the training data?
Experiments:
- What are the statistics of each dataset? I see Table 4 in the appendix, but it does not include the number of possible targets or time granularity, and what are the "Risk intervals" (that is not mentioned elsewhere)?
- What are the hyperparameters for baseline models, and is a hyperparameter search performed on each dataset for each architecture?
- How were the models trained on each dataset, and are the reported results on validation or test data?
- How did the experiments adapt NHP/THP/S2P2 for multi-target learning and prediction? Do the target-specific logits use the hidden state values in neural MTPPs, and if at or was used for NHP/THP/S2P2?
- Is Target NLL/Set NLL (or mark NLL) computed conditioned on the true event time or sampled event time?
- In Sec 6.3 lifecycle update analysis, how were the probabilities and predictions computed on a continuous timeline (e.g., is any grid point used)? Could the authors elaborate on why realization status becomes more uncertain over time, and does this relate to in Eq. 2?
- What are the T1 - T8 in Fig. 5a? They do not make sense to readers in terms of what they imply in e-commerce interactions in the Synerise dataset.
- What kind of shuffling was done in Fig. 5b, and what does the experiment intend to tell the readers? Shuffling the target-pair assignments does not sound like what the model is optimized for, so I would not be surprised if the NLL is worse. Why is it evaluated on validation likelihood?
- In Appendix B.3, the paper suggests a synthetic experiment where the ground-truth intensity is available. Could the authors elaborate on whether the events are sampled from a parametric Hawkes process, how the actions are sampled, or the parameters of the ground-truth model used for simulation, and how the intensity RMSE is defined for a continuous function?
Minor issue that does not affect assessment:
- Typo at the beginning of Sec 3.3.
10 comments
## W2 — Experimental details and reproducibility
Thank you for highlighting these concerns. We have expanded the experimental settings, clarified the baseline adaptations and metric definitions, and released anonymous code. We also added a comparison using Chang et al.’s released predictor and Instacart data. The main results now report means and sample standard deviations across three seeds.
Architectures and hyperparameters. Appendices E.2 and E.3 specify the architectures, training settings, and checkpoint selection. For mobility, NHP, THP, S2P2, and Action-Transformer use 16-dimensional representations and up to 32 available actions. Action-Transformer has one attention layer with two heads. These models use Adam at , with epoch budgets and regularization selected for Action-Transformer through inner-fold validation and shared with the other three encoders. Action-Feature uses a three-hour action window, Adam at , and its own inner-fold selection.
For commerce, encoders process up to 32 interaction records and produce 32-dimensional representations. Action-Transformer has two attention layers with four heads each; Action-Feature mean-pools interaction embeddings. All models use the same log-normal time-head and nonempty Bernoulli basket-head architectures, with independently trained parameters. The basket head has one 32-unit GELU hidden layer. Training uses Adam at , batches of 512 intervals, five epochs, and validation Joint NLL selection. We do not conduct separate hyperparameter searches for NHP, THP, and S2P2.
Evaluation at observed event times. Target NLL and Set NLL use the observed event time and preceding history:
They do not use sampled event times. For basket prediction, is the recursive distribution before refinement and after refinement. Both retain time and history conditioning, as clarified in Section 4.1 and Appendix E.1.
Multi-target baseline adaptation. We retain NHP, THP, and S2P2 as history encoders and replace their original heads with the shared time and basket-head architectures. At purchase time , NHP decays its state to , THP uses a causally masked query at , and S2P2 uses its left-limit state before the event impulse. The basket head combines this pre-event representation with item representations and candidate features. The current basket is used only as the label. Each encoder and its heads are trained jointly using Joint NLL.
Comparison using Chang et al. (2024). We use the authors’ released conditional-independence predictor and processed Instacart department data. We train on 8,192 official training users, select checkpoints using 2,048 official validation users, and evaluate on the complete official test split: 26,192 users and 449,861 events. Additive and pairwise refinement use the same frozen predictor and matched training and selection settings.
Test Set NLL is without refinement, with additive refinement, and with pairwise refinement across three seeds. Both refinements yield small improvements, while ranking scores change little. This comparison uses training and validation subsets, rather than reproducing the original full-data training. Instacart has no prospective-action records, so it tests set prediction rather than lifecycle inference. Appendix F.6 provides the settings and results.
NDCG@10 for unordered baskets. We rank items by their predicted marginal inclusion probabilities:
Observed basket items have binary relevance; their recorded order is not used. We exclude Other from ranking and compute ideal DCG using the original basket size, capped at ten. Thus, NDCG evaluates the predicted ranking without requiring an ordered ground-truth basket. Appendix E.1 gives the definition.
Code and uncertainty reporting. Code and configurations are available in our anonymous repository. Three-seed standard deviations measure training variability. Paired test comparisons additionally use 2,000 user-cluster bootstrap resamples, conditional on the fitted models.
## M1 — M4
M1 — History dependence of the baseline intensity
Is the base intensity only a function of time, or is it also conditioned on event history?
Thank you for this question. The baseline intensity depends on both time and observed history. Explicitly, this dependence is:
The event rate uses a history-conditioned log-normal hazard. The mark model uses history-derived destination features for mobility and history and candidate-item features for commerce. Section 3.1 and Appendix E.3 specify these components.
M2 — Lifecycle state updates
As a core component, how does the binary lifecycle state get updated between 0 and 1 when it sees more observations?
Thank you for this question. We infer the posterior pending probability rather than directly observing the binary state . New actions start with , and explicit cancellations set .
No-event update. For an at-risk interval without an event, let . Then
This accounts for hidden expiry and the evidence provided by observing no event. Outside at-risk intervals, only expiry is propagated.
Event update. For an event with mark at time ,
All responsibilities use the same pre-event state, and all probabilities are updated simultaneously. We retain these marginals in a factorized Bernoulli approximation. Section 3.2 and Appendix B give the derivations; Section 6.3 and Appendices F.1–F.3 evaluate state recovery and inference accuracy.
M3 — Action-age kernel and temporal influence
What is in Eq. 2, is it a function or a parameter? What does this imply about the effect of a previous action on future events over time?
Thank you for this question. is a function of action age with learnable parameters. For mobility,
Shared learned mappings of action features produce and ; is learned and shared across actions. The rate rises to a peak and then declines. The action’s predictive contribution is
so its influence depends on both action age and the inferred pending probability.
For commerce, rates are piecewise constant over four action-age bins and depend on action features and interval-start history. Section 3.1 and Appendix E.3 specify the parameterizations.
M4 — Observed action targets
How does an observed distribution look like (Sec 3.1)? In other words, how does such a distribution present in the training data?
Thank you for this question. Each action records a single target , represented by the point-mass distribution
It is the specified destination for mobility or the item added to the cart for commerce. Thus, is observed rather than learned.
The action-source mark distribution is for mobility. For commerce, an event may contain several items, so
where is the recorded item if represented in the candidate set, and Other otherwise. The pending probability is inferred separately. Sections 3.1 and 4.1 define these quantities.
## W1 — Method clarity and implementation details
Thank you for your detailed review and constructive suggestions. We appreciate your recognition of the novelty and motivation of our work. We have revised Sections 2–4 to clarify notation, model definitions, and training, and expanded the appendices with derivations and implementation details. We address the specific points below.
Notation. We now consistently use Ht− and Ht+ for histories before and after an observation. The symbol m denotes an event mark, while v denotes an individual target: m=v for single-target events and m=V for set-valued events.
Parameter optimization. For mobility, we train the recursive model from scratch using user-averaged Joint NLL. For commerce, we first train the history encoder and prediction heads, then freeze the selected predictor and train the action-rate network. Both stages select checkpoints by validation Joint NLL. Pairwise refinement is trained subsequently with the recursive predictor frozen, using the objective in Section 4.4. No pending-state labels are required. Appendix E.3 specifies the optimization and selection settings.
Architecture for ℓv. For commerce, each candidate item’s inclusion logit is produced by a network with one 32-unit GELU hidden layer. Inputs comprise the 32-dimensional history representation, the item representation, their elementwise product, and augmented candidate features. The item representation combines a hashed embedding with a linear projection of these features, which include cart status, action age, and a learned gate. Appendix E.3 provides the architecture details.
History dependence. The table below summarizes the conditioning of each component. History includes only observations available at the prediction point; commerce action rates use the interval-start history representation. The target distribution Qi is fixed by the observed action, whereas the pending probability πi(t) is updated recursively.
We explain the individual modeling choices in our responses to the methodological questions below.