acceptodds
Under review as a conference paper at ICLR 2027

Learning Collective Dynamics with Differentiable Gaussian Representations

Abstract

Social-media interactions and market transactions arise from individuals with different response propensities: contact opportunities determine which individuals generate observed events, and accumulated experience changes later responses. Predicting collective dynamics from aggregated event and behavior counts therefore requires learning how individual responses sum to current totals and how feedback shapes later behavior. Because population type composition, contact selection, and experience-induced propensity changes correspond to the components, intensity reweighting, and translation of a Gaussian mixture, we represent response heterogeneity with a trainable Gaussian mixture. Evaluating the intensity of arrivals (recorded contacts or transactions) and conditional behavioral probabilities at the same latent type, the model integrates their product to predict arrival counts, joint behavioral probabilities, and behavioral totals; observed or expected outcomes then update the state. Reparameterized quadrature and cross-day recurrence let predictive losses jointly train the representation, observation functions, and feedback parameters. We also derive conditions for count-likelihood sufficiency and parameter equivalence, and analyze cross-day propagation of quadrature error. On four KuaiRand-Pure and Online Retail II windows, the model has lower joint behavioral negative log-likelihood (NLL) than a DeepAR adaptation with a joint-behavior head, with 1.1232 versus 1.3466 in the KuaiRand-Pure standard-recommendation window; in Retail 2010, one-day count MAE averaged over three basket marks is 4.71 versus 6.88. In the standard-recommendation ablation, learning the distribution reduces behavioral NLL by 10.82% versus a fixed Gaussian; in a controlled dynamics experiment, removing dynamics raises mean joint KL from 0.0340 to 0.2577.

Primary area:
learning on time series and dynamical systems
Keywords:
collective dynamics, Gaussian representations, aggregate observations, response prediction, differentiable learning, social simulation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_QGQuOpenReview ↗
Rating
4(+1)
Confidence
2(+1)
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper presents differentiable Gaussian dynamics (DGD), which makes three contributions related to understanding population-level behavioral dynamics from aggregate observations.

  • They use a mixture of Gaussians in a latent space UU (containing hidden preferences) to model heterogeneity along distinct cohorts.
  • This latent distribution is then used to model observable behaviour using an autoregressive logistic model followed by differentiable numerical integration.
  • Transition parameters are used to predict response shifts over the following time stamp. These parameters are jointly optimized using a composite loss involving NLL terms, forecasting losses, and regularization. This method is tested on two datasets and outperforms neural-network-based baselines.
Critical strengths
  • This paper shows that a simple structured probabilistic model outperforms more complex GRU-based models. The experimental advantage is impressive, and the role played by each parameter in the model makes this approach much more interpretable.
  • Experiments seem to justify the complexity of the theoretical results. The ablations provide strong evidence that the proposed modelling components are useful. The error bounds also seem non-vacuous (but we need more experiments to convincingly show this).
  • The methods developed in this paper have applications to other areas in ML/statistics of interest to the reader. Gaussian representation, differentiable aggregration, and feedback recurrence seem like commonly used methods in related communities.
  • I went through the proofs and they seem correct to me.
Critical weaknesses
  • The presentation needs improvement. Section 3 introduces a substantial number of variables and parameters across its subsections, often without sufficient explanation of their roles or relationships within the overall framework. This makes the methodology difficult to follow. It would be helpful to have, for example, a table of each variable.
  • At the moment, the method assumes that individuals have already been assigned to cohorts based on historical information. This limits its applicability to settings where meaningful cohort assignments are not readily available. It would be interesting to know how DGD can be extended to perform these cohort assignments as well.
  • Some of the experimental choices are not sufficiently discussed. For example, why did the authors conclude that two Gaussian components are sufficient? Also, why is a 1D latent space sufficient?
  • For HH binary behaviours, there are 2H2^H possible patterns. This can make the computation and optimization of conditional behaviour expensive.
Questions for authors

Can the authors provide empirical computations of the error bounds derived in Appendix C? At the moment, its practical relevance remains unclear. It would be helpful to also compare the theoretical bounds with the empirical errors in the experimentals. This would clarify how vacuous the bounds really are. Also, I'm curious how the bounds scale with the number of quadrature nodes.

2 comments
AuthorsOpenReview ↗

Response to Reviewer QGQu

Thank you for checking the proofs.

W1. Readability of Section 3. The revised Section 3 will open with a table that gives each symbol's meaning, parameter group and first equation. We will also add the regularization formula, the behavior ordering, with its sensitivity to reordering in the appendix, the initialization, and the Cholesky lower bound.

W2. Cohorts are defined in advance. Cohorts are the model's units, and Appendix D.1 gives the partition rules, which are fixed before training. DGD can be extended by treating cohort membership as a latent variable that depends on individual features and training the soft assignment with the same aggregate likelihood; the revised Limitations will discuss this.

W3. Why K=2 and d=1. Over K∈{1,2,3,4} and Q∈{5,7,11}, moving from K=1 to K=2 gives most of the gain (Standard 1.2565 → 1.1232), and larger K adds nothing consistent. For d, we fitted d=2 (10 seeds) and d=3. On bucket 1 in Standard, with 9 training days, d=2 lowers joint NLL by 0.015 [0.003, 0.032] relative to d=1 and is 0.018 [0.009, 0.029] below a discrete model with the same number of distribution parameters; on five further buckets d=2 is lower than d=1 on four and lower than the discrete model on all five. On bucket 1, between d=1 and d=2 validation selects d=2 in Standard and d=1 in Random, and d=3 does not lower test NLL below d=2; the revised paper will report these choices. Our reply to LfFR (W4) has the details.

W4. Cost of enumerating 2H2^H patterns. For d=1, the forward pass costs O(K,Q,2HH)O(K\\,Q\\,2^H H) per cell-day, and the largest intermediate tensor has shape [cells, K, nodes, 2H2^H, H]. Single-thread forward-plus-backward time:

ChangeTensor sizeMeasured time
H from 5 to 8 (Q=7)×12.8×6.7
Q from 7 to 25 (H=7)×3.57×3.66

Time grows linearly with the number of nodes, and more slowly than the tensor as H increases. DGD has 452 parameters on KuaiRand and 148 on Retail. A full single-thread fit takes 24–68 seconds. The enumeration limits how large H can be; the revised paper will state this and report parameter count, fit time and memory separately.

Question. How large are the error bounds in practice, and how do they scale with the number of nodes? On the trained models, the local probability bound is about the same size as the actual quadrature error for Q≥7, and the gradient bound is within a factor of 1–3 wherever the error is above floating-point precision. In Standard the probability error falls from 6×10⁻⁴ at Q=7 to 7×10⁻⁷ at Q=25; this check uses a high-order reference integral. The recursive bound in Appendix C.3 is about six orders of magnitude wider than the actual error, because the per-step Lipschitz factor exceeds 1 and compounds over the horizon. The revised Appendix C will present it as an inequality next to a bound-versus-error table.

Thank you for your reply. Your response addresses several of my concerns, so I am happy to increase my score to a 4.

Reviewer_hoTjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Differentiable Gaussian Dynamics (DGD), a structured model for forecasting aggregate arrival counts and joint behavioral outcomes. Population heterogeneity is represented by a Gaussian mixture whose components share a time-varying translation. An arrival-intensity model and an autoregressive logistic response model map this distribution to expected counts and joint behavioral probabilities. A deterministic recurrence updates memory, fatigue, response shifts, and arrival states using observed or predicted outcomes. Parameters are learned jointly through numerical integration and backpropagation through time.

The principal contribution is the integration of these components into a compact model trained from aggregate joint-pattern counts without requiring persistent individual trajectories. Validation includes four windows from KuaiRand-Pure and Online Retail II, synthetic experiments, representation and dynamics ablations, and theoretical and numerical analyses. DGD improves behavioral likelihood over the evaluated recurrent neural baselines and reduces retail behavioral-count errors. Supplementary code provides implementations, comparison pipelines, and tests.

Critical strengths
  • A coherent model for a practically relevant observation regime. DGD jointly predicts activity volume and correlated behavioral outcomes from aggregate observations. Its compact population representation and explicit recurrence provide a plausible inductive bias for short time series, where flexible neural models may be difficult to estimate reliably. The combination is potentially useful even though its individual components have established precedents.

  • Encouraging predictive results and substantive implementation support. DGD achieves lower joint behavioral NLL than DeepAR-joint across all four evaluated windows. Retail behavioral-count MAE improves from 6.88 to 4.71 in 2010 and from 5.77 to 5.38 in 2011. The parameter-matched representation comparisons are valuable. The supplementary implementation further includes tests of normalization, finite-difference gradients, aggregate scoring, and forecast information boundaries. These provide evidence of implementation care and increase confidence in the evaluated computational properties.

Critical weaknesses
  • The incremental benefit of the distinctive architecture is insufficiently established. The paper combines established ideas from heterogeneous-response modeling, aggregate learning, and observation-driven time series. Such a combination can constitute a useful contribution, but its advantage over close alternatives needs stronger empirical attribution. In Table 4, the matched discrete mixture achieves NLL 1.1339 versus 1.1232 for DGD, with similarly close marginal and cohort-level metrics. This supports learned heterogeneity more strongly than it establishes an advantage for the Gaussian representation. Appendix B.2 also establishes equivalence to a separated readout under its stated assumptions, making the independently fitted separated-readout results particularly relevant. Finally, the no-dynamics ablation removes arrival-residual correction and behavioral feedback together. Its improvement therefore does not isolate the contribution of memory, fatigue, and response-shift dynamics.

  • The available empirical evidence leaves substantial uncertainty about robustness and performance relative to structured alternatives. KuaiRand uses nine and four training days in its two windows and a single user hash bucket out of 64. The three-day recurrence ablation has only five forecast origins, while the main tables report training-seed variability. The supplementary code includes weekly seasonal arrival-count baselines, broader discrete-mixture comparisons, separated-readout models, and calendar-block bootstrap analysis. These are valuable inclusions, but their numerical outputs are not supplied in the archive. Consequently, the available evidence does not establish whether the gains persist against these alternatives or across additional population samples and temporal windows. This matters because the central case for acceptance rests on the predictive value of the proposed combination.

Questions for authors
  • Could the authors report held-out predictive metrics for the independently fitted separated-readout models and the matched discrete mixtures across the available windows? How should the practical benefit of intensity–behavior coupling be interpreted alongside Appendix B.2, and does intensity clipping materially affect that equivalence in the fitted models?

  • How much of the recurrence benefit remains when arrival-residual correction is retained but memory, fatigue, and response-shift feedback are removed? This distinction would materially affect whether the results support the proposed behavioral dynamics.

  • Could the authors provide the seasonal arrival-count baseline results and calendar-block bootstrap intervals implemented in the supplementary pipeline? Are evaluations available across additional user buckets or temporal windows? These results would help distinguish consistent improvements from sensitivity to the selected samples and dates.

3 comments
AuthorsOpenReview ↗

Response to Reviewer hoTj (1/2)

Thank you for the three concrete requests. You wrote that "the central case for acceptance rests on the predictive value of the proposed combination." With at most one-fifth of the parameters of the selected DeepAR-joint models, the combination lowers joint NLL by 14.7–15.9% on KuaiRand and 5.1–7.7% on Retail, and the gap holds across 6 user buckets and 6 Retail cut-offs. On real data the gain comes from the learned cohort distribution, whose Gaussian form is lower than a parameter-matched discrete mixture in all six Standard user samples at d=2, and from the arrival-error state; when responses change over time, as in the controlled dynamics, the gain comes from the response states (Q2, W1). The contact-weighted readout identifies the arrival-weighted distribution and fits lower than the separated one in three of four KuaiRand comparisons (Q1).

Q1. Separated readout, matched discrete, and truncation. Held-out joint NLL, DGD minus variant, so negative values favor DGD. Bucket 1 and Retail use 10 seeds; buckets 2–6 use 3 seeds each. ✓ marks a 95% interval excluding 0 when seeds and calendar blocks (3 days for KuaiRand, 7 for Retail) are resampled together 5,000 times; for buckets 2–6, buckets are resampled as well.

VariantStandardRandomRetail 2010Retail 2011Buckets 2–6, StandardBuckets 2–6, Random
Separated readout−0.012 ✓+0.002 ✓−0.002+0.001−0.014 ✓−0.021 ✓
Matched discrete (d=1)−0.011−0.003−0.000+0.002−0.011 ✓+0.016

At d=2, which validation prefers to d=1 for Standard on bucket 1, the Gaussian model is 0.018 [0.009, 0.029] below the matched discrete mixture over 10 seeds, and it is lower on each of the five new buckets (0.010 [0.004, 0.018] pooled, 5 seeds per bucket). Validation selects d=1 in Random.

The clip on log intensity at [−16, 10] fired 26 times in 352,800 node evaluations. When we apply the B.2 transformation to 40 trained models and reevaluate them with 61 nodes, NLL changes by 10⁻¹¹–10⁻¹⁴ in Random and Retail and by 1.9×10⁻⁴ in Standard, or 3.5×10⁻⁸ once the clip is removed. The two readouts therefore represent the same functions on the trained models, and the table compares how the two parameterizations fit: the contact-weighted form is lower in three of the four KuaiRand comparisons (by 0.012, 0.014 and 0.021, all ✓) and within 0.002 in Random on bucket 1 and on Retail. Counts identify the arrival-weighted distribution of recorded behavior patterns, and the readout makes its distance from the population distribution explicit: in total variation, 0.385 under recommendation and 0.042 under random exposure.

Q2. Keeping the arrival-residual correction without response states. Retrained with 10 seeds in each window; variant minus full DGD, so negative values favor the variant:

VariantMetricStandardRandomRetail 2010Retail 2011
Arrival-error state onlyJoint NLL−0.009 ✓−0.017 ✓−0.000−0.001 ✓
Arrival MAE+0.7−21.9 ✓−0.33 ✓+0.26
Behavioral-count MAE−0.0−0.8−0.0+0.2
Response states onlyJoint NLL+0.037 ✓−0.005 ✓−0.001+0.000
Arrival MAE+15.9+589 ✓+4.0 ✓−1.5
Behavioral-count MAE+1.4+15.8 ✓+0.9−0.5

Keeping only the arrival-error state of Eq. 9, which adjusts the next day's arrival level using the previous day's log-count error, matches or improves full DGD's joint NLL in every window. In Random it alone lowers arrival MAE from 782 without dynamics to 183 (205 for full DGD), which separates DGD from the sequence models there (682–759); count NB NLL falls from 6.81 without dynamics to 2.63 for full DGD. On the real windows the per-cell response distributions barely move over the test days: a per-cell table frozen at the training cut-off is 0.008–0.009 nats above one updated daily on KuaiRand and is the lower of the two on Retail. The response states add no further gain there.

We ran the same split on the controlled dynamics generator, where responses change over time (10 seeds; both columns are errors against the generator):

ModelJoint KLOracle arrival-intensity error
Full recurrence0.02323.6
Arrival-error state only0.22121.1
Response states only0.014108.8
No dynamics0.278107.8

Removing the response states raises joint KL by 0.198 [0.136, 0.259], and the full recurrence is lower in all 10 seeds; removing the arrival-error state raises the oracle arrival-intensity error from 23.6 to 108.8. The revised Section 4.4 will report the retrained state groups and the no-dynamics model separately, on the real windows and on the controlled generator.

AuthorsOpenReview ↗

Response to Reviewer hoTj (2/2)

Q3 and W2. Robustness.

On the four original windows, every model now has 10 seeds under one stopping rule, and Random DGD's joint NLL is 0.6738 ± 0.0096.

For buckets 2–6, users are assigned by a hash of their ID, and we rebuilt cohorts and units and reselected every model on each bucket. DGD minus comparison, so negative values favor DGD; buckets 2–6 are merged, and their intervals also resample buckets:

ComparisonStandardRandomBucket 1 (10 seeds), Standard / Random
DeepAR-joint−0.230 ✓−0.067 ✓−0.214 / −0.116
GRU−0.087 ✓−0.083 ✓−0.091 / −0.129
Fixed Gaussian−0.105 ✓−0.019−0.128 / −0.025

DGD beats both sequence models on every new bucket.

We moved each Retail cut-off by 14 and 28 days. Against DeepAR-joint, joint NLL differs by −0.107 to −0.200 at the four new cut-offs (all ✓) and by −0.095 to −0.200 over all six. Against the historical baselines, count MAE is 2.0–7.7 lower at the new cut-offs.

Seasonal minus DGD, one-day arrival MAE, so positive values favor DGD:

WindowSame weekday last weekWeekly-seasonal EWMA
Standard−11.9 [−28.6, +2.5]+1.6 [−14.6, +19.9]
Random+482 [+362, +610]not estimable (4 training days)
Retail 2010+3.5 [+0.1, +7.1]+4.2 [+2.3, +6.5]
Retail 2011−0.7 [−4.5, +3.2]+0.8 [−3.0, +4.0]

One-day forecasts have 10 origins in Standard and 7 in Random, and three-day forecasts have 8 and 5. The revised appendix will give the full tables, including the seasonal baselines and the three-day horizon. Buckets 2–6 replicate the results over users at bucket 1's test dates, and the Retail cut-offs replicate them over time.

W1. Attribution. The gain on real data comes from the learned cohort distribution (−0.128 ✓ against the fixed Gaussian in Standard, −0.105 ✓ on the new buckets) and from the arrival-error state (Q2). The Gaussian family matches the discrete control at d=1, as in Section 4.3, and at d=2 it is lower in all six Standard user samples: by 0.018 [0.009, 0.029] on bucket 1 over 10 seeds and by 0.010 [0.004, 0.018] pooled over buckets 2–6. The contact-weighted readout represents the same functions as the separated one and fits lower in three of four KuaiRand comparisons (Q1). When responses change over time, the recurrence's gain comes from the response states (controlled split in Q2). The revised Section 4 will give each component its own paragraph.

Thank you for the additional experiments. The broader evaluation and ablations addresses my concerns. Because of this, I am raising my rating to 3 (Weak acceptance).

Reviewer_LfFROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DGD learns collective response dynamics from aggregate counts using three components. A per-cohort Gaussian mixture over latent propensity is made differentiable through reparameterized quadrature nodes. Contact and behavior aggregation then evaluates arrival intensity and behavioral probability at the same latent propensity and multiplies them before integrating. A feedback recurrence updates behavioral memory, fatigue, response shift and the arrival environment. The three parts are trained jointly by backpropagation through time under a negative-binomial count likelihood plus a multinomial pattern likelihood. Evaluated on two KuaiRand-Pure windows and two Online Retail II windows against DeepAR-joint, two GRU variants and three historical baselines, with representation and feedback ablations and two controlled generators.

Critical strengths
  • Method design. Because contact intensity and response probability share one latent variable and are combined before integration, the model represents the fact that heavily reached subpopulations dominate the log.

  • Novelty of the construction. The paper assembles an explicit population representation, a differentiable observation operator and a feedback recurrence into one end-to-end trainable object, so that errors on observed counts update the distribution, the observation functions and the state transitions within a single objective.

  • Experimental completeness. Beyond forecast comparisons on real data, the paper designs two controlled studies that test whether the learned object is the right one, asking whether cross-period response distributions can be recovered from cohort counts alone and whether the fitted model extrapolates to product attributes and prices never seen during training.

Critical weaknesses
  • W1. The reported configuration uses a one-dimensional latent propensity, so a scalar loading per behavior forces the individual-level correlation between any two of the seven KuaiRand behaviors to be exactly plus or minus one. This is implausible when one of the behaviors is negative feedback. The behavior-dependence matrix captures co-occurrence but is a population-level fixed effect and cannot express heterogeneous individual correlation.

  • W2. Sec. 3.4 states that the arrival loading, the component means and the shifts can be absorbed into one another, while the controlled experiments report only joint KL and marginal probability error, which are invariant to exactly those reparameterizations. Distribution recovery is therefore demonstrated, but representation recovery, which the explicit-representation framing requires, is not.

  • W3. The central design, one shared propensity multiplied before integration, is never ablated. The decisive control is a two-stage variant that keeps the Gaussian mixture and the feedback recurrence but predicts arrivals and conditional behavior separately.

  • W4. The number of mixture components, the latent dimension and the quadrature node count are never varied. The matched discrete variant reaches an NLL of 1.1339 against DGD's 1.1232, which suggests the operative factor is having some learnable within-cohort spread rather than a Gaussian one.

  • W5. The negative-binomial dispersion is fixed rather than learned, which hard-codes the count variance, and the count loss is then down-weighted to 0.05. Whether count fit is limited by misspecification or by a near-zero loss weight cannot be determined from what is reported.

  • W6. The selection of the four creators plus one pooled unit, the window boundaries and the cohort partitions is never stated. With five and one observation units respectively, these choices could determine the ranking.

  • W7. The 3D Gaussian Splatting framing is rhetorical rather than structural, given a one-dimensional propensity space, quadrature in place of rasterization, and no view consistency or density control. It obscures the fact that the model is mixed logit with reparameterized quadrature plus an exponential-smoothing recursion.

Questions for authors
  • What are the results for a latent dimension of two or more and for more mixture components on KuaiRand, and what sign does the fitted loading take for negative feedback against clicking in the one-dimensional case?

  • Given the absorption property noted in Sec. 3.4, can you report parameter-level recovery against the controlled generator rather than only joint KL?

  • What does the two-stage ablation in W3 yield, keeping the same mixture and feedback but predicting arrivals and conditional behavior separately?

  • What do ablations over the component count, latent dimension and node count show, and what changes with learned dispersion or a different loss weight?

  • Can the fixed origins be replaced by rolling-origin evaluation, and can the unit, window and cohort selection rules be stated?

4 comments
AuthorsOpenReview ↗

Response to Reviewer LfFR (1/3)

Thank you for the detailed questions.

W1 / Q1. ±1 correlations under a one-dimensional latent. The ±1 constraint holds only for the latent term λh⊤u\lambda_h^\top u. Each outcome also includes Bernoulli noise, the dependence matrix Ψ\Psi, and the memory and fatigue intercepts. We simulated complete individual outcomes from the fitted models, with moments within 0.014 of the exact ones. At d=1 the click–negative-feedback correlation is −0.056 to −0.017 in Standard and close to 0 in Random, and at d=2 it stays below 0.06 in absolute value. Ψ[hate,click]\Psi[\text{hate},\text{click}] is negative in every seed. As you note, Ψ\Psi is shared across the population, so one factor still cannot separate two groups that both click often but differ in negative feedback. This needs d≥2, which the d=2 model provides (W4). The sign of λhate\lambda_{\text{hate}} is the same in all 10 seeds of each window, negative under recommendation (Standard) and positive under random exposure (Random), and the revised paper will report it as a direction of each window's arrival-weighted distribution.

W2 / Q2. Identifiability. As you point out, Appendix B.2's absorption property leaves some transformations unidentified. Once these are removed, we found no exact invariant direction beyond those in B.2, but aggregate counts constrain the latent shape weakly. We measure the Wasserstein-2 distance between the fitted and generating centred distributions. Averaged over seeds, fits started at the generating parameters stay close to them (0.03–0.06), while fits from the default initialization and from a distant one both end at 0.30–0.38. All three reach a test KL of 0.0002–0.0003 with 320 training days. The controlled experiments therefore show that DGD recovers the predictive distribution, and the revised Limitations will discuss parameter recovery.

W3 / Q3. Multiply-then-integrate. The two-stage control, fitted on its own, is higher than DGD by 0.012–0.021 (✓) in three of the four KuaiRand comparisons and within 0.002 of it in Random on bucket 1 and on Retail (table under hoTj, Q1). The gaps are small because the two readouts represent the same functions: on trained models the equivalence stated in Section 3 holds to 1.9×10⁻⁴. As your first listed strength notes, the design lets frequently contacted subgroups dominate the log. The total-variation distance between logged and population behavior measures how much: 0.385 under recommendation and 0.042 under random exposure.

AuthorsOpenReview ↗

Response to Reviewer LfFR (2/3)

W4 / Q4. Why Gaussian components, and were K and d tuned? On bucket 1 in Standard (9 training days), validation prefers d=2 to d=1, and there the Gaussian model is 0.018 [0.009, 0.029] below a discrete mixture with the same number of distribution parameters over 10 seeds (1.1158 ± 0.0058 against 1.1339 ± 0.0087). On each of the five new buckets the Gaussian model is again lower (0.010 [0.004, 0.018] pooled, 5 seeds per bucket), and across the six user samples it is lower in 33 of 35 seed-matched pairs. A discrete support point is the Σ→0\Sigma\to0 limit of a Gaussian component, and with the same parameter budget the two representations spend it on different structure: two Gaussian components keep within-component spread, while the discrete mixture uses more support points (three at d=1 and four at d=2, with 11 effective distribution parameters per cohort for both at d=2). Two properties can account for the difference. Under exponential contact weighting a Gaussian component's mean shifts by Σkγ\Sigma_k\gamma, which a discrete point cannot do; on trained checkpoints it accounts for 26–37% of the absolute within- plus between-component mean shift from the population to the logged distribution. With d≥2 a component also carries covariance, which gives each type its own correlation structure (W1). At d=1 the two are close in all four original windows, as Section 4.3 showed for Standard. Validation prefers d=1 in Random, and d=3 does not lower test NLL below d=2. At d=1 the discrete fits also vary more: in the controlled choice experiment occasional poor fits raise its mean KL (0.00160 against 0.00037, and 0.00070 against 0.00047), and on real data its seed-to-seed standard deviation is larger in 10 of 13 settings, including five of the six Standard user samples (0.022 against 0.012 pooled) and five of the six Retail cut-offs.

3D Gaussian splatting uses anisotropic Gaussians instead of points for the same two reasons: a Gaussian has extent and an orientation given by its covariance, while a point has neither. The revised paper will report the validation-selected d for each window. For K, going from 1 to 2 gives most of the gain (Standard 1.2565 → 1.1232), 3 and 4 add nothing consistent, and at K=2, Q=5, 7 and 11 differ by at most 0.0004. The revised Section 4.3 will report these scans, and a new appendix section will explain the choice of Gaussian components.

AuthorsOpenReview ↗

Response to Reviewer LfFR (3/3)

W5 / Q4. Dispersion and count-loss weight. The paper fixes r=50. In an added experiment that learns r through a softplus parameter, it ended between 47.9 and 50.0 in all 36 runs, so we profiled it directly. On KuaiRand, validation count NB NLL keeps falling to the smallest values tried (r=5 in the main grid, r=3 in an extended one), and r=50 is 0.22–0.48 worse than r=5; on Retail it is lowest at 10–20. Retraining at the profiled values moves joint NLL by at most 0.005 and improves count NB NLL by 0.40–1.17 on KuaiRand (unchanged on Retail), while arrival and count MAE get worse. Setting α=1.0 raises joint NLL (Standard +0.059, Random +0.125) and lowers arrival MAE. So r=50 gives the best point forecasts and underestimates the KuaiRand count variance, and a lower r buys count likelihood at the cost of point error; the revised paper will report this trade-off.

W6 / Q5. Selection rules and origins. The rules are in Appendix D.1, and the revised paper will repeat them in the main text. On KuaiRand the units are the four creators with the most exposures in the warm-up period plus one pooled unit, and the cohorts cross four tiers of historical click rate with two tiers of exposure volume, plus new users; Retail has four customer cohorts and one store. Each Retail window trains on January–February, validates on March 1–14 and tests on the rest of March; KuaiRand Standard trains on April 13–21, 2022 and Random on April 25–28, and both test through May 8. Bucket 1 was used because its users are disjoint from the bucket used in earlier model development. Each test day is forecast one step ahead on a rolling basis with a fixed training cut-off, and the new buckets and cut-offs are reported under hoTj, Q3.

W7. Mixed logit with exponential smoothing, and the 3DGS analogy. DGD builds on mixed logit and on observation-driven recursions, and the revised Section 3 will state this lineage. It differs in three ways: the mixing distribution is a per-cohort Gaussian mixture trained from aggregate joint-pattern counts; the same latent variable enters the contact intensity, which separates the population distribution from the logged one; and the recurrence carries response states in addition to the arrival level. On the real windows the arrival level is the part that improves prediction, and in the controlled dynamics the response states are (hoTj, Q2). We keep 3DGS as the inspiration and will spell out the correspondence. Closure under the operator, within-component covariance and gradients to location and shape carry over; view consistency, adaptive density control and rasterization have no counterpart in DGD. We will also qualify the multi-view analogy in the introduction. In 3DGS, many views constrain the scene's geometry; in DGD, counts from many dates and units constrain the predictive distribution.

Thank you for your detailed response. I will keep my original score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response to All Reviewers (1/2)

Thanks to all three reviewers. DGD's joint NLL is lower than DeepAR-joint's in all four windows: by 15.9% in Standard, 14.7% in Random, and 7.7% and 5.1% in the two Retail years (all ✓). The selected DeepAR-joint models have at least 5.2 times as many parameters. DGD is also lower than the GRU (561 and 276 parameters against DGD's 452 and 148) in all four windows. The advantage over DeepAR-joint holds on all five new KuaiRand user buckets and at all four new Retail cut-offs, and the advantage over the GRU on all five buckets. On Retail, DGD's behavioral-count MAE is lower than every historical baseline at all six cut-offs. Throughout, ✓ marks a 95% interval that excludes 0 when training seeds and calendar blocks are resampled together.

When the population's responses change over time, the historical distributions' joint KL is 3–13 times DGD's in the controlled dynamics experiment (0.072–0.310 against 0.023), and DGD's oracle arrival-intensity error (23.6) is lower than that of every external comparison (69.8–125) in all seeds. In the controlled choice experiment, the Gaussian choice model fitted to aggregate choices predicts new attribute–price combinations from the same generating distribution with KL 0.00037–0.00047, against 0.034–0.037 for an attribute MLP and 0.062–0.064 for a point model.

On the four original windows every model uses 10 seeds under a single stopping rule, with checkpoint selection sealed before test scoring. The added experiments cover five KuaiRand user buckets, four Retail training cut-offs, seasonal baselines, a retrained split of the two recurrent state groups on the real windows and on the controlled dynamics generator, d=2 and d=3 latent spaces with a parameter-matched discrete control, and numerical checks of Appendices B and C. The new buckets and cut-offs use 3 seeds per setting, and 5 for the Gaussian–discrete comparison; the recurrence splits, the Retail readout comparison and d=2 on bucket 1 use 10, and parameter recovery uses 5.

AuthorsOpenReview ↗

General response to All Reviewers (2/2)

We compare each component with its closest alternative. Gaussian components against a discrete mixture with the same number of distribution parameters. In Standard, where the learned distribution matters most (0.128 ✓ below a fixed Gaussian), the Gaussian model is lower in all six user samples at d=2: by 0.018 [0.009, 0.029] on bucket 1 over 10 seeds and by 0.010 [0.004, 0.018] pooled over the five new buckets. At d=1 the two reach similar NLL in all four windows (Section 4.3 showed this for Standard), and in Standard and Retail the discrete fits vary more across seeds. Gaussian components also add a within-component mean shift under contact weighting, which accounts for 26–37% of the absolute within- plus between-component shift from the population to the log (reply to LfFR, W4). Contact-weighted against separated readout. The two represent the same functions, as Appendix B.2 states, and agree on trained models to within 1.9×10⁻⁴; fitted independently, the contact-weighted form is lower by 0.012–0.021 (✓) in three of the four KuaiRand comparisons and within 0.002 elsewhere (reply to hoTj, Q1). Recurrence. On real data the gain comes from the arrival-error state, which lowers Random arrival MAE from 782 to 183. In the controlled dynamics experiment, where responses change over time, removing the response states raises joint KL from 0.023 to 0.221 (by 0.198 [0.136, 0.259]; the full recurrence is lower in all 10 seeds) (reply to hoTj, Q2).

The revised paper will add a parameter column to the joint-NLL table, report the Retail count result as a range over cut-offs, and describe the controlled experiment as recovering the predictive distribution.

The revised introduction will state the contributions as follows.

  • A compact collective-dynamics model for aggregate counts. DGD connects a Gaussian population representation, contact–behavior aggregation and a feedback recurrence. With at most one-fifth of the parameters of the selected DeepAR-joint models, it has lower joint behavioral NLL than the learned sequence models in all four windows and on six KuaiRand user samples, and than DeepAR-joint at six Retail training cut-offs.
  • What aggregate counts identify. Aggregate pattern counts are sufficient for the event-level likelihood and identify the arrival-weighted distribution of recorded behavior patterns. We bound the quadrature error and its propagation across days; on trained models the local bounds are close to the actual error.
  • When a shared population representation helps. DGD is stronger when arrivals and behaviors are predicted together (lower behavioral-count MAE than every historical baseline at all six Retail cut-offs, by at least 54% at the original two) and, in controlled experiments, when responses evolve over time, where the gain comes from the response states. Retrained ablations locate the real-data gains in the learned cohort distribution (Standard), whose Gaussian form is lower than a parameter-matched discrete mixture at d=2 in six user samples, and in the arrival-error state.
AuthorsOpenReview ↗

Revised Paper Uploaded

We have uploaded the revised paper. The main text is 10 pages, and Appendices J–N are new. Locations refer to the revised PDF, and each change is followed by the reviewer point it answers.

Main text

  • Abstract and Section 1: results under the 10-seed protocol; the contributions as listed in our general response; the multi-view analogy stated for the predictive distribution (LfFR W7).
  • Section 2: the properties of 3D Gaussian Splatting that carry over to DGD (LfFR W7).
  • Section 3: lineage from mixed logit and observation-driven recursions; notation table (Table 1); the two state groups of the recurrence (Eqs. 8 and 9); behavior order and Cholesky floor (LfFR W7, QGQu W1).
  • Section 4.1: rules for units, cohorts and dates; the added user buckets and Retail cut-offs (LfFR W6, hoTj Q3).
  • Section 4.2: parameter column in Table 2; replication on buckets 2–6 and at the moved Retail cut-offs; count MAE at six Retail cut-offs (hoTj Q3).
  • Section 4.3: one paragraph per comparison: learned against fixed distribution, Gaussian against matched discrete components at d=1 and d=2 with the K and Q scans, contact-weighted against separated readout (hoTj Q1 and W1; LfFR W3 and W4; QGQu W3).
  • Section 4.4 and Table 4: retrained ablations of the two state groups on the real windows and on the controlled generator (hoTj Q2).
  • Section 5: Limitations on parameter recovery, predefined cohorts and the 2H2^H enumeration (LfFR W2; QGQu W2 and W4).

Appendix

  • A.3 and A.4: initialization, regularizer, sensitivity to behavior order (QGQu W1).
  • B.2: clipping frequency and the equivalence on trained models (hoTj Q1).
  • C.4 and Table C1: bounds against actual errors (QGQu).
  • D.2: protocol for the 10-seed runs and the added experiments.
  • E.3 and E.4: seasonal baselines and interval estimates (hoTj Q3).
  • J, Tables J1–J7: complete results, with the seasonal baselines, the three-day horizon, buckets 2–6 and the Retail cut-offs (hoTj Q3).
  • K and Table K1: Gaussian components and latent dimension (LfFR W4).
  • L: individual-level correlations, the sign of the negative-feedback loading, parameter recovery (LfFR W1 and W2).
  • M: dispersion and count-loss weight (LfFR W5).
  • N, Tables N1–N2: computational cost (QGQu W4).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.