acceptodds
Under review as a conference paper at ICLR 2027

Test the Record, Not the Imputation: Model Criticism of Temporal Point Processes from Aggregated Counts

Abstract

Event data such as neural spike trains and activity logs are often recorded only as counts per interval. Exact-time diagnostics for temporal point processes, such as time rescaling, must then impute event times. For history-dependent models, we show that the imputation can decide the verdict: different imputations of the same record can reverse the diagnostic. We instead criticize a model through the distribution it induces on the recorded counts. For continuous-time neural generators this distribution is implicit, so our test, ObsCritic, estimates count forecasts by simulation or particle filtering. It then accumulates signed residuals that show which counts are mispredicted after which histories. Calibrating the whole procedure by Monte Carlo keeps the test valid in finite samples, even when the forecasts are approximate. On synthetic data where frequencies and adjacent transitions of count categories stay fixed, ObsCritic detects changes in higher-order dynamics that imputation-based tests miss. A jointly calibrated library of lag features detects the same alternative. On this matched alternative, signed count residuals are substantially more powerful than a PIT-based diagnostic using the same forecasts. ObsCritic also detects misspecification of a Transformer Hawkes Process and, on real datasets, rejects lower-capacity generators more often. These results support evaluating generators through their distributions on recorded counts.

Primary area:
learning on time series and dynamical systems
Keywords:
temporal point processes, aggregated count data, goodness-of-fit testing, model criticism, Monte Carlo tests, particle filtering, Hawkes processes

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WBynOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies model criticism for temporal point processes when only binned event counts are observed. Instead of imputing one exact within-bin event-time sequence and applying standard time-rescaling diagnostics, the paper evaluates the distribution induced on the observed counts while treating the missing event history as latent uncertainty.

Critical strengths
  1. The central idea is simple and convincing: test the observed record, not an arbitrary imputation.
  2. The Monte Carlo calibration gives the procedure strong statistical grounding even when particle forecasts are approximate.
Critical weaknesses
  1. My main concern is scalability. The particle-filter implementation works well in the evaluated settings, but it is unclear how it behaves for long histories, high-dimensional latent states, or marked/multivariate TPPs where particle degeneracy may become more severe.
Questions for authors
  1. The authors needs to provide a more comprehensive study on the scalability/efficiency of the proposed methods, especially when dimensionality becomes high, sampling SMC can be burdensome.
  2. Has the author studied the accumlation of error over long horizons of SMC? How does that affect the power and applicability of the developed test.
  3. The current particle formulation marginalizes uncertainty over latent event histories while treating the fitted TPP parameters as fixed. Could the framework be extended to propagate parameter uncertainty as well, yielding a Bayesian posterior-predictive critic with uncertainty estimates on the discrepancy? How would such an approach relate to the current finite-sample Monte Carlo calibration, and would the additional nested inference remain computationally feasible?
4 comments
AuthorsOpenReview ↗

Response to Reviewer WByn (1/2): scaling and accumulation of SMC error

Thank you for the positive assessment. Your three questions go to the practical limits of the method, and the new Appendix A.6 answers them. We reply in two parts, with your comments quoted in italics.

W1. My main concern is scalability. The particle-filter implementation works well in the evaluated settings, but it is unclear how it behaves for long histories, high-dimensional latent states, or marked/multivariate TPPs where particle degeneracy may become more severe.

Q1. The authors needs to provide a more comprehensive study on the scalability/efficiency of the proposed methods, especially when dimensionality becomes high, sampling SMC can be burdensome.

On both points, the time to score a record grows linearly with its length, and particle degeneracy stays moderate as the latent dimension, the event rate and the number of event types grow (Appendix A.6).

SettingRange testedResult
Record length (recurrent generator of the main experiments)60, 240, 960 and 3,840 sMedian time to score one record is 1.1, 5.2, 18.1 and 69 s, linear in the length
Latent dimensionHidden size 3 to 5121.3–54 s for a record of 60 s; median ESS ratio 0.995–0.998 (minimum 0.58)
Dense bins1.4 to 50 events per binMedian ESS ratio 0.976–0.998 (minimum 0.28)
Multivariate Hawkes process2, 5, 10 and 20 event typesMedian ESS ratio 0.97, 0.94, 0.89 and 0.80

Here the ESS ratio is the effective sample size divided by the number of particles. Calibration multiplies the cost by the number of null groups, which run independently and in parallel.

We have not tested continuous marks, which would need a matching proposal. Appendix A.1 says so.

Q2. Has the author studied the accumlation of error over long horizons of SMC? How does that affect the power and applicability of the developed test.

The error does not accumulate, and power is nearly the same with particle forecasts and with exact forecasts. We checked this on a Markov-modulated Poisson process, for which the count forecasts can be computed exactly.

  1. Error over time. With 4 hidden states and records of 3,840 s (76,800 bins), the total variation distance between particle and exact forecasts is the same in the first and the last tenth of the record: 0.0020, 0.0010 and 0.0005 with 16, 64 and 256 particles.
  2. Size of the error. It shrinks roughly as N−1/2N^{-1/2}. With 64 particles it is about 60 times smaller than the noise of the 128 predictive draws.
  3. Power. For an alternative that doubles the relaxation rate of the hidden chain, the lag-2 statistic rejects as follows.
Record length60 s240 s960 s
4 states, particle / exact2% / 4%22% / 29%74% / 73%
32 states, particle / exact2% / 3%13% / 11%22% / 21%

Whatever the particle error, the Monte Carlo calibration covers the test as it is run. Particle error can cost power, but it cannot raise the false rejection rate. Path degeneracy does occur. It is harmless here because this model forgets its history, and we make no such claim for models with very long memory.

AuthorsOpenReview ↗

Response to Reviewer WByn (2/2): parameter uncertainty

Q3. The current particle formulation marginalizes uncertainty over latent event histories while treating the fitted TPP parameters as fixed. Could the framework be extended to propagate parameter uncertainty as well, yielding a Bayesian posterior-predictive critic with uncertainty estimates on the discrepancy? How would such an approach relate to the current finite-sample Monte Carlo calibration, and would the additional nested inference remain computationally feasible?

Yes, and it fits the existing calibration, keeps the finite-sample guarantee, and costs about as much as the current test. Each null group is simulated from one draw of the parameters, and every record is scored with particles that each carry a parameter label, so the forecast mixes over parameters. Under the posterior predictive null, the observed group is distributed like the simulated groups, and all of them are scored in the same way. The B+1B+1 statistics therefore remain exchangeable, and the Monte Carlo p-value stays valid in finite samples.

We tried this on an exponential Hawkes process fitted by maximum likelihood, with 8 bootstrap fits as the parameter draws (Appendix A.6). The nominal level is 4%.

Fixed fitRefit in every null groupPosterior predictive
False rejection, 30 s of training data11.3%6.3%5.0%
False rejection, 100 or 1,000 s2–5%2–5%2–5%
Time per testabout 45 s43–163 s43–68 s

Against a renewal alternative, the three versions have similar power.

Two limits remain. The parameter labels degenerate over long records (3 to 4 of the 8 survive after 40 s). For neural generators each refit takes minutes, and we have not tested that case. The Limitations now note that, when parameters are estimated from a short record, a rejection can also reflect estimation error.

I like the originality of the work and the fact that the paper is addressing an important question in the field. I will raise my score

Thank you for reading our responses and for raising your score. We are glad that you find the question important. Your comments led to the new experiments on long records, larger hidden states, multivariate processes and parameter uncertainty, and helped us improve the paper.

Reviewer_QKPzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper looks out how hypothesis testing cannot use time rescaling when data is logged in bins without precise timing.

Critical strengths

Point process modeling metrics are useful in a variety of fields. The paper points out obvious problems when time points are inaccurately imputed during the process of assessing model fitness.

Critical weaknesses

There are are some hard to read portions and some awkward phrasing (seems likely a result of LLM production).

Questions for authors

What do the phrases 'reverse the diagnostic' nor 'jointly calibrated library of lag features' mean? Why is the latter referred to as 'this matched alternative'?

What is the "observed sigmafield"?

Is the choice of words "reverse the rescaling verdict" common?

I've never seen the words "prequential" or 'evaluable' before, what do they mean?

Why does Algorithm 1 appear before the variables are defined in Section 3.2

Is there an explicit definition of the history Ft−1Y\mathcal{F}_{t-1}^Y that is missing from Equation 3?

How do G0G_0 and G1,…,GBG_1, \ldots, G_B relate to QTQ_T from (5) (maybe there is no sense of subscript like in Algorithm 1) so that Q0Q_0 and QbQ_b from 6 match?

3 comments
AuthorsOpenReview ↗

Response to Reviewer QKPz

Thank you for listing the unclear phrases one by one. Several passages in the submission were hard to read. We rewrote the abstract, the introduction and parts of the method section in plainer words, and added the definitions that were missing. The changes are in blue. Your questions are quoted in italics.

Q1. What do the phrases 'reverse the diagnostic' nor 'jointly calibrated library of lag features' mean? Why is the latter referred to as 'this matched alternative'?

These three phrases were unclear, and we have removed them from the paper. This is what they meant.

  1. "Reverse the diagnostic". The same record can pass time rescaling under one imputation and fail it under another. The abstract now gives the numbers instead. With the correct model, time rescaling rejects 4% of records under uniform imputation and 100% under imputation skewed toward one end of each bin. Across eight uniform imputations of the same 50 records, 9 records were accepted under some imputations and rejected under others.
  2. "Jointly calibrated library of lag features". We take the maximum of the statistics for several lags and calibrate that maximum as one statistic, so that the search over lags is accounted for. Section 5.2 now says this directly.
  3. "This matched alternative". The phrase meant the reordered records of Section 5.2. These records keep the frequency of every count and of every pair of neighboring counts. It did not mean the library, and the sentence did not make that clear.

Q2. What is the "observed sigmafield"?

It is the information in the counts recorded before bin tt, written Ft−1Y=σ(Y1,…,Yt−1)\mathcal F^Y_{t-1}=\sigma(Y_1,\ldots,Y_{t-1}). We removed the phrase, and Section 2 now introduces Ft−1Y\mathcal F^Y_{t-1} in those words.

Q3. Is the choice of words "reverse the rescaling verdict" common?

No. It is replaced by the rejection rates given under Q1.

Q4. I've never seen the words "prequential" or 'evaluable' before, what do they mean?

"Prequential" means that each forecast is judged by the next observation, and "evaluable" meant "can be computed directly". The first is an established term in forecast evaluation, short for "predictive sequential". The introduction now says "sequential forecasting", and the term appears only once, in the related work, with its reference. The second word is replaced by the plain phrase.

Q5. Why does Algorithm 1 appear before the variables are defined in Section 3.2

Algorithm 1 summarizes the particle implementation, so it stays with that subsection. A sentence before it now says where hth_t and QTQ_T are defined (Section 3.2) and where the calibration is described (Section 3.3).

Q6. Is there an explicit definition of the history Ft−1Y\mathcal F^Y_{t-1} that is missing from Equation 3?

Yes. The definition was given only in the introduction, and Section 2 now repeats it immediately before Eq. (3).

Q7. How do G0G_0 and G1,…,GBG_1,\ldots,G_B relate to QTQ_T from (5) (maybe there is no sense of subscript like in Algorithm 1) so that Q0Q_0 and QbQ_b from 6 match?

QbQ_b is the statistic QTQ_T of Eq. (5) computed on group GbG_b, so Q0Q_0 is the observed statistic. A group is the set of records that are tested together. G0G_0 is the observed group, and G1,…,GBG_1,\ldots,G_B are groups simulated from the model and passed through the same logger. Algorithm 1 now writes QTQ_T with its subscript.

On the phrasing. The wording in these places was poor. As stated in our AI use disclosure, we used AI assistants for language editing and translation. In this revision we replaced vague wording with concrete statements and numbers, and we will gladly revise any other passage you find unclear.

Thank you for the revisions. I think it must clearer now.

Once concern I have upon reading it afresh is that the skewed imputation seem completely artificial to me. I understand the main point that the counts are what should be evaluated, but why would any sensical imputation every create a skew rather than uniform in the bin? Or is it that the actually timing is skewed into a bin, which could happen if the bin edges were time-locked with a stimulus.

Can this be clarified?

Thank you for reading the revision again. You are right that no one would recommend the skewed imputation as a method. We use it as an extreme case. Both imputations match every recorded count, yet with the correct model one rejects 4% of records and the other 100%. So the counts alone cannot tell which of the two results to report.

Uneven placement does occur in practice. When stored times are rounded down to the start of a second or a minute and then used as event times, every event sits at the start of its bin. This is a known source of bias in time rescaling for spike trains recorded at bin resolution, and Haslinger et al. (2010) developed discrete-time rescaling to correct it. As you suggest, true times can also be uneven within bins when the bins are locked to a stimulus or a schedule. Uniform imputation then places events away from where a correct model puts them.

The main results do not rely on the skewed case. The imputation tests in Section 5 fill in times uniformly and are calibrated by Monte Carlo. They reject about 5% of records from the correct model as intended but miss the change in the reordered records of Section 5.2. In Table 1, Max imputation rejects 4.3% of them, against 67.7% for ObsCritic, and Table S3 shows similar gaps at longer lags.

Your question showed us that the introduction did not make this clear. We added two sentences there in blue, saying that the skewed case is an illustration and that the imputation tests in Section 5 use uniform imputation.

Reviewer_yjsjOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets goodness-of-fit (model criticism) for temporal point processes (TPPs) when events are recorded only as counts per interval. It makes a sharp conceptual point: exact-time diagnostics such as time rescaling require imputing event times, and for history-dependent generators the imputation can decide the verdict.Validation is extensive: a matched-alternative benchmark preserving count-category frequencies and adjacent transitions.

Critical strengths

The problem showing that imputation can reverse the verdict of a standard diagnostic for history-dependent TPPs (Fig. 1) and re-grounding criticism at the observable pushforward — forecasts conditioned on the recorded filtration, not an invented path — is a conceptual correction that the neuroscience/TPP diagnostics community should take seriously. The paper delivers both the argument and working machinery for it.

Whole-procedure Monte Carlo calibration of the exact algorithm as run, near-nominal null rates across many independent banks, a matched alternative that defeats frequency and first-order controls, negative controls (reversal, long-memory), and closed-form validation make the reported power numbers trustworthy and the method practically usable.

Critical weaknesses

Power is contingent on feature–alternative alignment, and the headline claim rests largely on one construction. The matched alternative is built to maximize a→b→a returns over Euler trails — precisely the second-order structure the method's lag-2 feature indexes. With the wrong lag the test is at size, and with an intercept feature power collapses to ~6%; the jointly calibrated library rescues power only within the included feature family.

Replication variance in the headline power (47.5%–67.7%) is unexplained; the real-data "capacity ladder" establishes only compatibility with the chosen count categories (no ground truth, unexplained non-monotonicity for Taxi at the intermediate capacity). The THP study tests a single controlled alternative; and A.5 shows power depends strongly on bin width/atom dictionary (24–60% across 25–100 ms bins).

Questions for authors

The main matched study gives 203/300 (67.7%) power but replications A and B give 95/200 (47.5%) and 129/200 (64.5%) with the same method and alternative type. What explains this spread (bank size/calibration count, seed effects, or the alternative construction)? Would the paper's conclusion change if the reported power were the worst replication?

Since power is near-zero with an intercept feature and requires the "right" lag, how should a practitioner choose history features and count atoms for a real dataset? Can features be selected data-dependently (e.g., from the calibration bank) without breaking the exchangeability/calibration argument, and is there any guarantee that the joint library covers the space of plausible misspecifications?

6 comments
AuthorsOpenReview ↗

Response to Reviewer yjsj (1/2): feature dependence, one construction, spread across replications

Thank you for the detailed review. Your questions led to most of the new experiments in this revision. We answer them in two parts, with your comments quoted in italics.

W1. Power is contingent on feature–alternative alignment, and the headline claim rests largely on one construction. The matched alternative is built to maximize a→b→a returns over Euler trails — precisely the second-order structure the method's lag-2 feature indexes. With the wrong lag the test is at size, and with an intercept feature power collapses to ~6%; the jointly calibrated library rescues power only within the included feature family.

You are right on both points. Power does depend on the history feature, and the Limitations now say so. The submission did rest on one construction, so we ran two further studies, and the main claim holds in both.

(a) Other lags, another kind of dependence, and a second construction (Appendix A.9, Table S3). We tested returns at lags 2, 3, 4, 6, 10, 20, 30 and 100, a dependence in which each count tends to be followed by a different count, and a second reordering that keeps every run of four consecutive count categories. In every row the imputation tests reject 2.2–6.8% of records, against 4.9% for records from the correct generator. ObsCritic at the true lag rejects 20–62%, and it rejects more often than the imputation tests in every row (paired McNemar test, p<10−15p<10^{-15}; 500 paths per row).

(b) Same forecasts and same lag: our residuals against a PIT autocorrelation check (Appendix A.10, Tables S4 and S5). This comparison uses the four-category reordering because one random reordering of this kind changes the autocorrelations at lags 1 to 10 by less than two standard errors. Neither test rejects such a reordering more than about 6% of the time.

  1. When each count category is given a different successor, ObsCritic is the stronger test. Over all 44 such assignments and lags 4, 5, 6, 8, 10 and 20, it rejects significantly more often in 246 of the 264 settings (168 after Holm correction) and never less often. Median power is 21–23% against 7–9%.
  2. When counts recur, the two tests are close (26.4% and 24.1% at lag 4).
  3. The data show why. The power of the autocorrelation check follows how far a change moves earlier and later counts in the same direction (rank correlation 0.84). The power of ObsCritic does not (0.05).
  4. If the lag is unknown and lags 1 to 10 are tested together, both tests are weak in these settings (below 13% in every row of Table S4). This is why the choice of feature is listed as a limitation.

One change to the abstract follows from this. The submitted abstract compared ObsCritic with Q20, which pools lags 1 to 20 and does not choose a lag. Section 5.1 now states this, and the abstract reports the same-lag comparison instead.

Q1. The main matched study gives 203/300 (67.7%) power but replications A and B give 95/200 (47.5%) and 129/200 (64.5%) with the same method and alternative type. What explains this spread (bank size/calibration count, seed effects, or the alternative construction)? Would the paper's conclusion change if the reported power were the worst replication?

Most of the spread comes from the calibration threshold, not from the alternative, and the conclusion does not change under the worst replication. Replications A and B were calibrated with 99 null statistics and the main study with 499, so their thresholds are much noisier.

StudyCalibration statisticsNull rejectionPower
Main49912/300 (4.0%)67.7%
Replication A993/200 (1.5%)47.5%
Replication B9925/200 (12.5%)64.5%

Replication A drew a high threshold and B a low one: their null rates fall at the 8.3rd and 98.9th percentiles of what this design produces. In repeated simulations of each design, about 90% of the variance in power with 99 calibration statistics comes from the threshold. The 5–95% range of power is 47.9–78.1% with 99 statistics and 59.9–72.2% with 499. A new sample of the main study (2,000 null and 1,000 alternative paths) gives 68.4% (95% interval 65.5–71.2%). The null rates of A and B are now reported in Table S1 and Appendix A.8.

Under the worst replication, A, ObsCritic at lag 2 rejects 47.5%, while the controls reject 3.0% (frequency), 5.5% (first-order table), 10.0% (max imputation) and 34.5% (Q20; paired counts 55 against 29, p=0.006p=0.006).

For the maximum over lags (46.5%), Section 5.2 and Table S9 now give the spread over other sets of 64 reference paths: mean 39.3%, 5–95% range 30.9–47.9%.

AuthorsOpenReview ↗

Response to Reviewer yjsj (2/2): Taxi, THP, bin width, choosing features

W2. Replication variance in the headline power (47.5%–67.7%) is unexplained; the real-data "capacity ladder" establishes only compatibility with the chosen count categories (no ground truth, unexplained non-monotonicity for Taxi at the intermediate capacity). The THP study tests a single controlled alternative; and A.5 shows power depends strongly on bin width/atom dictionary (24–60% across 25–100 ms bins).

The replication variance is answered under Q1 in the first part. The other three points follow.

(a) Real data and Taxi. We agree that real data show only compatibility, and the paper claims no more (Section 5.3 and Limitations). The Taxi pattern comes from these model families on these data, not from the test.

We checked this by repeating the comparison where the truth is known. We simulated records from the fitted three-component model, then fitted and tested each capacity level, 20 times.

Records simulated fromExponential rejectedOne-component rejectedThree-component rejected
Taxi fit15/20 (median ratio 1.88)20/20 (14.29)4/20 (0.43)
Yelp-Airport fit20/2018/202/20

With known truth, Taxi again shows the middle model doing worse, and Yelp-Airport is monotone. The exponential and one-component LogNormal generators have hazards of different shapes and are not nested, so the middle model need not fit better than the first.

Refits agree (Table S10). Over five random initializations, the one-component mixture is always rejected (ratios 10.9–15.2), while the ratio of the exponential generator ranges from 0.56 to 1.65 and depends on the fit.

The same study shows that a correct but estimated model is rejected more often than 5% (6 of 40 runs, pooled). The Limitations now note this.

(b) The THP study. It does test a single alternative, and Section 7 says so. The THP study is meant to show that the particle implementation runs on a modern generator. It is not evidence about THPs in general.

(c) Bin width and dictionary. Power does depend on both, and we think this is inherent: the logger decides what the record can show. We reran A.5 with 200 null and 200 alternative paths per width (the submission used 50) and added a width of 200 ms.

Bin width25 ms50 ms100 ms200 ms
Fixed dictionary of low counts33.0%64.5%66.0%18.0%
Quantile dictionary, 3 categories25.5%46.0%18.0%5.0%
Quantile dictionary, 5 categories25.5%67.0%14.5%8.5%

Power is highest when bins hold one to three events on average. Dictionaries cut at quantiles of simulated counts are no better. The best one matches the fixed dictionary at 50 ms (67.0% against 64.5%) and is weaker at every other width. A practical rule is to choose the width so that bins hold one to three events on average, and to use the fixed dictionary of low counts. We have checked this rule only for this generator and alternative.

Q2. Since power is near-zero with an intercept feature and requires the "right" lag, how should a practitioner choose history features and count atoms for a real dataset? Can features be selected data-dependently (e.g., from the calibration bank) without breaking the exchangeability/calibration argument, and is there any guarantee that the joint library covers the space of plausible misspecifications?

In short, features can be selected from data if the same rule is applied to the observed group and to every null group, no library is guaranteed to cover every misspecification, and we suggest features from domain knowledge or a small library.

Selection done this way keeps the calibration, because the B+1B+1 statistics then remain exchangeable. We checked several rules on 1,000 separate null paths, with 99 or 499 calibration groups.

Selection ruleNull rejectionValid
Maximum over a fixed library3.9–5.8%Yes
Select on the calibration groups, then rank against the same groups7.0% with 99 groups, 5.7% with 499Inflated with few groups
Select on the observed record only33–34%No

A larger library costs power. The maximum over a fixed library has 37–38% power with 2 features, 31% with 4, and 21–23% with 8 or 11.

On coverage, the submitted library has about 8% power for returns at lags 4 and 6, and the long-memory control (A.7) stays at the null level. Useful features come from domain knowledge, such as refractory or periodic structure. For the count categories, the rule given in (c) above applies.

Thank you for your answer. My questions have been answered. Considering the originality of the paper, the general applicability of the conclusions, and the thoroughness of the experiments, I will keep my score unchanged.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of the revision

We thank the reviewers for their careful reading. A revised PDF is uploaded. This note lists what changed, in the order of the paper. All changed text is in blue.

The method, the theory (Section 4), and the main experiments with their numbers are unchanged. The main text still ends on page 9.

  1. Abstract. It now states the numbers behind Figure 1. Its comparison with a PIT diagnostic is now made with the same forecasts and the same lag. The submitted abstract compared ObsCritic with Q20, which pools lags 1 to 20 and does not choose a lag.
  2. Introduction and Sections 2 and 3. Several phrases and coined terms are replaced by plain descriptions. Ft−1Y\mathcal F^Y_{t-1} is defined in Section 2, immediately before Eq. (3). A sentence before Algorithm 1 defines a group and says where hth_t, QTQ_T and the calibration are defined. Section 3.3 defines QbQ_b. The same kind of replacement appears in Sections 4 to 6, in the proof sketches, and in several table headings and captions of the appendix. These change wording only.
  3. Section 5.1. It now defines Max imputation and states that Q20 does not choose a lag.
  4. Section 5.2. It adds a comparison of our residual statistic with a PIT autocorrelation check at the same lag (details in Appendix A.10). It also gives the spread of the power of the maximum over lags (details in Table S9).
  5. Section 7, Limitations. Two sentences are added: power depends on the history feature, and when parameters are estimated from a short record, a rejection can also reflect estimation error.
  6. Appendix, new subsections. A.6 covers long records, latent dimension, multivariate processes and parameter uncertainty. A.9 (Table S3) gives imputation tests for dependence up to 100 bins back and under a second reordering. A.10 (Tables S4 and S5) gives the comparison of item 4.
  7. Appendix, additions to existing parts. A.5 reruns the logger width study with 200 paths per width. Table S1 and A.8 report the null rejection rates of replications A and B. Table S10 reports Taxi generators refitted from five initializations.
  8. Figures. No data changed. Text inside figures cannot be colored, so the edits are listed here. In Figure 3, the panel titles and legend now use the wording of the text. In Figures 2, 5 and 7, "model-null" now reads "null".

Since the first revision we made two changes, both in blue.

  1. We added two sentences to the introduction, after the reference to Figure 1. They say that the skewed imputation is an illustration and that the imputation tests in Section 5 use uniform imputation.
  2. We revised the reproducibility statement. It now points to Appendices A and C and says that code and machine-readable results will be released upon acceptance.

The rest of the paper is unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.