acceptodds
Under review as a conference paper at ICLR 2027

Architectural Leakage: Per-Timestep Sequence Models That Quietly See the Future

Abstract

Leakage in time-series evaluation is usually a data problem: a split that lets training see the test period, or a feature computed after the label. We document a form those checks cannot see, architectural leakage, in which the split and the features are clean and the model itself has a path from future inputs to past outputs. It arises whenever a per-timestep model conditions on a whole-sequence quantity—a bidirectional encoder, unmasked attention, symmetric padding, a pooled context vector—and it is invisible to split- and feature-level checks because the model is trained and scored under the same protocol. We give a model-agnostic detector, the future-perturbation score (FPS): perturb timesteps after , measure the change at timesteps ; a causal model scores exactly zero. It needs only forward(x, t): a replay on the truncated prefix detects the leak, and four value perturbations of the future narrow down what is read; a shuffle-only test would pass the form in the audited model, a mean-pooled context, because a mean is permutation-invariant. We then measure what such a leak is worth: on a monotone early-warning label, an oracle -bit channel carrying the onset admits a label-only score computed from the labels alone; on PhysioNet2019 a single such bit reaches (at least under a validation-selected partition), above every causal model we can train, and a bit predicted from whole-record summaries of the inputs already reaches . An audit of matched causal/non-causal pairs from four families (LSTM/BiLSTM, causal/symmetric TCN, masked/unmasked Transformer, running/whole-sequence pooling) finds every causal member of the four at – and every non-causal member beating its causal twin on all twenty splits, up to ; the gap survives per-patient scoring and the challenge utility. A routed additive model reported at AUROC over its causal baseline serves as the case study: a fair comparison gives , and reading its measurement process causally recovers AUROC on 19 of 20 splits ( normalised utility)—most of it without any router. The detector (under a minute per checkpoint) and the label-only scores (minutes on a CPU) are released.

Primary area:
learning on time series and dynamical systems
Keywords:
architectural leakage, causality, time series, early warning, sepsis prediction, evaluation protocol, mixture of experts, generalized additive models, benchmark audit

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_oWWdOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The authors highlight architectural leakage in per-timestep sequence models. The authors propose Future-Perturbation Score (FPS) as a detector, introduce label-only scores that quantify what a leaked bit is worth on monotone early-warning labels, and run a matched causal/non-causal audit across four architecture variants on PhysioNet 2019.

Critical strengths
  • I thought the introduction and related works were very well written. To the point and clear.
  • The details in the main paper and appendix is appreciated. I especially liked how the authors included the exact code implementation differences between different architectures in the paper.
Critical weaknesses
  • Although the intro and related works were well written, I thought the methodology and results were quite choppy sometimes, reading more like a log.
  • The authors mention this, but i think a key weakness is the paper's investigation comprises of one benchmark.
  • I thought the scope of the paper is a bit too large and tried to fit in too many experiments in the main paper.
Questions for authors
  • Have the authors tried running FPS on any published open-source sepsis or early-warning models that report per-timestep metrics?
  • Did any of the causal models improve meaningfully when given the same observation indicators that drove part of the routed model’s gain?
Reviewer_AeWsOpenReview ↗
Rating
4(+2)
Confidence
2
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper describes a form of leakage that checks on data splits and features will miss. A sequence model that predicts at every timestep can have an architecture that lets the output at step k read inputs that arrive after k. Bidirectional recurrence, attention without a causal mask, symmetric convolutional padding and a context vector pooled over the full sequence all allow it. Training and evaluation raise no error because the model is fit and scored under the same protocol. The authors propose a black-box test, the future-perturbation score, which changes the timesteps after k and measures how far the outputs at or before k move. A causal model scores exactly zero. Varying the perturbation type shows what the model is reading, and the authors demonstrate that a shuffle-only check misses a mean-pooled context.

The experiments use the PhysioNet 2019 sepsis data. A single oracle bit derived from the labels, carrying whether and roughly when a patient becomes septic, reaches 0.957 AUROC. The best causal models the authors could train reach 0.76 to 0.80, and a bit predicted from whole-record summaries such as length of stay reaches 0.86. In a matched-pair audit of four architecture families, each pair one line of code apart, the non-causal member beats its causal twin on all twenty splits by 6.2 to 19.2 points. The case study is the authors’ own concurrent submission, which reported a 14.2-point AUROC gain over its causal baseline. A fair comparison gives −0.3. The detector, the label-only scores and a checklist are released.

Critical strengths

Most readers know that a bidirectional encoder cannot run online, and the paper would add little if it stopped there. Its contribution is to compute, from the label distribution alone, what a side channel carrying the onset time is worth. On the PhysioNet 2019 data the answer is 0.957 AUROC for one bit, against 0.76 to 0.80 for every causal model the authors trained. That changes how published results on this benchmark should be read. Any per-timestep AUROC above the causal range now needs the leak ruled out before a reader accepts it as progress on sepsis prediction.

The input-recoverable variant is the more useful finding for clinical readers. A model can reach 0.86 from stay length and recording rates, with no physiology involved. Both inputs describe how the patient was managed. Stay length is fixed only when the admission ends, and recording rates rise when the care team becomes concerned. This shows that the leak is available to an ordinary model through routine fields of the record.

The case study is strong because the authors ran it on their own concurrent submission, with full configuration and training scripts. They report that the 14.2-point gain was entirely the leak and that a fair comparison gives −0.3. They also report that the remaining 1.7 points, from reading observation indicators causally, mostly need no router. The model had passed every standard leakage check, so the detector and the label-only score are shown catching something that existing practice missed.

The negative results stay in the paper, and the twenty-seed re-runs hold up under a multiplicity correction. Few groups publish a correction to their own headline number before anyone else finds it. If venues want this kind of reporting, a paper that does it should be rewarded for it at review.

Critical weaknesses

Every estimate of what a leak is worth comes from one dataset. The detector is general, but the “one bit” result, the 0.86 input-recoverable score and the 6 to 19 point gaps all describe the PhysioNet 2019 cohort. Its label switches on six hours before onset and stays on, and its ICU stays are structured so that length alone predicts the class. Section 5 then speaks of what architectural leakage is worth on “a monotone early-warning label” in general, having measured one such label.

I would ask for a second clinical per-timestep task where stay length carries less of the label. The authors’ synthetic pair does not fill this role because it confounds local signal with the leak. My specific concern is that the size of the leak depends on how care is recorded. These data come from a small number of US hospital systems with frequent lab draws and dense charting. In a hospital where labs are ordered less often, or where records are closed out differently at discharge and death, the measurement indicators and the stay length would carry different amounts of information. The magnitudes could be smaller or larger there, and the paper currently gives no way to tell. Until a second dataset is added, the audit shows that the problem exists and is large in one cohort, and the general claims in Section 5 should be worded to match.

A large share of the empirical content concerns one architecture from a manuscript readers cannot see. Sections 8 and C, five appendices, the gate ladder and the corrected ten-dataset benchmark all refer to it, and Tables 5 and 6 correct a table that is not available. If that manuscript is rejected or revised, this half of the paper loses its reference point. Including the original results table and the model code as supplementary material would let readers verify the correction on their own.

Questions for authors

Questions

  1. How much of the label-only score is stay length? Table 4 shows stay length alone reaching 0.917 at four bits with a logistic model, above the boosted model on all summaries (0.901). Does the one-bit result of 0.957 change materially if the score is computed over stays truncated to a common length? Stay length is set by discharge, transfer and death, all of which are decisions or outcomes that follow the prediction. I would like to see how much of the headline number survives once that information is removed.
  2. The input-recoverable probe uses whole-stay means and recording rates, and both depend on when the stay ends. What is the score if the probe sees only summaries over a fixed prefix, such as the first 24 hours? A causal model has access to exactly that. The gap between the two figures would state what the leak adds over a strong causal baseline.
  3. The 1.7-point gain for the causal indicator router is measured against a model without indicator columns. The paired comparison of router against indicators-as-inputs is +0.35 with an interval that spans zero. Should the abstract’s “recovers +1.7 AUROC” be read as indicator inputs recovering about 1.4 points, with routing adding nothing resolvable? If so, the recommendation in Appendix L to use indicator inputs by default belongs in the main text.
  4. Does the indicator signal hold equally across the hospital systems in the data and across patient subgroups? Indicators record when clinicians chose to measure. If some patients are monitored less closely, a model that leans on indicators will perform worse for them, and an aggregate AUROC would hide that.
  5. Does the corrected ten-dataset benchmark in Tables 5 and 6 use the same seeds and splits as the concurrent submission? Will that submission’s numbers be updated to match? If the two manuscripts disagree, readers need to know which is authoritative.

Suggestions:

  1. Apply the label-only score to a second clinical per-timestep benchmark where the label depends less on stay length, such as a deterioration task in eICU or MIMIC-IV, or circulatory failure in HiRID. A dataset from outside the United States would be especially informative because ordering and charting practices differ, and the size of the leak may differ with them.
  2. Run the detector on publicly released checkpoints from the 2019 challenge and from later papers that use bidirectional or attention encoders per timestep. The paper declines to name any published result as an artefact, which is a fair choice. A count of how many public models fail the test would still tell the field how much of the reported progress on this benchmark is real.
Reviewer_fmj1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper examines future-information leakage inside sequence-model architectures. It presents a diagnostic that checks whether earlier predictions change when future inputs are modified or removed, and measures how much this leakage can inflate early-warning performance. A detailed case study separates apparent architectural improvements from gains that remain after correcting the evaluation.

Critical strengths
  • Addresses an important evaluation problem that can remain even when training and test data are properly separated.
  • Provides a practical diagnostic that could be incorporated into model testing. The example showing why shuffling alone misses a pooled-context leak is particularly useful.
  • Includes comparisons across several model families and repeated splits. The additional patient-level and challenge-utility results strengthen the evidence that the observed effect is substantial.
  • Goes beyond identifying leakage by examining what remains after correction. The distinction between legitimate measurement-history information and future information is useful.
Critical weaknesses
  • The contribution needs to be stated more clearly. The restriction against future access is familiar, and the paper acknowledges earlier work on comparing full-sequence and prefix execution. I see the main value in the systematic audit, the comparison of diagnostic checks, and the measurement of performance inflation. The paper should explain more directly what these experiments establish beyond illustrating the known problem.
  • I am not fully convinced by the emphasis on the one-bit argument. The bit is constructed from the true outcome, so it is intentionally informative. This shows how vulnerable the benchmark can be, but does not directly establish what a trained model will recover. The authors distinguish the oracle example from input-derived leakage, which is helpful, but the headline claims still give the oracle result considerable weight. The input-derived experiments provide a stronger basis for the practical conclusions.
  • The main early-warning evidence comes from PhysioNet2019, where labels remain positive after onset. Testing multiple architectures strengthens the finding within this setting, but leaves open how much the size of the effect depends on the label structure, prediction horizon, and recording process. The synthetic experiment helps illustrate the mechanism, but broader empirical claims would benefit from another real early-warning task.
  • The paper combines a general auditing method with an extensive investigation of one routed model. It is sometimes difficult to separate the findings that apply broadly from those specific to that model. A more focused presentation would help readers understand which checks they should adopt and which architectural conclusions are limited to the case study.
Questions for authors
  • What does the audit add beyond existing prefix-replay checks? Please identify the main new findings and the experiments that support them.
  • How should readers interpret the oracle one-bit score alongside the input-derived results? A clearer explanation of what each establishes, and what it does not, would help.
  • Would the main performance-inflation result hold on another early-warning dataset? An additional evaluation would strengthen the paper; otherwise, please narrow the claims to reflect the current evidence.
  • The checklist in Section 7 is useful. An end-to-end example showing a failed check, the architectural correction, and the result after correction would make it easier to apply.
Reviewer_jJpyOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors created an extensive report of a known phenomenon centered primarily on a single benchmark: The phenomenon is that in a causal task (i.e. only access to data from timesteps <= t is allowed to make a prediction for timestep t) any channel that leaks information from timesteps > t can drastically improve the models performance. They propose the FPS score as a black-box evaluation to identify such future leakage in models and show that in the considered setting the score detects the leakage while simpler scores fail. They moreover do an extensive analysis demonstrating that tiny channels (as little information as 1 bit leaked) drastically improve over causal models and on the particular benchmark considered identify which information leakage is sufficient (+ demonstrate that neural networks learn to exploit this if given future access).

Critical strengths
  • On the considered benchmark, the authors do an in-depth analysis showing that if the data is sensitive to some information (here for example the stay length) leaking this tiny amount of information through a small channel from the future leads to substantial score gains.
  • FPS seems to be an improvement upon previous, simpler scores catching some leakage modes other scores don't catch.
Critical weaknesses
  • The studied phenomenon is trivial and well known. In any scenario, where future information helps with a causal prediction this information shall not be leaked and if done so is a bug that will improve upon causal baselines. Moreover, if the latent information necessary to boost the prediction is tiny, it is not surprising that a small channel from the future (as small as 1 bit) can be sufficient. This is a property of the task/data. In that sense, it is also not surprising that leaking the sequence length (correlates with hospital stay duration in the considered task) can be sufficient leakage.
  • If the paper would have done a major study identifying that many submissions in the field suffer from related bugs this could have been a contribution.
  • Proposing the FPS score, is a small contribution, yet, in my opinion, it does not meet the standards of a conference submission.
  • The paper, in many sections, is not well-written in that it is very hard to parse. The best example is the abstract.
  • To summarize: The known phenomenon is a bug wherever it occurs. Any leakage of privileged information that goes beyond the input data permissible to make a prediction, no matter how subtle, is a bug / cheat. In some way, there is not really a need for a score checking this since (as the authors acknowledge) just feeding in the data in a streaming fashion (which doesn't allow for such bugs being active) will catch the amount of information leaked. Modulo numerical precision, the streaming and forced (without leakage bugs) forward passes should produce exactly the same prediction. Of course, this is costly with long sequences. (FPS_trunc approximates this full check by comparing two forward passes - truncated prefix vs. full sequence - at a chosen cutoff). The core weakness of the presented work is that it does not add sufficient conceptual or empirical novelty on top of this well-known fact.
Questions for authors

Feel free to address the identified weaknesses. In particular, in case I missed a major contribution, I am happy to be made aware of it an adapt my review accordingly.

1 comment

Thank you for the review. We agree with much of it; the disagreement is mostly about what the paper claims.

We agree that any path from future inputs to a per-timestep output that must be issued online is a bug, that future information helps whenever the label depends on it, and that a one-bit channel being enough is a property of the label. We also agree that running the model in streaming fashion is the reference check. The paper does not present the two-run test as new. Section 2 cites Fonseca (2026), who perturbs only future data, and Saggese & Smith (2025), who compare batch with streaming execution, and says FPS is that two-run test applied to trained sequence models. A prefix replay is a streaming check at one cut, and the checklist recommends it.

The contribution is the benchmark of which checks miss which leaks and the measurement of how much these leaks are worth.

  1. The leaks are not written as future access. In the case-study model the leak is a router context averaged over the whole record. The released challenge entry that fails combines a bidirectional LSTM with per-record min–max scaling, backward fill and a softmax over the time axis (we did not ablate which part leaks). Such a model is scored causally by the official challenge harness, which replays prefixes, and still inflates an offline whole-record evaluation.

  2. The shortcuts used instead of a full streaming run miss some of them. On 14 synthetic leaks with causal twins (Table 1), a shuffle-only test catches 4, a shift-only test 10, and a truncation that zero-pads the future instead of cropping it catches 13. The one it misses reads the tensor length, so a check that masks rather than crops would likely not see it either. The prefix replay catches all 14 by construction. We flag FPS > 1e-6 to allow for round-off, and there are no false positives: all 125 causal checkpoints we scored give exactly 0 in float64, as you say they should (float32 does not; Appendix A).

  3. On this benchmark the size is large enough to change conclusions. Pairs that differ in one line of code gain +6.2 to +19.2 AUROC on all twenty splits, and the gap survives per-patient scoring, the challenge utility and training on one PhysioNet2019 hospital and testing on the other (3 splits each way). The routed model that started this was reported at +14 AUROC over its causal baseline. With causal context and matched augmentation it gives −0.3 at five seeds and −0.65 at twenty.

  4. The label-only score is a tool. We agree the one-bit result follows from the label. The oracle score is computed from labels alone, in minutes on a CPU, so anyone can check how exposed their own label is before training a model. What a real leak delivers is what the inputs reveal: on this benchmark a bit predicted from whole-record summaries of the inputs reaches 0.86, above every causal model we trained (the best, with the observation indicators as inputs, reaches 0.82).

A field-wide study would make a stronger paper, and we did not do one. Our evidence is narrower: the case-study model, and four released PhysioNet/CinC 2019 entries with trained weights (a selection we could run as released, not a screen). One of the four, run once on the whole record as an offline per-timestep evaluation runs it, fails and scores 4.5 AUROC points above its own prefix loop (0.673 against 0.628, Table 5). Its official score is not affected. We do not infer a failure rate from this, and we could not extend the screen before the discussion closes.

We agree about the abstract. For a final version we would replace it with a shorter one along these lines:

"Per-timestep sequence models can read future inputs through their architecture even when the split and the features are clean, for example through a bidirectional layer, unmasked attention, symmetric padding or a context pooled over the whole record. A prefix replay detects this: run the model on the record cropped at step k and compare its outputs up to k with those of the full run; a causal model gives identical outputs in float64. On 14 synthetic leaks, cheaper checks such as shuffling or zero-padding the future miss some. On PhysioNet 2019 sepsis prediction, causal and non-causal models that differ in one line of code differ by 6 to 19 AUROC points on all twenty splits, and one of four released challenge entries, evaluated offline on whole records, scores 4.5 points above its own prefix loop. A routed additive model reported at +14 AUROC loses its gain once the leak is removed; a causal design that reads its measurement history gives +1.7, most of it without the router."

Whether an empirical audit of this kind is enough for ICLR is a fair question, and it is yours to judge. We would only ask that the paper be read as one: the fact that future access helps and the two-run test are not what it claims, and the benchmark of checks and the measurements are.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.