acceptodds
Withdrawn from ICLR 2027

Interval-Width Drift: A Controlled Study of Localization, Detection, and Error Ranking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Can interval widths from a frozen imputer support variable localization, independent-window detection, and individual error ranking interchangeably? We study the evidence needed for each use. With draws from one coronary-trained flow imputer, raw-width localization AUROC on 40 paired PhysioNet 2012 encounters is 0.987 with shared imputation randomness, but 0.386 with independent randomness; disjoint groups of 40 give 0.447. The latter two estimates are compatible with chance. Monte Carlo-only controls locate a dependence on numerical noise cancellation before patient pairing is removed; paired localization alone is therefore insufficient monitoring evidence. Across window sizes 20–100, width-statistic responses remain small while a direct timing diagnostic increasingly detects the intervention. Source-validation pseudo-masking reveals worse reconstruction than simple linear interpolation and small, measurable widening after context removal. A separate encounter–feature-matched design changes the hidden positions; harder selections increase error with weak average response in both local width and the deployed representation, including a 50-draw sensitivity check. In a separate supervised error-ranking analysis, restoring probability information to the baseline nearly eliminates the apparent benefit of widths. We provide an evaluation protocol that tests these uses separately, varying patient pairing, random-stream coupling and baseline information. The findings delimit this imputer's behavior rather than establishing a universal failure of width monitoring.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Distribution Shift, Deployment Monitoring, Uncertainty Quantification, Imputation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_aiUuOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper tries to study whether interval widths from a probabilistic imputer could be used for feature localization, independent-window monitoring, and individual error ranking. Then it narrows down the contribution to an evaluation protocol, telling people not to assume that interval widths' success on the task of feature localization does not mean the same for the other two tasks. To evaluate the idea, the main experiment uses a frozen flow-based imputer on PhysioNet 2012, with additional supervised error-ranking experiments on NIH-DEP. The author also validates the results through sensitivity analyses on parametes such as various patient pairings and Monte Carlo random streams. For the 3 tasks, the results show that the good performance on feature location would disappear after removing the shared randomness. Weak performance is observed in independent-window monitoring, and the additional value of interval widths would vanish after using a stronger probability-based baseline in individual error ranking. From the disclosed limitation, we know the findings are limited by the weak imputer and evaluation settings.

Critical strengths

The empirical validation is good. The paper conducts sensitivity analysis and negative controls to further evaluate how components like patient pairing, Monte Carlo randomness, and window size would impact the performance.

Critical weaknesses
  1. Motivation is not straightforward to me. The paper tries to define an evaluation protocol to prevent us from making wrong assumptions that do not directly apply interval width to independent-window monitoring or individual error ranking based only on the results of its performance on feature localization. But this does not make sense, as I do not think people would naturally link the success in one case scenario to its scalability to other scenarios with different target and test units.
  2. The weak imputer being used is not strong enough to help us to identify whether the failure in some tasks is from interval width or from the poor estimation of the imputer. I don't know why the author did not test the idea with stronger or multiple probabilistic imputers before drawing broader conclusions.
Questions for authors
  1. For the motivation part or the contribution, I am just wondering whether the author identifies prior work that actually treats success on one of these tasks as evidence for success on the others? From my search through prior work, interval width or uncertainty estimation has been researched deeply for the 3 tasks separately, and I do not see any paper justifying that their evidence on the success of one task could be transferable to other tasks.

  2. For the experiment, I would suggest the authors test strong imputers to help us identify whether the value of interval width could be enhanced or removed by a stronger imputer.

  3. I did not find bibliography entries for the PhysioNet 2012 and 2019 datasets, as well as for the used Flow matching. It would be good to add a reference to each of them.

Reviewer_FCdFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

In this work, the authors analyze the relevance of monte carlo-based intervals from imputation with a fixed imputer. They analyze the use of those intervals for tasks related to detection changes in the data-generating process: localization, ie which variables are subject to drift, detection of the occurrence of a change in a time window.

Critical strengths

The problem is interesting, and the authors have provided a tremendous empirical work, with very interesting insights in detecting artefacts and diagnoses tools, that could be reused in this context or others.

Critical weaknesses

The paper is extremeley hard to read, the problem is not initially exposed in an intuitive and pedagogical way, the contributions are not clearly stated (except a few sentences that are extremely honest from the authors, such as "our contribution is a reusable evaluation protocol, instantiated as a controlled-case study". If that is the case, it should guide the reader through why this evaluation protocol is needed, what interesting aspects it catches thare are usually missed by other approaches, how to practically use it. It is also not clear how valid it is: more imputers should be tested, including very powerful imputation strategies like MICE with adequately tuned parameters, maybe also explore more datasets to show that the pipeline remains interesting in a variety of cases. A lot of analyses in the appendices are marked as "historical", the main messages and analyses should be really clarified, this really feels like a draft requiring a lot of polishing and presentation effort

Questions for authors

Questions are pretty straightfoward from the weaknesses section, but seem very difficult to address in the time frame and the format of the rebuttal, especially the re-writing part, so that the manuscript is readable

Reviewer_MULwOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether interval widths from a stochastic imputation model are useful for identifying affected variables, detecting distribution shifts, and ranking prediction errors. The experiments show that apparent usefulness can depend strongly on the evaluation design and comparison baseline.

Critical strengths
  1. Useful evaluation perspective. The distinction between identifying affected variables, detecting changes in new patient groups, and predicting errors is useful. Success on one task should not automatically be interpreted as success on another.

  2. Careful empirical comparisons. The authors investigate alternative explanations for favorable results and report limitations and negative findings transparently.

Critical weaknesses
  1. The broader contribution is not sufficiently clear. The paper provides reasonable evaluation recommendations, but I found it difficult to understand how substantially these advance existing practice. The motivation would be stronger with a clearer explanation of which existing claims or approaches the findings challenge.

  2. Limited generalizability. The central findings rely on one imputation model, which performs worse than simple interpolation in the reported quality assessment (Table 3). It is therefore difficult to distinguish a broader limitation of interval-width monitoring from shortcomings of this particular model.

  3. The presentation obscures the central message. The many experimental variants and repeated qualifications make the paper feel fragmented. A simpler organization around the main research questions would improve accessibility.

Questions for authors
  1. What is the main insight that readers should expect to transfer beyond the particular imputer and datasets studied?
  2. What evidence would help distinguish limitations of interval-width monitoring from limitations of the chosen imputation model?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.