acceptodds
Withdrawn from ICLR 2027

EventWorth: Risk-Controlled Event–Frame Prediction with Delayed Visual Anchors

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Conventional cameras often struggle under fast motion or challenging illumination, whereas event cameras capture fine-grained temporal changes that can complement frame observations for temporally causal video prediction. However, event quality varies across sensing conditions, and corrupted or unreliable events can degrade prediction rather than improve it. Existing event–frame methods mainly focus on exploiting events, but rarely determine whether they should be used for each prediction relative to a deployable frame-only fallback. This leaves negative-transfer risk largely uncontrolled. To address this problem, we propose EventWorth, a selector-agnostic framework for risk-controlled event use. EventWorth learns baseline-relative gain and harm scores from paired supervision provided by delayed future anchors. Validation data propose and freeze a selective policy, which is independently certified before deployment. Event assistance is enabled only after certification; otherwise, the system falls back to the frame-only branch. Under the stated round-level i.i.d. certificate assumption, EventWorth provides finite-sample control over the probability of deploying a selector whose selected harm risk exceeds a prescribed target. Experiments show that the analytic EventWorth policy is certified in all five seeds while retaining 50.44% development-test coverage, 1.30% harm, and 0.0220 system gain. These results show that selective event use can retain useful event information while controlling deployment risk.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Event Camera, Video Prediction, Selective Event Use, Risk Certification, Event–Frame Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_N6xzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes EventWorth, a risk-controlled framework for selectively using event-camera information in temporally causal video prediction. The main motivation is that event streams can improve prediction under challenging conditions, such as fast motion and low illumination, but unreliable events may instead degrade performance.

To address this issue, the authors employ a frame-only predictor and an event-assisted predictor and learn baseline-relative gain and harm scores using paired supervision from delayed future visual anchors. Based on these scores, a selection policy determines when event information should be used. The proposed framework further incorporates an independent statistical certification procedure based on Clopper–Pearson confidence bounds to control the risk of harmful event usage. If certification fails, the system falls back to the frame-only predictor.

Experiments on the RLED dataset show that EventWorth selectively uses event information while reducing harmful predictions and maintaining prediction gains.

Critical strengths
  1. The paper addresses a practically relevant issue in event-assisted video prediction, where unreliable event information may degrade prediction quality compared to a frame-only baseline.

  2. The proposed framework incorporates an independent risk certification procedure to control harmful event usage, providing a principled approach to selective event-based prediction.

Critical weaknesses
  1. Unclear practical motivation and significance. While the paper formulates selective event usage as a risk-control problem, the practical importance of this task is not sufficiently justified. In particular, the paper lacks concrete application scenarios or downstream evaluations demonstrating why such a selection framework is necessary.

  2. Limited technical novelty. The proposed method largely combines existing techniques, including gain/harm-based modality selection, threshold-based gating, and standard statistical risk certification. The technical contributions beyond applying these established concepts to event-assisted video prediction remain unclear.

  3. Restrictive theoretical assumptions. The risk-control guarantee relies on a round-level i.i.d. assumption, which may not hold for temporally correlated video data. This limits the applicability of the theoretical guarantees to realistic video prediction scenarios.

Questions for authors
  1. Could the authors clarify the practical applications of the proposed framework? In particular, what real-world scenarios would benefit from selectively switching between frame-only and event-assisted video prediction?

  2. Could the authors further clarify the key technical contributions of EventWorth beyond existing modality selection and statistical risk certification methods?

  3. Why does the proposed framework show substantially weaker results with learned predictors compared to analytic predictors? Could the authors provide additional experiments with stronger event-based video prediction models to demonstrate its generalizability?

Reviewer_DwmvOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper considers event-camera-augmented causal video prediction. In causal video prediction, the goal is to predict future video frames from past frames (or past latent-space representations of earlier predictions). The variant considered in this work uses event cameras as an additional information source. The work argues that existing methods leverage event camera data indiscriminately, regardless of whether it is useful or harmful (due to poor data quality) in a given case. The authors address this by proposing a theory-grounded method that can evaluate the usefulness of the event-camera data.

Critical strengths

The work provides a theoretical justification and derivation for its selection approach.

Critical weaknesses
  1. In its present form, Figure 1 is not really useful. Its font is unreadably small, and even zoomed in, it is not clear whether it helps clarify things or confuses even more.

  2. Several details are missing to reproduce this work. I checked this primarily with an LLM and double-checked its outputs. Instead of listing all the points that are missing, here is the prompt that I used in Claude Opus 5.5 Extra High to evaluate reproducibility:

    Please help me review this paper. As a starting point, check whether the work is reproducible. I.e., if you were to fully implement the paper from scratch (only reusing artifacts that the authors themselves reused from other sources), would you have enough information to fully reproduce the results? If not, give me a concise bullet-point list of any missing parameters, dataset information, or training details.

    The output involved numerous points, including several highly relevant ones such as missing network parameters.

  3. The motivation of this work is not fully clear, and it is not clear which research hypothesis it is testing. In other areas, it has already been established that providing some sort of confidence information to downstream tasks can improve performance. Moreover, the interesting basic question would be how a method like this performs compared to trying to learn the usefulness of the event-based branch. I.e., what is it that makes the high-level decision by the proposed method better than having the network choose itself.

  4. While the work briefly explains how event-cameras conceptually work, it does not introduce the formalism and nature of event measurements, making it essentially inaccessible to even the vision community, let alone the broader ICLR community. This is a lost opportunity because the underlying representation learning problem may be of more general interest.

  5. The experimental results seem not to involve any published baselines. Is it because no baselines exist (in which case the writing of the paper needs to be different, reflecting that it introduces a new problem to be considered) or because the authors omitted them (if yes, why?)

Questions for authors

I'd primarily like to know some of the authors' reactions to the weaknesses mentioned above.

Reviewer_hKzvOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The study focuses on when to use the event branch or go for the fallback of using only frames for event-assisted video prediction. The method here involves training the scoring function based on how well each of the two branches predicts compared to the next real frame and then testing the damage rate of the chosen rounds as against the frame baseline from an external certification data set. If it does not match the criteria, the system will fully fall back to using the frame only model.

Critical strengths
  1. The problem is consistent with practical application, and the approach taken here is to minimize negative transfer in relation to the frame baseline. Given the assumption that the rounds are independent, the theorem derivation is consistent, and the certificate upper bounds for the table can be calculated accordingly. The authors have honestly acknowledged the limitations associated with the rounds within the same video.
Critical weaknesses
  1. This certificate assumes i.i.d rounds but only uses 15 rounds per video across the 48 videos. The authors acknowledge this issue, however even under the assumption that the videos are independent units, even with no errors, 48 videos cannot certify a 5% risk (the upper bound is still 6.05%).

  2. Most of the certification methods follow the principles of Learn then Test, selective classification with guaranteed risk (Geifman & El-Yaniv, 2017) and Seldonian algorithms. These have not been cited, so the theoretical contribution is limited.

  3. The main results are using a simple analytical predictor. Only 1 seed out of 5 have their certification pass using a very small learned CNN. The evaluation is only performed on one dataset and does not include absolute errors.

Questions for authors
  1. If the entire video is considered as one independent unit or dependence assumptions in the video are provided, would the selector in the analytics setting be able to pass certification? For which deployment population will such a guarantee be given?

  2. What is the new statistical result of Theorem 1 compared to Learn then Test, and Seldonian algorithms, besides defining harm as the loss increase relative to the fallback?

  3. Will the selector remain certified and with positive gain if the predictor is changed for other event-assisted predictors using a fixed evaluation protocol? What are the absolute errors for both branches and the deployed system?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.