acceptodds
Withdrawn from ICLR 2027

A³Flow: Attention Residuals, Adaptive Uncertainty Gating, and Motion Aggregation for Unsupervised Optical Flow

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Despite the dominance of RAFT-style recurrent correlation models, unsupervised optical flow methods with uncertainty estimation often inherits three restrictive design choices: fixed feed-forward residual encoders lacking data-dependent routing across scales, a hand-crafted stop-gradient gate that suppresses rather than correct updates when uncertainty is miscalibrated, and local correlation lookup that struggles to provide reliable correspondence cues in occluded regions. We present A³Flow, which augments an uncertainty-aware recurrent unsupervised framework with three pathways: (i) AttnResEncoder, which replaces fixed residual fusion between encoder blocks with learned softmax attention over pooled preceding-block representations, drawing inspiration from depth-wise attention in large language models to enable cross-scale routing in CNN encoders; (ii) ResidualFiLM(Residual Feature-wise Linear Modulation), which generalizes the fixed one-sided sigmoid gate to a two-sided bounded affine modulation driven by predicted uncertainty, enabling richer uncertainty-conditioned correction rather than one-sided suppression of flow updates; and (iii) CP-GMA, a Content-and-Position Global Motion Aggregation module integrated into the motion encoder, which propagates motion features from confident to occluded pixels, providing global motion context beyond local correlation lookup. Extensive experiments on MPI-Sintel and KITTI demonstrate that our method achieves new SOTA among unsupervised methods, with pronounced gains in occluded regions while preserving highly reliable uncertainty estimates. Code will be made available upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Unsupervised optical flow, uncertainty estimation, attention residuals, feature-wise modulation.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_vVQkOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes A^3Flow, an unsupervised optical flow estimation method. It builds on U^2Flow by incorporating three modules: an attention residual encoder (AttnResEncoder), adaptive uncertainty gating via ResidualFiLM, and content-and-position global motion aggregation (CP-GMA). Experiments show promising results.

Critical strengths
  1. The methodology is generally sound, the training and evaluation protocols are standard, and the results are valid and promising.
  2. The method section explains the architecture of each proposed module in detail, supported by mathematical formulations.
Critical weaknesses
  1. The method section mainly describes what the proposed modules are. The motivations are stated but not supported by analysis or evidence specific to optical flow. The results are largely empirical and offer few insights. The performance improvements remain a black box, which makes the results less compelling. Unsupervised optical flow is a poorly constrained problem. I would expect improvements to come from heuristics or ideas rooted in traditional computer vision, i.e., methods whose effectiveness under such weak constraints can be understood, rather than treating the problem as pure engineering, as is often done in supervised optical flow.
  2. The ablation study in Table 2 is incomplete. If the three proposed modules are independent of each other, the ablation should cover all 8 possible combinations rather than only 4.
  3. The mathematical notation is inconsistent, imprecise, and at times confusing. Examples include, but are not limited to:
    • Lines 178–181 use I0I_0 and I1I_1 but F1→2F_{1\to2}, while Fig. 2 uses I1I_1 and I2I_2.
    • Eq. (2): Cflow′C'_\text{flow} is not defined.
    • Eq. (3): GAP\text{GAP} is not defined. The definition is also circular: on the left, xlx_l is defined in terms of klk_l and vlv_l, while on the right, klk_l and vlv_l are defined in terms of xlx_l.
    • Eq. (2) uses φ\varphi to denote the sigmoid function, while Line 284 uses φ\varphi to denote the key projection. α\alpha denotes both the uncertainty (log-variance) and the scalar coefficient in Eq. (8).
    • Eq. (10) is confusing: ii and jj index xix_i and xjx_j, but ixi_x and jxj_x are not defined.
    • Concatenation is written as concat\text{concat} in Eqs. (2) and (7), but as ∣∣\vert\vert in Line 291.
  4. The presentation could be more polished:
    • The figures are not in vector format (e.g., EPS/PDF) and may show artifacts when zoomed in.
    • Formatting is inconsistent: Sections 1–3 use “Fig.” (e.g., Line 161), while Section 4 uses “Figure” (e.g., Line 411).
    • Table 3: the best value in the second column (5.49) is not in bold. Table 4: the best values in the last two columns are not in bold.
    • Typo in Line 351: UnSAMFlow\text{UnSA}^{\text{MF}}\text{low} → UnSAMFlow\text{UnSAMFlow}.
Questions for authors

See weaknesses.

Reviewer_LkmUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes A3 Flow, an uncertainty-aware framework for unsupervised optical flow estimation that extends U2 Flow with adaptive feature fusion, uncertainty-conditioned modulation, and global motion aggregation. Experiments on MPI-Sintel and KITTI demonstrate improved optical flow accuracy over the baseline, supported by ablation studies, cross-dataset evaluations, and uncertainty analysis.

Critical strengths
  1. Clear problem formulation: The paper identifies three limitations in existing uncertainty-aware unsupervised optical flow methods and proposes targeted modifications to address them.

  2. Promising experimental results: A3 Flow consistently improves upon U2 Flow on the reported Sintel and KITTI benchmarks, demonstrating the potential effectiveness of the proposed approach.

  3. Comprehensive evaluation: The paper includes ablation studies, cross-dataset generalization, and uncertainty evaluation, providing multiple perspectives on the method's performance and limitations.

Critical weaknesses
  1. Limited novelty: Although the paper introduces three modifications, the overall framework appears to be an extension of U2 Flow rather than a substantially new approach. The experimental settings and figure styles also closely resemble those of U2 Flow, further reinforcing the impression that this work is primarily an incremental extension. Additionally, CP-GMA seems to largely reuse the original GMA [1] mechanism, while the paper provides neither a clear explanation of its methodological differences nor a direct comparison against the original GMA integrated into U2 Flow. The reported improvements are promising but may not be sufficient to establish the level of novelty expected at ICLR.

  2. Unclear multi-frame comparisons: Table 1 includes an Unsupervised + Multi-frame category, although the proposed method does not introduce a multi-frame variant. The motivation for including these comparisons and the fairness of comparing methods with different training and input settings are not sufficiently discussed.

  3. Limited benchmark coverage: The evaluation focuses on KITTI-2012, KITTI-2015, and MPI-Sintel, following the baseline U2 Flow. However, it does not include newer optical flow benchmarks such as Spring [2]. Evaluation on more recent benchmarks would provide stronger evidence of the method's generalization and competitiveness.

  4. Insufficient validation of the benefits of unsupervised learning: The paper motivates unsupervised optical flow partly by the high cost of obtaining ground-truth flow annotations. However, it does not experimentally demonstrate the practical benefits of leveraging additional unlabeled data. Moreover, modern supervised optical flow methods have achieved very strong performance across standard benchmarks, often substantially outperforming unsupervised approaches, as also reflected in Table 1. In this context, the practical significance and motivation of pursuing unsupervised optical flow deserve further consideration. The authors should better articulate the advantages of their unsupervised formulation beyond avoiding annotation costs and provide experiments examining the use of additional unlabeled data or the trade-off between annotation requirements and performance.

  5. Limited ablation analysis: The ablation study follows a progressive evaluation scheme, adding the three components sequentially. However, it does not evaluate each component independently or investigate different combinations of the proposed modules. Consequently, it remains unclear whether the reported improvements reflect the individual contributions of each component or their interactions, making it difficult to fully validate the effectiveness and complementarity of the proposed designs.

  6. Inconsistent uncertainty estimation: Although A3 Flow improves optical flow accuracy, its uncertainty estimation deteriorates on Sintel compared with U2 Flow, with AUSE increasing from 0.11 to 0.15 and Spearman's correlation decreasing from 0.66 to 0.56. This raises questions about whether the proposed modifications consistently preserve uncertainty quality across datasets. Further analysis is needed to explain this discrepancy and clarify the trade-off between flow accuracy and uncertainty estimation.

[1] Jiang, S., Campbell, D., Lu, Y., Li, H., and Hartley, R. Learning to Estimate Hidden Motions with Global Motion Aggregation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.

[2] Mehl, L., Schmalfuss, J., Jahedi, A., Nalivayko, Y., and Bruhn, A. Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 4981–4991.

Questions for authors

See weakness

Reviewer_eGJFOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper is an unsupervised 2D optical flow estimation paper, claiming SOTA. The main contributions are architectural (AttnResEncoder as a convolutional analogue of attention residuals, ResidualFiLM), and combining with existing off-the-shelf components to achieve better performance on this task. They validate with the most common benchmarks like sintel / KITTI.

Critical strengths

Justification mainly rests on SOTA performance on common optical flow benchmarks amongst self-supervised methods.

Critical weaknesses

I think overall, it could be argued against the paper that the architectural contributions (which are the only contribution, as the loss and even many architectural components are lifted from prior work) feel a bit incremental, and the bespoke nature of the amalgamated design choices make any findings in this paper difficult to transfer to other areas in computer vision. I understand that many of the feature space design choices are more common in optical flow literature, but it could be argued that the vision community at large continues to move towards a more performant strategy of vision transformers with less inductive bias and fewer bespoke choices / inductive bias in feature space, and this makes this work less future-proofed.

I think the application of attention residuals to CNNs with AttnResEncoder is maybe the part of the paper that has the greatest transferability to the broader computer vision community, since I understand the authors to be correct that in many applications where memory / flops / latency are important (or data is limited), larger vision transformers can suffer. However, since this paper is positioned as a pure SSL optical flow paper and don't have any extra ablations on this, the amount of value to the community the paper delivers on this specific architectural improvement is modest given, as aforementioned, how entangled the main benchmarks are with other architecture choices.

I want to be careful here, as there is real value that can be delivered from probing lower data / lower compute regimes. Given that the intro / related works somewhat position the value of this work and other CNN based methods as being the lower memory / flops / latency regime, it would be stronger support of the author's position to have a table (in the suppmat at the very least) making it clear that the gains in performance cant be attributed to just more flops / more parameters in a way where on-par performance could be obtained from taking an older architecture and making it slightly bigger / more expensive or something along those lines. In this respect, an appendix is sorely missing compared to things like U2Flow.

Questions for authors

I need help understanding if there is value delivered from this work beyond just slightly better performance on self-supervised optical flow. I think this is keeping the paper very borderline in my mind, as the architectural choices are merely justified with their performance on this one task, and the choices are very bespoke, and the paper in its entirety rests on the value of these architecture choices. Any further information / pointers to places in the paper (that might normally be in a supplementary material) that can ease any concerns about things like parameter count / flop counts being the primary driver of improved performance compared to prior work would be appreciated, given that this is purely an architecture paper.

Reviewer_BNzhOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, A3Flow modifies U2Flow in three designs, replacing fixed residual fusion with attention-based stage aggregation, the reliability gate with bounded affine modulation, and augmenting the motion encoder with global motion aggregation. It improves EPE on Sintel and flow error on KITTI, with additional gains in uncertainty quality and cross-dataset transfer.

Critical strengths
  1. The work equations, dimensions, and initialization details are generally sufficient to understand the three modifications. Limited innovation and unattractive graphics.
Critical weaknesses
  1. Important.

All three components adapt prior work and the mechanism analysis remains limited. The ablations and qualitative error maps do not isolate which motion or occlusion categories benefit, or compare simpler adaptive fusion, so the contribution rests mainly on composition and evaluation.

  1. The occlusion claim lacks direct quantitative support. The gap between Fl-all and Fl-noc gains is only an aggregate inference rather than an occluded-pixel metric, despite occlusion masks in Sintel and foreground splits in KITTI, where only qualitative error maps are reported.
Questions for authors

1.Please strengthen the innovation and motivation description of the article!!!!!

2.Provide more comprehensive experiments!!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.