- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a frequency-domain output representation for visual coordinate regression. Instead of regressing a coordinate directly with an MSE loss, the method maps the target coordinate to multi-frequency sine–cosine (Fourier) codes, supervises each frequency component separately with MSE, and then decodes the prediction back to the spatial domain via hierarchical phase unwrapping. The overall design is simple.
On patch localization (Tables 1 and 2), the method outperforms direct regression as well as other regression variants. It also improves over direct regression on several downstream vision tasks (Sec. 5.1: 2D pose estimation, homography estimation, and 3D pose estimation).
Critical strengths
S1. Importance of the problem. Coordinate regression is a fundamental problem in computer vision. Developing output representations that do better than plain MSE supervision is a worthwhile direction, as it can broaden the applicability of regression-based models across domains.
S2. A simple yet sound method. Rather than supervising a single scalar value with MSE, the method supervises each frequency component of the Fourier encoding in the frequency domain and recovers the coordinate through phase unwrapping. The approach is simple but sound, and it has clear potential to extend beyond vision to other coordinate regression problems.
Critical weaknesses
W1. Proposed method is not specific to vision. Although the experiments are conducted on visual benchmarks, nothing in the method itself is tied to the visual domain. This could be viewed as a strength, but for the same reason the method should also be compared against regression techniques developed in other domains.
W2. Insufficient baselines (comparison only with plain direct regression). In the experiments (Secs. 4 and 5), the main baseline is plain direct regression. The reported gains may therefore partly come from the increased number of output channels in the Fourier-based head. A comparison with classification-based approaches (i.e., predicting a probability distribution) that use the same number of output channels is needed.
W3. Missing citation. The paper does not cite the following prior work, which also performs regression by applying losses to individual frequency components:
- 3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction (Lee and Cho, NeurIPS 2024)
Please cite this work and discuss the similarities and differences.
W4. Lack of analysis and missing statistics. Tables 4 and 5 and Figure 3 show interesting results, but the experimental settings and statistics are not described in enough detail. Since the improvements could be due to run-to-run noise, please report the number of runs together with the mean and standard deviation.
In addition, there is no analysis of the qualitative results: what the failure cases are, and in which cases the method works well.
W5. Choice of K. Is K in Sec. 3.2 a fixed hyperparameter? Could learning K (or the frequencies) bring further benefits?
Questions for authors
This method demonstrates its effectiveness across a variety of visual coordinate regression tasks with a simple technique. I find this simple yet effective approach quite interesting and worth exploring further.
However, in terms of the overall completeness of the work, I cannot give an acceptance score at this stage.
I would be willing to raise my score if the authors adequately address W1, W2, W3, and W4 above.
3 comments
Post-rebuttal review of Paper ID 29499
I thank the authors for the detailed response and the additional experiments.
W1: partially resolved. W2: resolved. W3: resolved. W4: resolved. W5: resolved.
The authors have addressed most of my concerns, so I raise my score from 2 to 3.
Thank you for your thoughtful feedback. We are glad that our responses and revisions have addressed your main concerns, and we appreciate your time and consideration throughout the discussion.
(W1. Comparisons with regression methods developed in other domains)
We thank the reviewer for clarifying the scope of Fourier Output. We agree that Fourier output parameterization is not inherently vision-specific. To address the connection to regression methods developed beyond vision, we evaluate HL-Gauss, a histogram-based regression objective studied in Improving Regression Performance with Distributional Losses (ICML 2018), in patch localization experiments. We set the number of output channels to 8, the same as in Fourier Output. Across three seeds, the test MAE is 0.0431 ± 0.0006 for HL-Gauss and 0.0371 ± 0.0022 for Fourier. This indicates that Fourier Output still outperforms a general output representation.
(W2. Comparison with classification output representations)
We design two classification-based output representations for patch localization: a 128-channel one-hot distribution (SimCC-style) and an 8-channel two-hot classification distribution that represents the probabilities of the two nearest bins. As shown in the table, Fourier Output outperforms both classification-based output representations.
(W3. Wigner-D)
Thank you for pointing out this relevant work. We will add a citation to it. We agree that both methods share the general idea of predicting harmonic representations of geometric targets instead of directly regressing their original parameters. However, there are two key differences: 1) Wigner-D coefficients are designed for SO(3) rotations, while Fourier Output is designed for 1D translation; 2) Wigner-D requires a specialized equivariant architecture, while Fourier Output can be incorporated into existing direct coordinate regression architectures.
(W4. Protocols, number of runs, means, and standard deviations)
We reran the experiments with 3 seeds. The improvements persist for 2D pose, homography, optical flow, and VLM localization at 6K updates.
(W4. Failure cases in qualitative results)
We observe left–right limb confusion in some DeepPose predictions. Fourier Output corrects such errors in some examples where Direct fails, but also exhibits the reverse pattern. These cases suggest that ambiguity in joint identity remains a limitation despite the overall accuracy improvement. The analysis has been added to Appendix C.2 (Figure 6) of the revised paper.
(W5. Choice of K)
K is a fixed hyperparameter, and we study its effect in the ablation in Section 4.2. We tried a progressive learning algorithm that gradually increases K during training once learning at the lower frequencies has converged, but it does not improve performance. We will add this information to the final version of the paper.