- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposed to use a uniformly sampled bias distribution to replace the initialization in the sine-activated function. The authors claimed that the previous initialization methods might ignore the periodicity of the sine network. Instead, they argue for uniformly sampling bias on [0, 2pi], which will let the network have independent inputs to the weights or activations. In addition, the authors also tried to match the output distribution of the initialized network to the input distribution by tuning the initialized network weights. The authors have applied this strategy on 1d audio and 2d image fitting tasks, and the proposed method achieved good results.
Critical strengths
- This paper raised an important and interesting question: how to initialize the sine-activated network to make it effective for image/audio fitting? Generally, this is an important topic for a general sine-activated network.
- The authors have provided both theoretical analysis and quantitative results.
- The proposed method is simple and should be analyzed thoroughly to see if it has the potential to be generally used in sine-activated networks or other periodic-activated networks.
Critical weaknesses
- I feel like this paper has made relatively more general arguments than it actually proved. The authors have done experiments on 2d image and 1d audio fitting, which are lower-dimensional applications. It is sure that a paper could do experiments on these tasks, but the authors should narrow down the main claim to limit the discussion within these fields. The theoretical and quantitative results are not convincing to back up the authors’ claim.
- In addition, the entire paper should clearly state that it is based on the sine-activated network, not general to all architectures. The current version exaggerates the arguments and confuses the idea with a wider range of network structures.
- This paper’s analysis is based on the assumptions made in Lemma 1 that separate the activation and the bias from the input distribution. However, this is not valid for many vision tasks except for fitting. The authors did not provide any discussion, analysis, or experimental comparisons on this.
- This paper lacks a clear ablation study. Some of the recent methods are only mentioned in the related work section, but not compared in the main results.
- The paper writing needs substantial improvements. The entire paper is written in a dialog tone but with some strange words. The logic link between sentences needs to be reorganized, especially in the introduction section.
- The reference format is inconsistent. Papers are not correctly referenced.
Questions for authors
Please see my detailed comments above. My main concern is that this paper only focused on the sine-activated network for image/audio fitting tasks. However, the authors have exaggerated the proposed method’s performance and overclaimed for general use. The experiments are insufficient to back up the arguments. And the writing of the paper is unclear and needs a big improvement. I encourage the authors to rethink the motivations and decide what should be included in the paper and narrow down the argument to a specific setting.
5 comments
reminder #1
Dear reviewer eA2J, we are following up for your answers to these questions, as we look forward a constructive rebuttal discussion.
reminder #2
Dear reviewer eA2J, we are following up for your answers to these questions, as we look forward a constructive rebuttal discussion.
Preliminary clarifications
In this early phase of the rebuttal period, we seek clarifications so that we have concrete ideas to strengthen the pending revision.
Can you please clarify which "relatively more general arguments"? We are happy to refine the paper's claims or to strengthen the theory where it falls short of the claims.
Can you please clarify what claims are exaggerated? In our view, the primary contribution is Theorem 1, which is followed by a full proof in the main text.
Can you please identify the "strange words"? We prepared the main text of this paper entirely by hand, without AI, and are eager to improve the writing during rebuttal.
Can you also clarify which sentences lack a "logic link"? We take responsibility for the clarity of our writing and would be pleased to strengthen the Introduction in this way.
Can you give examples of unclear references?
Can you clarify which claims? We would like to strengthen our paper by aligning the claims better with evidence.