- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper describes an approach to improving recurrent networks by letting a representation of the input history influence hidden-to-hidden state transitions. Experimental results show that the proposed model shows good performance in comparison to several baselines.
Critical strengths
The paper seems to describe a method that allows state transitions in a recurrent network to be conditioned on a representation of the input history. This is an intriguing idea that to the best of my knowledge has not been widely explored in the literature.
Critical weaknesses
Most of the paper is complete gibberish. Even though I am quite familiar with recurrent networks there are many terms and statements whose meaning I do not understand and that should be introduced before use. A small sample: What does this mean: “The routed state current and encoded input form a bounded drive” What does this mean: “a recurrent cell selecting graph-supported skew operators from a learned K-factor family” What are “donors” and “donor maps”?
There is plenty of unused space that could have been used for providing a readable motivation, introduction of terms and discussion of intermittent results. So this is not a space issue.
In some of the experimental evaluations, results for comparing methods are below their published performance values and are instead based on a re-implementation. For the reported performance values to be meaningful they would need to be derived from detailed sweeps over hyperparameters.
Questions for authors
Line 107: If h_1 is the initial hidden state and h_k occurs before x_k, does that mean the input sequence starts with x_2?? This seems confusing.
Results are shown on seemingly random collections of compared-to models (for example, why are there GRU and Mamba results in the experiment from Table 3 (b) but not in the experiments from Table 3 (a), and vice versa for HyperRNN, NM-RNN and others?)
Section 4: What is the context for these propositions, how are they motivated, why are they useful?