- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper introduces training-free temporal residual connections: at each token and each layer, it adds a scaled version of the previous token's residual stream at the same level to the current residual stream. The residual connections are claimed to reduce degeneration in the generated text.
Critical strengths
The paper tests a simple intervention idea in a relatively simple inference settings. It compares the effect of its intervention on multiple models against helpful controls (random vectors, rotated or whitened signal). It is explicit about the failure cases, e.g. the collapse of greedy decoding.
Critical weaknesses
The paper is very hard to follow. Its definitions are scattered across the text, there are no results table, and quite a bit of the text reads as LLM-generated, with short sentences that lack context and are hard to interpret.
Questions for authors
Could you evaluate generated text on quality and coherence in addition to testing degenerated cases?