- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper's target is learned memory request prediction, where a neural network with small size is trained to predict future addresses that will be requested in a program. The paper's main claim is the fact that the accuracy of the prediction is not the only metric to care for; the timeliness of the prediction is just as important if not more, for if a prefetch request lands in the cache too early it will lead to premature cache eviction, and if it lands too late it may not be useful (because the actual memory request has already stalled).
The paper mentions that most learned based predictors target a single horizon: they predict the address requested by a fixed number of instructions ahead. This limited horizon of prediction breaks the timeliness of the prediction, a feature that early heuristic-based cache prefetching methods emphasized on. As a solution, the authors propose to predict multiple horizons of future requested addresses. To enable an efficient implementation, the authors propose a shared LSTM trunk which takes in a sequence of past memory access patterns as input, and generates a hidden representation vector as output. The hidden representation is then utilized by multiple linear layers (heads), each of which predicts the memory address accessed in a certain time horizon.
Critical strengths
-
I sincerely appreciate the inclusion of section 2, preliminaries, in the paper. It helped me a lot in understanding the scope of the paper. In addition, the writing of the paper is really good, with a clear structure that follows a purpose: to help the reader understand the problem at hand. Reading the paper I realize that the authors respect the readers time. I appreciate and encourage the authors to continue this style of writing.
-
The authors distinguish betweeen the long horizon cost minimization (J(pi)) and individual horizon prefetch predictors, which helps formalize the proper learning objective. They also explain clearly why the overall task is not a supervised task.
-
Experimental results are solid, the multi horizon prefetching method shows superior hit rate and lower delay compared to the related work.
-
The design of the LSTM input/output streams are reasonable
Critical weaknesses
- I could not understand how the authors connect prefetch delay to the cost function J(pi). More specifically, does the delay estimation require the authors to know which cache layer the requested memory block resides in? Or the delay is simply simulated through the dataset/simulator mentioned in the experiment section? Please clarify this in the paper.
- I could not connect the learning objective to the optimization of the cost function J(pi). To my understanding, the LSTM + heads training learns a number of prefetch address predictors. How is J(pi) backpropagated through this setup?
Questions for authors
- Is the set of requested addresses (offseted by the current program counter) dynamically updated during inference? if yes how? If no, please indicate that in the paper.
- Have you entertained using a transformer block with limited-horizon causal attention rather than the LSTM?
3 comments
Author-reviewer discussion period is ending soon
Dear Reviewer fB4q,
If you haven't done so, please read through authors' response and let authors know if you need further clarification or have more questions. Beginning 11/19, authors are not allowed to respond any more, so this is good time to review the response and ask more.
Thank you.
-AC
keeping my rating
authors provided sufficient response. I am keeping my score unchanged.
We thank the reviewer for reviewing our paper and providing valuable feedback.
Q1: No, we do not dynamically update the offsets during inference. We will indicate it in the paper.
Q2: We will include the Transformer based results in the paper. The transformer based model gave roughly the same performance as multihead while being a bulkier model and requiring more inference time.