- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
It appears to be an auto-research work, and the authors did not even proofread the manuscript. I am not able to understand the content and message of this paper. At a very high level, this work proposes Rowmax-H15, an implementation that patches flash attention and approximates softmax.
Critical strengths
At least the authors are honest that "Generative AI assistants (code-capable large language models) were used in the design, implementation, analysis, and writing of this work."
Critical weaknesses
- This is purely AI-generated and not readable. For example, what is JSD? Is it Jensen-Shannon Divergence? This terminology appears in the second paragraph of the introduction and is never defined, and it also serves as a core component in the analysis. The paper is full of unnecessary details that no human researcher would put in the main text. For example, sec. 3.1, "the test stream is cut into 97 aligned blocks of 2048 tokens (198,559 predictions)", and line 362, FA4 "commit 0dc2cb48"
Questions for authors
N/A. Just withdraw your paper.
Response
We respond to each point factually.
On “JSD”. You are right that the main text does not expand the acronym. It is the Jensen–Shannon divergence, as you inferred. Its formula is Eq. (21) in Appendix A.7, and Sec. 3.2 points to Appendices A.6–A.7, but it should have been defined at first use (line 044). We will revise the presentation.
On the details you call unnecessary. The block and prediction counts in Sec. 3.1 (97 blocks × 2,047 predictions = 198,559, the sample behind our primary NLL results and their bootstrap intervals) and the FlashAttention-4 commit (line 362) are there so that readers can reproduce and check our numbers. For the same reason, the Reproducibility Statement (lines 488–499) says that the code and results would be provided during the discussion period.
On AI use. We disclosed it because the ICLR 2027 AI Policy for Authors requires disclosure. The statement you quote continues (lines 503–520): the reported measurements, figures and tables derive from executed code and recorded experimental outputs, and no experimental result was supplied by an AI model; the authors chose the research questions and experiments, ran or supervised the evaluations and hardware measurements, checked the reported results against stored outputs, reviewed the manuscript before submission, and take responsibility for its content.
On whether the numbers are real. They are real. The supplementary material, uploaded during the discussion period as the Reproducibility Statement promised, contains the code, the per-block and per-run result files behind every figure and table, and a provenance map from each figure and table to its input files. One included script recomputes 794 numeric table cells of the paper from those files in a few seconds, without a GPU (python paper/tables/check_tables.py).
Your review does not identify any number that is inconsistent, implausible or irreproducible. If you have one in mind, please name it; the provenance map points to the exact files behind it. Otherwise, we respectfully ask you to withdraw the ethics flag. We ask this even though we are withdrawing the submission, because the review will remain public. We have also asked the Area Chair to assess it on the evidence.
For the public record, we intend to withdraw this submission, for two reasons. Reviewers o371 and M7it identified comparisons that require new kernel experiments, and we accept those points. And this review is a significant part of why we will not continue in this cycle: it alleges fabrication without identifying a single number, and it gives us nothing technical that a revision or an experiment could address. Withdrawing does not mean we accept your unsubstantiated allegation.
We stand by the reported results.