The presentation of the core methodology, throughout Section 3, is very poor, with many cases of undefined notation (across every equation) and generally unclear descriptions. The overall presentation is so unclear that any architectural design contributions of their proposal are completely obscured and reproducibility is impossible. Please see the itemized list further below.
Across all of the results tables (1-3), even though SIFR row is always bolded as the best, the advantage (difference in means) is often small compared to the reported standard deviations. For example, in the first column of Table 1, SIFR has an accuracy advantage over TSCeption of just 0.02, but the standard deviations of those cells are 1.26 and 0.31. In addition to providing these large tables of numbers (with 16*27 = 432 cells overall), the authors should also perform statistical analysis to check whether SIFR has statistically significant advantage. Given a rough eyeballing of the tables and seeing how many advantages that are within one std deviation, I would guess that significance is not established PhysioNet (table 1), due to that benchmark being saturated. However, there may be a significant advantage for the other two benchmarks (tables 2 and 3). Regardless, it should not have to be the job of the reviewer nor reader to parse and process 864 (= 432 * 2) numbers from these tables to check for statistical significance.
Some of the values in Table 1 appear to show impossible statistics. For DGCNN, the S2/Acc and S2/F1 are both reported as "99.36 ± 1.85" and for TSCeption, the S2/Acc is reported as "99.36 ± 1.35". Assuming (since Accuracy and F1 Score are calculated as percentages) that each sample must be between 0 and 100, the largest sample standard deviation (for N=3 trials, and dividing by (N-1)=2) possible (while being consistent with the mean of 99.36) is only approximately 1.11, which is realized when the three samples are 100, 100, and 98.08. Thus, these three cells appear to be mathematically impossible, which raises a serious concern about the accuracy and validity of their experimental results.
Table 1 also seems (frankly speaking) suspiciously unlikely. For this PhysioNet evaluation set, it appears that many methods nearly saturate the benchmark, with accuracy and F1 score often close to 100% and equal error rate close to 0%. Since several methods (including SIFR) perform similarly well (close to perfect), it should be harder to statistically distinguish them. The reported means between SIFR and several other methods are very small compared to the reported standard deviations. If these values are reasonably accurate, then it's quite remarkably lucky that SIFR just happened to take the best (slightly better) bolded value in each column. That is, one would expect that at least some of the closer sample mean advantages (across the many close pairwise comparisons) would have easily fallen the other way. At best, the authors may have had a bug in their code that led to inaccurately high standard deviations. The observation in the previous paragraph (of impossibly high std dev) would also be consistent with such a hypothesis.
Itemized list of notation and presentation issues
Note that this is not an exhaustive list. Instead, I'm just noting some issues to illustrate how prevalent that undefined notation and lack of clarity completely undermine the presentation of the paper.
In equation 1, μn, vn, and ϵ are undefined notation. Of course, the structure of the equation can lead one to guess that these must be the feature-wise mean and variance of each token, and that ϵ>0 is small value for numerical stability, however, it's just sloppy to not define symbols that are used, and this is the mildest case of undefined notation.
In equation 2, W1, W2, b1, and b2 are undefined notation, and it is also sloppy to not clarify what stacking γsess and βsess should mean in terms of how the dimensions of the right-hand-side of the equation are assigned to those two variables. Based on the overall context, one could guess that there must be 2D total dimensions and they must be split equally, but this should be made clear. Further, a critical issue with the undefined notation is that, even though one could guess that W1 and W2 are meant to be weight matrices and b1 and b2 are bias vectors, by not clarifying their dimensions, the intended low-rank nature of this operation is only indirectly inferred by the sentence following equation 2 that states "A second low-rank path" when referring to the next equation. Thus, one could guess that W1 should have dimensions r×D and W2 should be 2D×r, where r is some (undefined and unclarified) rank hyper-parameter that should be smaller that D? Is this correct? What is the setting for this hyper-parameter in the experiments?
In equation 3, bg is undefined. Is this a bias vector, a scalar, or something else? What value of q is used in the experiments? Does it coincide with the rank r above?
In equation 4, γid and βid are undefined. What are these set to in the experiments? Are they trainable parameters or constants?
In equation 5, yn is defined in the expression, but this quantity is not clarified with a name or purpose (i.e., it is not mentioned again in other equations). How is yn later used in the rest of the framework?
In equation 6, the meaning of the notation Nl and dln are undefined and unclear. Note that while dn is defined in equation 5, it is unclear what exactly it means to add another subscript for the expression dln. It appears that the intention is to express how the earlier descriptions of Section 3.3 is meant to describe the behavior of one FactorNorm module, and that now in Section 3.4, this is dealing with "four FactorNorm modules", so l appears to be indexing across these 4 modules. However, then why is the subscript n (which presumably may be intended to be over the N tokens) now averaged over Nl indices. What is Nl?
In equation 7, B, zsess,i, and pai are undefined. While the similar zsess was previously defined, it is unclear what adding the extra subscript i is meant to index, or why there is an apparent mean across B values. Is this meant to denote batch size? Similarly, pk was defined, but it becomes unclear what replacing that subscript with ai means? How is ai defined?
In equation 8 (and the immediately preceding text), the notation a, e, and η are not defined. What are these meant to represent? Why is it given that e⊥p=0?
In equation 9, the loss terms Lid and Lsess (note: these are actually "\mathcal{L}" in the paper, but it seems that this review form that does not support the mathcal font) are not clearly defined, nor is it clear how everything ultimately feeds into these terms. Figure 1 seems to imply that Lid is related to "ArcFace" as an "Identity head", but the details are unclear, and Lsess seems to be a cross-entropy loss, however, it is unclear how the "Residual pool x4" goes through a "Session head" and is ultimately used as part of a cross-entropy loss. The "Session head" does not appear to be mentioned or further clarified across the methodology section 3, nor is it clear how this "Residual pool x4" is formed.
Other relatively minor presentation issues (again, not an exhaustive list)
On lines 78-80 (in Section 2.1), the text "However, discriminative feature extraction does not by itself guarantee stability across sessions," is repeated twice in a row.
One lines 262 and 263, there are spaces missing after commas in "recognition,component" and "ACC,macro-F1".
Figures 1 and 2 both have small text that is difficult to read in a printed copy. Figure 2 has some very small text that is also light grey over the white background of the page (in the top-left section). This is completely unreadable on a printed page, and other small text in that Figure is just barely readable. A mitigating factor is that both Figures seem to be vector graphics, so viewing them on a computer screen and zooming in a lot will make them legible. However, another possibly related issue if that maybe the Figures (especially Figure 2) is not so efficiently encoded. When viewing this PDF on my (reasonably capable) laptop, the PDF rendering even becomes a bit sluggish and unresponsive when trying to zoom and pan around on this Figure. Overall, these issues make it quite inconvenient when trying to extract information from these Figures.
Further, Figure 2 is really multiple things thrown together. I recommend breaking it up into separate figures, each presented in larger format (to aid readability), and perhaps moving some into the appendix
On line 377 (end of page 7), there is a broken reference to "Appendix ??". It seems that this should point to Appendix B.2.
In line 632 (Appendix A, before equation 11), there is an inline equation "w=ap+e" that is in plain text mode, but should instead be put in math mode.
An additional caption below Table 7, states "Source: parameter counts from the existing implementation. This table measures model size, not runtime." Both of these statements are unnecessary and seem like the sort of gibberish that is common in AI-generated text.