W1. As a dataset paper, this submission simply does not let reviewers see the data itself. The entire selling point is the dataset, so at the very least people need to see what the samples look like, such as whether the Uyghur script is legible, how severe the perspective distortion from the phone camera is, and how hard it is to tell skilled forgeries apart. But looking through the whole paper, there is only one schematic figure, one pipeline diagram, one EER heatmap, and six tables full of numbers. There is not a single full-size sample image, and no fine-grained statistics table broken down by script, device, session, or authenticity class.
W2. The numbers inside the paper do not reconcile, and two of the inconsistencies bear directly on the experimental protocol. The definition of the EVAL40 test set is ambiguous. It says these 86 writers "provide at least five genuine signatures and five skilled forgeries per capture device in S1," but §3.3 says S1 collected "3 to 5 genuine signatures and 3 to 5 skilled forgeries per script." If the upper limit is 5, then "at least 5" is basically sitting right at the ceiling, and the phrase "per capture device" never appeared in the S1 description at all. So did S1 actually collect more than 5, or is the EVAL40 criterion simply miswritten? This test set is the only one used for every experiment in the paper, and if how it was selected cannot even be stated clearly, then the credibility of every number that follows takes a hit. There are also a few other places that do not add up. For example, DEV-MULTI claims 19,953 paired instances, but given its description of 3 scripts plus 2 digit-string types multiplied by 2 devices, how that number was derived is never explained. On the forger side it only says "each target writer was assigned four to five," without giving the forger-to-target ratio or explaining how this interacts with the writer-independent EER protocol. None of this is reproducible enough.
W3. Two of the four main conclusions are stated too strongly for the evidence to support. The third conclusion says "training on both devices gives the best results," but I checked Table 5 row by row and that is simply not the case. In some cells a single-device-trained model is clearly better. On top of that, the whole table has no significance testing, no multi-seed variance, no confidence intervals, and the test set is only 40 writers. Under a 40-writer protocol, a difference of one or two EER points is basically noise, yet the paper directly uses the word "proves" and claims that device diversity significantly improves generalization. That is clearly overstating it. Table 6 has the same problem. The fourth conclusion says "performance drops markedly over nine months," and the trend itself is solid. But the paper fails to control for an obvious confound. S1 was captured with two devices while S2 used only the phone. It does say testing is "restricted to EVAL40 samples captured by the same phone in both sessions," but that restriction is chosen post hoc, and if the phone capture conditions in S1 and S2 differ, for example lighting, distance, or whether the sheets were the same batch, then the so-called "time effect" is mixed with an acquisition-condition effect. To separate the two, you would at least need a within-session control, such as re-imaging different sheets from the same session, but the paper does not do this.
W4. There are also some scattered issues that together add up to something quite serious. First, the differentiation from the MSDS work at NeurIPS 2022 is insufficient. That work already has Han-script signatures, a cross-session gap, and digit strings, yet the paper only mentions in §2.1 that it is "restricted to Han script signatures," without doing a head-to-head protocol comparison or explaining clearly what TriSig-430 adds beyond tri-script coverage and paired devices. Second, the baselines in Table 2 are not really fair. The WD column has 5 methods and the WI column also has 5 methods, but only HTCSigNet appears in both, and that is precisely the one adopted as the "primary baseline," which then wins every single column. This kind of incompletely crossed comparison is hard to find convincing. Third, the paper says "no validation split is used," with all results taken from the checkpoint of the final epoch, which means the reported numbers depend on an arbitrary choice of epoch count, and others reproducing it could easily get different results. For a benchmark, this is a real flaw. Fourth, it does not fit ICLR's style well. The paper has no new learning method, no new objective function, no new architecture, and no theory. Its content is essentially benchmarking in a hyperparameter format using off-the-shelf backbones, whereas dataset papers at ICLR usually come with a learning-relevant finding or some kind of mechanistic insight, and this paper clearly has no contribution of that kind.