acceptodds
Withdrawn from ICLR 2027

TriSig-430: A Tri-Script, Multi-Session, Multi-Device Dataset toward Real-World Offline Signature Verification

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Offline signature verification (OffSV) systems in real-world use must remain reliable when a query differs from the enrolled template in script, acquisition device, or time of writing. Existing OffSV datasets are mostly single-script, single-session, and single-device, so they cannot measure robustness to these shifts. We introduce **TriSig-430**, a writer-aligned tri-script offline signature dataset designed for this setting. It contains 76,374 real handwriting images from 430 writers, including 60,286 signature images and 16,088 content-controlled digit-string images. Each writer signs in three scripts: Han, Latin, and Arabic(Uyghur). A paired subset captures the same physical handwriting sheets with both a flatbed scanner and a smartphone, enabling acquisition-domain evaluation without confounding device changes with rewriting. A longitudinal cohort of 178 writers contributes samples from two sessions separated by approximately nine months, providing complementary sparse-enrollment and dense-sampling regimes. We define train-test protocols for matched and cross-script verification, cross-device transfer, few-shot enrollment, cross-session robustness, and content transfer between signatures and digit strings. TriSig-430 is intended as a unified dataset for learning and evaluating writer representations under realistic script, temporal, content, and device shifts.

Primary area:
datasets and benchmarks
Keywords:
Offline signature verification, Multi-script handwriting, Writer representation learning, Dataset

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RwxnOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposed TriSig-430, a new writer-aligned triscript offline signature dataset with three contribution: (1) rich intra-writer script diversity. (2) recollecting data from 178 writers after a nine-month gap. (3) digitizing 19,953 physical signature sheets written using both a flatbed scanner and a smartphone. Some state-of-the-art offline signature verification methods are evaluated on the proposed dataset and show the performance of the proposed dataset.

Critical strengths

(1) The proposed dataset collect the signature with a large scale from 430 writers, which is very suitable for large scale training for verification systems. (2) The multi-session design is very interesting for this field. Recollecting data from 178 writers after a nine-month gap may capture several writer behaviors from different times. (3) The evaluation from cross-device verification, cross-session verification, and cross-dataset validation is meaningful in these settings.

Critical weaknesses

(1) Collecting the signature from different language scripts with the same writer is not very meaningful for verification tasks. It is easy to know that the writer may very familiar to one or two languages scripts. If the writer sign the signature with an unfamiliar language script, the signatures may distort with several strokes. It should give some analysis or experiments to discuss this situation.

(2) The statistical analysis is needed to show the characteristics of the proposed dataset. For example, the distribution of the writer may affect the behavior of different signatures. It is better to discuss different writer behaviors from the statistical analysis.

(3) More state-of-the-art offline signature verification methods should be evaluated and discussed in the experiment such as TransOSV (PR 2025) and CCASigNet (EAAI 2025).

(4) The cross-language experiment should be conducted and evaluated in the proposed dataset. Such as training on HAN language script and testing on LAT and ARA language scripts. From this experiment, we can see that the real writer behavior from different writers.

Questions for authors

Please see the weaknesses.

Reviewer_JL3COpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces TriSig-430, an offline signature verification (OffSV) dataset intended to expose robustness gaps that existing benchmarks cannot measure. The dataset contains 76,374 real handwriting images from 430 writers, of which 60,286 are signature images and 16,088 are content-controlled digit-string images. Two properties are claimed as novel relative to prior OffSV corpora:

  1. Writer-aligned tri-script coverage: every writer signs in Han, Latin, and Arabic (Uyghur, right-to-left), so script shift can be isolated from writer identity.
  2. Paired scanner–smartphone capture: the same physical sheets are digitized by both a flatbed scanner (HP LaserJet M227fdw, 600 dpi) and a smartphone (iPhone 13), yielding 19,953 instance-level pairs, so device shift is not confounded with rewriting.
  3. A nine-month longitudinal cohort: 178 of 430 writers are re-collected in a second session, enabling cross-session evaluation. The authors define three training splits (CORE-v1, DEV-MULTI, DEV-LONG) and one shared writer-disjoint test set (EVAL40, 40 writers), then evaluate within-domain verification, cross-dataset transfer, cross-script generalization, cross-content writer identification/retrieval, cross-device transfer, and cross-session robustness. Four headline findings are reported: (i) verification is worst on Uyghur; (ii) models trained on public datasets transfer poorly; (iii) training on both devices helps; (iv) performance degrades markedly over nine months.
Critical strengths
  1. Genuine and non-trivial data collection. 430 writers × 3 scripts × 2 sessions × 2 devices is a real logistical achievement. The nine-month re-collection cohort (178 writers) is rare, and I do not doubt that this was difficult to execute.
  2. The paired scanner–smartphone design is well motivated. Holding the physical sheet constant is exactly the right way to isolate acquisition-domain effects, and 19,953 instance pairs is a reasonable scale.
  3. Writer-aligned tri-script collection is the correct design for the cross-script question. Prior work (BHSig260, BRS-ID, Obaidullah et al. 2021) had to use non-overlapping signer populations; using the same writer for all three scripts genuinely fixes a known methodological weakness. The paper correctly identifies this.
  4. Uyghur/Arabic-script coverage is a real gap. The claim that underrepresented scripts are genuinely harder for current methods is supported by the ARA column consistently showing the highest EER in Table 2, and by Figure 3b where ARA bars sit above HAN and LAT bars for every training configuration.
  5. The cross-content negative result is interesting. Table 3's finding that Top-1 identification collapses when reference and query content differ, while NUM8 (same token for all writers) is harder than NUM6 even under matched content, is a clean and useful observation: current representations entangle writer identity with lexical content. This is the most ML-relevant result in the paper and deserves more prominence.
  6. Ethics and consent handling is described clearly and responsibly (written informed consent, separate forger consent, anonymization before release). This is well done.
Critical weaknesses

W1. As a dataset paper, this submission simply does not let reviewers see the data itself. The entire selling point is the dataset, so at the very least people need to see what the samples look like, such as whether the Uyghur script is legible, how severe the perspective distortion from the phone camera is, and how hard it is to tell skilled forgeries apart. But looking through the whole paper, there is only one schematic figure, one pipeline diagram, one EER heatmap, and six tables full of numbers. There is not a single full-size sample image, and no fine-grained statistics table broken down by script, device, session, or authenticity class.

W2. The numbers inside the paper do not reconcile, and two of the inconsistencies bear directly on the experimental protocol. The definition of the EVAL40 test set is ambiguous. It says these 86 writers "provide at least five genuine signatures and five skilled forgeries per capture device in S1," but §3.3 says S1 collected "3 to 5 genuine signatures and 3 to 5 skilled forgeries per script." If the upper limit is 5, then "at least 5" is basically sitting right at the ceiling, and the phrase "per capture device" never appeared in the S1 description at all. So did S1 actually collect more than 5, or is the EVAL40 criterion simply miswritten? This test set is the only one used for every experiment in the paper, and if how it was selected cannot even be stated clearly, then the credibility of every number that follows takes a hit. There are also a few other places that do not add up. For example, DEV-MULTI claims 19,953 paired instances, but given its description of 3 scripts plus 2 digit-string types multiplied by 2 devices, how that number was derived is never explained. On the forger side it only says "each target writer was assigned four to five," without giving the forger-to-target ratio or explaining how this interacts with the writer-independent EER protocol. None of this is reproducible enough.

W3. Two of the four main conclusions are stated too strongly for the evidence to support. The third conclusion says "training on both devices gives the best results," but I checked Table 5 row by row and that is simply not the case. In some cells a single-device-trained model is clearly better. On top of that, the whole table has no significance testing, no multi-seed variance, no confidence intervals, and the test set is only 40 writers. Under a 40-writer protocol, a difference of one or two EER points is basically noise, yet the paper directly uses the word "proves" and claims that device diversity significantly improves generalization. That is clearly overstating it. Table 6 has the same problem. The fourth conclusion says "performance drops markedly over nine months," and the trend itself is solid. But the paper fails to control for an obvious confound. S1 was captured with two devices while S2 used only the phone. It does say testing is "restricted to EVAL40 samples captured by the same phone in both sessions," but that restriction is chosen post hoc, and if the phone capture conditions in S1 and S2 differ, for example lighting, distance, or whether the sheets were the same batch, then the so-called "time effect" is mixed with an acquisition-condition effect. To separate the two, you would at least need a within-session control, such as re-imaging different sheets from the same session, but the paper does not do this.

W4. There are also some scattered issues that together add up to something quite serious. First, the differentiation from the MSDS work at NeurIPS 2022 is insufficient. That work already has Han-script signatures, a cross-session gap, and digit strings, yet the paper only mentions in §2.1 that it is "restricted to Han script signatures," without doing a head-to-head protocol comparison or explaining clearly what TriSig-430 adds beyond tri-script coverage and paired devices. Second, the baselines in Table 2 are not really fair. The WD column has 5 methods and the WI column also has 5 methods, but only HTCSigNet appears in both, and that is precisely the one adopted as the "primary baseline," which then wins every single column. This kind of incompletely crossed comparison is hard to find convincing. Third, the paper says "no validation split is used," with all results taken from the checkpoint of the final epoch, which means the reported numbers depend on an arbitrary choice of epoch count, and others reproducing it could easily get different results. For a benchmark, this is a real flaw. Fourth, it does not fit ICLR's style well. The paper has no new learning method, no new objective function, no new architecture, and no theory. Its content is essentially benchmarking in a hyperparameter format using off-the-shelf backbones, whereas dataset papers at ICLR usually come with a learning-relevant finding or some kind of mechanistic insight, and this paper clearly has no contribution of that kind.

Questions for authors
  1. EVAL40 criterion. Please reconcile "at least five genuine signatures and five skilled forgeries per capture device in S1" (EVAL40) with "3–5 genuine signatures and 3–5 skilled forgeries per script" (S1, §3.3). Exactly how many genuine and forged samples per script per device did S1 writers produce, and what fraction of the 178 two-session writers satisfied the stated threshold?
  2. Pair definition. Are the 19,953 "instance-level scanner–camera pairs" counted at the sheet level or the crop level? Do they include digit-string crops or signatures only? Please give the exact per-script, per-device, per-authenticity breakdown in a table.
  3. Session confound. In the S1→S2 comparison, were S1 and S2 smartphone images captured on the same physical sheets? Were lighting, camera distance, and operator controlled? What within-session baseline (e.g. S2 sheets re-imaged under S1-like conditions) rules out an acquisition-condition explanation for the observed degradation?
  4. Protocol-crossing in Table 2. Why do the WD and WI method sets differ, with only HTCSigNet in common? Can all methods be reported under both protocols?
  5. MSDS comparison. MSDS (Zhang et al., NeurIPS 2022) already provides Han signatures, cross-session data, and digit strings. Please state quantitatively what TriSig-430 enables that MSDS cannot, beyond script count and paired devices, ideally by re-running one protocol on both datasets.
  6. Model selection. With no validation split, how were epoch counts fixed, and how sensitive are the reported EERs to that choice? Please report a learning-curve or epoch-sensitivity analysis.
  7. Sample evidence. Given the absence of supplementary material, can the main text include at least one figure of full-resolution genuine/forgery crops for each script, and one scanner-vs-camera crop pair, so reviewers can assess data quality?
Reviewer_doCmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces TriSig-430, an offline signature dataset containing 76,374 handwriting images from 430 writers across Han, Latin, and Uyghur Arabic scripts. It includes skilled forgeries, content-controlled digit strings, 19,953 instances imaged by both a scanner and a smartphone, and a cohort of 178 writers sampled again approximately nine months later. The authors define writer-disjoint protocols for verification, identification, retrieval, and transfer across script, content, device, and session, and benchmark multiple existing methods.

Critical strengths
  1. The same-writer, three-script design enables cross-script analysis without changing the writer population. Together with the content-controlled samples, this makes the dataset useful for studying whether a model recognizes writing style or merely written content.

  2. Imaging the same physical handwriting instance with two devices provides a particularly useful control: acquisition differences can be studied without confounding them with a newly written sample. The longitudinal cohort adds another valuable dimension.

Critical weaknesses
  1. The release and privacy arrangements need clarification before the dataset contribution can be fully assessed. Participants were instructed to write full names, yet the paper describes the released images as anonymized and containing no personally identifiable information. An anonymous writer ID alone would not remove a readable name from an image. The manuscript reports consent but does not state whether an ethics board approved or exempted collection and public release of genuine signatures and skilled forgeries. Table 1 also calls the dataset publicly available, whereas the reproducibility statement promises future release.

  2. The benchmark evidence is less stable than the dataset design merits. The sole test split contains 40 writers, without uncertainty over alternative writer splits or training runs. Some claims about the benefit of training on both devices or sessions require controls for the amount and distribution of training data. Writer-specific names and NUM6 strings may also provide content cues, so the identification results alone do not establish style-based generalization.

Questions for authors
  1. Are the names visible in released signature images participants’ actual names, pseudonyms, or otherwise altered? What exactly did the consent cover, and was the study approved or exempted by an appropriate ethics review body?

  2. Can reviewers access the dataset, metadata schema, fixed trials, and license now? Please clarify the conflict between “publicly available” and “will be publicly released.”

  3. How sensitive are the main findings to the choice of 40 test writers and training seed? Are the joint-device comparisons controlled for training-set size, and can content-only cues be measured separately from writer style?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.