- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a compositional supervision method for cuneiform recognition by combining real sign crops with text-derived sequences to generate synthetic training data. Experiments across three recognition models show substantial reductions in Sign Error Rate (SER), while additional analyses investigate sequence training benefits and cross-domain generalization.
Critical strengths
- Extensive empirical evaluation supports the effectiveness of compositional pre-training.
- The paper provides useful analyses of training formats and cross-domain limitations.
Critical weaknesses
1.The proposed compositional synthesis method mainly combines existing sign crops and text sequences, following established synthetic data generation approaches in OCR. Its methodological contributions beyond prior work remain limited. 2.The multi-sign training improvements may partly result from changes in input length and CTC alignment rather than better sequence understanding. Moreover, the per-sign DeiT3 classifier outperforms SVTRv2* on both cross-domain benchmarks. 3.The main experiments use ground-truth sign boundaries and reconstructed line images, which simplify the recognition task and limit the applicability of the findings to real-world cuneiform recognition.
Questions for authors
1.How well does compositional pre-training transfer to naturally cropped cuneiform lines without oracle sign localization and reconstruction? 2.In the whole-side recognition experiment, how much of the performance improvement can be attributed specifically to compositional pre-training rather than the stronger detection, classification, language modeling, and decoding components? 3.How stable are the relatively small gains from preserving attested sign order and co-occurrence across independent Stage-A pre-training seeds?
2 comments
Thank you for your response. I will maintain my rating.
Contribution, real-world applicability, attribution and seed stability
Thank you for the review; each of your three questions is answered by a new experiment below.
W1. Contribution. What is new for cuneiform is the source of supervision: two resources that exist independently and at scale, isolated sign images and transliterated corpora (here 199,243 sign crops and 387,315 text lines), are turned into sequence-level training data without any paired line images. This extends prior cuneiform work on sign detection, classification and synthesis (Dencker et al., 2020; Mikulinsky et al., 2025; Bullenkamp et al., 2026). Composing crops into lines has precedents in OCR (e.g., Senthil et al., 2020; StackMix), and we do not claim a new synthesis algorithm. Our methodological contribution is a tested route from independently collected Assyriological resources to image-to-sequence training, with controlled evidence of which parts of this supervision matter and which transfer. Under tablet- and text-disjoint controls, the supervision lowers SER by 24-52 points across three architectures. Most of the gain comes from the multi-sign line format (23-46 points on strict subsets); attested sign order adds a smaller in-period gain. The gain also holds beyond reconstructed lines with oracle sign boundaries: on natural line images (89.2% to 52.8%), on automatically found lines, and on tablets whose images are outside all our data.
W3 / Q1. Without oracle localization. The benefit carries over. On natural line crops cut from the tablet image (no stitching, no sign boundaries given to the model), Stage-A lowers in-period SER from 89.2% to 52.8%, and by 19.0 and 7.0 points cross-period and on HT. It also holds with no line annotation at test time (outputs tagged OOV are excluded, as the references exclude out-of-inventory regions). With line regions from the system's predicted rows, whole-side SER falls from 90.6% to 55.7% in-period. With an independent line segmenter trained on tablets of other periods and used without correction, it falls by 29.5 points [27.4, 31.7] in-period and by 26.7 points [22.5, 31.0] on source-disjoint tablets (Section 6.1, Appendix I, Table 23).
Q2. How much of the whole-side gain is due to pre-training. In the full system, pre-training acts only through the sequence verifier, which lowers SER by 1.83 points in-period, 2.63 cross-period and 2.18 on the source-disjoint tablets, with every other component fixed. Much of this share comes from Stage-A: with both verifier weights chosen by the same DEV rule, the Stage-A verifier beats the no-Stage-A one by 1.48, 1.91 and 1.29 points (each CI above zero), and with both fixed at the paper's weight, by 1.28 and 1.36 points in-period and cross-period. The rest of the drop from 68.92% to 38.05% comes from the stronger detection, classification and LM decoding, as Section 6.3 states; the original baselines already use Stage-A recognizers, so that comparison does not isolate pre-training. Where the sequence model itself reads the image, Stage-A is worth far more: 35.8 points as a decoder on predicted crops (32.7 with the language model), and 27-35 points on automatically found natural lines (Appendix M, Table 28; Appendix I, Table 23).
Q3. Stability across Stage-A seeds. Stable in-period, seed-dependent off-domain. With two extra seeds, the in-period gain is positive for every seed (mean 4.64 points on all lines, across-seed 95% interval [3.09, 6.19]; 2.19 on the strict subset). Off-domain, seed 456 reverses it (strict means 1.28 cross-period and 1.01 on HT), and Section 6.2 now says so. The full table is in our reply to 4j2R (Q2) and in Appendix L. The paper's large effects, Stage-A itself and the line format (W1), do not rest on this contrast. Across the three Stage-A seeds, the shuffled model's strict-subset SER moves by at most 2.1, 5.7 and 4.5 points, far less than the 45.1, 46.2 and 23.1-point line-format gain (Appendix L).
W2. Input length and CTC alignment. Output length explains only part of the line-format gap. The single-sign model does under-emit on multi-sign lines, yet on lines where both models output exactly the right number of signs, the line-format model is still far more accurate per position: 86.7% vs. 63.8% in-period, 64.7% vs. 23.8% cross-period and 31.9% vs. 11.9% on HT. The SER gap is already 33.4 points on 3-sign lines, the shortest we evaluate (Section 6.2, Appendix Q). We do not claim linguistic understanding; the attested-vs.-shuffled test (Q3) is the direct test of text structure. On DeiT3, the two models are complementary: rescoring DeiT3's top-5 candidates with the Stage-A SVTRv2* beats DeiT3 alone on all three sets (21.0% to 14.6% in-period, 27.8% to 25.1% cross-period, 62.4% to 61.7% on HT; on the strict subsets, 22.2% to 17.0% in-period and 28.3% to 26.1% cross-period), and with the no-Stage-A model the off-domain gains disappear (Section 6.1, Table 2, Appendix N).