acceptodds
Under review as a conference paper at ICLR 2027

Domain-Bootstrapped OCR for Low-Resource Traditional Chinese Painting Inscriptions

Abstract

Traditional Chinese Painting Inscriptions (TCPIs) carry critical information for authorship attribution, artwork dating, provenance tracing, and cultural interpretation, making automatic recognition of such inscriptions an important task for OCR in cultural heritage research. However, TCPIs pose challenges beyond standard document or scene-text OCR due to severe domain shifts, including free-form brush calligraphy, complex background interference, faded ink, and other degradations. These challenges are amplified in low-resource settings, where high-quality annotated TCPI data are scarce and expensive to obtain. To address these issues, we propose a domain-bootstrapped OCR pipeline tailored for low-resource TCPIs. Original paintings are divided into small and large groups and cropped into TCPI patches. The smaller group is used to construct a high-quality seed dataset through multi-model candidate generation and expert verification. A general-purpose OCR model is fine-tuned on this seed dataset to produce an initial TCPI-adapted model, which then generates pseudo-labels for the larger unlabeled group. The same OCR model is subsequently retrained via bootstrapped training on these pseudo-labeled samples using optimized strategies to further enhance recognition performance. Experimental results show that the TCPI-bootstrapped model achieves a clear performance advantage () over existing state-of-the-art methods. Ablation studies further demonstrate that recognition performance consistently improves as the pseudo-labeled dataset expands, highlighting the effectiveness and scalability of the proposed domain-bootstrapping pipeline. The model and benchmark will be released upon acceptance.

Primary area:
datasets and benchmarks
Keywords:
Traditional Chinese Painting Inscription, OCR, Vision-Language Models, Bootstrap

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EqwdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a domain-bootstrapped OCR pipeline for recognizing Traditional Chinese Painting Inscriptions (TCPIs) under limited high-quality supervision. It first constructs a small seed dataset through a machine–expert annotation protocol that combines candidate transcriptions from three pretrained OCR models with expert verification. A general-purpose OCR model is then fine-tuned on the seed dataset to obtain a TCPI-adapted model, which generates pseudo-labels for a larger unlabeled collection. After filtering these pseudo-labels, the resulting expansion dataset is used to train the TCPI-bootstrapped model for domain-specific inscription recognition.

Critical strengths
  1. The paper addresses a meaningful and challenging cultural heritage application. TCPI recognition involves expressive calligraphy, irregular layouts, background interference, and image degradation, making it a practically valuable setting for studying low-resource OCR adaptation.
  2. The pipeline provides a practical approach to constructing domain-specific supervision. Combining multi-model candidate generation, expert verification, TCPI adaptation, and pseudo-label expansion offers a coherent workflow for using a small verified seed dataset to support larger-scale training.
  3. The reported validation results show clear benefits from domain adaptation. On the 94-sample seed validation set, character-level F1 improves from 0.825 for the base olmOCR-2 model to 0.861 for TCPI-adapted and 0.868 for TCPI-bootstrapped. These results provide evidence that domain-specific supervision benefits TCPI recognition.
  4. The ablation studies provide useful empirical observations about training choices. The paper examines pseudo-labeled training scale, augmentation schemes, and FPFT/LoRA/RL strategies, with additional breakdowns by clarity, calligraphy, interference, and text length. These analyses reveal potentially useful domain-specific considerations, such as the adverse effect of rotation augmentation under the evaluated settings.
  5. The expert audit improves transparency regarding pseudo-label quality. The paper independently evaluates pseudo-labels against expert transcriptions on 463 expansion samples and reports both aggregate and length-specific results. This makes the remaining label noise explicit and provides useful evidence about the supervision used for bootstrapped training.
Critical weaknesses
  1. The paper states that approximately 12,000 paintings were used in the data construction pipeline, but it does not identify the public datasets, institutions, or websites from which the images were obtained. The collection procedure and applicable usage conditions are also unspecified. This leaves potential copyright and licensing concerns unresolved.
  2. The paper presents a complete and reproducible TCPI OCR pipeline, but it does not explicitly identify the final training configuration used for the TCPI-bootstrapped model in the main experiments. The Overall metrics in Table 2 match those of the “FPFT + W/o Rotate” configuration in Tables 3–5, leaving readers to infer the configuration indirectly.
  3. The paper uses a fine-tuned DINO-YOLO inscription detector to localize inscriptions within paintings, but it does not clarify the specific DINO and YOLO variants involved, how the components are combined, or which parameters are fine-tuned. It also does not report detector performance, such as mAP or a clearly defined [email protected]. Furthermore, the source and size of the detector fine-tuning dataset, preprocessing procedures, bounding-box annotation protocol, and training/validation split are not specified.
  4. Important training configurations are insufficiently reported across the three training strategies and experiments. Missing details include learning rates, optimizers, batch sizes, and training epochs; LoRA target layers, α\alpha, and rr; and the RL optimization algorithm, number of samples per input, sampling temperature, and regularization terms and coefficients. If a KL penalty is used, its formulation and coefficient should also be reported. These omissions hinder reproducibility and make it difficult to determine whether, and to what extent, the performance differences in Tables 2 and 3 are attributable to differences in hyperparameter configurations.
  5. The ablation studies in Tables 3–5 are primarily evaluated on the Overall set. Although this set can assess in-domain knowledge retention, approximately 90% of its samples were used during the initial adaptation stage, so it cannot provide an independent assessment of generalization. Even if the second stage does not initialize from the TCPI-adapted model, supervision from the seed training set can still influence the final model through pseudo-label generation. Consequently, the current ablations do not adequately establish the contribution of each component to generalization on unseen samples.
Questions for authors
  1. Could the authors provide the data sources, collection procedures, copyright or licensing information, and redistribution conditions discussed in Weakness 1?
  2. Regarding Weakness 2, could the authors explicitly specify the final training strategy, augmentation scheme, and model-selection criteria? Do the Validation and Overall results in Table 2 come from the same checkpoint?
  3. Could the authors provide the detector architecture, training data, fine-tuning procedure, and evaluation details discussed in Weakness 3?
  4. Regarding Weakness 4, could the authors report the important hyperparameter configurations for all three training strategies across the experiments?
  5. Regarding Weakness 5, could the authors report the corresponding ablation results on a completely held-out validation set or an independent test set?
  6. Could the authors provide the complete computation of the rule-based score used for pseudo-label filtering? The paper mentions special symbols and repetitions, but does not specify their precise definitions, penalty weights, normalization procedures, or handling of anomalous outputs. A formula or pseudocode would help make this step reproducible.
  7. In the second-stage training that produces the TCPI-bootstrapped model fθ2f_{\theta_2}, are the parameters initialized from the TCPI-adapted model fθ1f_{\theta_1}, the base olmOCR-2 model fθ0f_{\theta_0}, or another checkpoint? Do FPFT, LoRA, and RL use the same initialization? Does this stage use only the pseudo-labeled expansion dataset, or is the seed training dataset also included?
  8. In Table 2, TCPI-bootstrapped has an EM score that is 2.1 percentage points lower than TCPI-adapted on Validation, despite improvements in F1 and NES. Could the authors analyze possible reasons for this decrease after training on the expansion dataset, including whether pseudo-label noise, the training objective, or other configuration changes contribute to it?
  9. The paper reports that RL underperforms FPFT and LoRA overall and discusses its localized advantage on long inscriptions, but does not sufficiently analyze the reasons for its lower overall performance. Could the authors discuss whether the mismatch between the reward objective and evaluation metrics, pseudo-label noise, or training configurations may contribute to this result?
  10. The complete training experiments are conducted only with olmOCR-2, so the applicability of the pipeline to other OCR backbones remains unclear. Could the authors evaluate one or two additional trainable OCR models with different architectures or pretraining backgrounds to determine whether they also benefit from this pipeline?
Reviewer_za22OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on the problem of OCR of traditional Chinese painting inscriptions (TCPIs). These paintings are difficult in that they differ substantially from typical Chinese text/glyphs, and are a low-resource problem in that labeled data is scarce and expensive to obtain.

Given this problem, the paper proposes the following methodology: generate ground-truth labels on a small set of TCPIs (~1000), through a process of first applying off the shelf methods followed by expert verification. Subsequently, these labels are used to fine-tune an existing method (TCPI-adapted). These labels are then used in a self-distillation set-up, where TCPI-adapted is used to generate predicted labels on a set of unseens TCPIs, and these predicted labels are used as pseudo-labels to train a new system (TCPI-bootstraped).

TCPI-adapted and TCPI-bootstrapped are evaluated against existing methods on a small validation set, and some ablation studies are given.

Beyond the core system/method itself, the construction of a benchmark for TCPIs as well as released model are given as contributions of the paper.

Critical strengths

problem domain identification:

  • the results of Table 2 seem to indicate that TCPI is difficult for existing off-the-shelf methods
  • and additionally, from a small-to-moderate amount of domain-specific ground-truth, the results seem to non-trivially increase
Critical weaknesses

from my viewpoint, there are two primary weaknesses:

  • methodological novelty - the proposed system is to use the labels generated by an initial system (pseudo-labels) as pseudolabels to retrain a subsequent system. this is a common technique, referred to as self-distillation (which may be more appropriate than "bootstrapping", which in my understanding has a different meaning of resampling training instances with replacement typically to estimate variability/confidence intervals). so from this perspective, the paper is not introducing a novel method, but rather more of an application paper to a particular problem domain.

  • experimental validation / impact of proposed approach - in evaluating the proposed method, a central question is how much this improves over simply re-training with in-domain labels, ie comparing TCPI-bootstrapped versus TCPI-adapted. The only comparison given is in Table 2, on the small 94-sample validation set, and within this, the difference between these methods is quite small. Given the magnitude of the differences and sample-size, it is difficult to say if this result (improvement from bootstrap) is statistically significant. From Appendix D, it seems there is another set of 463 labeled samples available, it would be good to report the accuracy of these two methods on this set as well. And/or other means could be used to estimate confidence interval, such as training multiple models from different subsets of the available seed labels.

Questions for authors

If I am interpreting the details in 4.1 Experimental Setup correctly, it seems like 2000 paintings are initially considered as the seed set, but after the pipeline is applied, only 942 verified samples are generated? What is the reason for seemingly more than half of the paintings not obtaining a usable sample? Does this indicate fundamental failure modes with the off-the-shelf methods used to do the initial proposed labeling? If so, it seems to indicate this would be a fruitful direction to pursue, and perhaps it would be worthwhile for human annotators to directly provide ground-truth, which might give better coverage of these difficult cases.

Reviewer_XZkzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper addresses the challenging task of optical character recognition (OCR) for traditional Chinese painting inscriptions (TCPIs), which suffer from diverse calligraphic styles, complex background interference, and limited annotated data. The authors propose a domain-bootstrapped OCR framework that integrates expert-verified seed dataset construction, domain-specific model adaptation, confidence-based pseudo-label expansion, and iterative retraining. The paper also introduces a TCPI benchmark consisting of 942 human-verified samples and an additional 4,943 pseudo-labeled samples. Extensive experiments compare the proposed approach with existing OCR systems and multimodal large language models, supported by ablation studies on training strategies, data scales, and visual characteristics. The proposed method achieves an F1 score of 0.851 on the full seed dataset and 0.868 on the validation subset, demonstrating the potential of domain bootstrapping for low-resource cultural heritage text recognition.

Critical strengths

The paper addresses a challenging and underexplored OCR problem involving traditional Chinese painting inscriptions, where complex calligraphic styles, background interference, and limited annotations make recognition particularly difficult. A key strength is the proposed domain-bootstrapped pipeline, which combines expert-verified seed annotations, domain adaptation, and pseudo-label expansion to reduce the dependence on large-scale manual labeling. The authors also provide a relatively comprehensive evaluation against various OCR models and multimodal large language models, together with ablation studies examining different training strategies and challenging data characteristics. These contributions demonstrate the practical value of the proposed approach and its potential to support low-resource OCR research in cultural heritage applications.

Critical weaknesses

Despite its practical relevance, the paper has several limitations concerning benchmark reliability, methodological novelty, and experimental validation. First, the evaluation lacks a sufficiently large and fully independent test set, as the reported overall results are obtained on the seed dataset that was also used during model adaptation, while the separate validation set contains only 94 samples. This raises concerns about the generalizability of the reported improvements. Second, the proposed bootstrapping framework primarily integrates established techniques, including supervised fine-tuning, pseudo-label generation, and iterative training, without introducing substantial methodological innovations. Furthermore, the performance improvement over the domain-adapted model is relatively modest on the validation set, and the exact-match accuracy even decreases, making the effectiveness of the bootstrapping strategy less conclusive. Additional independent evaluations and more rigorous analyses of pseudo-label quality would strengthen the paper's claims.

Questions for authors

1.The paper evaluates the proposed method on a 94-sample validation set and the full 942-sample seed dataset, which partially overlaps with the training data. Could the authors clarify how data independence is ensured throughout model training, selection, and evaluation? How can the reported results reliably demonstrate generalization without a fully independent test set? 2.According to Table 2, the proposed bootstrapped model achieves only a modest F1 improvement over the TCPI-adapted model on the validation set (0.861 to 0.868), while the exact-match score decreases from 0.223 to 0.202. Could the authors explain whether these improvements are statistically significant and provide evidence that the bootstrapping strategy consistently improves recognition performance? 3.The proposed framework relies heavily on automatically generated pseudo-labels, whose quality directly affects subsequent training. Although the authors provide a human verification analysis, could they further clarify how the confidence threshold is determined and how residual annotation errors influence model performance? What evidence supports the reliability of the expanded training dataset? 4.The proposed pipeline combines established techniques, including expert-assisted annotation, supervised fine-tuning, pseudo-label generation, and iterative training. Could the authors clarify the key methodological contributions beyond conventional semi-supervised learning and self-training frameworks? In particular, which components are primarily responsible for the observed performance improvements?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.