acceptodds
Under review as a conference paper at ICLR 2027

Beyond Pixels: SVG Token Prediction for Historical Document Restoration

Abstract

Historical document restoration (HDR) aims to recover damaged content while preserving the original appearance of historical documents. Diffusion-based and MLLM-based methods have made promising progress in HDR. However, they still provide limited control over character geometry and unintentionally alter preserved character structures. Scalable Vector Graphics (SVG) provide a vector-based representation of character geometry through explicit paths and coordinates. To leverage this representation, we reformulate HDR as **SVG token prediction**, where surviving character structures are retained and only the missing parts are recovered in vector space. To support this formulation, we introduce **VECodex**, the first large-scale vector-based HDR dataset, containing 35.7M aligned restoration instances across four realistic damage types. Its construction combines memory-guided multi-agent quality control with Boolean shape splitting for precise raster–vector alignment. We further propose **Script**, which jointly utilizes pixel-level visual evidence and vector-level character structure to predict missing SVG tokens while preserving observed paths. Experiments on **VECodex** show that **Script** achieves state-of-the-art performance in both character restoration and final document quality, substantially improving Mask IoU and Mask B-F by 23.74% and 38.36% over the strongest baseline, respectively. These results demonstrate the potential of explicit vector-space structure recovery as a new direction for HDR beyond direct pixel generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Historical document restoration, SVG token prediction, document image restoration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2Fi9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper formulates historical character restoration as predicting missing SVG tokens while retaining observed paths. It introduces VECodex, with 2.23 million source samples and 35.7 million simulated restoration instances, and Script, combining raster evidence, partial SVG context, and background inpainting. Experiments report substantial character and masked-region improvements, with ablations for raster evidence and composition order.

Critical strengths
  1. The representation directly addresses preservation of surviving geometry. Table 2's masked-region IoU improvement from 56.39% to 80.13% shows that the advantage goes beyond retaining undamaged strokes. This is a promising formulation for controlled character restoration.

  2. The aligned dataset is substantial, and all degradation variants of a source remain in one split. The matched SVG-only ablation also shows a useful cross-modal effect: raster evidence helps most under water stains, where faint stroke information survives.

Critical weaknesses
  1. The evaluated inputs assume clean surviving geometry. Section 3.2 constructs observed SVGs by splitting the undamaged source, while Section 4 assumes those paths are available at inference. No damaged-input vectorization or noisy-path evaluation establishes how much performance survives the intended scan-to-restoration pipeline. This limits practical applicability without invalidating the vector-conditioned task.

  2. The source of the gains is incompletely isolated. Table 2 does not explain how raster baselines were adapted to VECodex or their training budgets. Table 3 establishes the value of adding raster evidence, but not of observed SVG context relative to a matched backbone. Consequently, the gains cannot yet be attributed specifically to the proposed representation rather than backbone and training differences.

Questions for authors

How are observed paths obtained from an actual damaged scan, and what happens when these extracted paths replace the source-derived inputs?

Were Table 2 baselines trained on the same split? A matched SVG predictor without observed SVG context would be a particularly useful additional control.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 2Fi9

We thank the reviewer for the constructive comments. We evaluated SVGs extracted from damaged scans and added controls for training conditions and observed SVG context.

✅ Please find rebuttal figures HERE.

The evaluated inputs assume clean surviving geometry... No damaged-input vectorization or noisy-path evaluation establishes how much performance survives the intended scan-to-restoration pipeline.

Q1. How are observed paths obtained from an actual damaged scan, and what happens when these extracted paths replace the source-derived inputs?

(1) Observed Path Extraction. At inference, observed paths are extracted from the damaged image I~\tilde{I} and the damage mask MM. The multi-agent pipeline of Sec. 3.1 first determines whether HAT×4 super-resolution is needed. It then extracts and cleans the character foreground, removes foreground inside MM, and vectorizes the remainder with Potrace. No undamaged image or complete source SVG is used in this extraction process.

(2) Performance with Extracted Paths. We replaced the source-derived observed paths with extracted paths and evaluated the same SCRIPT checkpoint on 1,600 test instances (400 per damage type), without retraining.

Table R1. Source-derived vs. extracted observed paths.

Observed pathsMask IoUMask B-F1
Source-derived (submission)81.23%80.76%
Extracted from I~\tilde{I}79.56%79.13%

These limited reductions indicate that SCRIPT retains most of its restoration performance with paths extracted from damaged inputs. Qualitative comparisons are provided (Fig. R1, HERE).

We also re-extracted the paths and re-ran the complete pipeline with masks dilated, eroded, or shifted by 4 pixels. Mask IoU remains between 74.49% and 76.22% under these perturbations (Table R5). Case studies on naturally damaged manuscripts further evaluate the complete pipeline under real-world conditions (Fig. R2, HERE).

The source of the gains is incompletely isolated... Table 3 establishes the value of adding raster evidence, but not of observed SVG context relative to a matched backbone.

Q2. Were Table 2 baselines trained on the same split? A matched SVG predictor without observed SVG context would be a particularly useful additional control.

In the original submission, all baselines in Table 2 used publicly available pretrained checkpoints without additional fine-tuning on VECodex. For a matched comparison, we have now fine-tuned the strongest baseline, LaMa, on the same VECodex training split and adopted a unified evaluation protocol using the same foreground extraction and Potrace pipeline. SCRIPT still outperforms fine-tuned LaMa by 19.62 percentage points in Mask IoU (80.13% vs. 60.51%, Table R2). On the 1,600 test instances with matched input information, SCRIPT achieves 79.56% Mask IoU, compared with 61.75% for LaMa (Table R3).

Matched SVG predictor without observed SVG context. As suggested, we trained two models that differ only in whether the observed SVG is given as input. Both use the same backbone (Qwen3.5-9B + LoRA), the same 20,000 training instances and the training settings of the paper (LoRA rank 16, learning rate 2×10⁻⁴, eight gradient-accumulation steps, 3 epochs), and both predict the missing paths. They are evaluated on the 1,600 instances of Table R1.

Table R8. Matched backbone, data and budget, with and without observed SVG context.

InputMask IoUMask B-F1
Image + mask + observed SVG64.88%64.12%
Image + mask only19.45%17.12%

With the backbone, training data and budget held fixed, removing the observed SVG context reduces Mask IoU from 64.88% to 19.45% and Mask B-F1 from 64.12% to 17.12%. Without the observed paths, the model has to place new paths in absolute coordinates from visual features alone, and its strokes rarely align with the surviving ones. Even a much larger MLLM without any fine-tuning, when given the observed SVG (Qwen3.8-Max, 34.86%, Table R3; Fig. R4, HERE), outperforms this trained model without it. The gain therefore comes from the observed SVG context, not from the backbone or the training budget. Both 20,000-instance models score below the full SCRIPT (79.56%, Table R1), which shows in addition that training on the full VECodex matters.

We hope these controls address the concerns and would appreciate reconsideration of the rating.

Dear Reviewer 2Fi9,

I hope this message finds you well. With less than 22 hours remaining in the discussion period, we wanted to check whether our response has addressed your concerns. If you have any further questions or feedback, please let us know. We value your perspective and would be happy to clarify any remaining points.

Thank you for taking the time to review our paper.

Reviewer_iHcnOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces an approach to historical document restoration (HDR) based on SVG representations of document characters, aiming to better preserve their original style and geometric structure. Specifically, the paper first proposes a pipeline for synthesizing damaged character data and constructs VECODEX. It then introduces SCRIPT (Structured Cross-modal Restoration via Image–Path Tokens), which uses Qwen3.5-9B adapted with LoRA to recover missing character structures through SVG token prediction. The recovered characters are subsequently combined with paper-background restoration in pixel space using SDXL 1.0 inpainting to produce the final restored images.

Critical strengths
  1. Formulating HDR as SVG token prediction is an interesting approach. SCRIPT preserves the observed SVG paths and predicts only the missing SVG tokens, providing explicit constraints on character geometry. Furthermore, separating character recovery from paper-background restoration and protecting the recovered characters during background inpainting help reduce unnecessary modifications to existing strokes and restored structures.
  2. VECODEX represents a substantial data contribution, containing approximately 2.23 million source samples and 35.7 million aligned restoration instances across four simulated damage types and four damage ratios. Boolean shape splitting aligns damage regions in pixel space with the corresponding observed SVG and missing SVG in vector space, providing valuable training and evaluation data for character restoration using SVG representations.
  3. SCRIPT achieves strong results in both character restoration and final document image quality under the synthetic damage settings evaluated on VECODEX. For example, it achieves 80.13% Mask IoU and 79.87% Mask B-F1, exceeding LaMa by 23.74 and 38.36 percentage points, respectively. The ablation studies also support the benefit of pixel–SVG joint reasoning, particularly for water stains, where some visual evidence of the damaged strokes remains available.
Critical weaknesses
  1. VECODEX is an important contribution of this paper. However, the manuscript does not clearly specify which datasets, collecting institutions, or websites provided the source character images, nor does it describe how these images were obtained or their licensing and redistribution conditions. Without this information, readers cannot adequately assess the reproducibility of the dataset construction process or the conditions governing its subsequent use.
  2. The paper does not report evaluation results on real, naturally occurring damage rather than simulated damage. Naturally occurring damage in historical documents is often more complex than the simulated degradations considered in the paper.
  3. The proposed model takes a pixel-level damage mask as input. In practical applications, manually annotated masks often contain boundary inaccuracies, which may limit the model’s applicability in real-world settings. The paper does not analyze its sensitivity to such mask boundary errors.
  4. The observed SVG used in the experiments is constructed from a complete reference character. However, the paper does not clearly explain how this input can be obtained at inference time using only a damaged image and an estimated damage mask, further limiting its practical applicability. The paper also does not evaluate the effects of foreground extraction and vectorization errors on restoration performance. It therefore remains unclear whether the reported results would hold under input conditions achievable in practice.

Overall, I consider the model’s practical applicability in real-world settings to be limited. I would raise my rating if the authors could provide convincing evidence that the model is effective in such settings.

Questions for authors
  1. Could the authors provide the data sources, acquisition procedures, copyright and licensing information, and redistribution conditions discussed in Weakness 1?
  2. As discussed in Weakness 2, could the authors report evaluation results on real, naturally occurring damage? If a suitable real-world dataset is unavailable, could they provide one or two case studies of restoration on actual damaged historical documents?
  3. As discussed in Weakness 3, could the authors evaluate the effects of perturbing the damage masks, for example through dilation, erosion, and spatial shifts, on restoration performance?
  4. As discussed in Weakness 4, could the authors clarify how the observed SVG can be obtained during real-world inference using only a damaged image and a damage mask with imprecise boundaries? Could they also compare restoration performance using observed SVG constructed from complete reference characters with performance using observed SVG extracted solely from damaged images and estimated masks?
  5. The paper introduces a complex and time-consuming memory-guided multi-agent construction framework to determine whether samples require HAT×4 super-resolution and provides a qualitative explanation for this design. However, it lacks quantitative comparisons with simpler strategies that always or never apply super-resolution, as well as an ablation comparing the framework with and without human-validated memory references. Could the authors provide these comparisons to establish whether the additional computational cost yields sufficient improvements in data quality?
  6. Each restoration instance currently contains only one damage type. Can the model still correctly restore a character when multiple damage types occur within the same character?
3 comments
AuthorsOpenReview ↗

Response to Reviewer iHcn (1/2)

We thank the reviewer for the constructive review, and for stating clearly what evidence would change the assessment. The central concern is whether SCRIPT works in practice, so we evaluated the complete pipeline under the conditions the reviewer listed: real damage, imprecise masks, observed SVG extracted from the damaged image, and mixed damage.

✅ Please find rebuttal figures HERE.

Q1. Could the authors provide the data sources, acquisition procedures, copyright and licensing information, and redistribution conditions?

All source images come from 432 Dunhuang manuscripts, provided by the Dunhuang Academy under a research cooperation agreement. The agreement permits releasing the cropped single-character images, the SVGs, the damage masks and the damage parameters; we will release them, together with the construction and evaluation code, upon publication. These details will be added to Sec. 3.

Q2. ...could the authors report evaluation results on real, naturally occurring damage? If a suitable real-world dataset is unavailable, could they provide one or two case studies of restoration on actual damaged historical documents?

Yes. We evaluated the complete SCRIPT pipeline on a historical manuscript with naturally occurring damage that was not included in VECodex. The characters were obtained from Pelliot chinois 3381 (Qinfu yin, Bibliothèque nationale de France), where stains obscure parts of the strokes. Damage masks were automatically estimated and manually refined, while observed SVG paths were extracted directly from the damaged scans, without relying on synthetic inputs. SCRIPT restores the obscured strokes while preserving the surviving geometry (Fig. R2, HERE). Two Dunhuang manuscript experts qualitatively evaluated the results and confirmed their visual consistency with the surviving strokes.

...manually annotated masks often contain boundary inaccuracies, which may limit the model's applicability in real-world settings...

Q3. ...could the authors evaluate the effects of perturbing the damage masks, for example through dilation, erosion, and spatial shifts, on restoration performance?

SCRIPT remains robust to the evaluated mask perturbations and consistently outperforms fine-tuned LaMa. We evaluated 1,600 test instances (400 per damage type) by dilating, eroding, or shifting the damage masks by 4 pixels (2–11% of crop size). For each perturbation, we re-ran the complete pipeline, including observed SVG extraction, using the perturbed mask. Evaluation was performed within the original ground-truth mask.

Table R5. Mask IoU under mask perturbations (1,600 test instances).

MaskSCRIPTLaMa (fine-tuned on VECodex)
Accurate79.56%61.75%
Dilated by 4 px76.22%59.89%
Eroded by 4 px74.49%56.16%
Shifted by 4 px75.16%55.65%

SCRIPT’s Mask IoU decreases from 79.56% to 74.49% in the worst case, while its advantage over fine-tuned LaMa ranges from 16.33 to 19.51 percentage points. Erosion and spatial shifts cause larger decreases because parts of the damaged region fall outside the restoration mask.

Q4. ...could the authors clarify how the observed SVG can be obtained during real-world inference using only a damaged image and a damage mask with imprecise boundaries? Could they also compare restoration performance using observed SVG constructed from complete reference characters with performance using observed SVG extracted solely from damaged images and estimated masks?

At inference, VobsV_{\text{obs}} is extracted from the damaged image and the mask alone; the complete reference is used only to build training targets. The pipeline of Sec. 3.1 chooses the processing route for I~\tilde{I}, extracts the character foreground, clears it inside MM, and vectorizes the rest with Potrace. The submission evaluated SCRIPT with VobsV_{\text{obs}} cut from the complete reference; Table R1 gives the comparison the reviewer asked for, on the same 1,600 instances and without retraining. The extracted setting includes all foreground-extraction and vectorization errors (Fig. R1, HERE).

Table R1. Observed SVG from the complete reference vs. extracted from the damaged image.

Observed SVGMask IoUMask B-F1
From the complete reference (oracle)81.23%80.76%
Extracted from I~\tilde{I}79.56%79.13%

Extraction costs only 1.67% Mask IoU and 1.63% Mask B-F1. With imprecise masks, VobsV_{\text{obs}} is re-extracted with the perturbed mask, so Table R5 already reports the combined effect: at most 5.07% Mask IoU for a 4-px boundary error.

AuthorsOpenReview ↗

Response to Reviewer iHcn (2/2)

Q5. The paper introduces a complex and time-consuming memory-guided multi-agent construction framework... However, it lacks quantitative comparisons with simpler strategies that always or never apply super-resolution, as well as an ablation comparing the framework with and without human-validated memory references.

Our results demonstrate that adaptive super-resolution selection substantially outperforms fixed strategies, achieving 99.1% routing accuracy compared with 55.7% and 44.3% for never and always applying super-resolution, respectively.

Why not a fixed pipeline. A fixed pipeline (Otsu binarization + Potrace) applied to the naturally damaged characters introduces spurious strokes from stains, paper speckles and bleed-through, and fragments faint strokes (Fig. R3, HERE). In contrast, our pipeline preserves the strokes and removes the contamination, as shown in the second column (Fig. R2, HERE). Such errors would otherwise propagate into both the training targets and the observed SVG at inference.

Always vs. never super-resolution. We randomly sampled 1,000 source images and produced both versions of each, with and without HAT×4. Two annotators (PhD students in the humanities and in computer science) independently judged which version better preserves the original strokes; they agreed on 998 images (99.8%).

Table R7. Routing accuracy on 1,000 annotated images.

StrategyCorrect routing
Never HAT×455.7%
Always HAT×444.3%
Ours (multi-agent)99.1%

Of the 440 images our framework sent to HAT×4, 437 were judged correct; of the 560 it kept at native resolution, 554 were. Neither fixed strategy works: without super-resolution, dense characters in low-resolution crops (e.g. 田) merge into solid blobs, while super-resolving clean crops amplifies stains and background contamination (Sec. 3.1). The planning agent makes this decision with the human-validated references retrieved by the memory agent (Sec. 3.1), so 99.1% is the accuracy of the complete framework. Its cost is paid once, offline, when the dataset is built; at inference, only route selection and extraction are run.

Without memory, hard-metric routing selects 612 of 1,000 images for super-resolution, including at least 169 unnecessary cases (≥16.9% of the test set), potentially introducing fragmented strokes in the resulting SVGs.

Q6. Each restoration instance currently contains only one damage type. Can the model still correctly restore a character when multiple damage types occur within the same character?

Yes. We evaluated mixtures of two, three, and four damage types at a total damaged character area of 30%. We tested all six two-type combinations (15% per type) and one combination each with three and four types, using 1,600 test instances per combination. For each mixture, the single-type reference is the mean score of its constituent damage types at 30% damage in Fig. 5.

Table R6. Character IoU / B-F1 (%) under mixed damage at 30% total damaged character area, with 1,600 test instances per combination.

Damage combinationIoU (single-type avg.)IoU (mixed)B-F1 (single-type avg.)B-F1 (mixed)
Water stain + wormholes83.2583.6877.7579.18
Water stain + edge loss83.0083.9677.5078.34
Water stain + tears84.0083.7878.5078.61
Wormholes + edge loss88.7588.1283.2583.95
Wormholes + tears89.7591.5084.2582.86
Edge loss + tears89.5088.8384.0085.01
Wormholes + edge loss + water stains85.0081.2479.5076.15
All four types86.3880.4780.8875.78

The six two-type combinations remain close to their single-type references, while the tested three- and four-type mixtures show larger reductions.

The reviewer noted that convincing evidence of effectiveness in real-world settings would raise the rating. We hope the results on real damaged manuscripts (Fig. R2, HERE), imprecise masks (Table R5), extracted observed SVG (Table R1; Fig. R1, HERE) and mixed damage (Table R6) provide this evidence, and we are happy to run further checks during the discussion.

The author’s rebuttal is quite detailed; however, incorporating all these results into the paper would necessitate substantial changes to its core content. After careful consideration, I believe it would be more appropriate to revise the paper and submit it to a future conference. Consequently, I will not be changing my score or confidence level—though, should the paper ultimately be accepted, I would certainly be fine with that.

Reviewer_tPTtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SCRIPT, a historical document restoration framework that reformulates character reconstruction as the prediction of missing Scalable Vector Graphics (SVG) tokens rather than direct pixel generation. The method combines a damaged raster image, a damage mask, and observed SVG paths to predict missing character geometry, while preserving existing paths. A subsequent SDXL-based inpainting stage restores the surrounding document background. The authors also introduce VECODEX, a large-scale dataset comprising 2,231,419 source character image–SVG pairs and 35,702,704 synthetic restoration instances across four damage types. The dataset is constructed using a memory-guided multi-agent quality-control pipeline and Boolean shape splitting to align raster damage regions with missing vector geometry. Experiments on VECODEX report substantial improvements over four raster-based restoration baselines. SCRIPT achieves 88.20% Character IoU, 80.13% Mask IoU, and 79.87% Mask B-F1. Ablation studies further suggest that raster evidence improves SVG prediction, particularly under simulated water stains. Overall, the paper presents an interesting representation-based approach and a potentially valuable dataset. However, important concerns remain regarding the availability of SVG inputs at inference time, the fairness of the experimental comparison, and the generalization of the findings beyond oracle-conditioned synthetic restoration.

Critical strengths

S1. A meaningful structural formulation for character restoration. The paper addresses a relevant limitation of pixel-based restoration: accurately recovering damaged character strokes without modifying preserved structures. The proposed formulation uses explicit SVG geometry and predicts missing paths rather than regenerating the entire character. The Boolean shape splitting procedure in Section 3.2 provides a well-defined mechanism for partitioning character geometry into observed and missing regions. This makes the supervision signal geometrically consistent within the constructed benchmark. The formulation represents a meaningful methodological direction by introducing explicit character geometry into an otherwise predominantly raster-based restoration pipeline. S2. A large-scale, carefully structured benchmark with encouraging results. VECODEX contains over 2.2 million source character samples and approximately 35.7 million restoration instances. Four degradation families and four damage ratios provide a reasonably broad controlled evaluation setting. Section 3.3 specifies that all synthetic variants of the same source sample remain in the same split, avoiding direct leakage between different damage realizations of an identical source character. Table 2 reports improvements across multiple structural and perceptual metrics. Table 3 also provides evidence that raster observations benefit SVG prediction, particularly for simulated water-stain degradation. These results suggest significant potential under the evaluated input assumptions.

Critical weaknesses

W1. The observed SVG input is constructed using pristine source geometry, creating a fundamental evaluation and inference gap. This is my primary concern. In Section 3.2, Equations (1)–(3), the authors obtain a complete SVG representation of the source character and partition its filled geometry using the synthetic damage region. Specifically, the observed geometry is defined as G_obs = G \ B, where G denotes the complete source character geometry and B is the damage region. This observed SVG is then used as an input to SCRIPT (Section 4.1, Equation (6)). Therefore, the evaluated system receives precise surviving vector geometry derived from the complete source character, which would not ordinarily be available for an unknown damaged document. First, the paper does not establish a practical inference procedure for obtaining such SVG paths from genuinely damaged documents. The dataset construction pipeline vectorizes the complete source before synthetic degradation; it does not establish that equally accurate surviving geometry can be extracted from already damaged characters, particularly under stains, overlapping strokes, fragmented contours, and severe background contamination. Second, the baseline comparison is not information-equivalent. Section 5.1 explicitly states that LaMa, ZITS, and SDXL receive the damaged image and mask, whereas SCRIPT additionally receives observed SVG paths. DiffHDR receives a different printed-character prior. Consequently, the substantial gains in Table 2 may partly reflect high-quality additional geometric information rather than the superiority of the missing-SVG-token predictor. The SVG-only ablation in Table 3(a) does not resolve this concern because it also receives source-derived observed SVG paths. The authors should demonstrate how observed SVG paths are obtained from damaged images without pristine source geometry, evaluate with automatically extracted inputs, and introduce appropriate information-matched baselines. Without these experiments, the results primarily establish performance on an oracle-conditioned SVG completion benchmark rather than a deployable historical document restoration system. This substantially weakens the central claim. W2. Synthetic evaluation and dataset quality analysis do not adequately establish real-world restoration performance. Sections 3.2, 3.3, and 5.1 describe restoration instances produced by applying four simulated degradation families at selected ratios to source character image–SVG pairs. This controlled construction is useful, but the main experiments do not demonstrate performance on an independently collected set of genuinely damaged historical documents. Real historical material may exhibit interacting degradation mechanisms, uncertain damage boundaries, bleed-through, background contamination, missing strokes, and ambiguous characters that differ from the simulated replacements or fading. The dataset quality assessment is also insufficiently detailed for a central data contribution. Section 3.3 reports that 96.13% of audit images assigned a pass decision require no additional enhancement before vectorization. This statistic does not measure the fidelity of generated SVGs against verified character geometry. No independently annotated SVG fidelity scores or detailed error breakdowns for missing strokes and background contamination are provided. Although the manuscript states that variants of a source sample remain in the same split, it is unclear whether samples from the same manuscript, document, or closely related writing style are grouped together. Sample-level independence does not necessarily establish document-level independence. Biases in extracted source SVGs may also affect both training targets and character-evaluation references. An independent assessment of vectorization quality, clarification of the split protocol, and real-world or external benchmark evaluation are needed.

Questions for authors

Q1. How are observed SVG paths obtained from genuinely damaged documents at inference time? The observed SVG in Section 3.2 is built from a complete source SVG. Does SCRIPT require a prior vector representation of the undamaged character? If not, please explain the complete procedure for extracting V_obs from damaged raster input alone, and report performance with automatically extracted rather than oracle observed SVG geometry. This directly affects the practical applicability of the method. Q2. How much of the reported improvement is due to privileged geometric information rather than SVG token prediction? Table 2 compares methods receiving different forms of information. Could the authors provide information-matched comparisons, for example supplying raster baselines with a rasterization of the observed SVG or comparing against vector completion baselines operating on the same partial geometry? A suitably adapted partial-shape reconstruction baseline, such as a method related to SketchRevive (2026), would also clarify the independent contribution of the proposed predictor. Q3. What evidence supports generalization beyond synthetic damage and automatically generated SVG references? Please clarify whether train/test splits are independent at the manuscript or document level, and quantitatively assess the geometric fidelity of automatically produced SVG references against independently checked annotations. Can the complete system be evaluated on real damaged historical documents, including extraction of SVG geometry and masks, rather than receiving those inputs from the synthetic construction process? Q4. Can the authors better isolate the contribution of the background restoration procedure? Table 3(b) reports an improvement in PSNRM from 10.56 dB to 17.16 dB for vector-first inpainting. Please specify how masks, protected regions, and final compositing differ between configurations. Is the difference due to operation order, protection masks, or changes in what SDXL sees? An ablation isolating these factors would strengthen the interpretation.

4 comments
AuthorsOpenReview ↗

Response to Reviewer tPTt (1/3)

We thank the reviewer for the careful and constructive review. Additional experiments show that SCRIPT does not require an undamaged reference at inference. Using observed SVGs extracted from damaged images and masks, it still achieves 79.56% Mask IoU and 79.13% Mask B-F1 on 1,600 test instances.

✅ Please find rebuttal figures HERE.

...the evaluated system receives precise surviving vector geometry derived from the complete source character...

Q1. How are observed SVG paths obtained from genuinely damaged documents at inference time? ...Does SCRIPT require a prior vector representation of the undamaged character?

No. At inference, SCRIPT takes only the damaged image I~\tilde{I} and the damage mask MM, exactly as the inpainting baselines do; it never sees the undamaged character. The complete source SVG in Sec. 3.2 is used only to build training targets, i.e. the missing paths SCRIPT learns to predict, just as clean images supervise an inpainting model such as LaMa.

How VobsV_{\text{obs}} is obtained at inference. It is extracted from the damaged raster alone:

  1. Route selection. The multi-agent pipeline of Sec. 3.1 analyzes I~\tilde{I} and decides whether HAT×4 super-resolution is needed. The accept/reject gate, which only curates source data, is skipped.
  2. Extraction. The character foreground is extracted and cleaned as in Sec. 3.1.
  3. Masking. The foreground inside MM is cleared, so that faded strokes under water stains count as missing, not as observed.
  4. Vectorization. The remaining foreground is vectorized with Potrace.

What this costs. The submission used VobsV_{\text{obs}} cut from the source SVG. We re-ran SCRIPT with extracted VobsV_{\text{obs}} on 1,600 test instances (400 per damage type), without retraining.

Table R1. Oracle vs. extracted VobsV_{\text{obs}}.

Observed SVGMask IoUMask B-F1
Oracle (cut from the source SVG, as in the submission)81.23%80.76%
Extracted from I~\tilde{I}79.56%79.13%

Mask IoU drops by only 1.67% and Mask B-F1 by 1.63%: the reported performance does not rely on privileged geometry. The extracted paths closely match the oracle ones, and are smoother along the mask boundary, because Potrace fits the cut where the oracle follows pixel steps (Fig. R1, HERE). Q3 shows the same procedure on real damaged manuscripts.

Sec. 3.2 and Sec. 4 of the revision will state how VobsV_{\text{obs}} is obtained at test time and report both settings.

AuthorsOpenReview ↗

Response to Reviewer tPTt (2/3)

...the baseline comparison is not information-equivalent...

Q2. How much of the reported improvement is due to privileged geometric information rather than SVG token prediction? ...A suitably adapted partial-shape reconstruction baseline, such as a method related to SketchRevive (2026), would also clarify the independent contribution of the proposed predictor.

Very little. Privileged geometry accounts for 1.67% Mask IoU, whereas SCRIPT leads the strongest baseline by 17.81% when both receive identical information. An advantage could come from three sources: privileged geometry, the evaluation protocol and the training data. We controlled for each.

(1) Privileged geometry. Replacing the source-derived VobsV_{\text{obs}} of the submission with one extracted from I~\tilde{I} changes Mask IoU by only 1.67% (Table R1).

(2) Evaluation protocol and training data. We matched both. First, baseline outputs were binarized with a fixed threshold (L < 128), while SCRIPT's vector output was compared with vector references; all outputs now pass through the same extraction and Potrace before scoring. Second, the baselines used public checkpoints while SCRIPT was trained on VECodex; we fine-tuned LaMa, the strongest baseline, on the VECodex training split. LaMa followed by Potrace forms a pixel-completion-then-vectorization pipeline, similar to SketchRevive [1].

Table R2. LaMa under the same evaluation and after fine-tuning (full test set, 3,570,272 instances, setting of Table 2).

MethodMask IoUMask B-F1
LaMa, submission evaluation (L < 128)56.39%41.51%
LaMa, same Potrace evaluation58.10%50.23%
LaMa fine-tuned on VECodex, same Potrace evaluation60.51%58.77%
SCRIPT80.13%79.87%

The matched evaluation and fine-tuning raise LaMa from 56.39% to 60.51% Mask IoU, and SCRIPT still leads by 19.62% Mask IoU and 21.10% Mask B-F1.

(3) Same information, different predictors. Existing HDR methods are raster-only and none accepts partial vector geometry, so we built information-matched comparisons from the two closest families, on the 1,600 instances of Table R1:

  • Pixel completion: fine-tuned LaMa. Since VobsV_{\text{obs}} is now computed from I~\tilde{I} and MM, LaMa, which receives I~\tilde{I} and MM, has exactly the same information as SCRIPT. For the same reason, rasterizing VobsV_{\text{obs}} for raster baselines would add nothing beyond I~\tilde{I}.
  • Vector completion: Qwen3.8-Max, a much larger MLLM from the same family as our backbone, given I~\tilde{I}, MM and VobsV_{\text{obs}} as SVG path data, exactly like SCRIPT, in fast and thinking modes (prompt HERE).

Table R3. Same information for all methods (1,600 instances, extracted VobsV_{\text{obs}}).

MethodMask IoUMask B-F1
LaMa fine-tuned on VECodex61.75%58.98%
Qwen3.8-Max, fast, 1-shot25.65%17.04%
Qwen3.8-Max, thinking, 1-shot34.86%18.42%
SCRIPT79.56%79.13%

With identical inputs, SCRIPT leads fine-tuned LaMa by 17.81% and Qwen3.8-Max (thinking) by 44.70% Mask IoU. Qwen3.8-Max always returns valid SVG paths but places them inaccurately (Fig. R4, HERE), consistent with the observation in VecGlypher (Huang et al., 2026) that general-purpose LLMs struggle to draw glyphs. The remaining gap therefore measures the contribution of the proposed predictor, which is trained on aligned pixel–vector data and reads the damaged raster and the observed paths jointly.

The revised Table 2 will use the matched evaluation, and Tables R1–R3 will be added.

References

[1] Ran Zuo et al. SketchRevive: Fine-Grained Pixel-to-Vector Sketch Completion with Diffusion-Prior-Guided Multimodal LLMs. CVPR, 2026.

AuthorsOpenReview ↗

Response to Reviewer tPTt (3/3)

Synthetic evaluation and dataset quality analysis do not adequately establish real-world restoration performance.

Q3. ...clarify whether train/test splits are independent at the manuscript or document level, and quantitatively assess the geometric fidelity of automatically produced SVG references... Can the complete system be evaluated on real damaged historical documents, including extraction of SVG geometry and masks...?

Yes. We evaluated the complete SCRIPT pipeline on real damaged historical manuscripts and ensured manuscript-level independence between training and test sets. The results demonstrate SCRIPT’s effectiveness in restoring damaged characters in real-world scenarios.

Real damaged manuscripts. We restored characters from Pelliot chinois 3381 (Qinfu yin, Bibliothèque nationale de France), a manuscript outside VECodex, where naturally occurring stains cover parts of the strokes. Masks are estimated automatically and corrected by hand, and the observed SVG is extracted from the scan as in Q1; nothing comes from the synthetic pipeline. SCRIPT completes the covered strokes and leaves the surviving ones unchanged (Fig. R2, HERE).

Split. The 8:1:1 split is made at the manuscript level over 432 Dunhuang manuscripts, so no manuscript appears in more than one split.

Quantitative Evaluation of SVG Fidelity. On 1,000 randomly sampled source images, we compare the rasterized automatic SVGs with the character foregrounds in the original raster images, obtaining a mean IoU of 98.76%. Two experts in Dunhuang manuscripts also reviewed the SVG references and restored characters and judged them faithful to the original strokes and helpful for restoration.

Q4. Can the authors better isolate the contribution of the background restoration procedure? ...Is the difference due to operation order, protection masks, or changes in what SDXL sees?

The improvement in PSNR$_M$ primarily comes from protecting the restored strokes (+4.64 dB), with additional gains from providing them as contextual guidance to SDXL (+1.96 dB). To clarify Sec. 5.3: both settings in Table 3(b) paste the restored character first; they differ in the mask given to SDXL:

SettingSDXL maskProtected regionFinal image
DirectMMnoneSDXL output inside MM
Vector-first (ours)Mbg=M⊙(1−T^)M_{\text{bg}} = M \odot (1-\hat{T})restored strokes T^\hat{T}SDXL output inside MbgM_{\text{bg}}, restored strokes kept

In the Direct setting, the restored strokes are masked out of SDXL’s input and are not protected during inpainting. Setting B isolates stroke preservation from contextual guidance by adding the restored strokes after inpainting. The strokes are thus retained in the final image without being provided as context to SDXL.

Table R4. Isolating the two factors (full test set).

SettingStrokes protectedSDXL sees strokesPSNR$_M$LPIPS
A. Direct: paste, then inpaint MMnono10.560.24
B. Inpaint MM, then paste the strokesyesno15.200.19
C. Vector-first (ours): paste, then inpaint MbgM_{\text{bg}}yesyes17.160.18

Protecting the strokes accounts for +4.64 dB PSNR$_M$ (A → B), and letting SDXL see them adds another +1.96 dB (B → C); LPIPS improves from 0.24 to 0.19 to 0.18. Without the strokes as context, SDXL has to fill the whole damaged region on its own and produces implausible paper texture. The order matters only through this second factor, since strokes pasted after inpainting cannot be seen by SDXL. Sec. 5.3 of the revision will describe the three settings and report Table R4.

We hope these results resolve the reviewer's concerns, in particular the primary one (W1): SCRIPT works from the damaged image and the mask alone, at a cost of 1.67% Mask IoU. If so, we would be grateful if the reviewer would consider raising the score.

Dear Reviewer tPTt,

I hope this message finds you well. With less than 22 hours remaining in the discussion period, we wanted to check whether our response has addressed your concerns. If you have any further questions or feedback, please let us know. We value your perspective and would be happy to clarify any remaining points.

Thank you for taking the time to review our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.