acceptodds
Desk rejected at ICLR 2027

When Transcript Supervision Cannot Locate Characters: Set-Valued Cuts and Sparse Anchors for CTC OCR

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Connectionist temporal classification (CTC) learns transcripts by summing monotone alignments, but its character peaks need not identify physical separators. We formalize adjacent-character segmentation as an ordered, one-dimensional, set-valued cut task: a cut is accepted between the projected right edge of one character and the left edge of the next, including their overlap. We establish a two-world lower bound when identical observable word images and transcripts admit separated valid-cut sets, and derive an anchor-recovery relation under fixed linear features and full-rank design. A recognition-conditioned SVTRv2 system combines CTC decoding, a bounded residual cut head, interval supervision from sparse character boxes, and an optional soft interval-gated CTC objective. On 61,842 source-disjoint SynthText words of length 2–5, 1% cut supervision raises midpoint ±2-frame all-cut word accuracy from 69.52% to 90.32%; dense supervision reaches 95.17%. The corresponding strict transcription-and-all-cuts rates are 64.78%, 84.15%, and 88.76%. Controlled synthetic experiments reproduce the lower-bound and rank-transition predictions, while OCR results quantify the gain from sparse geometric anchors.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
CTC‌, Character Interval, Weakly Supervised Localization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9pEAOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper defines an acceptable cut between adjacent characters as the interval joining their projected inner box edges: $ S_i=[\min(R_i,L_{i+1}),\max(R_i,L_{i+1})]. $

It proves a conditional two-world lower bound when identical image-and-transcript observations admit separated valid-cut sets, followed by a fixed-feature least-squares recovery relation for geometric anchors. An SVTRv2-based CTC recognizer supplies the predicted word length; a bounded-residual head predicts ordered cuts around equally spaced priors. Sparse interval supervision trains this head, with an optional interval-gated CTC objective.

Evaluation uses source-image-disjoint SynthText words of length 2–5. At 1%1\% training-anchor supervision, all-cut word accuracy under a midpoint tolerance of ±2\pm2 feature frames rises from 69.52%69.52\% to 90.32%90.32\%. The paper separately reports recognition, joint transcription-and-cut success, and interval membership.

Critical strengths
  1. The paper separates distinct outcomes. Recognition accuracy, cut accuracy, and joint success are reported separately, and inference uses predicted rather than ground-truth word length (pp. 3 and 6–8).
  2. Source-level splitting is appropriate. Splitting by source image addresses correlations between word crops. The paper also acknowledges geometric labels in pilot/forecast reserves and the need to account for pretrained supervision (pp. 3 and 7).
  3. The negative gate results are informative. The optional gated objective does not consistently improve performance, and the authors explicitly report degradation at the 0.1%0.1\% and 1%1\% budgets rather than claiming a universal benefit (p. 9).
Critical weaknesses
  1. The lower bound assumes the central ambiguity rather than demonstrating it for OCR. Theorem 1 assumes identical observable distributions with separated valid sets and obtains a half-distance bound through the triangle inequality. This is a valid conditional argument, but the paper does not establish a realistic glyph-generation or annotation mechanism producing the required separation. Synthetic worlds constructed to satisfy the assumption do not establish its prevalence in actual OCR (pp. 4–5).
  2. The recovery theorem has limited novelty and limited connection to the model. Under fixed features, full-rank design, and independent zero-mean noise, the expression $ \operatorname{MSE}_T=\sigma^2\operatorname{tr}(\Sigma_TG_A^{-1}) $ is the standard least-squares prediction-variance calculation. The subsequent margin bound is also direct. The learned OCR representation, bounded residuals, sorting, and interval-valued supervision do not satisfy this model without additional argument (pp. 5–6).
  3. The headline metric does not directly validate the proposed geometric task. At 1%1\% anchors, per-cut midpoint-tolerance accuracy is 92.80%92.80\%, but interval membership is 57.28%57.28\%; with dense supervision, the corresponding values are 95.58%95.58\% and 72.05%72.05\%. The broader ±2\pm2-frame tolerance does not certify valid-set membership or uncontaminated character crops (p. 8, Table 3).
  4. The evaluation lacks breadth and decisive baselines. It uses one seed and only short SynthText words. There is no real-image or long-word evaluation and no adapted comparison with the cited CRAFT, WordSup, TextScanner, or SIGA approaches. Equally spaced cuts, CTC-derived alignments, and simple anchor-trained regressors are also important missing controls (pp. 3 and 7–9).
  5. Some promised diagnostics and supervision details are missing. The cap oracle is discussed without numerical results. The 4,012 pilot/forecast words have geometric annotations, but their actual influence on model or hyperparameter selection needs clarification. The precise cut construction in the zero-anchor baseline should also be stated.
  6. Qualitative evidence is weak. Archived cut coordinates without the source word images cannot demonstrate whether predicted boundaries agree with visible glyph structure. The editorial remnants further reduce the manuscript's readiness, although they do not establish AI authorship or research misconduct.
Questions for authors
  1. Can you give a realistic image-generation or annotation model in which identical image-and-transcript observations induce separated valid-cut sets? What evidence suggests this ambiguity matters in the evaluated data?
  2. What are the interval-based per-cut, all-cut, and joint transcription-and-all-cuts results, with source-level uncertainty? How are wrong predicted lengths handled in every metric, including the Interval column?
  3. How does the method compare with equally spaced cuts, CTC-derived alignment cuts, a simple anchor-trained regressor, and adapted character-localization methods under matched supervision?
  4. What is the cap-oracle performance, and how much of the remaining error comes from limited reachable cuts versus learning failure? Can you add real-image, long-word, and multi-seed evaluations?
  5. Exactly how were the pilot/forecast geometric labels and any pretrained character-box supervision used? Can you report the total geometry-label budget affecting training and model selection, and provide original-image qualitative examples?
Reviewer_5QD5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper formulates adjacent-character segmentation in CTC OCR as an ordered, one-dimensional, set-valued cut task. For adjacent projected character boxes [L_i, R_i] and [L_{i+1}, R_{i+1}], the accepted cut set is S_i = [min(R_i, L_{i+1}), max(R_i, L_{i+1})], which treats gaps and overlaps under one convention. Two theorems are proved in the main text: (i) a two-world lower bound — for annotation worlds that induce the same law of the observations Z, inf_B max_w E_w d(B, S_w) >= (1/2) E_Q d(S_0, S_1), with the boundary case noted that intersecting sets give a zero bound, so the result does not transfer to an interval-hit metric; and (ii) a fixed-design linear recovery identity MSE_T = sigma^2 tr(Sigma_T G_A^{-1}), where G_A is the Gram matrix of purchased anchors. The model augments an SVTRv2 + RCTC recognizer with a bounded tanh residual cut head over K-1 priors p_i = T*i/K induced by the CTC-decoded length, trained with an interval Huber loss on sparse character boxes and an optional soft interval-gated CTC term (training-time regularizer only; all arms decode raw logits at test time). On 61,842 source-disjoint SynthText words of length 2-5 with 150,197 internal cuts, 1% cut supervision raises GT-midpoint +/-2-frame all-cut word accuracy from 69.52% to 90.32% and strict end-to-end success from 64.78% to 84.15%; dense supervision reaches 95.17% and 88.76%. Interval membership is much lower (57.28% and 72.05%; median valid interval 0.98 frames). Two synthetic experiments reproduce the theorem predictions numerically (lower bound 1.375 predicted vs 1.384 observed; MSE 0.0539 vs 0.0586 at m=32).

Critical strengths
  1. The theory is stated conditionally and its boundary is self-declared, which is rare and makes the contribution verifiable. Theorem 1 is explicitly conditional on observational equivalence and does not claim that every real glyph boundary is invisible; Theorem 2 is explicitly a fixed-design linear statement that "does not assume that a nonlinear OCR head automatically obeys it"; the prior-residual parameterization is declared a modeling choice rather than a consequence of the theorems. Each formula is then checked inside its stated regime: two-world lower bound 1.375 predicted vs 1.384 observed (h=0) and 0.0627 vs 0.0710 (h=2); linear recovery 0.0539 vs 0.0586 at m=32 and 0.0104 vs 0.0102 at m=128. This is a genuinely checkable theoretical contribution rather than a decorative one.
  2. The evaluation protocol separates observables that are usually conflated, and the sparse-anchor effect is measured precisely under it. Splits are source-image disjoint with a sealed hidden set; predicted cut counts come from CTC decoding rather than ground-truth length, so a wrong predicted length fails the whole-word event; cut, whole-word, transcription, joint, and interval outcomes are reported in separate columns; and the annotation ledger separates pretraining boxes, localization labels, and calibration data, with the budget percentages explicitly defined over the current anchor pool only. Under this protocol the reported effect is unambiguous: +20.80 points of all-cut word accuracy for 1% of training cuts, with recognition accuracy essentially unchanged (89.92% -> 89.61%).
Critical weaknesses
  1. All OCR experiments are on synthetic data and short words, so the external validity of the central empirical claim is unestablished. The corpus is a 30,000-source SynthText subset restricted to reference lengths 2-5, where character quadrilaterals are exact by construction and rectification is well conditioned. But the paper's framing is about what CTC transcript supervision cannot identify in general, and about applications that need character crops or editable word images. No real scene-text dataset (Total-Text, COCO-Text, ICDAR) is evaluated. Because the difficulty of the task is largely created by the synthetic rendering pipeline, neither the 69.52% baseline nor the +20.80 gain can be assumed to transfer; this single limitation is what prevents me from treating the empirical contribution as established.
  2. There is no baseline of any kind for the cut task, and only one seed is available. Section 2.2 itself states that adapting CRAFT, SIGA, TextScanner, or WordSup to the same split and metric "is necessary for direct comparison" and that a fair detector baseline must convert two-dimensional output into one cut per neighbouring predicted character, but no such baseline appears in Sections 6 or 7. Every reported gain is therefore relative to the same head without geometric supervision, and the practically decisive question — whether buying 1% character boxes beats an off-the-shelf weakly supervised character detector — is left unanswered. In addition, the archive has a single seed (17), so no variance is available for any difference in Table 3, and the gate ablation is inconsistent across budgets (−1.04, +0.28, −0.99 end-to-end points at 0.1%, 0.5%, 1%), which the authors appropriately decline to interpret as a recognition gain.
Questions for authors

Clarifications (answers could change my assessment):

  1. Dense per-cut accuracy is 95.58% at +/-2 frames but interval membership is only 72.05%, and the median valid interval is 0.98 frames. Why is the +/-2-frame point metric the primary comparison rather than interval membership? For the stated application (glyph-pure crops), is 57.28% interval membership at 1% anchors actually usable, and what value would it need to reach?
  2. Please report multiple seeds, and an ablation that removes the learned residual so only the uniform priors T*i/K remain. The zero-anchor arm already achieves 35.39% interval hits, which suggests the prior carries a substantial share of the localization; separating prior from learned correction is needed to attribute the gain to the purchased anchors.
  3. The paper requires that the provenance of any pretrained recognizer be disclosed before making a zero-character-supervision claim. Did the SVTRv2 initialization use character-level boxes at any stage of its pretraining, and what does the 0% row therefore mean?
  4. Anchors are drawn at random rather than by the conditioning criterion of Theorem 2. Do the OCR results change if the budget is selected by that criterion, and is a rank transition observable as in the synthetic experiment?

Suggestions for future work (not affecting my assessment): Real scene-text evaluation with detector baselines converted to ordered cuts, and multi-seed reporting; active anchor selection driven by the smallest eigenvalue of G_A; a full-width cut detector to close the reported gap between the dense arm and the cap oracle.

Reviewer_3XTDOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper separates CTC transcription from character localization. A valid cut is the interval between the inner edges of two adjacent boxes. Theorem 1 is a two-world set-distance lower bound when the image and transcript are shared but the valid sets are disjoint. Theorem 2 is fixed-design linear regression for purchased anchors. Neither is claimed to govern the network. The model is SVTRv2 with K−1 uniform priors, a bounded residual, and an interval loss on sparse SynthText boxes. An optional gate does not help.

Critical strengths
  1. The central idea is right. A CTC model can read the word correctly and still not know where the characters are, because the loss sums over alignments instead of marking a separator. Treating localization as a question that the transcript does not answer is a good way to pose the problem.

  2. The supervision idea fits that gap. A cut is an acceptable interval, not one annotator midpoint, and only a few of those intervals need to be labeled. That is a sensible way to add the missing geometry without boxing every character.

Critical weaknesses
  1. The theory does not show that transcript supervision cannot locate characters. Theorem 1 assumes two indistinguishable worlds and applies the triangle inequality. Theorem 2 assumes fixed full-rank features, which the authors say the OCR head does not satisfy. Table 1 only checks those formulas.

  2. The experiment does not measure the defined task. The abstract reports ±2-frame all-cut accuracy, 69.52% to 90.32%, while the median valid interval is 0.98 frames and per-cut interval membership at 1% is 57.28%. All-cut accuracy under the interval rule is not reported. The study is one seed on synthetic words of length 2–5, without CRAFT, WordSup, TextScanner, or SIGA. It is also unclear whether the 0% cuts are CTC peaks or uniform priors, and whether SVTRv2 already saw character boxes.

  3. Authoring notes are left in the paper and should not be there. Figure 1 says “All labels and shapes remain editable in Word.” Figure 2 says “Editable Word-native flowchart.” Figure 3 says “Native editable Office line chart.” Figure 5 says “Editable comparison.” These are production instructions, not scientific captions.

Questions for authors
  1. Are the 0% cuts CTC alignment peaks, or uniform priors with an untrained residual?
  2. What are all-cut and end-to-end accuracy under interval membership at each budget?
  3. Which character boxes, if any, were used before the anchor budgets in Table 3?
  4. On this hidden split, how do CRAFT, WordSup, TextScanner, or SIGA score as K−1 ordered cuts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.