acceptodds
Under review as a conference paper at ICLR 2027

Every Bit Counts: Resource-Efficient Learning for Endangered Language Translation

Abstract

Despite rapid advances in large language models (LLMs), endangered languages remain severely underserved due to their limited representation in pretraining corpora and the scarcity of standardized linguistic resources. This challenge is particularly acute for machine translation, where existing approaches typically address resource integration, data augmentation, and learning optimization in isolation, lacking a systematic framework that connects them under extreme resource scarcity. To the best of our knowledge, we present the first unified framework that connects these three stages through model confidence as a shared signal, using Manchu translation as a case study. Rather than applying confidence uniformly, we adapt its role to the objective of each stage. First, Dictionary-guided Knowledge Correction (DKC) leverages bilingual lexical resources to correct high-confidence errors inherited from pretraining. Second, Reverse Lexical Mapping Augmentation (RLMA) expands limited supervision by promoting diversity across confidence levels. Third, Confidence-Aware Progressive Selection (CAPS) prioritizes low-confidence examples that remain insufficiently learned and progressively introduces them according to the evolving model state. On Manchu translation, our framework improves BLEU from 16.40 to 26.38. The results demonstrate the complementary roles of confidence in correcting unreliable knowledge, diversifying augmented supervision, and prioritizing unresolved examples, providing a unified approach to exploiting scarce linguistic resources for endangered language translation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
endangered Language translation, Dictionary-guided Knowledge Correction, Reverse Lexical Mapping Augmentation, Confidence-Aware Progressive Selection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tCcZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the data scarcity challenges in the field of endangered machine translation. It uses a fine-grained data augmentation method, along with bilingual dictionary augmentation, lexical augmentation, and confidence-aware example selection. The examples selected by a pre-trained ByT5 model show a positive effect on the translation metrics.

Critical strengths
  1. An easy-to-apply unsupervised data selection method based on the confidence generated by pre-trained models
  2. Illustrate that data augmentation is an effective method for data-scarce languages like Manchu–Mandarin
Critical weaknesses
  1. first the base model is limited to one pre-trained model, which cannot show the generality of this data augmentation method
  2. The endangered languages may not be suitable for this data augmentation method; more languages are needed to support this method.
  3. Currently, LLMs are able to receive large context to do context learning on machine translation. In this paper, the dataset is small enough to be loaded and used to test this current paradigm with SOTA models like GPT-6.1 and DeepSeek-4.1-flash, which lack the capacity to support this data augmentation method and supervised training.
Questions for authors

NA

2 comments
AuthorsOpenReview ↗

We added NLLB and Qwen to form a three-architecture, four-task comparison, extended experiments to Mandarin→Manchu, Yi→Mandarin, and Swahili→English with reported BLEU gains, added LLM comparisons with DeepSeek-4.1-flash and GPT-6 (BLEU 31.43 and 33.94), transparently reported the associated data-contamination risk, and reiterated that this paper focuses on training deployable models under constrained resources.

We thank the reviewer for the suggestions on model generality, language coverage, and LLM comparison. We added two architectures and three translation tasks, and ran an initial LLM comparison that revealed a data-contamination risk, reported transparently below.

Critical Weaknesses Weakness 1: First the base model is limited to one pre-trained model, which cannot show the generality of this data augmentation method.

We agree that using only ByT5 is insufficient. We added NLLB-200-distilled-600M and Qwen2.5-1.5B-Base, forming a three-architecture comparison across Manchu→Mandarin, Mandarin→Manchu, Yi→Mandarin, and Swahili→English. On Manchu→Mandarin, DKC raises BLEU for ByT5, NLLB, and Qwen from 16.00, 15.82, and 0.85 to 18.02, 19.10, and 8.11. Selection preferences also differ by architecture: ByT5 and NLLB favor high-confidence samples, whereas Qwen favors low-confidence samples, revealing model dependence (Sections 4.3.1–4.3.2; Appendices A–C).

Weakness 2: The endangered languages may not be suitable for this data augmentation method; more languages are needed to support this method.

Manchu is itself endangered, with extremely scarce parallel data and no mature tokenization or alignment tools, which is precisely why augmentation under limited resources matters. We added ByT5 Mandarin→Manchu, ByT5 Yi→Mandarin, and NLLB Swahili→English. BLEU rises from 16.00 to 25.40 (+9.40) for ByT5 Manchu→Mandarin, from 14.15 to 18.01 (+3.86) for Mandarin→Manchu, and from 19.59 to 23.22 (+3.63) for Yi→Mandarin. NLLB Swahili→English provides a richer-knowledge control, and selection preferences vary by language. These gains reflect the overall training scheme rather than augmentation alone (Section 4.3; Appendices A–B).

Weakness 3: Currently, LLMs are able to receive large context to do context learning on machine translation. In this paper, the dataset is small enough to be loaded and used to test this current paradigm with SOTA models like GPT-6.1 and DeepSeek-4.1-flash, which lack the capacity to support this data augmentation method and supervised training.

Our work targets trainable models exploiting a dictionary plus few parallel sentences, a different resource assumption from long-context in-context learning. We agree an independent LLM comparison matters: DeepSeek-4.1-flash and GPT-6 achieve BLEU of 31.43 and 33.94. However, we found a data-contamination risk. For "emu aniya de duin forgon bi", the reference is "一年有四季" and DeepSeek-4.1-flash outputs "一年有四季。"; for "mini gebu batu sembi", the reference is "我的名字叫巴图" and it outputs "我的名字叫巴图。". Both match after ignoring final punctuation, including the noun "batu". These results clarify our question: improving deployable models under limited compute and supervision via dictionary adaptation, augmentation, and training scheduling. Future work will add stricter comparisons and contamination detection.

Questions for Authors NA

No separate questions were raised; the comments on generalization, language scope, and LLM comparison are addressed in Weaknesses 1–3.

We thank the reviewer for the broader perspective and candid criticism. As a team new to this direction, we had not fully considered cross-model applicability, cross-lingual validation, or the relationship with frontier LLMs. Your comments led us to extend to three architectures and multiple tasks and to add an initial LLM comparison, helping us rethink the paper's positioning. Constrained by capability, compute, and time, we have not completed all ideal validation, especially the strict long-context comparison, for which we are sorry. Your suggestions have opened new directions we will pursue in future work.

Sincerely, The Author Team

Thanks to the authors, my questions have been answered, and I will keep the score.

Reviewer_9FNFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a unified, confidence-centric framework for extremely low-resource machine translation, demonstrated on Manchu-->Mandarin. The framework comprises three stages: (i) Dictionary-guided Knowledge Correction (DKC) to correct pretrained lexical knowledge using a bilingual dictionary, (ii) Reverse Lexical Mapping Augmentation (RLMA) to generate synchronized bilingual augmentations via reverse dictionary lookup, and (iii) Confidence-Aware Progressive Selection (CAPS) to schedule augmented examples based on evolving model confidence. The empirical results are encouraging, with sensible gains at each stage. The approach is practical and resource-conscious, and the formulation is clear and reproducible in principle.

Critical strengths
  • This paper tackles an important and challenging problem of machine translation for endangered languages under extreme data scarcity.
  • The use of model confidence as a single signal to coordinate three resource-efficient stages is conceptually clean in its end-to-end framing for endangered languages.
  • The framework is designed with realistic constraints where monolingual corpora or multilingual pretraining coverage is scarce or unreliable. The gains are meaningful for this setting (~+10 BLEU from baseline to full system).
Critical weaknesses
  • The experiment is somewhat limited, and the evidence base is narrow. All experiments are confined to a single direction on a small, likely in-domain dataset.
  • Baselines against well-known dictionary-based augmentation methods and minimal semi-supervised techniques are missing.
  • Several crucial design choices (high-confidence selection for DKC, RLMA acceptance thresholds) are insufficiently justified.
Questions for authors
  • Why does DKC prioritize high-confidence dictionary entries instead of low-confidence ones?
  • Could you please report variance (std or 95% CI) over several runs and perform significance testing, especially for the small differences between RLMA vs. MLA and CAPS vs. Random?
  • How robust are the results (e.g., ~+10 BLEU from baseline to full system) to alternative uncertainty measures?
  • Do the improvements hold in the reverse direction (Mandarin-->Manchu) or on an out-of-domain set?
  • Could you please strengthen the empirical validation (e.g., broader baselines, significance, generalization tests) and provide deeper analyses of the confidence choices?
3 comments
AuthorsOpenReview ↗

We have extended the experiments to three architectures (ByT5, NLLB, and Qwen) and four translation tasks, added controlled comparisons of RMP, SLS, TLS, RLMA, and SSR together with three sets of three-seed experiments (differences not significant), and reported gains in reverse translation and cross-lingual settings.

We thank the reviewer. The concerns on narrow evidence, missing baselines, and unjustified design choices have shaped the revision. We respond below.

Critical Weaknesses

Weakness 1: The experiment is somewhat limited, and the evidence base is narrow. All experiments are confined to a single direction on a small, likely in-domain dataset.

We agree and added ByT5 (Mandarin→Manchu, Yi→Mandarin), NLLB (Manchu→Mandarin, Swahili→English), and Qwen (Manchu→Mandarin), covering three architectures and four tasks, plus Raw versus DKC scorer comparisons. Static preferences are conditional: high-confidence favored in ByT5 and NLLB Manchu→Mandarin and ByT5 Yi→Mandarin; low-confidence favored in ByT5 Mandarin→Manchu and NLLB Swahili→English; Qwen differs (Section 4.3.2; Appendices A–C).

Weakness 2: Baselines against well-known dictionary-based augmentation methods and minimal semi-supervised techniques are missing.

We added RMP, SLS, TLS, RLMA, and SSR, compared on H300H_{300} and L300L_{300} under identical settings. No method dominates consistently, and confidence gains vary with model and task, so we treat these as controlled conditions for studying confidence (Section 4.3.2; Appendix A). Classical and semi-supervised baselines remain missing; we restrict our conclusions accordingly and will add them in future work.

Weakness 3: Several crucial design choices (high-confidence selection for DKC, RLMA acceptance thresholds) are insufficiently justified.

We confirm the original DKC description did not match the implementation and apologize. DKC now uses all valid dictionary entries without filtering; comparing Raw and DKC scorers, DKC improves translation but does not consistently guide selection better. The RLMA 10%–30% threshold was empirical and removed; we now build H300H_{300} and L300L_{300} with traceability constraints (paraphrase length difference at most two characters, RapidFuzz Top-30, S≥45S \geq 45) (Sections 3.1–3.3, 4.3.1–4.3.2).

Questions for Authors

Question 1: Why does DKC prioritize high-confidence dictionary entries instead of low-confidence ones?

DKC does not filter entries; it uses all valid bilingual pairs. The corrected description is in Section 3.2. Relatedly, models with weak source-language knowledge favor high-confidence original sentences, whereas ByT5 Mandarin→Manchu, NLLB Swahili→English, and Qwen Manchu→Mandarin favor low-confidence samples. This is model- and task-dependent, and does not justify dictionary-entry filtering, a different supervision stage (Section 4.3.2; Appendix A).

Question 2: Could you please report variance (std or 95% CI) over several runs and perform significance testing, especially for the small differences between RLMA vs. MLA and CAPS vs. Random?

Three multi-seed experiments (ByT5 RLMA; NLLB RLMA and SLS; seeds 42, 43, 44) are reported in Table 4 and Appendix C. On H300H_{300}, ByT5 RLMA gives 23.33±0.8623.33 \pm 0.86 (CDC) and 23.92±0.8423.92 \pm 0.84 (CAC); NLLB RLMA gives 21.24±0.6921.24 \pm 0.69 and 21.37±0.2521.37 \pm 0.25; paired t-tests show no significance (p>0.05p > 0.05). Direct tests for the 0.07 and 0.46 gains remain incomplete due to compute and time limits, so we do not treat them as decisive.

Question 3: How robust are the results (e.g., ~+10 BLEU from baseline to full system) to alternative uncertainty measures?

We use average conditional log-probability. Tests with predictive entropy remain incomplete due to compute and time limits, but we expanded validation to three architectures and four tasks, compared scorers before and after DKC, and ran multi-seed experiments for some PAS configurations. Static selection shows conditional preferences, while progressive ordering has no stable optimal direction.

Question 4: Do the improvements hold in the reverse direction (Mandarin-->Manchu) or on an out-of-domain set?

Yes for the reverse direction: Manchu→Mandarin rises from 16.00 to 25.40 (+9.40) and Mandarin→Manchu from 14.15 to 18.01 (+3.86), each with the best PAS configuration. A reliable out-of-domain set is hard to build given scarce Manchu data, so we tested transferability: ByT5 Yi→Mandarin rises from 19.59 to 23.22 (+3.63), with NLLB and Qwen further supporting applicability.

Question 5: Could you please strengthen the empirical validation (e.g., broader baselines, significance, generalization tests) and provide deeper analyses of the confidence choices?

We extended to three architectures and four tasks. Static augmentation compared H300H_{300} and L300L_{300} with scorers before and after DKC and with SSR, RMP, SLS, TLS, and RLMA as controls, revealing conditional preferences that dictionary adaptation may shift. Progressive training with three-seed repeats shows PAS gains in most conditions but no stably optimal ordering, indicating that static selection value and dynamic ordering value are not equivalent. Remaining gaps under experience, compute, and time constraints will be addressed in future work.

Sincerely, The Author Team

Thank you for your efforts and response. I am still concerned about its novelty and overall technical contributions. I would like to maintain my initial assesment of "2: Weak rejection."

AuthorsOpenReview ↗

Clarification of the Framework and Empirical Contributions

We sincerely thank you for taking the time to reconsider our submission and for your candid assessment. We fully respect your assessment and acknowledge that the novelty and overall technical contributions of our work could have been articulated more clearly.

We would nevertheless like to clarify that our contributions lie in two complementary aspects. First, we propose a simple, effective, and resource-efficient framework for extremely low-resource machine translation that integrates bilingual dictionary supervision, lexical augmentation, and progressive training without relying on extensive parallel corpora or specialized source-language processing tools. Although these components are not individually claimed to be entirely novel, their systematic integration provides a practical approach to translation under severe resource constraints.

Second, our extended experiments investigate whether a pretrained model's self-generated confidence can reliably guide data utilization under extremely limited supervision. We find that confidence-based preferences in static augmentation depend on the model and language conditions, that dictionary adaptation does not necessarily improve the utility of confidence-based selection, and that confidence ordering does not consistently benefit progressive training. These findings provide additional empirical insights into the conditions under which confidence-based data utilization is effective.

We recognize that these contributions may not fully address your concerns regarding technical novelty, and we have accordingly clarified our claims and the scope of our conclusions.

Thank you again for your thoughtful feedback. Your comments have been valuable in helping us refine both the presentation and positioning of our work.

Sincerely, The Author Team

Reviewer_CLbBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Manchu-to-Mandarin translation with very limited training data. The pipeline combines Dictionary-guided Knowledge Correction (DKC), Reverse Lexical Mapping Augmentation (RLMA), and Confidence-Aware Progressive Selection (CAPS), all using model confidence as a shared signal. The full method improves BLEU from 16.40 to 26.38.

Critical strengths
  1. The paper addresses an important problem. Manchu has very limited translation resources.
  2. The method makes good use of bilingual dictionary resources and does not require Manchu tokenization or word alignment.
  3. The experiments show clear improvements over the base model. The paper also evaluates the three methods separately.
Critical weaknesses
  1. The benefit of high-confidence selection in DKC is not fully clear. DKC improves translation performance. However, it is unclear whether the improvement comes from dictionary training or high-confidence selection. A comparison with random and low-confidence selection would help clarify this.
  2. RLMA shows very small gains over MLA. In Table 2, RLMA improves BLEU by only 0.07 over MLA. Both methods have the same chrF score. RLMA also performs worse on ROUGE-L. These results provide limited support for the proposed replacement strategy.
  3. The additional benefit of CAPS is relatively small. In Table 3, random progressive selection already improves BLEU from 25.00 to 25.92, while CAPS further increases it to 26.38. This suggests that a substantial portion of the improvement comes from progressive training itself. The paper should report standard deviations and significance tests.
  4. Some methodological details need clarification. The paper does not clearly explain how high-confidence entries are selected in DKC. The 10%–30% confidence difference used in RLMA also needs a clearer definition. These details are important for reproducibility.
Questions for authors
  1. How does DKC compare with random selection and low-confidence selection?
  2. Why does RLMA show only a small improvement over MLA? Have the authors checked the quality of the augmented sentences?
  3. Is the improvement from CAPS over random progressive training statistically significant?
  4. How is the 10%–30% confidence difference calculated?
1 comment
AuthorsOpenReview ↗

We acknowledge that the DKC description did not match the implementation and has been changed to full-dictionary supervision; that RLMA shows no stable advantage and has been repositioned as a low-cost bilingual augmentation method; and that the CAPS gain is not significant and has been redefined as PAS. We have also removed the 10%–30% confidence threshold, adopted the H300 and L300 candidate pools, and added multi-seed experiments.

Weakness 1

The benefit of high-confidence selection in DKC is not fully clear. DKC improves translation performance. However, it is unclear whether the improvement comes from dictionary training or high-confidence selection. A comparison with random and low-confidence selection would help clarify this.

We confirm the original description did not match the implementation and apologize. Since the pretrained model lacks sufficient Manchu knowledge, confidence cannot reliably reflect entry correctness, so filtering is unjustified. DKC now uses all valid dictionary entries without filtering. Comparing Raw and DKC scorers shows DKC improves translation but does not consistently guide later selection better (Sections 3.2, 4.3.1, 4.3.2).

Weakness 2

RLMA shows very small gains over MLA. In Table 2, RLMA improves BLEU by only 0.07 over MLA. Both methods have the same chrF score. RLMA also performs worse on ROUGE-L. These results provide limited support for the proposed replacement strategy.

We agree these results cannot show RLMA is superior. RLMA and MLA did not control what is augmented. We unified the pipeline, split MLA into source-side (SLS) and target-side (TLS) substitution, standardized random masking perturbation (RMP), retained RLMA, and added selected-sample replay (SSR). RLMA does not consistently beat the others, so we position it as a low-cost bilingual augmentation method. Section 4.3.2 compares them on H300H_{300} and L300L_{300} (Section 3.3, Appendix A).

Weakness 3

The additional benefit of CAPS is relatively small. In Table 3, random progressive selection already improves BLEU from 25.00 to 25.92, while CAPS further increases it to 26.38. This suggests that a substantial portion of the improvement comes from progressive training itself. The paper should report standard deviations and significance tests.

We agree that +0.46+0.46 BLEU cannot establish a stable advantage. We extended to ByT5, NLLB, and Qwen on four tasks and repeated ByT5-RLMA, NLLB-RLMA, and NLLB-SLS with seeds 42, 43, 44 across H300H_{300}, L300L_{300}, and U300U_{300}, comparing the confidence-descending curriculum (CDC) with the confidence-ascending curriculum (CAC). Table 4 and Appendices B–C report means and standard deviations. No stable direction emerged, so gains likely come from staged optimization. We redefine CAPS as Progressive Augmentation Scheduling (PAS) (Sections 3.4, 4.3.3).

Weakness 4

Some methodological details need clarification. The paper does not clearly explain how high-confidence entries are selected in DKC. The 10%–30% confidence difference used in RLMA also needs a clearer definition. These details are important for reproducibility.

(1) DKC uses all valid dictionary pairs (see Weakness 1). (2) The 10%–30% range was empirical and removed since no range could be shown optimal. We instead build explicit H300H_{300}/L300L_{300} pools. Retained RLMA conditions: paraphrase length difference at most two characters, RapidFuzz Top-30, and S≥45S \geq 45.

Questions For Authors

Question 1: How does DKC compare with random selection and low-confidence selection?

This was a typographical error, corrected and addressed under Weakness 1.

Question 2: Why does RLMA show only a small improvement over MLA?

The edge was incidental; standardized methods show RLMA instead offers several low-cost augmentation variants at once (Weakness 2).

Question 3: Have the authors checked the quality of the augmented sentences?

Manual inspection showed generated pairs generally preserve meaning under RLMA's dictionary constraints: reverse lookup, string matching, length checks, similarity filtering, ranking, and substitution caps. Example 1: Mini boo tokso I dergi ergi de tehebi / 我家住在村东头 → Mini boo gashan I dergi ergi de tehebi / 我家住在乡村东头. Example 2: minde majige icihiyara baita bi / 有点事情要办 → minde majige icihiyara weile bi / 有点事要办. These ensure traceability but not full semantic consistency; large-scale annotation remains future work.

Question 4: Is the improvement from CAPS over random progressive training statistically significant?

The original gain came from a single run. Repeats with seeds 42, 43, 44 show no stable advantage for CDC or CAC (Table 4; Appendices B–C), unlike the conditional preferences in the static stage. CAPS is therefore redefined as PAS (Section 4.3.3).

Question 5: How is the 10%–30% confidence difference calculated?

Originally, ΔC=Caug−CorigCorig×100%,\Delta C=\frac{C_{\mathrm{aug}}-C_{\mathrm{orig}}}{C_{\mathrm{orig}}}\times 100\%, with CaugC_{\mathrm{aug}} and CorigC_{\mathrm{orig}} the confidences of the augmented and original sentences. Since no interval can be shown optimal, we removed the calculation and threshold, instead ranking original sentences to build H300H_{300} and L300L_{300} (Sections 3.1, 3.3, 4.3.2).

We thank the reviewer again and will incorporate these suggestions in future work.

Sincerely, The Author Team

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.