acceptodds
Under review as a conference paper at ICLR 2027

DocGRPO: Adaptive Adequacy and Fluency Preference Optimization for LLM-Based Document-Level Machine Translation

Abstract

Large language models (LLMs) have shown strong potential for machine translation, yet preserving source information while maintaining coherent discourse remains challenging in document-level machine translation (DocMT). Our preliminary analysis shows that word-alignment-based coverage declines as source documents grow longer, whereas fluency improves and human quality judgments follow a non-monotonic pattern. Coverage alone, however, cannot distinguish genuine omissions from contextually appropriate ellipsis and implicit references, motivating the joint consideration of adequacy and fluency. We propose DocGRPO (ynamic bjective oordination with roup elative olicy ptimization), a reinforcement-learning framework that combines an alignment-based coverage reward for source-content preservation with a discourse-aware fluency reward. To further alleviate the tension between adequacy and fluency, DocGRPO adapts reward weights to training progress and source-document length while combining group-relative and reference-relative advantages to guide policy updates. Experiments across five translation directions and two model scales show improvements over strong document-level baselines. Human evaluation further confirms better overall translation quality, while ablations support the benefits of complementary rewards and adaptive weighting.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Document-Level Machine Translation, Preference Optimization, Reinforcement Learning, Translation Adequacy, Discourse Fluency

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WkpqOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DocGRPO, a reinforcement learning framework designed to optimize document-level machine translation by dynamically balancing source-content adequacy and discourse fluency. The authors identify a non-monotonic trend where coverage-based adequacy and fluency often conflict, particularly as document length increases. To address this, paper introduce a fused reward with weighting mechanism and a dual relative advantage strategy that utilizes both policy-generated candidates and human references as anchors. Experiments demonstrate that DocGRPO outperforms other baselines in several machine translation metrics, with human evaluation confirming higher translation quality and a significant reduction in over- and under-translation errors.

Critical strengths
  • The paper introduces a novel stage- and length-aware adaptive weighting mechanism that effectively balances the inherent tension between adequacy and fluency in document-level machine translation.

  • The proposed reward formulation demonstrates a high correlation with human direct assessment scores, confirming that the model’s optimization objectives are effectively aligned with perceived translation quality and discourse naturalness

Critical weaknesses
  • The manuscript lacks technical details regarding the word alignment process used to compute the coverage score. Please clarify whether off-the-shelf tools (e.g., FastAlign, GIZA++) or other mapping skills were employed.

  • The evaluation is restricted to a single domain (News Commentary), which limits the claims regarding the model's robustness. Providing additional experiments on other evaluation datasets is necessary to demonstrate the generalizability of the proposed framework.

  • The description of the human evaluation process is insufficient. Please specify the number of annotators per sample, the inter-annotator agreement metrics, and the aggregation strategy used to derive final quality scores from multiple human judgments.

  • Utilizing large-scale models like Qwen3.6-27B and mBERT during the training to calcualte reward incurs significant computational and memory overhead. The authors should provide a throughput analysis or an ablation study on the reward model size to justify this design, especially given the potential memory bottlenecks during the training.

  • While the paper combines group-relative and reference-relative advantages, it is unclear how much each component contributes to the overall policy improvement. The authors should provide an ablation study that evaluates the model performance when using only one of these advantages, to justify the necessity of the dual-advantage formulation.

Questions for authors
  • Are there any existing studies that have applied the GRPO framework specifically to machine translation tasks? It would be better to include among baselines.

  • To better establish the efficacy of the proposed rewards, could you compare DocGRPO against baselines optimized directly with automatic metrics like BLEU or COMET? It would be insightful to see if the performance gains are uniquely tied to your dual-reward design.

  • Could you provide an analysis of the training dynamics, such as reward convergence or policy stability throughout the training process? Insights into how the adaptive weighting schedule impacts these dynamics would greatly strengthen the paper's contribution.

5 comments
AuthorsOpenReview ↗

Response to Reviewer Wkpq: Part 1/3

We really appreciate you for your precious time in reviewing our paper and providing helpful comments. We have carefully studied all the comments and tried our best to address every one of them. Below, we address each concern and question point by point:


W1:Clarification of Word Alignment Implementation.

We thank the reviewer for pointing out the missing implementation details. We employ a two-level alignment procedure. Specifically, (1) Segment-level Alignment: We first adopt the dynamic programming algorithm from Align-then-Slide (Guo et al., 2025) to establish source–target segment correspondences, accommodating mismatched sentence boundaries in document-level translation. (2) Word-level Alignment: We then employ the publicly available awesome-align toolkit (Dou and Neubig, 2021) to extract word-level alignment pairs within each aligned segment pair. The resulting alignments are aggregated into the document-level alignment set MiM_i. We have add these implementation details and the corresponding references to Section 3.2 in the revised manuscript.


W2: Cross-domain Generalization.

We thank the reviewer for this valuable comment. To address this concern, we conducted additional experiments on two out-of-domain datasets, TED Talks (spoken presentations) and OpenSubtitles (movie subtitles), using both Qwen3-4B and Qwen3-8B. Importantly, all models were trained exclusively on News Commentary and directly evaluated on these datasets without additional domain-specific fine-tuning:

BackboneDatasetMethodZh→EnDe→En
4BTEDDocMT-SFT23.5/80.8/42.129.4/83.2/47.6
WPO24.7/81.4/43.030.1/83.7/48.6
DeMPT26.3/82.8/45.632.4/84.7/50.8
DocGRPO28.0/84.1/48.333.6/86.0/53.4
4BOpenSubtitlesDocMT-SFT—27.3/81.4/44.8
WPO—28.2/82.5/45.9
DeMPT—30.1/83.4/48.2
DocGRPO—31.3/84.1/49.3
8BTEDDocMT-SFT26.1/83.0/45.032.3/85.6/50.4
WPO27.2/83.5/46.433.4/86.4/51.1
DeMPT29.1/85.2/48.535.0/87.0/53.8
DocGRPO31.0/86.5/51.736.7/88.0/56.6
8BOpenSubtitlesDocMT-SFT—30.2/83.9/47.5
WPO—31.3/84.3/48.8
DeMPT—32.9/85.9/50.6
DocGRPO—34.3/86.6/52.5

Results are reported as d-BLEU / COMET / BlonDe (↑).

The results demonstrate that DocGRPO consistently outperforms all baselines across both out-of-domain datasets and model scales, supporting its effectiveness beyond the News Commentary domain.

More interestingly, DocGRPO achieves substantially larger improvements in discourse-sensitive translation quality on TED than on News Commentary. For example, with Qwen3-8B, the BlonDe improvements over DeMPT increase from +0.9/+0.7 on News Commentary to +3.2/+2.8 on TED for Zh→En and De→En, respectively.

We hypothesize that this stronger benefit is related to the discourse characteristics of spoken-language translation. Compared with formal written news, TED talks and conversational subtitles can involve more context-dependent expressions, implicit references, and ellipsis, where optimizing explicit word-level coverage alone may be insufficient. Our joint coverage–fluency optimization is designed to better accommodate such phenomena.

These findings not only demonstrate the cross-domain transferability of DocGRPO but also provide additional empirical support for the motivation behind coordinating adequacy and discourse fluency.

AuthorsOpenReview ↗

Response to Reviewer Wkpq: Part 2/3

W3:Clarification of Human Evaluation.

We thank the reviewer for highlighting the insufficient description of our human evaluation procedure.

Each translated document was independently evaluated by two bilingual annotators with professional translation experience. Both annotators evaluated all document translations and assigned overall Direct Assessment (DA) scores on a 0–100 scale.

We assessed inter-annotator agreement using ICC(2,1) (two-way random-effects, absolute-agreement, single-measure ICC), obtaining values of 0.89 for Zh→En and 0.92 for De→En, indicating good and excellent agreement, respectively.

For score aggregation, the final DA score of each document was computed as the arithmetic mean of the two independent ratings. System-level DA scores were then obtained by averaging the document-level scores across the evaluation set. For under-translation and over-translation errors, disagreements were resolved through adjudication, and the final error counts were averaged across documents.

We have incorporated these details into Appendix C of the revised manuscript to improve transparency and reproducibility.


W4: Computational Overhead and Flexibility of Fluency Reward Estimation.

We thank the reviewer for highlighting the computational and memory overhead associated with reward estimation. We agree that using a 27B model introduces additional training costs, but would like to emphasize that the 27B scorer is an optional auxiliary component rather than an indispensable part of DocGRPO.

Our fluency reward primarily focuses on sentence-level discourse coherence, estimated by mBERT with a weight of 0.7. The 27B model provides a complementary token-level fluency signal with a smaller weight of 0.3. Given the importance of cross-sentence coherence in DocMT, we intentionally assign greater emphasis to the former.

Importantly, the 27B scorer can be entirely removed when computational efficiency is prioritized, without modifying the core optimization framework. To verify this, we conducted an additional ablation using only the mBERT-based fluency reward:

Fluency RewardZh→EnDe→EnEs→EnAverage
mBERT Only32.1/85.4/50.145.2/88.0/58.851.4/87.7/63.042.9/87.0/57.3
mBERT + 27B (Full)32.4/85.7/51.245.1/88.3/60.151.7/87.9/64.143.1/87.3/58.5

Results are reported as BLEU / COMET / BlonDe (↑).

Removing the 27B scorer results in only modest average decreases of 0.2 BLEU and 0.3 COMET, although BlonDe decreases by 1.2 points. These results demonstrate that the proposed optimization framework remains effective without the large fluency scorer, offering a practical option to trade some discourse-sensitive performance for lower training resource requirements.

Furthermore, both mBERT and the 27B model are frozen reward estimators used exclusively during offline training. Neither is required during inference, meaning that their computational and memory overhead does not affect the deployed translation model.


W5: Contribution of Group-relative and Reference-relative Advantages.

We thank the reviewer for this insightful comment. To address this concern, we conducted additional controlled ablations on Qwen3-4B across three translation directions (Zh→En, De→En, and Es→En), comparing the standard group-relative advantage, its scale-matched variant, and the full dual-advantage formulation:

MethodZh→EnDe→EnEs→EnAverage
Group Only(AiA_i)30.3/82.1/48.842.8/86.8/58.347.1/85.2/61.340.1/84.7/56.1
GRPO (2× Scale, 2Ai2A_i)31.0/82.9/49.543.7/87.2/58.847.9/85.8/62.440.9/85.3/56.9
DocGRPO (Ai+AirA_i + A_i^r)32.4/85.7/51.245.1/88.3/60.151.7/87.9/64.143.1/87.3/58.5

Results are reported as BLEU / COMET / BlonDe (↑).

Compared with Group Only, the full dual-advantage formulation improves average BLEU, COMET, and BlonDe by 3.0, 2.6, and 2.4 points, respectively. Furthermore, even after matching the advantage scale, DocGRPO achieves additional improvements of 2.2 BLEU, 2.0 COMET, and 1.6 BlonDe.

These results indicate that introducing reference-dependent calibration provides empirical benefits beyond simply increasing the magnitude of group-relative advantages. The group-relative component provides within-group discrimination, while the reference-relative component introduces a reference-dependent group offset into policy optimization.

AuthorsOpenReview ↗

Response to Reviewer Wkpq: Part 3/3

Q1&2: GRPO-based MT Methods and Metric-based Reward Baselines.

We thank the reviewer for these valuable suggestions. Recent studies have explored applying GRPO to machine translation, such as MT-R1-Zero (Feng et al., 2025), which employs mixed reward signals for translation optimization, and GRRM (Yang et al., 2026), which investigates group-relative reward modeling for MT.

However, directly optimizing automatic quality metrics may not always align with human-perceived translation quality. For example, MO-GRPO (Ichihara et al., 2026) identifies reward hacking risks in multi-objective GRPO, while GRRM (Yang et al., 2026) highlights the limitations of pointwise quality metrics for group-relative optimization.

Motivated by these concerns, DocGRPO does not directly optimize BLEU or COMET. Instead, it explicitly coordinates source-content coverage and discourse fluency to improve document-level translation quality.

To validate this design, we conducted additional experiments using Qwen3-8B on Zh→En, comparing GRPO variants directly optimized with BLEU and COMET against DocGRPO. Given the potential evaluation bias associated with optimizing and evaluating using the same metric, we additionally report independent human Direct Assessment (DA) scores.

MethodBLEU ↑COMET ↑Human DA ↑
GRPO-BLEU37.886.180.2
GRPO-COMET34.890.581.4
DocGRPO35.289.185.1

As shown above, GRPO-BLEU and GRPO-COMET achieve the highest scores on their respective optimization metrics. However, these improvements do not necessarily translate into better human-perceived translation quality. DocGRPO achieves the highest human DA score (85.1), outperforming GRPO-BLEU and GRPO-COMET by 4.9 and 3.7 points, respectively.

These results reveal a discrepancy between optimizing individual automatic metrics and improving human-perceived translation quality, consistent with concerns about metric overoptimization. They further support our motivation for explicitly coordinating source-content preservation and discourse fluency rather than maximizing a single automatic quality metric.


*Q3: Training Dynamics Analysis

We thank the reviewer for this insightful suggestion. To investigate this process, we analyze two aspects of training dynamics: (1) the evolution of coverage and fluency rewards, (2) policy stability measured by KL divergence and clipping statistics. These analyses help clarify whether the proposed weighting strategy effectively coordinates the two optimization objectives while maintaining stable policy updates. We have included the training dynamics and further discussion in Appendix F of the revised manuscript.


References

  • Feng, Z., et al. (2025). MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning. arXiv:2504.10160.
  • Ichihara, Y., et al. (2026). MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems. Transactions of the Association for Computational Linguistics, 14, 1867–1882.
  • Yang, S., et al. (2026). GRRM: Group Relative Reward Modeling for Machine Translation. Proceedings of EMNLP 2026, 51934–51956.
Reviewer_PvEiOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DocGRPO, a reinforcement learning framework for document-level machine translation that jointly optimizes translation adequacy and fluency. The method introduces coverage- and fluency-based rewards, together with an adaptive weighting strategy that dynamically balances the two objectives during training. It also combines group-relative and reference-relative advantages to guide policy optimization. Experiments on multiple language pairs show consistent improvements over supervised fine-tuning and existing preference optimization baselines, particularly for long-document translation.

Critical strengths
  1. The method is well motivated. Combining coverage and fluency rewards, with weights that vary by training stage and document length, offers an interesting way to balance between preserving source information and producing coherent translations.

  2. Results across five translation directions and two model sizes, together with automatic metrics, human evaluation, and ablations, support the effectiveness of the approach.

Critical weaknesses
  1. The proposed framework mainly combines existing techniques (GRPO, reward engineering, and adaptive weighting) rather than introducing a fundamentally new optimization method. The contribution appears incremental. The core adaptive reward balancing strategy lacks strong theoretical justification, and it is unclear whether the proposed schedule is optimal or generalizes beyond the evaluated settings.

  2. The evaluation only considers X→English translation. The lack of experiments on English→X directions raises concerns about the generalizability of the proposed method across different translation settings.

Questions for authors
  1. The experiments only report results with English as the target language. Additional results with non-English target languages would help demonstrate the generalizability of the approach.

  2. Although the coverage reward is defined mathematically, its implementation is not sufficiently explained. Please clarify how MiM_i is computed in practice.

3 comments
AuthorsOpenReview ↗

Response to Reviewer PvEi

We really appreciate you for your precious time in reviewing our paper and providing helpful comments. We have carefully studied all the comments and tried our best to address every one of them. Below, we address each concern and question point by point:


W1: Novelty and Methodological Contribution.

DocGRPO builds upon established techniques, including GRPO and adaptive reward weighting. However, we would like to clarify that our primary contribution lies in a new optimization perspective for document-level machine translation, rather than introducing a fundamentally new reinforcement learning algorithm or model architecture.

Our key novelty can be summarized as:

  1. A novel optimization perspective for DocMT: We propose dynamically coordinating source coverage and discourse fluency according to their distinct optimization characteristics, rather than treating them as equally prioritized objectives throughout training.

  2. Stage- and length-aware reward weighting: We introduce a curriculum-inspired strategy that progressively shifts optimization from coverage to fluency, with an earlier emphasis on fluency for longer documents.

  3. Reference-relative advantage calibration: We incorporate reference-dependent calibration into GRPO to improve policy optimization beyond conventional group-relative advantage estimation.


W1&2&Q1: Generalizability to English-to-Non-English Translation.

We thank the reviewer for this valuable suggestion. We followed the suggestion and conducted additional experiments on three representative reverse translation directions (En→Zh, En→De, and En→Es) using Qwen3-4B:

MethodEn→ZhEn→DeEn→EsAverage
DocMT-SFT30.2 / 82.534.8 / 84.140.6 / 85.335.2 / 84.0
DeMPT32.1 / 84.036.3 / 85.642.4 / 86.836.9 / 85.5
DocGRPO33.8 / 85.037.7 / 87.043.6 / 87.838.4 / 86.6

Results are reported as BLEU / COMET (↑). BlonDe is not applicable to non-English target languages, as its original implementation relies on English-specific linguistic features.

As shown above, DocGRPO consistently outperforms both DocMT-SFT and DeMPT across all three directions. In particular, compared with DeMPT, DocGRPO achieves average improvements of +1.4 BLEU and +1.1 COMET, demonstrating that its effectiveness extends beyond English-target translation.


Q2:Clarification of Alignment Implementation.

We thank the reviewer for pointing out the missing implementation details. In practice, we compute the coverage reward through a two-level alignment procedure to accommodate the flexible sentence boundaries and non-monotonic sentence segmentation commonly observed in LLM-based document translation.

First, segment-level alignment. We adopt the dynamic programming-based alignment algorithm from Align-then-Slide (Guo et al., 2025) to establish source–target segment correspondences. Specifically, this algorithm identifies an optimal alignment path based on semantic similarity, accommodating mismatched sentence boundaries and many-to-one correspondences.

Second, word-level alignment. For each aligned source–target segment pair, we employ awesome-align (Dou and Neubig, 2021) to extract word-level alignment pairs. These alignments are aggregated into the document-level alignment set MiM_i.

We have incorporated these implementation details into Section 3.2 in the revised manuscript.

References

[1] Guo et al. (2025). Align-then-Slide: A Complete Evaluation Framework for Ultra-Long Document-Level Machine Translation. arXiv:2509.03809.

[2] Dou and Neubig (2021). Word Alignment by Fine-tuning Embeddings on Parallel Corpora. EACL 2021, pp. 2112–2128.

Thanks for the authors' clarifications and additional results to address my concerns. I am happy to raise my score to 3.

Dear Reviewer PvEi,

We sincerely appreciate your positive feedback and your decision to raise your overall score from 2 to 3. We are pleased that our responses have addressed your concerns. Thank you again for your valuable suggestions, recognition of our work, and kind support.

Sincerely,
The Authors

Reviewer_RYjzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DocGRPO, a reinforcement-learning framework for document-level machine translation that jointly optimizes source-content coverage and target-language fluency. Its main components are an alignment-based coverage reward, a fluency reward combining adjacent-sentence coherence and language-model perplexity, a reward-weighting schedule depending on training progress and source length, and a combination of group-relative and human-reference-relative advantages.

Experiments cover five translation directions into English using Qwen3-4B and Qwen3-8B. The paper reports improvements over supervised, preference-optimization, and single-reward baselines on BLEU, COMET, and BlonDe. Additional validation includes reward-weighting ablations, analyses across document lengths, and human evaluation of overall quality and over-/under-translation for Chinese–English and German–English.

Critical strengths
  1. A relevant objective with complementary evaluation The paper addresses the important tension between preserving source information and producing coherent document translations. Evaluating automatic translation quality alongside human judgments and omission/addition errors is valuable because improvements in fluency alone would not establish translation reliability.

  2. Consistent gains under controlled model settings Experiments span two model sizes and five translation directions, with shared backbones and data settings across the reproduced baselines. The single-reward and adaptive-weighting comparisons provide useful evidence that the combined training recipe improves performance beyond either reward alone.

Critical weaknesses
  1. The practical advantage over directly using an LLM remains unclear. The experiments focus on adapted models and do not include a straightforward whole-document prompting baseline using the original backbones or a strong larger LLM. In particular, the 27B model already used for fluency scoring is not evaluated as a direct translator. Consequently, the results establish improvements within the chosen adaptation setting, but leave the quality–cost trade-off motivating the additional training pipeline unresolved. This matters because the gains over the strongest reported baseline are relatively modest, while the method requires multiple sampled translations and additional reward-model computation during training.

  2. The distinctive optimization mechanisms are insufficiently isolated and justified. The reference-relative advantage is a central contribution, but its independent effect is not ablated. Algebraically, the final advantage equals twice the group-relative advantage plus a group-dependent offset, making a scale-matched GRPO control particularly relevant. In addition, the stated weighting schedule reaches a fluency coefficient of one for longer documents: for 1,536 source tokens, this occurs at approximately 51% of training. The explicit coverage reward then disappears. Although the complete method performs well, the paper does not establish why this behavior is appropriate for preserving source information or whether retaining a nonzero coverage weight would work better.

Questions for authors
  1. Do you have results for whole-document prompting with the original Qwen3 backbones and the 27B fluency-scoring model? A comparison including translation quality, inference cost, and training cost would materially clarify the method’s practical advantage.

  2. What improvement comes specifically from adding the human-reference-relative term? Please compare against ordinary GRPO and a scale-matched control using twice the group-relative advantage, while keeping the reward schedule and other settings fixed.

4 comments
AuthorsOpenReview ↗

Response to Reviewer RYjz: Part 1/2

We sincerely appreciate you for the thorough evaluation of our work and the constructive comments and suggestions. Below, we address each concern and question point by point:

W1&Q1: Practical Advantages over Direct LLM Translation and Quality–Cost Trade-offs

To address this concern, we conducted additional experiments with the original Qwen3-4B/8B models and the larger Qwen3.6-27B model, together with an efficiency comparison against our strongest baseline, DeMPT.

(1) Comparison with Direct Whole-Document Prompting.

We evaluated direct whole-document prompting across all five translation directions under the same evaluation settings. The results are summarized below (BLEU / COMET / BlonDe):

ModelZh→EnFr→EnDe→EnEs→EnRu→EnAverage
Qwen3-4B (Prompting)21.1/80.6/41.233.0/79.5/48.239.2/84.3/53.643.1/80.1/55.320.4/77.9/40.831.4/80.5/47.8
Qwen3-8B (Prompting)22.7/81.9/43.935.1/80.1/50.340.9/85.9/55.444.8/81.3/56.423.6/79.4/42.633.4/81.7/49.7
Qwen3.6-27B (Prompting)29.3/83.7/47.340.1/82.3/52.643.1/87.1/57.147.3/83.1/59.330.3/82.5/46.438.0/83.7/52.5
DocGRPO (4B)32.4/85.7/51.243.2/84.3/57.845.1/88.3/60.151.7/87.9/64.132.1/84.1/49.640.9/86.1/56.6
DocGRPO (8B)35.2/89.1/53.845.0/86.8/60.847.6/91.0/64.555.9/91.6/67.435.7/87.2/52.943.9/89.1/59.9

The results show that DocGRPO substantially outperforms direct prompting with the corresponding backbones. More importantly, **DocGRPO-4B even surpasses the much larger 27B model, while using approximately 85% fewer backbone parameters. These findings demonstrate that our adaptation pipeline enables smaller LLMs to achieve competitive or even superior document-level translation quality compared with substantially larger models used directly.

(2) Quality–Cost Trade-off.

DeMPT enhances document-level translation through decoding-enhanced multi-phase prompt tuning, which introduces additional trainable prompts and a multi-stage decoding process to improve contextual modeling. However, this design increases the number of trainable parameters and introduces additional inference overhead.

In contrast, DocGRPO directly optimizes the translation policy through reinforcement learning, requiring only lightweight LoRA adaptation and a single-stage decoding process at inference time. We provide a full comparison with DeMPT in the following Table:

MethodBLEU ↑COMET ↑BlonDe ↑Trainable Params ↓Decoding Stages ↓Inference Time (s/doc) ↓Training Time (h) ↓
DeMPT40.185.256.0~3.0%37.124.9
DocGRPO40.986.156.6~1.0%14.837.3

Compared with DeMPT, DocGRPO achieves consistently better translation quality while reducing trainable parameters by approximately 67% and inference latency by 32.4% (1.48× speedup). Although DocGRPO requires approximately 50% more training time (on 8 ×\times A100 GPU) due to additional candidate sampling and reward computation, this overhead is incurred only during offline training. At inference time, DocGRPO requires neither reward models nor multiple candidate generations, and performs translation through a single decoding stage.

AuthorsOpenReview ↗

Response to Reviewer RYjz: Part 2/2


W2&Q2: Insufficient Ablation and Justification of the Proposed Optimization Mechanisms

To address the concerns, we conducted two additional sets of ablation experiments on Qwen3-4B across three translation directions (Zh→En, De→En, and Es→En).

(1) Independent Contribution of Reference-relative Advantage.

Our dual relative advantage can be reformulated as: A^i=Ai+Air=2Ai+μG−RrefσG+ϵ\hat{A}_i = A_i + A_i^{r} = 2A_i + \frac{\mu_G - R_{\mathrm{ref}}}{\sigma_G + \epsilon}. This decomposition indicates that the reference-relative component introduces both advantage scaling and a reference-dependent group offset. To distinguish their effects, we additionally evaluated GRPO (2× Scale), which doubles the group-relative advantage while excluding the reference-dependent offset:

MethodZh→EnDe→EnEs→EnAverage
Group Only (AiA_i)30.3/82.1/48.842.8/86.8/58.347.1/85.2/61.340.1/84.7/56.1
GRPO (2× Scale, 2Ai2A_i)31.0/82.9/49.543.7/87.2/58.847.9/85.8/62.440.9/85.3/56.9
DocGRPO (Ai+AirA_i+A_i^r)32.4/85.7/51.245.1/88.3/60.151.7/87.9/64.143.1/87.3/58.5

Results are reported as BLEU / COMET / BlonDe.

As shown in the table, doubling the group-relative advantage yields gains of 0.8 BLEU, 0.6 COMET, and 0.8 BlonDe, whereas DocGRPO further improves these metrics by 2.2, 2.0, and 1.6 points, respectively. These results demonstrate that DocGRPO's gains extend beyond advantage scaling, highlighting the benefit of reference-dependent calibration, which adjusts policy optimization without altering within-group candidate rankings.

(2) Rationale and Effectiveness of Adaptive Reward Weighting.

On the one hand, our design is motivated by the observation that discourse fluency optimization becomes increasingly challenging as document length increases, due to more complex cross-sentence dependencies, coreference, and ellipsis. Therefore, we prioritize fluency optimization earlier for longer documents to better develop the model's discourse fluency capabilities, rather than applying a uniform schedule across document lengths.

On the other hand, a zero coverage weight for longer documents does not imply that coverage optimization disappears entirely. Shorter documents continue to receive explicit coverage rewards throughout training, helping maintain the model's source-content preservation capabilities.

To empirically assess this design, we additionally evaluate maximum fluency weights of 0.8 and 0.9, ensuring that at least 20% and 10% of the coverage weight, respectively, remains allocated to coverage across all documents:

SettingZh→EnDe→EnEs→EnAverage
λmax⁡=0.80\lambda_{\max}=0.8032.9/84.9/48.045.7/87.5/58.151.8/87.0/62.243.5/86.5/56.1
λmax⁡=0.90\lambda_{\max}=0.9032.8/85.4/49.245.5/87.6/58.951.8/87.3/63.043.4/86.8/57.0
λmax⁡=1.00\lambda_{\max}=1.00 (Original)32.4/85.7/51.245.1/88.3/60.151.7/87.9/64.143.1/87.3/58.5

Results are reported as BLEU / COMET / BlonDe on Qwen3-4B

The results show that retaining higher coverage weights for longer documents during later training substantially degrades discourse-level translation quality, as reflected by the consistent decline in BlonDe. In contrast, the original setting λmax⁡=1.0\lambda_{\max}=1.0 achieves the highest COMET and BlonDe scores, supporting our design choice to prioritize fluency optimization for longer documents.

AuthorsOpenReview ↗

Gentle Reminder: Discussion Period Approaching Its End

Dear Reviewer RYjz,

Thank you for your thoughtful review and constructive feedback. We have carefully addressed all your concerns and questions in our official responses and incorporated the corresponding revisions into the manuscript.

As the discussion period is approaching its end, we would sincerely appreciate it if you could take a moment to review our responses and let us know whether they have adequately addressed your concerns. We would be happy to provide any further clarification if needed.

Thank you again for your valuable time and consideration.

Sincerely, The Authors

Area_Chair_aNf9OpenReview ↗

Reminder: Discussion deadline approaching

Dear Reviewer,

Thank you for your time and effort in reviewing Submission 23167. As the discussion deadline is approaching, could you please read the authors’ response and any additional evidence, reply in the discussion to indicate whether your concerns have been addressed, and update your evaluation accordingly as soon as possible? Thank you for your contribution to the review process.

Best regards, Area Chair

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Rebuttal and Manuscript Revision

We sincerely appreciate all reviewers for their constructive feedback. We have substantially revised the manuscript, with all major revisions highlighted in blue, and conducted additional experiments to address the reviewers' concerns. The main concerns and corresponding revisions are summarized below:

1. Reviewer RYjz:

  • Direct LLM comparison and efficiency: We added direct whole-document prompting baselines using Qwen3-4B/8B and Qwen3.6-27B, and quantified the quality–cost trade-off against DeMPT, including trainable parameters, inference latency, and training time (Table 1 and Appendix K, Table 10).
  • Reference-relative advantage: We introduced a scale-matched GRPO baseline to isolate the contribution of reference-dependent calibration, demonstrating improvements beyond simple advantage scaling (Section 3.3 and Appendix E, Table 6).
  • Adaptive reward weighting: We conducted additional ablations with capped fluency weights (0.8 and 0.9), validating the benefit of prioritizing discourse fluency for longer documents (Section 3.3 and Appendix L, Table 11).

2. Reviewer PvEi:

  • Contribution clarification: We refined the Introduction to emphasize our primary contribution: dynamically coordinating source coverage and discourse fluency through stage- and length-aware optimization and reference-relative calibration (Section 1).
  • Reverse translation directions: We extended evaluation to En→Zh, En→De, and En→Es, demonstrating effectiveness beyond English-target translation (Appendix G, Table 7).
  • Alignment implementation: We detailed the two-level alignment procedure combining dynamic-programming-based segment alignment and awesome-align for word-level alignment (Section 3.2).

3. Reviewer Wkpq:

  • Cross-domain generalization: We added experiments on TED Talks and OpenSubtitles using both model scales, demonstrating robust improvements without domain-specific fine-tuning (Section 5.1, Table 2).
  • Reward and optimization ablations: We evaluated GRPO-BLEU and GRPO-COMET with human assessment (Appendix D, Table 5), isolated the contribution of reference-relative advantages (Appendix E, Table 6), and examined the effectiveness of removing the 27B fluency scorer (Appendix I, Table 9).
  • Training dynamics: We added analyses of reward evolution, KL divergence, clipping statistics (Appendix J, Figure 9).
  • Evaluation transparency: We clarified human annotation protocols, inter-annotator agreement, and score aggregation, and expanded discourse-sensitive case studies (Appendix C and Appendix H, Table 8).

Overall, we believe the above revisions strengthen the empirical evidence for the effectiveness of our work, clarify the contributions of its core optimization mechanisms, and improve its reproducibility, generalizability, and practical relevance. We would be happy to continue the discussion if there are any further questions or comments.

AuthorsOpenReview ↗

Summary of Discussion

We sincerely appreciate the Area Chair and all reviewers for their time and efforts throughout the review process. We are especially grateful to Reviewer PvEi (2 → 3) and Reviewer Wkpq (3 → 4) for recognizing the value of our work and raising their ratings. We also appreciate Reviewer RYjz's valuable feedback. Although the reviewer has not participated in the discussion, we believe our detailed point-by-point responses and substantial manuscript revisions have thoroughly addressed the concerns raised.

We would like to reiterate three key contributions of our work:

  1. Novel optimization perspective: We identify an inherent tension between source coverage and discourse fluency in DocMT and propose a DocGRPO that dynamically coordinates these complementary objectives through stage- and length-aware reward weighting and reference-relative advantage calibration.
  2. Strong empirical evidence: Extensive experiments and newly added ablations, direct LLM comparisons, cross-domain evaluations, and efficiency analyses consistently validate the effectiveness and practicality of DocGRPO.
  3. Broader impact: Our findings offer new insights into balancing competing objectives in document-level translation and may inspire future research on discourse-aware reinforcement learning.

We sincerely appreciate the reviewers' recognition and thoughtful suggestions again. We hope our comprehensive rebuttal and strengthened manuscript will support a favorable final assessment by the Area Chair.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.