acceptodds
Under review as a conference paper at ICLR 2027

Morphology-Consistent Preference Optimization for Indic Language Models

11 views

Abstract

Preference optimization has become the standard practice for aligning large language models, yet every dominant objective scores whole responses or raw subword tokens. The mismatch is acute in morphologically rich Indic languages, where one grammatical unit can decide rejection while the rest of the response stays correct. Case suffixes, agreement and tense-aspect-mood markers, honorific forms, and script-sensitive vowel signs all carry that weight. Subword tokens do not fill that role either as one token may conflate a stem with a suffix, and one morpheme may split across tokens. A token-level objective therefore spreads its signal past the unit that decided the preference. We introduce MorphDPO, which moves the unit of alignment from a flat token sequence to a latent morpheme lattice. The resulting reward replaces the sequence-level Direct Preference Optimization (DPO) log-ratio with a normalized morpheme-lattice reward. The new reward is tokenizer-agnostic, marginalizes over uncertain analyses, and reduces exactly to DPO under uniform weights. The underlying tilted-policy log-ratio keeps morphology weighting a valid change of policy geometry rather than an unnormalized token reweighting. Furthermore, we build three components, namely counterfactual partial-order preferences from rule-generated minimal pairs, a paradigm-neighborhood objective enforcing consistency across valid inflections, and a group-distributionally-robust objective over language-script-morphology groups. Across eight Indic languages on the Indic-specialized Sarvam-1 2B, MorphDPO attains the best composite accuracy () and win-rate () among preference objectives, with the highest morphology minimal-pair accuracy and a group gap tied with DPO for the smallest. The same lead holds on Llama-3.2 3B and Qwen3 1.7B, proving that morphology rather than the token is the right unit of preference alignment for morphologically rich languages.

Primary area:
optimization
Keywords:
Morphology, Indic Language, Preference Optimization, Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_gGHAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This research proposes a preference optimization framework that places greater emphasis on specific grammatical units in order to improve overall win-rate on Indic language tasks. This framework, MorphDPO, creates candidate morphological analyses that are mapped to a model’s existing tokens with character-span overlap creating response score averages over them. It also combines a locally normalized reward, auto-generated contrasts, and some other objectives. The evaluation section fine-tunes three small model backbones across eight languages measuring response score rank against synthetic perturbations. The authors show their proposed method achieves high accuracy on their synthetic tasks and slightly improve on DPO.

Critical strengths

S1. The initial motivation for the task is interesting. It makes sense that some languages may be more susceptible to smaller changes than others in terms of overall error or acceptability of a good response. Trying to map these specific language features onto existing tokenizers provides a way of adding additional linguistic information into preference optimization.

S2. In terms of range of experiments the authors seem headed in the right direction, showing comparisons that ablate across languages and model backbones. But there are concerns about the training setup addressed in the weaknesses section and additional comparison between Morph-heavy non non-morph-heavy languages could help with W1.

Critical weaknesses

W1. For the same reason the motivation is strong, the choice of intervention is not. The authors discuss MorphTok, which makes intuitive sense to me as the first place to start, and beyond that it may be helpful to ground inexisting research on the structure of languages (e.g. WALS study) and not just focus on Indic languages. The comparisons provided insert morphological boundaries while retaining the existing tokenizer and embeddings which provides limited evidence for the advantage of changing the preference objective over changing the representation. More grounding in linguistic research would also help explain which morphological properties motivate the selection of these Indic languages. They have different grammatical structures, and it would be useful to understand which distinctions existing tokenizers and preference objectives miss, and why MorphDPO should address them. This would make the proposed mechanism clearer and help establish where its benefits might generalize beyond the evaluated languages.

W2. It’s difficult to separate the benefit of the new proposed objective from the training setup in this work. It seems like baselines are not being trained on the same datasets according to the appendix. “baselines that do not use counterfactual data train on the 44,160 reference pairs (2,760 steps), while MorphDPO trains on reference plus counterfactual pairs (101,538 examples), which accounts for its larger step count.” It would be a much cleaner comparison to ensure all optimizers run on the exact same data and the rest of the training environment is fixed.

Questions for authors
  1. Which conclusions about the tilted-policy normalization and paradigm consistency are supported by the experiments that were run?
  2. Can the authors describe the analyzer and checks used to validate the analyses and generated preferences - in particular if any human verification occurred according to a predetermined protocol?
2 comments

Thank you for the clear and thoughtful review. We have run new experiments and added the linguistic grounding you asked for. We hope this addresses your concerns, and that you will consider raising your score.

W1. Linguistic grounding, and objective versus representation

Typology: To address this, the revision adds a typology table from WALS features 26A (prefixing vs suffixing), 30A (genders), 49A (cases), 81A (word order) and 102A (verbal person marking), and links each feature to a corruption type. In brief:

  • All eight languages are verb-final and strongly suffixing. The errors we target therefore sit in word-final material, often a single vowel sign.
  • The languages differ in which suffixes carry agreement. Hindi, Marathi and Gujarati verbs agree in gender and number. Tamil, Telugu and Kannada verbs agree in person, number and gender. Bengali has no grammatical gender, and Malayalam verbs carry no subject agreement.

Tokenizers trained mostly on other languages split these suffix stacks at arbitrary points. A sequence-level reward then spreads the signal from one wrong suffix across the whole response, and MorphDPO is designed to concentrate it again. The typology also predicts where the method should not help: for example, there are no gender pairs for Bengali. We will state these predictions so the claims about generalisation are concrete.

Objective vs representation: We agree that MorphTok-DPO keeps the original vocabulary, so it tests re-segmentation rather than a new representation. We see the two approaches as complementary, not competing. We will frame our claim as "a morphology-aware objective on a fixed tokenizer", and list vocabulary extension combined with MorphDPO as a natural next step.

W2. Baselines on the same data

To address this, we trained every variant on Sarvam-1 at full scale with a fixed environment. CF denotes our morphology-aware counterfactual pairs, and DPO+CF is DPO trained on exactly MorphDPO's data and steps (6,346):

Sarvam-1DataCompositeWorst-Group
DPOreference only0.93940.7660
DPO+CFref + CF0.96240.8511
MorphDPO, submittedref + CF0.94130.7660
MorphDPO, exact log Zref + CF0.95660.8298

Our morphology-aware counterfactual pairs are a key part of the framework, and the objective is designed to exploit them. With exact log Z, MorphDPO is level with same-data DPO on composite (−0.006 [−0.015, +0.006]) and slightly ahead on substitution pairs (0.9797 vs 0.9788). DPO+CF will be added to the main table for all three backbones.

Q1. Which conclusions about log Z and paradigm consistency are supported?

Log Z: In answer to your question: the full-scale run with exact log Z (table in W2) improves composite by 1.5 points and worst-group by 6.4 points over the submitted configuration. The overall win-rate gain is significant (McNemar p < 0.001). The 500-example subset ablations, now with three seeds, were too small to show this reliably, so the full-scale result is the stronger evidence.

Paradigm consistency: The paradigm-consistency term is part of the general formulation, and the reported configuration uses λ_par = 0. We will state this clearly in the abstract and present the term as a principled extension, with its empirical study left to future work.

Q2. Analyzer and checks

Analyzer: In answer to your question: each whitespace word is looked up in the UniMorph table for its language and split into stem and feature-bearing suffix (score 1.0). A rule-based suffix split gives an alternative analysis (score 0.5). The posterior is the normalised scores, which are fixed trust levels rather than calibrated probabilities. UniMorph covers Hindi, Bengali, Gujarati, Kannada and Telugu, and suffix rules cover Hindi, Tamil and Bengali. We will add a per-language coverage table.

Automatic checks: The target rule must fire, and the corrupted text must pass a length check. Pairs carry a reliability weight of 0.8 (0.7 for word order) to reflect possible noise.

Thank you again. We hope these results and clarifications address your concerns, and we would be grateful if you could raise your score.

Thanks for responding. I won't be changing my score, but I appreciate the iteration here and think with some further work you'll have a solid contribution to make.

  1. I like the idea of a new table linking corruption types to WALS features
  2. Baselines on the same data show DPO+CF gives the best improvement, so I think the main contribution is really the counterfactual data here.
  3. Response to Q2 was describing automated checks but not checks from experienced language speakers in each language who could provide additional audit.

I think 2/3 outweigh 1 overall.

Reviewer_gsxcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MorphDPO for preference optimization in morphologically rich Indic languages. It combines morphology-based weighting, synthetic preference pairs, and group-level robustness objectives. Experiments evaluate preference ranking across eight languages and three language-model backbones.

Critical strengths
  1. Improving grammatical correctness in underrepresented languages is a meaningful research problem.

  2. The evaluation covers multiple languages and backbones and reports results by error type and disadvantaged group, rather than only average performance.

Critical weaknesses
  1. The specific need for morphology-aware optimization is not established. The motivation is plausible, but the paper does not show why standard DPO trained on the same morphological preference pairs would be insufficient. Additional data, length normalization, and group weighting also change simultaneously. A same-data DPO baseline and a uniform-weight version of MorphDPO are needed to isolate the benefit of linguistic structure.

  2. The main experiments do not evaluate several central contributions. All full-scale runs disable the tilted-policy normalization term, and assign zero weight to paradigm consistency. The ablations also fail to establish consistent benefits from the normalization correction and analysis marginalization. Consequently, the main results cannot substantiate the paper’s emphasis on these components.

  3. The reliability of component-level conclusions remains limited. Several ablation gains fall within the reported run-to-run variation. Although paired significance tests are provided, they do not measure training variability. Multiple independent training runs would strengthen the conclusions, particularly given the small composite-accuracy gain over DPO on Sarvam-1.

Questions for authors
  1. Which morphological analyzer is used, and how are candidate analyses and their probabilities obtained?

  2. Do the improvements persist with faithful baselines, identical training data and budgets, and multiple training seeds?

1 comment

Thank you for pointing out exactly which controls were missing. We have now run them, and we hope the results below address your concerns and that you will consider raising your score.

W1. Is morphology-aware optimisation needed?

To address this, we ran both controls you asked for, plus the full-objective run, all on Sarvam-1 at full scale. CF denotes our morphology-aware counterfactual pairs, and DPO+CF is DPO trained on exactly MorphDPO's data and steps (6,346):

Sarvam-1DataCompositeWorst-Group
DPOreference only0.93940.7660
DPO+CFref + CF0.96240.8511
MorphDPO, uniform weightsref + CF0.94730.7872
MorphDPO, submittedref + CF0.94130.7660
MorphDPO, exact log Zref + CF0.95660.8298

We read three things from this.

  1. Our morphology-aware counterfactual pairs are a key contribution of the framework and bring large gains. The objective is designed around these minimal pairs, and the two work best together.
  2. The tilted-policy correction brings MorphDPO level with same-data DPO. The composite difference is −0.006 [−0.015, +0.006], and on substitution pairs MorphDPO is slightly ahead (0.9797 vs 0.9788).
  3. Uniform weights perform about the same as morphology weights. This is consistent with our analyzer's limited coverage (see Q1): most words currently get weight 1. We see better analyzers as the clearest way to strengthen this component.

We will add this full ladder to the main paper and revise the contribution statement to match.

W2. Central components are not evaluated at full scale

To address this, we trained MorphDPO on Sarvam-1 at full scale with the exact log Z correction on and nothing else changed. Composite rises from 0.9413 to 0.9566, worst-group from 0.7660 to 0.8298, and the gap falls from 0.234 to 0.170. The overall win-rate gain is significant (McNemar p < 0.001), so the tilted-policy correction does help at full scale. The effect holds when the adapter is re-scored with fixed functions instead of its own reward: composite stays between 0.945 and 0.958 under all three rules. The paradigm-consistency term is part of the general formulation, and the reported configuration uses λ_par = 0. We will make this explicit and present the term as a principled extension for future work.

W3. Ablations and training variance

We agree, and we have rerun the subset ablations with three seeds (42–44). At this 500-example scale, the component differences are mostly within seed variation. The clearest trend is reward normalisation, which improves worst-group by 5.7 points on average (in 2 of 3 seeds). We see the full-scale run in W2 as the more informative test. For the full test set, MorphDPO beats DPO on discordant pairs 117 vs 83 (McNemar p = 0.019). We will report McNemar separately from training variance, as you suggest, and add three full-scale seeds for DPO, DPO+CF and MorphDPO.

Q1. Which analyzer is used, and how are analyses and probabilities obtained?

To answer your question: we split each response on whitespace and look up each word in the UniMorph inflection table for its language. A hit gives a stem/suffix split with the entry's features (score 1.0). A hand-written suffix list gives an alternative split (score 0.5), kept only when it differs from the first. The posterior q(a|y) is the normalised scores: (2/3, 1/3), or a point mass when the two splits agree. These are fixed trust levels, not calibrated probabilities. UniMorph tables cover Hindi, Bengali, Gujarati, Kannada and Telugu, and suffix rules cover Hindi, Tamil and Bengali. On test responses, the UniMorph hit rate ranges from 0.1% (Hindi, Telugu) to 8% (Bengali), and the rule-based splits reach 21% of Bengali and 7% of Tamil words. We will add this description and the coverage table to the paper.

Q2. Do the improvements persist with faithful baselines, identical data and seeds?

On identical data, MorphDPO with exact log Z matches DPO+CF on composite and is slightly better on substitution pairs (W1). Seeds are covered in W3. We will relabel the incomplete TDPO and ORPO baselines as variants and add the faithful objectives in the camera-ready.

Thank you again. We hope these new results address your concerns, and we would be grateful if you could raise your score.

Reviewer_f9uEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MorphDPO, a morphology-aware preference optimization method for Indic language models. It constructs a latent morpheme lattice using a morphological analyzer, maps morphological features to token weights through character-span overlap, and marginalizes over candidate analyses. The formulation introduces a normalized tilted-policy log-ratio and reduces to DPO under specific conditions. It also incorporates counterfactual partial-order preferences, a paradigm-neighborhood objective, and group-DRO over language-script-morphology groups. The full-scale configuration uses a morphology-weighted, length-normalized reward with the log Z correction and paradigm-neighborhood objective disabled. Experiments on eight Indic languages and three backbones with at most 3B parameters report the highest composite accuracy and overall win-rate among the compared methods on the constructed test set, with worst-group win-rate tied with DPO on Sarvam-1 and higher than the baselines on Llama-3.2 and Qwen3.

Critical strengths
  1. The research question is clear and practically significant. It focuses on the impact of localized morphological errors on response quality in Indic languages, highlighting the granularity mismatch between subword tokens and morphemes.
  2. The method design is intuitive. It uses character-span overlap to construct a soft token-morpheme alignment and marginalizes over candidate analyses, enabling morphology-aware preference scoring without changing the tokenizer.
Critical weaknesses
  1. The full-scale configuration disables the log Z correction and the paradigm-neighborhood objective, using a morphology-weighted, length-normalized reward without the tilted-policy correction. The ablation benefit of log Z falls within the run-to-run spread and does not replicate on Llama-3.2 or Qwen3, leaving the empirical benefits of these theoretical components unverified. Please report full-scale results with these components enabled or qualify the corresponding claims.

  2. The fairness of the comparisons and the validity of the evaluation need further justification. MorphDPO uses additional rule-generated counterfactual pairs, covering the same corruption types as the test set, and approximately 2.3 times the training examples and optimization steps of baselines without counterfactual data. The shared learning rate and β were selected in a search that included MorphDPO-specific parameters, and some baselines omit key components, such as the sequential-KL term in TDPO and the SFT term in ORPO. Please provide comparisons with the same data and budget, complete baseline objectives, and comparable hyperparameter tuning. Evaluation ranks pairs using each method’s own reward, without separating training improvements from scoring changes or assessing generation quality. Approximately 82 percent of the test pairs involve word-order perturbations. The remaining corruptions use regular-expression substitutions without human verification, and some rules are reused across languages with different scripts. Please include evaluation under a common scoring function, human validation of the corrupted pairs, and complementary generation evaluation. The overall win-rate gain over DPO on Sarvam-1 is only 0.28 percentage points, and results across multiple training seeds are not reported. Please report sample counts for each language-script-morphology group and results across multiple seeds, particularly for worst-group win-rate. Experiments cover only three backbones with at most 3B parameters, so stronger model comparisons or narrower claims are needed.

Questions for authors

Please see the Weaknesses part.

1 comment

Thank you for a careful and constructive review. We ran the experiments you asked for, and we think they make the paper clearly stronger. We hope the answers below address your concerns and that you will consider raising your score.

W1. Log Z correction and paradigm term at full scale

To address this concern, we trained MorphDPO on Sarvam-1 at full scale with the exact log Z correction on and nothing else changed:

Sarvam-1CompositeMin-PairWorst-GroupGap ↓
MorphDPO, submitted (log Z off)0.94130.98050.76600.2340
MorphDPO, exact log Z0.95660.98360.82980.1702

The correction adds 1.5 composite points and 6.4 worst-group points. The overall win-rate gain is significant (82 vs 44 discordant pairs, McNemar p < 0.001). So the theoretical component does help at full scale, which the 500-example subset ablations were too small to show. The paradigm-consistency term is part of the general formulation, and the reported configuration uses λ_par = 0. We will make this explicit and present the term as a principled extension for future work.

W2. Fairness and validity of the comparisons

Same data and budget: To answer this directly, we trained DPO on exactly MorphDPO's data (reference + counterfactual pairs, 6,346 steps), which we call DPO+CF. It reaches 0.9624 composite on Sarvam-1. With exact log Z, MorphDPO's composite difference to it is within the bootstrap interval (−0.006 [−0.015, +0.006]), and its substitution-pair accuracy is slightly higher (0.9797 vs 0.9788). This also highlights that our morphology-aware counterfactual pairs are an integral part of the method: the objective is designed to exploit these minimal pairs, and the two work best together. We will present data generation and objective as one framework, and add DPO+CF to Table 1 for all three backbones.

Tuning and baseline objectives: DPO+CF performs strongly with the shared learning rate and β, so the shared setting does not seem to put baselines at a disadvantage. We will still add per-method tuning. We will relabel TDPO and ORPO as "uniform-token TDPO" and "ORPO without SFT", and add the faithful objectives in the camera-ready.

Common scoring: We re-scored the new adapters with three fixed rules: DPO log-ratio, length-normalised log-ratio and mean policy log-prob. Exact-log-Z MorphDPO scores 0.945–0.958 composite under all three, so its gains do not come from its own reward function. We have also set up a generation-based evaluation on 742 held-out prompts (chrF, script fidelity, attested-inflection rate), which will be included in the revision.

Test composition: You are right that 9,813 of the 11,978 pairs are reference or word-order pairs. On the 2,165 substitution pairs alone, where morphology matters most, MorphDPO beats DPO by 2.7 points (0.9737 vs 0.9469). Substitution rules fire only for Hindi, Marathi, Bengali, Tamil and Telugu, and we will report coverage per language. Margin, seeds and group sizes: Against DPO, MorphDPO is right on 117 pairs where DPO is wrong, versus 83 the other way (McNemar p = 0.019). There are 30 groups (language × corruption type), with 11 to 2,115 pairs each and a median of 151. The worst group, Tamil number agreement, has 47 pairs, so one pair moves its win-rate by 2.1 points. We will add the full group table. The subset ablations now have three seeds, and we will add three full-scale seeds for DPO, DPO+CF and MorphDPO.

Scale: We will scope our claims to LoRA preference tuning of 2–3B models on eight Indic languages.

About your questions

Your questions point to the weaknesses, so we hope the answers above cover them. In short: exact log Z helps at full scale, and with it MorphDPO is on par with same-data DPO and stronger on substitution pairs. We are adding common scoring, substitution-only results and seeds.

Thank you again. If these results address your concerns, we would be grateful if you could raise your score.

Comments on the paper

1 comment

Dear Reviewers,

Thank you for the review. The authors have responded to the points you raised, and the response period closes on 18 November (AoE). I would be grateful if you could post a brief reply on whether your concerns are resolved, partly resolved, or still open, so that the authors have a fair chance to address them. If anything still needs clarifying, please ask now while they can still respond.

Regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.