- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This research proposes a preference optimization framework that places greater emphasis on specific grammatical units in order to improve overall win-rate on Indic language tasks. This framework, MorphDPO, creates candidate morphological analyses that are mapped to a model’s existing tokens with character-span overlap creating response score averages over them. It also combines a locally normalized reward, auto-generated contrasts, and some other objectives. The evaluation section fine-tunes three small model backbones across eight languages measuring response score rank against synthetic perturbations. The authors show their proposed method achieves high accuracy on their synthetic tasks and slightly improve on DPO.
Critical strengths
S1. The initial motivation for the task is interesting. It makes sense that some languages may be more susceptible to smaller changes than others in terms of overall error or acceptability of a good response. Trying to map these specific language features onto existing tokenizers provides a way of adding additional linguistic information into preference optimization.
S2. In terms of range of experiments the authors seem headed in the right direction, showing comparisons that ablate across languages and model backbones. But there are concerns about the training setup addressed in the weaknesses section and additional comparison between Morph-heavy non non-morph-heavy languages could help with W1.
Critical weaknesses
W1. For the same reason the motivation is strong, the choice of intervention is not. The authors discuss MorphTok, which makes intuitive sense to me as the first place to start, and beyond that it may be helpful to ground inexisting research on the structure of languages (e.g. WALS study) and not just focus on Indic languages. The comparisons provided insert morphological boundaries while retaining the existing tokenizer and embeddings which provides limited evidence for the advantage of changing the preference objective over changing the representation. More grounding in linguistic research would also help explain which morphological properties motivate the selection of these Indic languages. They have different grammatical structures, and it would be useful to understand which distinctions existing tokenizers and preference objectives miss, and why MorphDPO should address them. This would make the proposed mechanism clearer and help establish where its benefits might generalize beyond the evaluated languages.
W2. It’s difficult to separate the benefit of the new proposed objective from the training setup in this work. It seems like baselines are not being trained on the same datasets according to the appendix. “baselines that do not use counterfactual data train on the 44,160 reference pairs (2,760 steps), while MorphDPO trains on reference plus counterfactual pairs (101,538 examples), which accounts for its larger step count.” It would be a much cleaner comparison to ensure all optimizers run on the exact same data and the rest of the training environment is fixed.
Questions for authors
- Which conclusions about the tilted-policy normalization and paradigm consistency are supported by the experiments that were run?
- Can the authors describe the analyzer and checks used to validate the analyses and generated preferences - in particular if any human verification occurred according to a predetermined protocol?
2 comments
Thanks for responding. I won't be changing my score, but I appreciate the iteration here and think with some further work you'll have a solid contribution to make.
- I like the idea of a new table linking corruption types to WALS features
- Baselines on the same data show DPO+CF gives the best improvement, so I think the main contribution is really the counterfactual data here.
- Response to Q2 was describing automated checks but not checks from experienced language speakers in each language who could provide additional audit.
I think 2/3 outweigh 1 overall.
Thank you for the clear and thoughtful review. We have run new experiments and added the linguistic grounding you asked for. We hope this addresses your concerns, and that you will consider raising your score.
W1. Linguistic grounding, and objective versus representation
Typology: To address this, the revision adds a typology table from WALS features 26A (prefixing vs suffixing), 30A (genders), 49A (cases), 81A (word order) and 102A (verbal person marking), and links each feature to a corruption type. In brief:
Tokenizers trained mostly on other languages split these suffix stacks at arbitrary points. A sequence-level reward then spreads the signal from one wrong suffix across the whole response, and MorphDPO is designed to concentrate it again. The typology also predicts where the method should not help: for example, there are no gender pairs for Bengali. We will state these predictions so the claims about generalisation are concrete.
Objective vs representation: We agree that MorphTok-DPO keeps the original vocabulary, so it tests re-segmentation rather than a new representation. We see the two approaches as complementary, not competing. We will frame our claim as "a morphology-aware objective on a fixed tokenizer", and list vocabulary extension combined with MorphDPO as a natural next step.
W2. Baselines on the same data
To address this, we trained every variant on Sarvam-1 at full scale with a fixed environment. CF denotes our morphology-aware counterfactual pairs, and DPO+CF is DPO trained on exactly MorphDPO's data and steps (6,346):
Our morphology-aware counterfactual pairs are a key part of the framework, and the objective is designed to exploit them. With exact log Z, MorphDPO is level with same-data DPO on composite (−0.006 [−0.015, +0.006]) and slightly ahead on substitution pairs (0.9797 vs 0.9788). DPO+CF will be added to the main table for all three backbones.
Q1. Which conclusions about log Z and paradigm consistency are supported?
Log Z: In answer to your question: the full-scale run with exact log Z (table in W2) improves composite by 1.5 points and worst-group by 6.4 points over the submitted configuration. The overall win-rate gain is significant (McNemar p < 0.001). The 500-example subset ablations, now with three seeds, were too small to show this reliably, so the full-scale result is the stronger evidence.
Paradigm consistency: The paradigm-consistency term is part of the general formulation, and the reported configuration uses λ_par = 0. We will state this clearly in the abstract and present the term as a principled extension, with its empirical study left to future work.
Q2. Analyzer and checks
Analyzer: In answer to your question: each whitespace word is looked up in the UniMorph table for its language and split into stem and feature-bearing suffix (score 1.0). A rule-based suffix split gives an alternative analysis (score 0.5). The posterior is the normalised scores, which are fixed trust levels rather than calibrated probabilities. UniMorph covers Hindi, Bengali, Gujarati, Kannada and Telugu, and suffix rules cover Hindi, Tamil and Bengali. We will add a per-language coverage table.
Automatic checks: The target rule must fire, and the corrupted text must pass a length check. Pairs carry a reliability weight of 0.8 (0.7 for word order) to reflect possible noise.
Thank you again. We hope these results and clarifications address your concerns, and we would be grateful if you could raise your score.