acceptodds
Desk rejected at ICLR 2027

REAL: Repairability-Aware Latent Alignment for Low-Resource Languages

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Multilingual language models remain less accurate in low-resource languages. Translation-based approaches change the input language, and fine-tuning updates model weights; activation steering preserves both but can also damage correct answers. We introduce REAL, an inference-time method that proposes cross-lingual latent interventions and selects among their executed outcomes using predicted repair and damage. REAL improves low-resource answers without changing the backbone, translating the test question, or querying paired high-resource states. On all 14,042 official MMMLU examples in each of four Yoruba/Swahili model–language settings, REAL improves Qwen3-8B and Llama-3.1-8B-Instruct by 6.65–11.12 percentage points. Across 56,168 predictions, accuracy rises by 9.04 points while preserving 98.56% of initially correct answers. With the same candidate library, verification adds 2.01 points over base-only selection and 1.51 over confidence gating. Permuting exact residual correspondence retains the gain under separately calibrated field-specific verification, whereas low-resource-only residuals trail by 4.53 points. On a full-test, automatically translated Tibetan extension, REAL gains 3.55 and 2.51 points across the two models. On AfriQA free generation, the library contains 3.38–8.06 F1 points of preserving-oracle headroom across four models, although the tested selectors do not deliver reliable gains. Together, these results establish repair-aware selection as an effective way to turn cross-lingual interventions into accuracy gains for low-resource choice scoring.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multilingual language models, low-resource languages, cross-lingual activation steering, representation learning, selective intervention

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_be8aOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes REAL, a method to improve multilingual MCQA performance using inference time scaling. The method works in 2 steps. First step is to generate multiple candidates. Here the authors propose adding perturbation to the hidden states of the question tokens. The perturbation vectors are generated offline using high and low resource non-test questions. This setup is used to generate three candidate answers to each question.

For selection, the authors train classifiers that essentially select the best answer from the pool of candidates. The selection is framed as predicting repair vs damage: REAL switches to the best perturbed answer only if predicted repair minus λ times predicted damage passes a threshold tuned on validation data, and otherwise keeps the original answer.

The authors' main results are on MMMLU test sets in Yoruba and Swahili with Qwen3-8B and Llama-3.1-8B-Instruct. The results show improvements of up to 11.12 pp. The paper compares against fixed perturbations (which reduce accuracy), a base-feature-only selector (+7.02), a confidence gate (+7.52) and an NLLB translation baseline. Finally the paper reports smaller gains on Tibetan and no reliable gain on free-form AfriQA.

Critical strengths
  • The paper shows rigorous and well-documented evaluation. The paper evaluates on the full official MMMLU test sets and reports paired bootstrap confidence intervals. The authors also carefully keep the bank, classifier and test data disjoint and documented (Table 7). The paper also reports negative results faithfully, for example fixed perturbations reduce accuracy, the AfriQA results, and more. This matters to my assessment because I trust the reported numbers, even where I disagree with how they are interpreted.
Critical weaknesses
  • The gain is not shown to come from the cross-lingual perturbations as several results suggest the perturbations mostly act as noise that changes the answer, while the classifier decides when to switch:
    • Shuffled and covariance-sampled residuals do as well as the matched ones (Table 4).
    • The base-only selector already gets +7.02 of REAL’s +9.04 (Table 3).
    • About 98% of the answer changes happen on questions the base model got wrong, and of those changed answers 36% become correct, close to the 33% you would get by picking one of the other three options at random (computed from Table 3, L089 and Appendix E, L802–803).
    • On free-form AfriQA, where you cannot just switch to another option, the selectors give no reliable gain.

This matters because the title and main claims credit cross-lingual alignment. Without these controls, the gain could come entirely from switching away from low-confidence answers.


  • The paper is hard to read and the method is overly complex.
    • The perturbation alone has many tuned settings (PCA size, removed language directions, cluster and neighbor counts, temperature, layer, scale and cap). On top of that come the three-perturbation library, two classifier stages, λ and the threshold.
    • Key baselines are not defined. The confidence gate is described only as ‘a validation-tuned confidence/margin gate’ (L300).
    • The text relies on terms without giving context about them first, such as preserving oracle (used in the abstract and introduction before Section 3 defines it) and SemSub (never spelled out), and run labels such as P25 and P29 (L727) that appear before or without explanation.
    • Tables 1, 3 and 4 are repeated as Tables 8, 9 and 10.

This matters because it makes the contribution hard to assess.

Questions for authors
  • [Suggestion] Can you run the same classifier on candidates from temperature sampling, and on random perturbations of the same norm? If these match REAL, then it tells us the gains comes from the selection step alone.

  • [Clarification] How are the confidence/margin gate and the base-feature-only selector defined?

  • [Suggestion] I wonder how an off-the-shelf reward model compares with your trained classifier when picking among temperature samples, or among your perturbations.

Reviewer_UtUcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces REAL, an inference-time framework for improving low-resource language performance with a frozen language model. REAL constructs cross-lingual latent interventions, executes a small candidate set, and selects an output using separately predicted repair and damage probabilities, with the original prediction as a fallback. Across four model–language settings on full-test MMMLU, it reports accuracy gains of 6.65–11.12 percentage points. Residual-field controls suggest that exact instance correspondence is unnecessary under field-specific calibration. The demonstrated benefits are primarily limited to multiple-choice scoring; reliable improvements in free generation remain unestablished.

Critical strengths
  1. Consistent empirical gains with meaningful selection controls. REAL improves all four full-test settings across two model families and two languages, achieving a pooled gain of 9.04 percentage points while preserving 98.56% of initially correct predictions. On the same candidate library, it exceeds base-feature-only selection and confidence gating by 2.01 and 1.51 points, respectively, whereas unconditional interventions reduce pooled accuracy (Tables 1 and 3). These results substantiate the value of post-intervention selection and provide the strongest evidence supporting acceptance.

  2. Informative controls that refine the explanation for the gains. Matched, correspondence-permuted, and covariance-sampled cross-lingual residual fields achieve similar gains (+9.91/+10.09/+9.98 points), exceeding low-resource-only residuals (+5.38 points; Table 4). With each field independently calibrated, these results show that exact instance correspondence is unnecessary for the observed system-level improvement. This finding adds scientific value beyond performance gains by identifying reusable cross-lingual residual structure as a more plausible explanation than precise instance-level matching.

Critical weaknesses
  1. The specific contribution of repair/damage modeling is insufficiently isolated. The comparisons establish the usefulness of post-intervention evidence, but do not distinguish the proposed dual-target formulation from a conventional supervised selector using the same evidence. A matched baseline predicting candidate correctness or net accuracy gain, with identical features, supervision, model capacity, and candidate budget, is missing. This limits confidence that the repairability formulation itself provides a substantive methodological advantage, rather than the gains primarily reflecting richer inputs to a learned selector.

  2. Superiority on the accuracy–damage tradeoff remains unestablished. Confidence gating achieves a 7.52-point gain with 100% preservation, whereas REAL achieves 9.04 points with 98.56% preservation: 1,131 additional repairs accompany 282 additional damages (Table 3). This establishes higher net accuracy at a different risk level, but not a superior accuracy–damage frontier. Since protecting initially correct predictions motivates the method, validation-calibrated frontiers or matched-damage comparisons are important for assessing whether its additional complexity improves risk-sensitive decision-making.

Questions for authors

N/A

Reviewer_tYRxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes REAL, an inference-time framework for improving low-resource language performance through cross-lingual latent interventions and repair-aware candidate selection. REAL first constructs a library of activation transports using cross-lingual residual representations. For each input, it executes a subset of candidate interventions and employs separately trained repair and damage predictors to determine whether an intervention should replace the original prediction.

The method is evaluated primarily on MMMLU using Qwen3-8B and Llama-3.1-8B-Instruct across Yoruba and Swahili. REAL achieves accuracy improvements of 6.65–11.12 percentage points, with an average gain of 9.04 points while preserving 98.56% of initially correct predictions. Additional experiments investigate the importance of cross-lingual residual structure, verification strategies, calibration transfer, and generalization to Tibetan and free-form question answering.

Critical strengths
  1. A meaningful approach to selective activation steering.

The paper addresses an important limitation of activation steering: improving representation alignment does not necessarily improve downstream predictions and may damage initially correct answers. Explicitly modeling repair and damage provides a useful perspective on selecting interventions while preserving correct predictions. The separation between candidate generation and post-intervention verification is conceptually well motivated.

  1. Strong empirical improvements with extensive controlled evaluations.

The improvements on MMMLU are substantial and consistent across the four primary model–language settings. The experiments use complete test sets, disjoint calibration and evaluation data, and several controlled comparisons. In particular, REAL outperforms base-feature-only selection and confidence-based gating by 2.01 and 1.51 percentage points, respectively, demonstrating the practical value of post-intervention information. The residual-field ablations and cross-language calibration experiments also provide useful insights into the framework.

Critical weaknesses
  1. The contribution of the repair/damage decomposition is not sufficiently isolated.

Although the experiments demonstrate the benefit of post-intervention verification over base-only selection and confidence gating, they do not clearly establish whether separately modeling repair and damage is necessary for these improvements. In particular, the paper lacks comparisons against simpler supervised selectors that use the same base and post-intervention features to directly predict candidate correctness or expected accuracy improvement.

Consequently, it is difficult to determine whether the observed gains primarily arise from the proposed repair-aware formulation or from incorporating additional post-intervention information into a supervised candidate selector. This distinction matters because the repair/damage decomposition is presented as a central methodological contribution.

  1. Limited evidence of effectiveness beyond multiple-choice answer selection.

The main results are concentrated on multiple-choice MMMLU, where the verifier can exploit a relatively structured answer space and option-level probability features. Although the paper investigates free-form generation on AfriQA, the proposed selectors do not achieve statistically significant improvements, despite substantial headroom demonstrated by the preserving oracle.

This gap raises concerns about whether the verification mechanism generalizes to more realistic generation settings, where correctness is harder to assess from model outputs and hidden-state features. The Tibetan extension provides additional language coverage, but it remains a multiple-choice task based on automatically translated questions. Therefore, the current evidence supports the effectiveness of REAL for multiple-choice scoring more strongly than its broader applicability to low-resource language tasks.

Questions for authors

Q1 (Corresponding to Weakness 1). Could the authors compare REAL against a simpler supervised selector trained to directly predict candidate correctness or the damage-adjusted accuracy improvement, using exactly the same base/post-intervention features, calibration data, and candidate library? Such a comparison would help clarify how much of REAL's improvement comes specifically from separately modeling repair and damage, rather than from supervised post-intervention candidate selection in general.

Q2 (Corresponding to Weakness 2). Given the substantial oracle headroom on AfriQA but the lack of statistically significant gains from the learned selectors, what are the primary factors limiting verification in free-form generation? Is the main bottleneck the quality of post-intervention features, the amount of calibration data, or the formulation of the repair/damage prediction objective? Additional analysis identifying this bottleneck would help clarify the extent to which REAL can generalize beyond multiple-choice answer selection.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.