acceptodds
Under review as a conference paper at ICLR 2027

MAPLE: Multilingual Alignment of Predictive Latent Embeddings

Abstract

Enhancing Visual-Language Models (VLMs) with external tools improves perception but incurs substantial overhead. Common solutions rely on specialised supervision; however, these are predominantly English-centric and fail when scaling to multilingual spaces. We propose MAPLE (Multilingual Alignment of Predictive Latent Embeddings), a representation-learning framework based on predictive alignment that learns from expert tool-use trajectories in the latent space, eliminating the need for explicit tool invocation at inference time. Specifically, MAPLE learns multilingual predictive embeddings, bypassing reconstruction-based methods that generate latent tokens and suffer from a training-inference mismatch, as well as limited support for multi-step tool use. By preserving the standard vision-language generation pipeline, our framework is model-agnostic and straightforward to train. Crucially, it naturally supports trajectories with multiple tool calls across languages. We evaluate MAPLE on a comprehensive suite of perception benchmarks and introduce their corresponding multilingual versions, demonstrating that it matches or outperforms standard tuning and reconstruction-based latent-reasoning baselines across multiple languages. Finally, we show that reconstruction methods learn static embeddings, demonstrating that predictive alignment is a more principled and globally scalable paradigm.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multilingual, GEPA

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_1nawOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

MAPLE extends predictive latent learning from visual tool-use trajectories to multilingual settings, adding a cross-lingual consistency objective while preserving standard tool-free inference. Experiments across multilingual visual perception benchmarks and several VLM backbones report improvements in accuracy and cross-lingual consistency.

Critical strengths

The paper reports substantial improvements in multilingual accuracy and consistency across several backbones. The component ablations and correct-consistency evaluation provide useful evidence that these gains extend beyond simply producing identical answers across languages.

Critical weaknesses
  • Limited methodological novelty over Pearl. Predictive trajectory alignment, next-latent regularization, prediction tokens, and the inference procedure closely follow Pearl (Adhikari and Lapata, 2026). The main additions are multilingual supervision and cross-lingual consistency. The paper does not sufficiently establish a substantive methodological insight beyond this extension, which limits its significance despite the empirical gains.
  • The added objective is insufficiently characterized. Table 4 supports the usefulness of CLLC, but its advantages over alternative cross-lingual alignment objectives and its interaction with trajectory learning remain underexplained. Additional training costs and effects on broader VLM capabilities are also unclear, limiting assessment of the extension’s practical value and wider applicability.
Questions for authors
  • Does the “w/o CLLC” variant faithfully reproduce Pearl with the same multilingual data and training budget? What implementation or tuning differences remain?
  • Which objectives are retained in w/o�0�w/o �0�, and why do its reported results exactly match SFT? Does this comparison isolate the contribution of cross-lingual alignment without trajectory prediction?
  • What additional training time and memory does CLLC require? Is there evidence about its effects on general VLM capabilities beyond the evaluated perception tasks?
Reviewer_eVe2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper proposes MAPLE, which trains vision-language models to internalize visual tool-use trajectories, removing the need for tool calls at inference time.

  • This paper replaces latent-token reconstruction with predictive alignment to the trajectory embedding, plus a cross-lingual consistency objective.

  • This paper introduces multilingual versions of perception benchmarks in 11 languages and reports gains in accuracy and cross-lingual consistency, especially for low-resource languages.

Critical strengths
  • MAPLE keeps the standard decoding pipeline: it makes no tool calls and generates no latent tokens or extra tokens at test time, while still learning from multi-step tool-use supervision. This matters because the latency and error-propagation costs of explicit tool use limit its practical deployment, and the approach applies to different model architectures without changes.

  • The paper evaluates fine-grained visual reasoning across 11 languages, including several low-resource ones, and introduces consistency metrics that check whether a model gives the same answer across languages.

Critical weaknesses
  • Limited novelty relative to prior work. The core method closely overlaps with PEARL (Adhikari & Lapata, 2026), which already proposes learning from tool-use trajectories through predictive embedding alignment, with tool-free inference. The paper’s main addition, multilingual training with a cross-lingual consistency objective, is weakly motivated, since visual tool operations are inherently language-independent. As a result, the methodological contribution appears incremental.

  • The central claim of internalizing tool use is not well supported. The alignment target is the final hidden state of the full trajectory, which includes the answer. The predictive objective therefore teaches the model to anticipate the answer as well as the effect of tool operations. Since the target is discarded at inference and no analysis isolates the contribution of tool-use information (e.g., a target without the answer, or probing for tool-revealed content), the gains may reflect answer-level self-distillation rather than internalized tool use. This undermines the paper’s main claim and its explanation of why the method works.

Questions for authors
  • Relation to PEARL: The core method appears closely related to PEARL (Adhikari & Lapata, arXiv:2604.08065, 2026). PEARL also learns from expert tool-use trajectories through predictive embedding alignment, avoids tool calls at inference, and critiques reconstruction-based latent reasoning. Apart from the multilingual data and the cross-lingual consistency loss (Section 2.3), what does MAPLE contribute methodologically? How does MAPLE compare with PEARL experimentally under the same setting?

  • Motivation for multilingual tool use Visual tool operations are language-independent, and MAPLE trained on English data only already improves non-English accuracy substantially (Table 2). What makes multilingual tool use a distinct problem? Does the cross-lingual consistency loss provide similar gains in plain multilingual fine-tuning without tool trajectories?

  • Answer information in the alignment target: Section 2.1 states that the final trajectory step contains the answer, and the target is the last-token hidden state of the trajectory view. The objective may therefore teach the model to anticipate the answer rather than the effect of tool operations. Could the authors report: (a) results with a target that excludes the answer; (b) results with a target built from the answer alone, without the tool states; and (c) a probing analysis showing that the learned representations encode tool-revealed content (e.g., information from the cropped region)?

  • Figure 4 (Section 4.5): This figure is titled “Synthetic UMAP,” and its legend lists only languages, without distinguishing input-view from trajectory-view embeddings or explaining what each cluster represents. Is the figure computed from actual model embeddings? If so, could the authors clarify the markers and clusters, so that the figure can support the claim that MAPLE removes language-based clustering?

1 comment

As the discussion period is approaching the end and the authors do not give a response to my concerns, I will keep my rating as weak rejection.

Reviewer_qXvzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MAPLE aims to reduce language-dependent accuracy gaps in visual question answering without inference-time tools. It trains a VLM to predict representations of expert tool trajectories and applies cross-lingual latent consistency to translated questions. Inference uses the original image and question without tool execution or latent decoding. Experiments report improvements over SFT and tool-learning baselines, with component ablations, representation diagnostics, and results across several backbones and seeds.

Critical strengths
  1. MAPLE retains standard VLM inference, and the Table 13 reports the same relative decoding latency as SFT. This is a useful serving benefit. The additional backbone and seed experiments provide supporting evidence beyond a single model run.
Critical weaknesses
  1. The methodological contribution beyond Pearl is insufficiently established.
  • Pearl (COLM 2026) already learns predictive embeddings from expert tool trajectories while preserving standard VLM inference. MAPLE’s main addition is multilingual adaptation with cross-lingual consistency. Table 4 demonstrates useful gains from this addition, but the paper does not sufficiently establish a distinct methodological advance over the existing predictive-trajectory formulation.
  1. The experiments do not isolate the value of tool trajectories or establish competitiveness with direct alignment methods.
  • SFT and the CLLC ablation retain trajectory supervision; a matched QA/explanation target with the same losses, backbone, answer information, and training budget is missing.
  • Recent alternatives such as MRRE (ACL 2026) and ANCHOR (arXiv:2608.15085, August 2026) also warrant explicit positioning and a suitable alignment comparison. Their different settings require task-compatible evaluation.
  • Table 13 supports the inference claim, while preparation and training costs remain unquantified.
  1. The evaluation is insufficient for the breadth of the multilingual-alignment claims.
Questions for authors
  1. What task-compatible direct alignment comparison supports the relative performance claim, considering MRRE and ANCHOR’s different settings? Please provide any existing comparable results and the additional preparation and training costs.

  2. What evidence shows that tool-trajectory targets provide an advantage over QA or explanation targets under the same backbone, answer information, alignment losses, and training budget? Please clarify which trajectory inputs and objectives remain in the ablations.

1 comment

I received no author response. My main concern remains whether training on tool trajectories offers an advantage over training on multilingual QA or explanations under the same conditions. The reported gains from cross-lingual consistency leave that question open. I maintain my rating of 1 (Clear rejection).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.