- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes mRewriter-R1, an adaptive multilingual query rewriting framework based on agentic reinforcement learning (RL), aiming to improve the reasoning performance of large language models across different languages.
Critical strengths
The strongest aspect of this paper is its encouraging empirical performance across multiple multilingual reasoning benchmarks. As reported in Table 1, mRewriter-R1 achieves average accuracies of 73.9% on MGSM, 77.6% on XCOPA, and 80.2% on BELEBELE, outperforming the strongest reported baselines by 4.2, 1.7, and 2.6 percentage points, respectively.
Critical weaknesses
W1. Insufficient evidence that RL-based adaptive operator selection is responsible for the performance improvements The main weakness is that the experimental design does not adequately isolate the contribution of the proposed adaptive RL mechanism.
The paper emphasizes learning when and how to apply different rewriting operators. However, according to Section 2.2 and Appendix G, the three rewriting aspects are executed in a predefined order: language, structural, and semantic. The model therefore learns operator selection within each aspect, rather than learning the overall sequence of aspects or the number of rewriting stages.
More importantly, the existing comparisons primarily involve direct inference or previously proposed rewriting approaches. Although the ablation studies show that removing operators or individual aspects degrades performance, these experiments do not establish whether reinforcement learning provides substantial advantages over simpler approaches using the same operator space.
W2. Incomplete validation of semantic faithfulness and inconsistencies in the experimental protocol A second important concern is whether the rewriting process reliably preserves the original problem semantics and whether the experimental setup is sufficiently documented. The proposed method is optimized using downstream answer correctness rather than an explicit semantic-preservation objective. Consequently, improved answer accuracy does not necessarily guarantee that the rewritten query faithfully preserves all the information in the original input.
Although the authors provide a semantic-preservation analysis, the reported results indicate that critical-number token recall is 93.28%, while only 89.07% of queries retain all original numerical tokens. These statistics do not necessarily imply semantic errors, since some numerical transformations may be harmless. Nevertheless, they suggest that numerical information is not always preserved verbatim. The paper primarily relies on token-level preservation statistics, semantic similarity, and a limited number of qualitative examples. These evaluations are insufficient to establish whether the rewriting process consistently preserves mathematical relationships, logical conditions, and the original question intent, particularly in cases where numerical information is omitted or modified.
This issue is especially important for mathematical reasoning, where small changes to numerical constraints can fundamentally alter the problem.
Questions for authors
- Could the authors provide controlled comparisons against simpler methods using the same rewriting operators and three-stage pipeline, such as a training-free LLM-based operator selector, a heuristic operator-selection strategy, or a fixed rewriting pipeline without RL training?
- Could the authors clarify the exact MGSM training and evaluation splits, particularly the treatment of Telugu?
5 comments
Q2.
Thank you for pointing out this ambiguity. We clarify that the MGSM training set contains 10 languages (10,300 instances) constructed from multilingual GSM8K, whereas evaluation covers 11 languages with 250 instances each (2,750 in total). Specifically, Telugu is excluded from training and included only in evaluation, serving as an unseen-language test of cross-lingual generalization.
We also identified a reporting error in Appendix Table 4, where the MGSM test total was incorrectly listed as 2,500 instead of 2,750. We will correct this value and explicitly distinguish the training and evaluation languages in the revised manuscript.
We sincerely thank the reviewer again for the constructive feedback and hope that our clarifications and additional experiments adequately address your concerns and encourage you to reconsider your assessment.
Dear Reviewer qasp,
We sincerely thank you again for your thoughtful comments and constructive feedback. We have carefully addressed your concerns in our rebuttal and hope that our responses have helped clarify the questions you raised.
As the discussion period is approaching its deadline, we would be truly grateful if you could take a moment to review our responses and share any further thoughts. If you have any remaining questions or concerns, we would be glad to provide additional clarifications or evidence.
If our responses and additional experiments have adequately addressed your concerns, we would sincerely appreciate your reconsideration of the overall rating.
We greatly appreciate your time and effort in reviewing our work and look forward to hearing your feedback.
Best regards,
The Authors
We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. We carefully address the concerns below and provide additional experimental evidence and clarifications to strengthen our work.
Thank you for this insightful comment. We first clarify that the three rewriting aspects follow a predefined order (Language → Structural → Semantic), while the operator within each aspect is adaptively selected based on the current query state and preceding rewriting outcomes. Thus, our adaptivity lies in query-dependent operator selection and sequential rewriting, rather than learning the ordering or number of rewriting aspects. We will clarify this distinction in the revised manuscript.
To address your concern, we introduce two additional controlled baselines:
These experiments help distinguish the benefit of the fixed multi-stage framework from the additional improvements associated with adaptive selection and RL optimization.
As shown above, Non-RL Adaptive outperforms Fixed Operator by 2.94 percentage points, demonstrating the benefit of query-dependent operator selection over a fixed rewriting strategy. Moreover, mRewriter-R1 further improves over Non-RL Adaptive by 2.78 points, indicating that RL optimization provides additional gains beyond training-free adaptive rewriting.
Thank you for raising this important concern. We clarify that the previously reported 89.07% measures exact numerical-token preservation rather than numerical semantic correctness. Differences in numerical tokens may arise from semantically equivalent transformations rather than actual information loss.
To address this concern, we conduct two additional analyses. First, we examine queries with numerical-token changes to distinguish semantically equivalent transformations from genuine alterations of numerical information. Second, we directly evaluate the preservation of three key elements essential to mathematical reasoning: mathematical relationships, logical conditions, and the original question intent. We assess semantic equivalence by comparing each original query with its rewritten version using an independent LLM-based evaluator, without providing ground-truth answers or downstream predictions.
The results show that most numerical-token changes correspond to semantically equivalent transformations rather than genuine numerical errors, explaining why exact token preservation underestimates numerical faithfulness. Furthermore, the additional evaluations provide quantitative evidence that the rewriting process preserves the key mathematical relationships, logical conditions, and question intent in the majority of cases. While occasional deviations remain, these analyses provide direct evidence of semantic preservation beyond token-level matching and embedding similarity.
We will incorporate these results and the corresponding evaluation protocol into the revised manuscript.