acceptodds
Under review as a conference paper at ICLR 2027

Stay True to Your Roots: Reference-Anchored Policy Optimization for Faithful GEO

Abstract

Generative search is redefining web visibility: being retrieved no longer suffices if a source goes unrepresented in the LLM-generated answer. This creates a new optimization problem for content creators—improving visibility among competing sources without compromising faithfulness to the original content. Existing generative engine optimization (GEO) approaches largely optimize visibility in isolation, neglecting faithfulness, and rely on manually designed edit instructions with limited coverage. We introduce Reference-Anchored Policy Optimization (RAPO), a reinforcement learning framework that generates multiple candidate rewrites but measures progress against the original source rather than other sampled rewrites. RAPO anchors advantage estimation to the source reward, so positive signals reflect improvement over the original content, while retaining within-group standard-deviation normalization for stable scaling. To capture the competitive nature of generative search, we introduce a competition-aware visibility signal that evaluates a source against its strongest competitor and combine it with faithfulness in a composite reward. Across GEO-Bench, Researchy-GEO, and E-commerce, RAPO improves Overall visibility over the source by 9.34, 15.82, and 7.15 points with better faithfulness, outperforming prompt-based and trainable RL baselines. Ablations and analysis show source anchoring is essential, RAPO generalizes across rewriter model scales and generative engines, and benefits sources with lower rank in LLM context most, while varying the faithfulness weight reveals a controllable visibility–faithfulness trade-off. Code is released at https://anonymous.4open.science/r/RAPO-4190/README.md.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Generative Engine Optimization, Reference-Anchored Policy Optimization, Reinforcement Learning, Faithful Content Optimization, Generative Search

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_B5DAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is about GEO by rewriting source documents to increase their visibility in generative search while preserving their content. It trains a rewriter using a reward that combines visibility relative to the strongest competing source with a faithfulness score. The main change to GRPO is to compare each sampled rewrite's reward with the original document's reward, while retaining normalization by the sampled group's standard deviation. The paper also extracts editing guidelines from successful rewrites for use with an off-the-shelf LLM. Experiments on three GEO benchmarks report improved visibility and faithfulness over several rewriting baselines.

Critical strengths
  1. The proposed optimization strategy by comparing rewrites with the original document is a good choice for the GEO task. The controlled GRPO comparison reports improvements in both visibility and faithfulness.
  2. The experiments are pretty thorough, including model-size ablations, transfer across generative engines, and a useful demonstration that successful edits can be distilled into reusable instructions.
  3. The provided repository contains the necessary code and scripts for auditing.
Critical weaknesses

I find the intellectual and technical contribution fairly narrow.

  1. The main technical idea is a simple tweak to GRPO: replace the sampled group mean reward with the original document's reward when computing advantages, while keeping the same normalization. AutoGEO already combines GRPO with a semantic-preservation reward.

  2. The empirical improvements are modest. At the same 1.7B model size, RAPO improves visibility over AutoGEO-Mini by only 0.33 and 0.78 points on GEO-Bench and Researchy-GEO, respectively. Under the controlled GRPO comparison, the gains are 1.48 and 1.16 points on the reported 0–100 scale. The faithfulness scores have bigger margins, but the main metric for GEO seems to be visibility...

Questions for authors
  1. Why should comparing rewrites with the original document lead to better learning than comparing them with each other (i.e. GRPO)? Could you provide some technical or theoretical justification?

  2. Could RAPO score better simply by removing important information and making the remaining text easier to cite? Do the resulting generative search results actually become better?

6 comments
AuthorsOpenReview ↗

Author Response for Reviewer B5DA (1/4)

We thank the reviewer for the positive assessment and for recognizing that comparing rewrites to the original document is a good fit for GEO, that the controlled GRPO comparison improves both visibility and faithfulness, and for the thoroughness of our experiments, including model-size ablations, generative-engine transfer, and the provided code repository. Below, we highlight RAPO's novel technical contributions beyond reference-anchoring with a component-by-component ablation (W1), give the theoretical and empirical justification for source-anchoring over group-mean centering (Q1), clarify how to read the visibility and faithfulness margins together under matched scale and controlled GRPO comparisons (W2), and rule out omission-driven visibility gains with a direct baseline comparison (Q2). We hope these address the reviewer's concerns and would welcome any further questions.


W1: RAPO's three technical contributions each measurably contribute, beyond a single centering-term tweak

W1 - "The main technical idea is a simple tweak to GRPO: replace the sampled group mean reward with the original document's reward when computing advantages, while keeping the same normalization. AutoGEO already combines GRPO with a semantic-preservation reward."

We would like to note and highlight that RAPO's proposed approach has three novel technical contributions, not one: (i) a competition-aware visibility formulation that scores a rewrite relative to the other sources competing for the same result slot, (ii) a faithfulness reward and metric that explicitly checks rewrites against the source, and (iii) a reference-anchored advantage that centers learning on the source document itself rather than on sampled peer rewrites. The table below ablates each one separately on a fixed Qwen3-8B backbone, changing only the named component per row while holding the other two fixed, and we discuss each contribution afterward in light of this table, empirically and in contrast to related methods.

ContributionDescriptionComparison (same Qwen3-8B backbone)BenchmarkOverall Visibility (without → with)Faith. (without → with)
Competition-aware visibilityScores a rewrite's visibility relative to its competing sources in the same result set, rather than in isolation.Raw visibility → competition-aware visibilityGEO-Bench26.25 → 28.84 (+2.59)75.52 → 77.45 (+1.93)
Researchy-GEO32.69 → 35.52 (+2.83)78.14 → 80.32 (+2.18)
Faithfulness rewardAn LLM-judge reward checks each claim in a rewrite against the source, penalizing unsupported additions during training. This is not just a reward term, but proposes faithfulness itself as a metric GEO approaches generally omit, and shows that optimizing visibility alone lets faithfulness quietly degrade.αF = 0.0 → αF = 0.3 (Table 5)GEO-Bench23.21 → 28.84 (+5.63)71.18 → 77.45 (+6.27)
Researchy-GEO31.80 → 35.52 (+3.72)76.89 → 80.32 (+3.43)
Reference-anchored advantageCenters the advantage on the source document's own reward instead of the sampled group mean, so a rewrite is reinforced only if it improves on the source.Advantage centered on group mean (GRPO, Table 3) → centered on source reward (RAPO)GEO-Bench27.36 → 28.84 (+1.48)75.36 → 77.45 (+2.09)
Researchy-GEO34.36 → 35.52 (+1.16)78.04 → 80.32 (+2.28)
  • Competition-aware visibility is itself a distinct contribution of the paper, not a byproduct of reference-anchoring or the faithfulness reward: it changes how visibility itself is scored. Concretely, for a target document scored against the same N-1 competitor documents its rewrite would actually face in the result set, it computes VCAV = V(target) / (V(target) + V(strongest competitor)), where V is raw visibility (Eq. 1). This ratio is above 0.5 only when the target beats the single toughest competitor, not merely the average one, so a rewrite is rewarded for standing out against the other sources it is actually competing with, rather than scoring it in isolation. Its standalone contribution (+2.59 and +2.83 Overall Visibility points, with the other two components held fixed) is among the largest of the three, showing that modeling visibility this way is itself a core driver of RAPO's gains.

Answer to be continued ...

AuthorsOpenReview ↗

Author Response for Reviewer B5DA (2/4)

  • Faithfulness reward: markedly more effective at semantic preservation than AutoGEO's heuristic reward. AutoGEO-Mini is a Qwen3-1.7B model fine-tuned with GRPO using a heuristic, rule-based reward (Sec. 4), which checks for semantic preservation without verifying individual claims. RAPO's faithfulness term F(x̃, x; J) instead is computed by an LLM judge in two steps (Appendix A.2): it first extracts the factual claims in the rewrite, then checks each claim against the original source, and the score is the fraction of claims verified as consistent with it. This claim-level check is applied to every sampled rewrite during training, so unsupported additions are penalized directly in the reward, rather than only caught when they happen to violate a fixed heuristic rule. Our qualitative analysis (Appendix A.10, Table 17) shows exactly this difference in practice: on the same source document, RAPO's rewrite preserves every cited figure and still gains +18.4 visibility, while AutoGEO-Mini's heuristic reward lets it fabricate a purity-grading table with invented grades and symbols ("Coiny," "PM") and a false "penny gold" statistic, losing -24.7 visibility. At the same Qwen3-1.7B scale, this gap holds in aggregate too: RAPO reaches 75.90 and 75.71 faithfulness on GEO-Bench and Researchy-GEO, versus 63.27 and 61.38 for AutoGEO-Mini (+12.63 and +14.33 points).

  • Reference-anchored advantage: an empirical and conceptual improvement over GRPO. Reference-anchoring gives the largest faithfulness gain (+2.09 and +2.28 points) and also improves overall visibility (+1.48 and +1.16 points), since each rewrite is scored, and penalized for drifting, against the source's own reward; the faithfulness term reinforces this same objective. The GRPO variant in the second row isolates just this centering change, since it already shares RAPO's full composite reward, including competition-aware visibility and faithfulness (Table 3), and differs only in how the advantage is centered. Two points show the centering still matters on its own. First, RAPO still outperforms this variant, as well as Dr. GRPO, DAPO and Reinforce, on both metrics despite the identical reward, so the gain must come from centering, not the reward. Second, centering changes what the advantage means: group-mean centering only signals whether a rewrite beat its sampled peers, while reference anchoring signals whether it improved on the source (Eq. 7), so a group where every rewrite regresses yields no positive advantage at all (Sec. 3.2).

Thus the three components (competition-aware visibility, the reference-anchored advantage, and the faithfulness reward), taken in combination as well as individually, and including a faithfulness mechanism that is both conceptually distinct from and empirically stronger than AutoGEO's rule-based approach, together constitute the paper's technical contribution. We have included this component ablation in Appendix A.5 (Table 10), referenced from the main paper.


Q1: Theoretical and empirical justification for reference-anchoring over group-mean centering (GRPO)

Q1 - "Why should comparing rewrites with the original document lead to better learning than comparing them with each other (i.e. GRPO)? Could you provide some technical or theoretical justification?"

Group-mean centering tells a rewrite only whether it beats the other sampled rewrites in its group, not whether it improves on the source. Because centering is relative to the group's own mean, a group still splits into above-mean (positive advantage) and below-mean (negative advantage) rewrites even when every rewrite in that group agrees in direction relative to the source, i.e., all of them improve on it or all of them regress from it. This gives two failure cases:

  1. All rewrites are worse than the source. The above-average rewrites still regress, yet they receive positive advantage and are reinforced. For example, with a source reward of 0.50 and rewrite rewards of 0.30, 0.35, 0.40 and 0.45 (group mean 0.375), GRPO reinforces the 0.40 and 0.45 rewrites even though both are worse than the source.
  2. All rewrites are better than the source. The below-average rewrites still improve on the source, yet they receive negative advantage and are suppressed.

In both cases the update points the wrong way relative to the actual objective (improve over the source), because the comparison target is the sampled group. Source anchoring removes this by construction: Âᵢ > 0 iff rᵢ > r̄(x,q) (Eq. 7), so a rewrite is reinforced only if it improves on the source. Because r̄ is a fixed reference and not a statistic of the current policy, it does not rise as the policy changes, whereas the group mean moves with the policy and always produces roughly half positive and half negative advantages regardless of absolute progress.

AuthorsOpenReview ↗

Author Response for Reviewer B5DA (3/4)


W2: Visibility and faithfulness gains are joint and consistent at matched scale and under the controlled GRPO comparison

W2 - "The empirical improvements are modest. At the same 1.7B model size, RAPO improves visibility over AutoGEO-Mini by only 0.33 and 0.78 points on GEO-Bench and Researchy-GEO, respectively. Under the controlled GRPO comparison, the gains are 1.48 and 1.16 points on the reported 0–100 scale. The faithfulness scores have bigger margins, but the main metric for GEO seems to be visibility..."

In both of the settings i.e. the matched-scale comparison against AutoGEO-Mini and the controlled GRPO comparison, comparisons should be read and interpreted as joint gains in visibility and faithfulness. RAPO improves Overall Visibility and also improves faithfulness with good margins as discussed below -

  • RAPO outperforms AutoGEO-Mini at matched 1.7B model scale (Tables 1 and 6): Both methods use the same Qwen3-1.7B backbone here, isolating the effect of RAPO's reward/advantage design from any difference in model scale. RAPO improves Overall Visibility over AutoGEO-Mini from 23.88 to 24.21 on GEO-Bench (+0.33 points; +1.4% relative gain) and from 30.59 to 31.37 on Researchy-GEO (+0.78 points; +2.5% relative gain). Measured against the unoptimized source (Vanilla), RAPO's relative visibility gain is +59.2% vs. AutoGEO-Mini's +55.3% on Researchy-GEO, and +24.2% vs. +22.5% on GEO-Bench. Faithfulness improves over AutoGEO-Mini from 61.38 to 75.71 on Researchy-GEO (+14.33 points; +23.3%) and from 63.27 to 75.90 on GEO-Bench (+12.63 points; +20.0%).
  • Controlled GRPO comparison (Table 3, same reward, backbone and training setup): Under this matched setup, RAPO improves Overall Visibility over GRPO from 27.36 to 28.84 on GEO-Bench (+1.48 points; +5.4% relative gain) and from 34.36 to 35.52 on Researchy-GEO (+1.16 points; +3.4% relative gain). Faithfulness improves over GRPO from 75.36 to 77.45 on GEO-Bench (+2.09 points; +2.8%) and from 78.04 to 80.32 on Researchy-GEO (+2.28 points; +2.9%). Both methods substantially outperform the unoptimized source (Vanilla) on visibility, and RAPO's margin over Vanilla is larger than GRPO's: +47.9% (RAPO) vs. +40.3% (GRPO) on GEO-Bench, and +80.3% (RAPO) vs. +74.4% (GRPO) on Researchy-GEO.

Regarding "the main metric for GEO seems to be visibility": we agree that visibility is GEO's primary task-level objective, but optimizing it in isolation can degrade faithfulness. Raising citation likelihood changes the content, and without a faithfulness signal those changes need not preserve the meaning, facts and claims of the original document. Several of our baselines show this pattern: Statistics Addition and Quotation Addition raise visibility over Vanilla but reach only 62.30 and 72.15 faithfulness on GEO-Bench, and AutoGEO-Mini reaches 63.27. The goal stated in Section 1 is therefore to improve visibility keeping faithfulness intact as much as possible, and RAPO's results should be assessed on both axes together. A method that increases visibility while also improving faithfulness over its matched baselines addresses this goal more fully than a visibility gain alone. Reference-anchoring is also one of three contributions (with competition-aware visibility and the faithfulness reward, addressed in W1 above), so its standalone visibility margin should be read alongside the separately ablated visibility contribution of competition-aware visibility (+2.59 and +2.83 Overall Visibility points).


Q2: RAPO's gains are not from deleting information, confirmed via a direct baseline comparison which simplifies content

Q2 - "Could RAPO score better simply by removing important information and making the remaining text easier to cite? Do the resulting generative search results actually become better?"

To test this, we refer to a baseline already in our main results: GEO-Bench's "Easy-to-Understand" strategy (Table 1), which simplifies the source language into shorter, easier-to-cite text. This is the baseline corresponding to the omission-driven visibility gain the reviewer describes, and it performs much worse than RAPO: simplification alone yields a much smaller visibility gain than RAPO at comparable faithfulness, so a strategy built around shrinking content is itself an inferior way to gain visibility. Moreover, RAPO's Word and Pos scores rise rather than fall, which indicates that RAPO's own gains are not coming from shrinking content either. We report the numbers (from Table 1) in the following table:

Answer to be continued ...

Reviewer_9C1dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper studies generative engine optimization (GEO). The goal is to rewrite a document so that an LLM-based search engine cites it more, while the rewrite stays faithful to the original. The method, RAPO, is a GRPO variant that computes advantages relative to the reward of the original document instead of the mean reward of the sampled rewrites. The reward combines two scores. One is visibility measured against the strongest competing source, and the other is faithfulness judged by an LLM. A Qwen3-8B rewriter trained with RAPO improves visibility over the original document by 7 to 16 points on three benchmarks. The paper also distills RAPO rollouts into a list of rewriting guidelines (RAPO-Distill) that GPT-4o can follow without any training.

Critical strengths

S1. Faithfulness is part of the objective, and the evaluation is broad. Prior GEO methods mostly optimize visibility alone, and the qualitative examples show that this leads to fabricated facts. The experiments are more thorough than usual in this area. They cover three benchmarks, two engines not used in training, three model sizes, and four alternative RL methods.

S2. RAPO-Distill is practical. It turns RL rollouts into readable guidelines for an off-the-shelf model. This avoids serving a trained policy and makes the learned edits easy to inspect. It also gives the best results in the paper.

Critical weaknesses

W1. The main idea is weakly motivated, and its gain is small. The paper argues that GRPO’s group-mean baseline is misaligned with improving over the source. The paper’s own Eq. 9 shows why this does not hold. Subtracting a baseline that does not depend on the sampled rewrite leaves the expected gradient unchanged. The group mean is also such a baseline (up to a constant factor), so GRPO and RAPO optimize the same objective. The failure cases in Section 3.2 only flip the sign of individual advantages. They do not change what is learned.

The results match this. With the same reward, GRPO already reaches 27.36 and 34.36 Overall on GEO-Bench and Researchy-GEO, against 28.84 and 35.52 for RAPO. Only about 1.2 to 1.5 points of the headline gains therefore come from the proposed advantage, and these are single runs without seeds. Three further results weaken the case:

  • RAPO without std-normalization is worse than GRPO.
  • The competition-aware visibility score is never ablated.
  • At the 1.7B scale of AutoGEO-Mini, RAPO’s visibility gain over it nearly disappears (24.21 vs 23.88 and 31.37 vs 30.59, Table 6).

W2. The faithfulness evaluation is circular and too lenient. RAPO is trained and evaluated with the same faithfulness metric and the same judge (GPT-4o-mini). The baselines never optimized against this judge.

The metric also appears to count new claims as faithful whenever they do not directly contradict the source (Appendix A.2). The paper’s own material suggests this happens:

  • In Table 14, RAPO adds a 2023 Pew Research statistic that is not in the source.
  • The guidelines distilled from high-faithfulness rollouts tell the rewriter to add precise figures, named entities and named sources (DOs #4 to #6).

These are the same errors the paper criticizes in the baselines.

W3. Limited novelty, and the new part is not what drives the results. The only algorithmic change over GRPO is replacing the group mean with a fixed baseline, namely the reward of the original document. Using the reward of a reference output as a policy-gradient baseline is a known idea. For example, ReMax uses the reward of the greedy output as its baseline. The other components are not new either. AutoGEO-Mini already trains a GEO rewriter with GRPO, and LLM-judge faithfulness rewards are common. As W1 shows, most of the improvement comes from this existing recipe applied to a larger model, not from the anchored advantage. The paper therefore lacks a contribution that is both clearly novel and clearly effective, which is the main reason for my score.

Questions for authors

Clarifications:

Q1. What are αV\alpha_V, τ\tau and δ\delta, and how were hyperparameters such as αF\alpha_F selected?

Q2. Are “idk” verdicts counted as faithful, as the text describes, or as unfaithful, as Eq. 12 states? What is the idk rate for RAPO and the baselines?

Q3. The source reward averages six engine runs before applying the nonlinear VCAVV_{\text{CAV}}, while each rewrite uses a single run. For an unchanged copy of the source, is the average advantage close to zero?

Q4. Why does αF=0\alpha_F = 0 give lower visibility than αF=0.3\alpha_F = 0.3 (Table 5)? If training is reliable, removing the faithfulness term should not reduce visibility.

Additional experiments that could change my assessment:

Q5. Mean and standard deviation over at least three seeds for GRPO, DAPO and RAPO.

Q6. RAPO trained with raw visibility instead of competition-aware visibility.

Q7. Faithfulness measured with an independent judge plus a small human study, and RAPO-Distill run without the DOs.

14 comments
AuthorsOpenReview ↗

Author Response for Reviewer 9C1d (1/9)

We thank the reviewer for the detailed technical engagement and for recognizing that faithfulness is part of our stated objective, the breadth of our evaluation (3 benchmarks, 2 held-out generative engines, 3 model sizes, 4 RL baselines), and RAPO-Distill's practicality and strong results. Below, we clarify the scope of Eq. 9 and confirm the resulting empirical gain is stable across seeds (W1 + Q5), add the missing competition-aware visibility ablation (W1 + Q6), clarify the std-normalization ablation by decomposing it into its two underlying design axes (W1), show RAPO's visibility and faithfulness gains both hold at matched 1.7B scale (W1), validate faithfulness under independent judges and a human study (W2 + Q7), clarify the idk-verdict contribution to faithfulness and compare idk rates across methods (W2 + Q2), pin down RAPO's three contributions' technical differences from ReMax and AutoGEO-Mini (W3), clarify hyperparameter selection (Q1) and provide analysis to check advantage using original source copies (Q3), explain the non-monotonic αF–visibility relationship (Q4), perform and report the RAPO-Distil ablation without DOs result (Q7). We hope these address the reviewer's concerns and would welcome any further questions.


W1 + Q5 - Eq. 9 does not establish GRPO/RAPO equivalence, and RAPO's multi-seed gains over GRPO/DAPO are stable

W1 - "The paper argues that GRPO's group-mean baseline is misaligned with improving over the source. The paper's own Eq. 9 shows why this does not hold. Subtracting a baseline that does not depend on the sampled rewrite leaves the expected gradient unchanged. The group mean is also such a baseline (up to a constant factor), so GRPO and RAPO optimize the same objective. The failure cases in Section 3.2 only flip the sign of individual advantages. They do not change what is learned.

With the same reward, GRPO already reaches 27.36 and 34.36 Overall Visibility on GEO-Bench and Researchy-GEO, against 28.84 and 35.52 for RAPO. Only about 1.2 to 1.5 points of the headline gains therefore come from the proposed advantage, and these are single runs without seeds."

Q5 - "Mean and standard deviation over at least three seeds for GRPO, DAPO and RAPO."

We clarify the theoretical claim and other aspects by answering in four parts. (1) We measure the consequence over three seeds, confirming the empirical effect is stable and not a single-run artifact. (2) We derive what Eq. 9 gives for the source reference r̄ (baseline visibility) and for the group mean μG (mean of visibility scores across G rollouts), and we agree with the reviewer that the two share the same expected gradient direction at a fixed θ. (3) We then state our claim explicitly: despite this, GRPO and RAPO are different update rules, because they assign different advantages to the same finite group, and training follows those realized updates. We prove this with an exact identity relating the two advantages. (4) We show what the difference means for individual rewrites, with a worked example.

Part 1: the measured consequence across seeds. The reference-anchored advantage is one of three contributions of the paper. The 1.2 to 1.5 point difference the reviewer cites is the portion attributable to this component alone, while the competition-aware visibility formulation and the faithfulness reward contribute separately (see W1 below). To check that the difference is reliable, we re-ran GRPO, DAPO, and RAPO from three random seeds under an identical reward and training budget. GRPO and DAPO use the same competition-aware visibility and faithfulness reward (Eq. 2) as RAPO, and GRPO differs from RAPO only in the centering term, so the RAPO–GRPO comparison isolates the reference-anchored advantage. We report mean ± population standard deviation over the three seeds:

MethodGEO-Bench Overall VisibilityGEO-Bench Faith.Researchy-GEO Overall VisibilityResearchy-GEO Faith.
GRPO26.76 ± 0.0974.02 ± 0.2633.65 ± 0.1776.92 ± 0.24
DAPO26.98 ± 0.1074.58 ± 0.3032.73 ± 0.1775.82 ± 0.16
RAPO (Ours)28.46 ± 0.3177.35 ± 0.1235.10 ± 0.3480.29 ± 0.13

Answer to be continued ...

AuthorsOpenReview ↗

Author Response for Reviewer 9C1d (2/9)

Across the three seeds, RAPO improves both metrics:

  • Visibility: The margin holds on every seed. Per-seed RAPO Overall Visibility (28.08–28.84 GEO-Bench, 34.69–35.52 Researchy-GEO) stays above GRPO's (26.69–26.88, 33.43–33.84) and DAPO's (26.84–27.08, 32.51–32.93). Each method's seed-to-seed standard deviation in Overall Visibility is ≤0.34, more than four times smaller than RAPO's 1.45–2.37 point gap over GRPO and DAPO.
  • Faithfulness: RAPO's 77.35 ± 0.12 / 80.29 ± 0.13 versus GRPO's 74.02 ± 0.26 / 76.92 ± 0.24 is a 3.33/3.37-point gap, more than ten times any seed's standard deviation. This is consistent with Parts 3 and 4 below. The source is by definition perfectly faithful to itself (F(x,x;J) ≡ 1, the maximum attainable value), so anchoring advantages to the source's own reward r̄(x,q) rewards rewrites for staying close to that ceiling every time a rewrite is scored. Group-mean centering compares each rewrite only with its siblings, so a group whose rewrites have all drifted from the source can still assign positive advantage to some of them.

Part 2: what Eq. 9 gives for r̄ and for μG (where we agree). Notation: sᵢ = ∇θ log πθ(x̃ᵢ|x), and J(θ) = E[r] is the expected reward, so ∇J = E[sᵢ·rᵢ]. We use two facts. The log-derivative identity ∇θπ = π∇θ log π gives E[sᵢ] = Σ ∇θπ = ∇θ1 = 0. And for any baseline b that takes the same value for every rewrite x̃ (it may depend on θ),

E[sᵢ·b] = b·Σ ∇θπθ(x̃|x) = b·∇θ Σ πθ(x̃|x) = b·∇θ1 = 0.

  • Source reference. r̄(x,q) is computed from the fixed source and cached before the group is sampled, so it is the same for every rewrite. Hence E[sᵢ(rᵢ − r̄)] = ∇J. This is Eq. 9.

  • Group mean. Split μG = rᵢ/G + B₋ᵢ, where B₋ᵢ = (1/G)Σⱼ≠ᵢ rⱼ. B₋ᵢ is independent of x̃ᵢ, so E[sᵢ·B₋ᵢ] = E[sᵢ]·E[B₋ᵢ] = 0. The own-reward part does depend on x̃ᵢ, and E[sᵢ·rᵢ/G] = ∇J/G. Hence

    E[sᵢ(rᵢ − μG)] = ∇J − ∇J/G = ((G−1)/G)·∇J.

We therefore agree with the reviewer: at a fixed θ, GRPO and RAPO have the same expected gradient direction, up to the constant factor (G−1)/G. Eq. 9 certifies r̄ exactly, and it shows nothing against μG beyond this factor.

Part 3: our claim, that GRPO and RAPO are nevertheless different learning rules. Same expected gradient direction does not mean the same update. Training uses realized updates from finite groups (G=6), and for the same group the two advantages differ exactly:

A^iRAPO−A^iGRPO=[(ri−rˉ)−(ri−μG)]/(σG+δ)=(μG−rˉ)/(σG+δ)=:cGÂᵢ^{RAPO} − Âᵢ^{GRPO} = [(rᵢ − r̄) − (rᵢ − μG)] / (σG + δ) = (μG − r̄) / (σG + δ) =: c_G.

The shift c_G is the same for every rewrite in the group. The realized updates differ by c_G·(1/G)Σᵢ sᵢ. Two properties follow.

  • GRPO is blind to the group's position relative to the source. Σᵢ(rᵢ − μG) = 0 in every group, so GRPO's advantages are unchanged when a constant is added to every reward in the group. A group in which every rewrite improves on the source and a group in which every rewrite regresses can therefore produce identical GRPO updates. In RAPO, c_G carries exactly this information: c_G > 0 when the group is on average better than the source and c_G < 0 when it is worse.
  • Source-relative sign. By Eq. 7, Âᵢ^RAPO > 0 if and only if rᵢ > r̄(x,q). Under GRPO, Âᵢ > 0 if and only if rᵢ > μG, which says nothing about the source.

As a consistency check (treating σG as fixed), E[c_G·(1/G)Σᵢ sᵢ] = (1/(σG + δ))·E[μG·s̄] = ∇J/(G(σG + δ)), where s̄ = (1/G)Σᵢ sᵢ and the r̄ term vanishes because E[s̄] = 0. This is exactly the (1/G)∇J piece that Part 2 showed GRPO's self-inclusion removes. The two estimators agree in expectation, and what differs is the realized per-group signal.

Part 4: what the difference means for individual rewrites. Take source reward r̄ = 0.50 and a group of four rewrites (G=4 for readability; δ ignored; σG = 0.056 in both rows):

GroupRewrite rewardsGRPO advantagesRAPO advantages
All regress0.30, 0.35, 0.40, 0.45−1.34, −0.45, +0.45, +1.34−3.58, −2.68, −1.79, −0.89
All improve0.60, 0.65, 0.70, 0.75−1.34, −0.45, +0.45, +1.34+0.89, +1.79, +2.68, +3.58

GRPO produces identical advantages for the two groups. In the first it reinforces the 0.40 and 0.45 rewrites, although both are worse than the source. In the second it suppresses the 0.60 and 0.65 rewrites, although both improve on it. RAPO gives no positive advantage to any regression and no negative advantage to any improvement. Each term sᵢ·Âᵢ reinforces or suppresses that specific rewrite according to its sign, so the failure cases in Section 3.2 affect which rewrites are reinforced during training. They are not only a change of sign convention.

Answer to be continued ...

AuthorsOpenReview ↗

Author Response for Reviewer 9C1d (3/9)

Part 2 also does not settle what the trained policy learns, because Eq. 9 is a statement about expectations at a fixed θ. Each real update is computed from one group, divided by σG, clipped by the PPO ratio and passed through the optimizer. These operations are nonlinear, so the constant factor (G−1)/G does not pass through them unchanged (clipping acts on the product ρᵢÂᵢ, for example), and equal expected gradients do not imply equal updates (this is also seen in other GRPO variants: Dr. GRPO, which removes the length and std normalization terms, and DAPO, which changes the loss normalization and clipping, alter the weighting and scale of the per-sample gradient while using the same reward signal, yet they reach different results from GRPO in Table 3, with Overall Visibility of 27.36/34.36 for GRPO, 25.13/32.43 for Dr. GRPO and 27.51/34.18 for DAPO on GEO-Bench/Researchy-GEO). Since the policy determines the distribution of the next group, differences in realized updates accumulate over training.

The headline single-seed comparison the reviewer cites is therefore representative of a stable, reproducible gap and not a favorable outlier. Together with Parts 3 and 4, which show that the two advantages differ in the signal each rewrite receives, this indicates that both the mechanism and the measured effect are real, and not an artifact of run-to-run variance. We have revised the PDF to include this full multi-seed table and the training curves in Appendix A.6 (Table 11), referenced from the main paper (line 376).


W1 + Q6 - Added the competition-aware visibility ablation, isolating its independent contribution

W1 - "The competition-aware visibility score is never ablated."

Q6 - "RAPO trained with raw visibility instead of competition-aware visibility."

To isolate the contribution of the competition-aware visibility term, we trained a RAPO variant that replaces VCAV in both the reward and the cached source reference with raw (non-competitive) visibility V(d,q;E), keeping reference-anchoring, std-normalization, and the faithfulness term identical. Removing competition-awareness results in degradation in visibility, while faithfulness drops only mildly because the reference-anchored advantage and faithfulness term still constrain drift. We report the numbers in the following table:

GEO-BenchResearchy-GEO
VariantWordPosOverall VisibilityFaith.WordPosOverall VisibilityFaith.
RAPO with competition-aware visibility28.9228.7328.8477.4535.5435.6135.5280.32
RAPO with raw (non-competitive) visibility26.4126.1926.2575.5232.8532.7432.6978.14

Removing the competitive comparison costs 2.59 Overall Visibility points on GEO-Bench and 2.83 on Researchy-GEO. Competition-aware visibility is therefore a separate source of RAPO's gains. In generative search, a source competes with other retrieved documents for attribution, and raw visibility ignores this context. VCAV instead rewards a rewrite for outperforming its strongest competitor in the same answer, which is the quantity a content creator needs to improve. It also improves faithfulness in both settings, by 1.93 and 2.18 points. In this ablation the reference-anchored advantage and the faithfulness term are unchanged, so the faithfulness penalty still limits factual drift when the visibility signal is less discriminative, and this bounds how far the raw-visibility variant can drift from the source despite its lower visibility.


W1 - Decomposing reference-anchoring and std-normalization into a 2x2 ablation explains why RAPO w/o std-norm underperforms GRPO

W1 - "RAPO without std-normalization is worse than GRPO."

The RAPO w/o std-norm ablation (Table 3) tests whether the σG normalization that RAPO inherits from GRPO is still needed once the baseline is the source reward r̄(x,q). Comparing RAPO w/o std-norm with GRPO changes two things at once: the baseline (source reward instead of group mean) and the normalization (none instead of σG). To separate them, we arrange the Table 3 runs as a 2×2 design, where all four cells use the same reward and training setup:

BaselineWith std-norm: Overall Visibility (GEO-Bench / Researchy-GEO)With std-norm: Faith. (GEO-Bench / Researchy-GEO)Without std-norm: Overall Visibility (GEO-Bench / Researchy-GEO)Without std-norm: Faith. (GEO-Bench / Researchy-GEO)
Anchoring on Group mean μG27.36 / 34.36 (GRPO)75.36 / 78.0425.13 / 32.43 (Dr. GRPO)74.97 / 77.18
Anchoring on Source reward r̄28.84 / 35.52 (RAPO)77.45 / 80.3226.13 / 33.46 (RAPO w/o std-norm)74.72 / 78.28
Reviewer_RsvLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors of this paper propose RAPO, that is a reinforcement‑learning framework for Generative‑Engine Optimization (GEO). Different from conventional group‑relative advantage estimation, RAPO computes advantages anchored to the reward of the original source document, rather than the mean of sampled rewrites. It adopts a composite reward combining competition‑aware visibility against rival sources and a faithfulness term that discourages factual drift from source content. The authors evaluate their method on three benchmarks, and compare against prompt‑based and RL‑based GEO baselines. The ablation studies are conducted over reward weights, model scales and generative engines. Besides the trained RL policy, the authors introduce RAPO‑Distill: extracting reusable rewriting guidelines from high‑quality RAPO rollouts, which can be directly injected as prompts for off‑the‑shelf LLMs without further RL fine‑tuning.

Critical strengths
  1. RAPO anchors advantage estimation to the original source’s reward, so learning signals reflect genuine improvement relative to the source itself, which fits the core requirement of faithful source rewriting for generative search. It further designs a competition‑aware visibility reward together with faithfulness regularization for this multi‑objective setting.
  2. Experiments span three heterogeneous benchmarks, with ablations on faithfulness weight, rewriter model scales, and different generative engines. Systematic comparisons against prompt heuristics and existing RL baselines (e.g., GRPO, DAPO) verify effectiveness and cross‑setting generalizability.
  3. This work demonstrates that high‑quality RL rollouts can be distilled into interpretable rewriting guidelines. These guidelines can be deployed purely via prompting on arbitrary LLMs, decoupling heavy RL training cost from inference‑time usage, which is a practically valuable direction for GEO research.
Critical weaknesses
  1. There exist incomplete baseline comparisons and model‑scale confounding:
    • Several highly‑relevant recent GEO baselines are missing from main experiments: FeatGEO, Mind Reader, MAGEO, AgentGEO; some of them are originally evaluated on GEO‑Bench and are direct competitors.
    • Main experiments mix different model sizes across compared systems. Under matched model scale, RAPO’s empirical gains shrink substantially. Without fully controlled comparisons, it is hard to robustly establish RAPO’s real superiority over state‑of‑the‑art GEO approaches. There is also no simple untrained Qwen‑8B prompt‑rewriting baseline, making it difficult to quantify how much performance gain originates from RL optimization versus vanilla prompting.
  2. GEO is formulated as constrained optimization, yet it is solved by fixed‑weight scalarized reward rather than hard constraint. Faithfulness scores remain notably lower than original source, e.g. only 67.49 on E‑commerce benchmark. There is risk that visibility gains come at the cost of factual fidelity.
  3. Faithfulness evaluation fully relies on GPT‑4o‑mini, which is also the reward judge during training. There is no independent human or disjoint‑LLM validation to confirm scores reflect real‑world factual consistency, instead of judge‑specific biases.
  4. The paper provides no clear mechanism explanation for why a competing fidelity objective boosts visibility performance.
  5. It remains unclear whether RL‑produced trajectories are necessary for distilling high‑quality rewriting guidelines. Could guidelines extracted from high‑quality non‑RL rewrites (e.g., Claude‑generated candidates without RL training) achieve comparable performance? Proper control experiments are missing.
Questions for authors
  1. Could you add comparisons against closely‑related published GEO baselines including FeatGEO, Mind Reader, MAGEO and AgentGEO under matched settings? If full comparison is infeasible, please explicitly elaborate key differences that prevent direct apples‑to‑apples evaluation. Also add an untrained Qwen3‑8B prompt‑rewriting baseline to disentangle RL gains from vanilla prompting.
  2. Why does activating faithfulness reward bring extra visibility improvement on Researchy‑GEO? Please analyze whether this originates from optimization stability, reduction of harmful hallucinated edits, or other behavioral changes of rewritten content.
  3. The theoretical derivation proves that subtracting fixed source reward preserves expected gradient, but the practical advantage still applies group‑wise std normalization. Does this group‑dependent normalization keep the theoretical property intact? Please provide full derivation together with uncertainty analysis.
  4. For RAPO‑Distill: please conduct control experiment: extract rewriting guidelines from high‑quality non‑RL rewrites (e.g. Claude outputs without RL rollouts). Compare guideline quality & downstream rewriting performance against guidelines distilled from RAPO RL trajectories on same model backbone, to verify whether RL trajectories deliver tangible extra value.
  5. Please supply multi‑seed results and training curves for RAPO and GRPO under identical training budget, to verify whether source‑anchoring yields consistent, robust empirical advantages rather than single‑run fluctuation.
8 comments
AuthorsOpenReview ↗

Author Response for Reviewer RsvL (1/7)

We thank the reviewer for the detailed feedback and for recognizing RAPO's source-anchored design, its competition-aware faithfulness reward, and RAPO-Distill's practical value. Below, we add matched-setting comparisons against FeatGEO, MAGEO, and Agentic-GEO (W1+Q1), clarify the model-scale comparison against AutoGEO-Mini (W1), add an untrained prompt-only baseline to isolate RL's contribution (W1+Q1), address the E-commerce faithfulness concern (W2), add independent LLM-judge and human-study validation of faithfulness (W3), explain why faithfulness improves visibility up to αF=0.3 with a worked example (W4+Q2), clarify that the unbiased-estimator property was stated for the reward, not the post-normalization advantage (Q3), run the requested RL-vs-non-RL control experiment for RAPO-Distill (W5+Q4), and report multi-seed results (Q5). We hope these address the reviewer's concerns and would welcome any further questions.


W1 + Q1: Added comparisons against recent GEO baselines (FeatGEO, Mind Reader, MAGEO, AgentGEO)

W1 - "Several highly‑relevant recent GEO baselines are missing from main experiments: FeatGEO, Mind Reader, MAGEO, AgentGEO; some of them are originally evaluated on GEO‑Bench and are direct competitors."

Q1 - "Could you add comparisons against closely‑related published GEO baselines including FeatGEO, Mind Reader, MAGEO and AgentGEO under matched settings? If full comparison is infeasible, please explicitly elaborate key differences that prevent direct apples‑to‑apples evaluation."

To directly address this, we evaluated FeatGEO and MAGEO under the exact same GEO-Bench and Researchy-GEO protocol used for our main results (same context size N=5, same GPT-4o-mini generative engine, same fixed competitor set), reporting Word/Pos/Overall Visibility and Faith. alongside RAPO run under an identical setup; Agentic-GEO is already reported under this identical protocol in our Table 1 under RL-based trainable baselines and is reproduced below for completeness. For Mind Reader, we were unable to produce a matched-setting comparison, and we explain the concrete obstacle.

RAPO outperforms FeatGEO, MAGEO, and Agentic-GEO by a wide margin on visibility while being roughly 5–30 points higher on faithfulness. We report the numbers in the following table:

GEO-BenchResearchy-GEO
MethodWordPosOverall VisibilityFaith.WordPosOverall VisibilityFaith.
FeatGEO25.8726.1426.0557.1032.9132.6232.8454.60
MAGEO26.1826.3126.2455.1228.3127.9428.5050.32
Agentic-GEO23.0823.2623.8372.4026.4826.5126.5074.38
RAPO (Ours)28.9228.7328.8477.4535.5435.6135.5280.32

Even where FeatGEO and MAGEO deliver competitive raw visibility, RAPO still improves on them. On GEO-Bench, RAPO's Word and Pos scores exceed FeatGEO's by 3.1 and 2.6 points and MAGEO's by 2.7 and 2.4 points. On Researchy-GEO, RAPO leads FeatGEO, the strongest visibility baseline there, by a consistent 2.6–3.0 points across Word, Pos, and Overall Visibility (and MAGEO by 7.0–7.7). These visibility gains come with much larger faithfulness gains: FeatGEO and MAGEO remain at roughly 50–57 Faith. on both benchmarks, 20–30 points below RAPO (77.45 and 80.32), reinforcing that these baselines pursue visibility largely independent of faithfulness. FeatGEO trails RAPO by 2.8 points Overall Visibility on GEO-Bench and 2.7 on Researchy-GEO, and by 20.4 and 25.7 points in faithfulness. MAGEO trails RAPO by 2.6 points Overall Visibility on GEO-Bench and 7.0 on Researchy-GEO, and by 22.3 and 30.0 points in faithfulness. Agentic-GEO attains the highest faithfulness among baselines (72.40 / 74.38) but the lowest visibility, trailing RAPO by 5.0 and 9.0 points Overall Visibility and 5.1 and 5.9 points in Faith. on GEO-Bench and Researchy-GEO, consistent with our reward design explicitly weighting faithfulness during RL rather than leaving it as a side effect.

Answer to be continued ...

AuthorsOpenReview ↗

Author Response for Reviewer RsvL (2/7)

We could not run a matched-setting comparison with Mind Reader (Chen et al., 2026) because its code and artifacts are not publicly available, so we could not reproduce it under our GPT-4o-mini, competitor-controlled setting within the rebuttal window. Its reported numbers also cannot be compared directly with ours: its evaluation setting differs substantially from ours. It uses a Qwen3-30B generative search engine rather than GPT-4o-mini, and the absolute scores are far higher (Overall Visibility in the high-30s to high-50s even for its AutoGEO-API baseline, versus at most ~35.5 for any method in our Table 1, including RAPO). A gap this large for a baseline points to differences in benchmark construction and evaluation protocol, not just in method quality. We have added FeatGEO, MAGEO, and Mind Reader to Related Work (main paper) and included the FeatGEO and MAGEO comparison in Appendix A.4 (Table 9); Agentic-GEO is reported in Table 1.


W1: RAPO's gains hold at matched model scale, removing the scale confound

W1 - "Main experiments mix different model sizes across compared systems. Under matched model scale, RAPO's empirical gains shrink substantially. Without fully controlled comparisons, it is hard to robustly establish RAPO's real superiority over state‑of‑the‑art GEO approaches."

RAPO-1.7B and AutoGEO-Mini use the identical Qwen3-1.7B backbone (AutoGEO-Mini from Table 1, RAPO-1.7B from Table 6), so comparing them removes model size as a confound. Measured against the unoptimized source (Vanilla), RAPO improves both sides of the visibility/faithfulness trade-off on this same backbone. On visibility, RAPO raises overall visibility from 19.50 to 24.21 on GEO-Bench (+24.2% relative gain over Vanilla) and from 19.70 to 31.37 on Researchy-GEO (+59.2%), compared with +22.5% and +55.3% for AutoGEO-Mini. RAPO's relative visibility gain over the source is therefore higher than AutoGEO-Mini's on both benchmarks. At the same time, RAPO is substantially more faithful: it scores 75.90 and 75.71 on GEO-Bench and Researchy-GEO, versus only 63.27 and 61.38 for AutoGEO-Mini (+12.63 and +14.33 points). So at matched scale, RAPO's gains do not shrink relative to AutoGEO-Mini. It leads on both visibility and faithfulness.

We'd like to re-emphasize that faithfulness is an important aspect of GEO, so visibility gains should be compared in conjunction with impact on faithfulness. It measures the fraction of claims in a rewrite that are grounded in the original source, so a low score means the optimizer is injecting content the source never stated, such as invented statistics, dates, or named entities (see AutoGEO-Mini's fabricated bill number, governor, and wage detail in Appendix A.10, Tables 18-19). The edits that most raise citability (precise figures, named authorities, quotable claims) are the same ones that invite fabrication, so an optimizer rewarded for visibility alone can inflate its score by drifting from the source. Our αF ablation (Table 5) shows this directly, as raising αF to 0.5 buys faithfulness at the cost of Overall Visibility. RAPO thus achieves a larger visibility gain over the source while preserving considerably more of its content compared to AutoGEO-Mini at the same backbone scale. The larger RAPO models in Table 6 widen the visibility margin further (Qwen3-8B: 28.84 and 35.52 Overall Visibility, i.e. +47.9% and +80.3% over Vanilla), showing that the source-anchored advantage scales with model capacity.


W1 + Q1: Added an untrained prompt-only Qwen3-8B baseline isolating RL gains

W1 - "There is also no simple untrained Qwen‑8B prompt‑rewriting baseline, making it difficult to quantify how much performance gain originates from RL optimization versus vanilla prompting."

Q1 - "Also add an untrained Qwen3‑8B prompt‑rewriting baseline to disentangle RL gains from vanilla prompting."

To isolate how much of RAPO's improvement comes from RL training versus simply prompting a larger, untrained Qwen3-8B model, we ran a zero-shot "Base Qwen3-8B" rewriting baseline: no RL fine-tuning, same rewrite instruction prompt used to seed RAPO training, evaluated under the identical protocol. Untrained prompting alone recovers only a fraction of RAPO's gain and lags well behind in faithfulness. We report the numbers in the following table:

MethodGEO-Bench Overall VisibilityGEO-Bench Faith.Researchy-GEO Overall VisibilityResearchy-GEO Faith.
Vanilla19.5010019.70100
Base Qwen3-8B (untrained prompt)22.5866.8527.5164.45
RAPO (Ours)28.8477.4535.5280.32

Answer to be continued ...

AuthorsOpenReview ↗

Author Response to Reviewer RsvL (3/7)

Untrained Qwen3-8B prompting raises Overall Visibility over the original content by only 3.08 points on GEO-Bench and 7.81 on Researchy-GEO. RAPO raises it by 9.34 and 15.82 points over the same original content, so its total gain is roughly 3.0× and 2.0× that of prompting alone. Equivalently, RL training adds a further 6.26 and 8.01 Overall Visibility points beyond the untrained Qwen3-8B, which accounts for 67% and 51% of RAPO's total gain. The untrained model's faithfulness (66.85/64.45) is also 10.6 and 15.9 points below RAPO's (77.45/80.32), indicating that without the reference-anchored, faithfulness-aware reward, naive instruction-following by a larger model drifts further from the source rather than becoming more grounded in it. This confirms RAPO's gains are not simply an artifact of using a larger rewriter model, but stem from the RL optimization itself.


W2: Clarifying interpretation of the E-commerce faithfulness number

W2 - "GEO is formulated as constrained optimization, yet it is solved by fixed‑weight scalarized reward rather than hard constraint. Faithfulness scores remain notably lower than original source, e.g. only 67.49 on E‑commerce benchmark. There is risk that visibility gains come at the cost of factual fidelity."

On E-commerce, RAPO achieves both the highest Overall Visibility (26.27) and the highest faithfulness (67.49) among all GEO methods we compare. Vanilla, the unedited source, trivially scores 100 on faithfulness because it makes no edits, so it serves as a reference point rather than a competing method. The meaningful comparison is against other methods that rewrite the source for the same task. By this comparison, RAPO does not trade faithfulness for visibility: it leads on both. We report the E-commerce numbers from Table 1 in the following table:

MethodOverall Visibility ↑Faith. ↑
Vanilla19.12100
Technical Terms22.3466.13
Cite Sources22.9557.25
Keyword Stuffing20.8064.44
Unique Words22.9864.56
Authoritative22.3465.11
Easy-to-Understand21.8865.33
Statistics Addition20.8643.16
Quotation Addition21.7961.59
Fluency Optimization23.7167.06
AutoGEO-Mini22.7850.20
Agentic-GEO23.1761.38
RAPO (Ours)26.2767.49

Faithfulness of 100 is attained only by the unedited source, so any rewrite that improves visibility will score below it. The relevant question is whether RAPO's visibility gains come at a disproportionate fidelity cost. On E-commerce they do not. RAPO achieves the highest Overall Visibility (26.27), +7.15 over the unedited source and +2.56 over the next best method (Fluency Optimization, 23.71). Its faithfulness (67.49) is the highest among all methods that edit the source. Several baselines reach lower visibility at markedly lower faithfulness: Cite Sources 57.25, AutoGEO-Mini 50.20, Statistics Addition 43.16. RAPO therefore matches the most faithful baseline while delivering a clearly larger visibility gain, so its visibility gain is not bought with factual fidelity.


W3: Faithfulness scores validated across multiple LLM judges and a human study, ruling out judge circularity

W3 - "Faithfulness evaluation fully relies on GPT‑4o‑mini, which is also the reward judge during training. There is no independent human or disjoint‑LLM validation to confirm scores reflect real‑world factual consistency, instead of judge‑specific biases."

To address the concern that faithfulness scores reflect judge-specific bias rather than genuine factual consistency, we re-scored the rewrites of each method below with two independent LLM judges that play no role in RAPO's reward or training: GPT-5.4-mini and Claude-Sonnet-4.6, alongside the original GPT-4o-mini judge. Under every judge and on both benchmarks, RAPO has the highest faithfulness among these methods. We report the Faith. scores in the following table:

GEO-BenchResearchy-GEO
MethodGPT-4o-miniGPT-5.4-miniClaude-Sonnet-4.6GPT-4o-miniGPT-5.4-miniClaude-Sonnet-4.6
Cite Sources72.4361.6070.4372.9560.8271.44
Quotation Addition72.1567.3271.1872.3863.4770.92
AutoGEO-Mini63.2757.3762.4861.3855.4461.14
Agentic-GEO72.4068.1571.3274.3866.8772.84
RAPO (Ours)77.4573.4677.9280.3271.1879.47

Answer to be continued ...

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.