- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
In this work, the authors look at generative engine optimization as a repeated game between content suppliers, who rewrite their documents to be cited more by LLM answer engines, and a platform that defends against such rewrites. The authors model the interaction as a repeated Stackelberg game with partial monitoring. The proposed mechanism, VCR, adds a reward channel on top of suspicion-based defense. Experiments cover three benchmarks (E-commerce, GEO-Bench, Researchy-GEO), three answer engines and five attackers, with five rounds per game. The main metric is Net = Def + Welf. VCR has the highest Net in all nine engine/dataset settings, 12.1 points above the strongest baseline on average.
Critical strengths
These are the primary strengths of the work:
I. The problem is timely and the framing is sensible.
II. Most GEO work treats the supplier side as one-shot optimization. Modeling the platform and supplier as adapting to each other over rounds, is a useful point for both the GEO and the RAG-robustness communities.
III. The idea of rewarding something the platform can verify from document versions, rather than only filtering, is a reasonable design direction and connects naturally to the strategic classification literature that separates gaming from genuine improvement.
IV. The experimental coverage is broad for this area where the authors use three datasets, three answer engines, five different attackers, per-round results, and an attempt to test whether VCR can be gamed through rule disclosure or fabricated supporting pages.
V. The qualitative rule trajectories (Tables 3, 9, 10) are interesting, and they give some insight into how supplier strategies change under the different defenses.
Critical weaknesses
Given the strengths of the work, I still think there are some weaknesses that needs to be addressed:
I. I believe that the main metric proposed by the author does not measure what the paper claims. By the authors' own simplification in App. E,
When the attack raises target visibility (), Net is strictly increasing in , the target's visibility after defense. So under this metric the best "defense" is the one that keeps attacked documents as visible as possible. Applying no defense () gives Net , which is simply the attack's lift, and any mechanism that boosts the rewritten targets further would score higher still. The "no defense" row is not reported in Table 1, which makes the problem hard to see.
II. Because and are recomputed per arm and per round, Net also depends on how effective the attack is in each arm. Backing these ratios out of Table 1 (Gemini, E-commerce), in the VCR arm but only about in the Prompt arm. Most of VCR's Net advantage therefore seems to come from the supplier's rewrites being more visible in VCR's arm, not from better defense. Since VCR directly rewards rewrites with supported content and moves them earlier in the context, this is partly the mechanism scoring itself.
III. The "win-win" claim also rests on a ±5-point "operational tolerance". VCR's Welf is negative in 8 of the 9 settings, and the full-VCR ablation has Welf , outside the band. The only metrics that relate directly to answer quality (Table 2, Table 7, GEU) show small differences (e.g., answer average .520 vs .506–.515), come from a single gpt-4o-mini judge, and have no confidence intervals.
IV. The statistical evidence and the theory are both weaker than presented where App. E says prior repeated runs had a standard deviation of about 5 points, yet each main result is a single pipeline run, and the intervals in Table 4 come only from bootstrapping over queries.
V. Please add a no-defense row to Table 1. Given the identity Net , how do you justify Net as a defense metric when it increases with the defended visibility of attacked documents? Would the conclusions hold under a metric that rewards defense directly, such as the fraction of attack-induced lift removed, , which App. E mentions but does not report?
VI. The reward checks support against the previous version of the document, and the supplier controls that version. What happens under a simple two-step attack, where the supplier first publishes fabricated specifics and then "surfaces" them in the next version?
Questions for authors
Please see the weaknesses above for the questions to address.
3 comments
Rebuttal (Part-2)
pi9f-WIV
Concern: Does query bootstrap capture variability from relearning rules and regenerating revisions, and how strong are the theoretical claims?
Response:
-
We repeated the full pipeline: We agree that question bootstrap alone cannot capture variation from learning rules and generating revisions. We added three independent five-round runs, regenerating revisions and relearning rules. The 12 test questions, six separate training questions, and initial targets were held fixed.
Method Fifth-round target citation share, mean of three runs Across-run standard deviation No defense 49.25% 9.60 % Prompt 43.42% 5.69 % Hard reject 32.29% 9.91 % Keyword scrub 43.82% 11.05 % VCR 46.95% 7.66 % -
We report distinct uncertainty sources: We report pipeline variation, question sampling, and call failures separately because each analysis addresses a different source of uncertainty.
Uncertainty Reported analysis Interpretation Relearning and generation Individual results, sample standard deviations, and paired t intervals across three runs Pipeline variation with questions and target selection fixed Question composition Paired bootstrap clustered by question Conditional on the three realized runs Call failure Full-schedule results and completed-case sensitivity analysis Avoid interpreting failure-induced zero exposure as defense success
Changes: Added independent pipeline reruns and failure sensitivity in Section 5.3 and Appendices J.2 and K.3; the theoretical conditions are stated in Appendices F.2 and G.
pi9f-WV / Q
Concern: Do the conclusions survive a no-defense baseline and a measure of attack-gain removal?
Response:
-
The metric comparison narrows the conclusion: Following your suggestion, we added no defense and computed incremental exposure removal,
(ga−gd)/(ga−g0). Here, g0, ga, and gd denote original undefended, revised undefended, and post-defense exposure; small or nonpositive attack gains make the ratio unreliable.Answer model, original label Dataset No defense Prompt† Hard reject Keyword scrub VCR Gemini-2.5-Flash-Lite E-commerce 0.0% 399.8% No positive gain No positive gain 88.6% Gemini-2.5-Flash-Lite GEO-Bench 0.0% No positive gain 3530.9% -100.2% 136.9% Gemini-2.5-Flash-Lite Researchy-GEO 0.0% 646.6% 4599.1% 48.4% 124.1% GPT-4o-mini E-commerce 0.0% No positive gain ≈1771.8% ≈-269.6% ≈111.0% GPT-4o-mini GEO-Bench 0.0% No positive gain No positive gain No positive gain ≈184.9% GPT-4o-mini Researchy-GEO 0.0% Near-zero gain ≈-2061.8% ≈-2565.0% ≈148.9% Claude-Haiku-4.5 E-commerce 0.0% No positive gain No positive gain ≈27.0% ≈109.7% Claude-Haiku-4.5 GEO-Bench 0.0% ≈-3519.0% No positive gain ≈-77.6% ≈114.0% Claude-Haiku-4.5 Researchy-GEO 0.0% No positive gain Near-zero gain ≈230.5% ≈106.5% Answer model VCR exposure-change range Hard reject exposure-change range Gemini-2.5-Flash-Lite −3.3% to +2.2% −63.3% to −33.2% GPT-4o-mini −4.7% to −1.2% −67.5% to +20.0% Claude-Haiku-4.5 −4.8% to −1.7% −85.7% to −5.6% ≈ denotes rounded inputs; † retains original labels whose run identities remain unresolved. With separately generated revisions, VCR stays closer to original exposure than Hard reject, but this does not isolate defense strength on identical content.
-
Shared documents clarify the tradeoff: We applied the same measures to the shared-document control in WII, where all 900 scheduled answers completed.
Method on shared documents Original, undefended g0 Revised, undefended ga Post-defense gd Incremental exposure removal Change from original No defense 35.40% 50.40% 50.40% 0.00% +15.00% Prompt 35.40% 50.40% 39.56% 72.28% +4.16% Hard reject 35.40% 50.40% 33.11% 115.26% −2.29% Keyword scrub 35.40% 50.40% 39.25% 74.34% +3.85% VCR 35.40% 50.40% 45.15% 34.98% +9.76 % VCR weighted-exposure difference from baseline Mean Paired 95% interval across three runs No defense −5.25% [−11.60, 1.10] Prompt +5.60% [−8.81, 20.00] Hard reject +12.05% [4.37, 19.72] Keyword scrub +5.90% [−5.70, 17.51] VCR retains more exposure than Hard reject, while Hard reject brings exposure closer to the original level. Only the VCR–Hard reject retention difference has a three-run interval excluding zero. These shared-document answers lack independent quality assessment, so retained exposure alone does not establish reader benefit.
Rebuttal (Part-3)
pi9f-WVI / Q
Concern: Can an attacker first publish false details and later highlight them to obtain historical rewards?
Response:
-
Historical support alone can reward falsehoods: We agree that a false claim planted in an earlier version can appear historically supported when highlighted later. We tested this with a Pro attacker across three publications, using disclosed rules and feedback, and complete, incomplete, or missing trusted benchmark references.
Trustworthy reference Original VCR: ever wrongfully rewarded over three rounds With trustworthiness checks Final-round legitimate reward rate: original VCR → combined method Complete 11.1%–18.5% 0.0%–7.4% 25.0%→50.0% Incomplete 11.1%–22.2% 0.0%–11.1% 0.0%→0.0% Missing 14.8%–29.6% 0.0%–3.7% 11.1%→0.0% Trustworthiness checks reduce wrongful rewards, but missing references can also eliminate legitimate credit. The reported ranges bound unknown outcomes and are not confidence intervals.
-
Reward checks can also reduce accuracy: We also checked whether blocking wrongful rewards harms answers by removing useful content. With incomplete references, 23 of 27 paired attack cases completed; accuracy was 17.4 percentage points below penalty-only (95% interval: [−31.8, −4.2]).
Reference Method Answer accuracy Targeted wrong-answer rate Complete Penalty only 76.0% 0.0% Complete Original VCR-style reward 68.0% 0.0% Complete Trustworthiness checks + VCR 76.0% 0.0% Incomplete Penalty only 65.4% 3.8% Incomplete Original VCR-style reward 62.5% 4.2% Incomplete Trustworthiness checks + VCR 54.2% 0.0% Missing Penalty only 65.4% 3.8% Missing Original VCR-style reward 69.6% 0.0% Missing Trustworthiness checks + VCR 69.2% 3.8% These paired cases differ from the pooled final-round results above. Correct document D2 was removed because missing support was treated as contradiction; withholding unsupported rewards should preserve otherwise usable content.
Changes: Added adaptive false-history tests and separated reward eligibility from content filtering in Section 5.4 and Appendices J.4 and K.5.
Rebuttal (Part-1)
Thank the reviewer for the insightful suggestions. We will answer the question as follows.
pi9f-WI
Concern: Can Net rank no defense above VCR because it increases with post-defense target exposure?
Response:
Net does not establish defense superiority: We agree that when the attack gain is positive, the Net metric increases with post-defense target exposure. To test whether Net ranks suppression appropriately, we compared VCR with no defense on the same revisions.
The no-defense baseline scores higher despite removing none of the added exposure. We now interpret Net as an exposure composite and report exposure removal and independent quality separately.
Changes: Added attack-lift removal alongside Net in Section 5.1 and Appendix K.2.
pi9f-WII
Concern: Do different supplier revisions confound the comparison, making Net reward stronger attacks rather than better defense?
Response:
Shared documents control for supplier revisions: We agree that different supplier revisions confound the original comparison. We therefore replayed the same final documents under all five methods to compare their treatment of identical content.
VCR retains more exposure than Hard reject on these shared documents, so this difference is not solely due to different revisions. Its differences from Prompt and Keyword scrub remain uncertain.
Matched history tests the reward: The shared-document control still leaves the effect of access to history unresolved. We therefore added a separate control with identical historical information and no ranking reward.
With this information held fixed, the reward increases citations with Flash-Lite, while the Pro estimate remains uncertain.
Changes: Added shared-document and equal-history controls in Section 5.2 and Appendices J.1–J.2.
pi9f-WIII
Concern: Can the ±5-point tolerance band support a win–win claim without uncertainty estimates for quality differences?
Response:
Quality claims now include intervals: We agree that the ±5-point tolerance band does not establish equivalence or a win-win outcome. We therefore recomputed paired, question-clustered 95% intervals for document-score differences on a 0–100 scale.
The intervals support all three E-commerce gains, while two GEO-Bench comparisons remain uncertain.
Answer quality requires a separate evaluation: Better documents need not produce better answers, so we had Pro and DeepSeek blindly score the same 384 answers. We report 95% intervals clustered by question.
Neither judge finds consistent VCR improvement over penalty-only. Both rate Pro-generated answers below no defense, so the revised interpretation separates document gains from answer-quality benefits.
Changes: Added paired document-quality intervals and independent answer scoring in Section 5.2 and Appendices J.3 and K.4.