- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper is about a side effect of generative engine optimization (GEO). When a source is rewritten to get cited more often by a generative search engine, it also tends to get cited for unrelated queries that the content owner never wanted to target. The authors call this off-target visibility. It is hard to remove because the target and non-target queries often rely on the same text, so simply deleting the optimized parts can hurt the target queries too. Also, the edits have to be made offline before any user query is known.
Their method, OFF-GEO, learns one shared, document-agnostic editing rule from engine feedback. Training data are documents paired with a desired query and a related undesired query. An LLM proposes candidate rules based on failure cases, and a constrained successive-rejects procedure picks the rule that keeps the most desired visibility while meeting a minimum reduction in undesired visibility. At inference, the learned rule is applied to new sources, with no test queries or engine feedback needed.
They evaluate on the AutoGEO E-commerce benchmark and on GEO-Bench. On E-commerce, undesired visibility drops by about 10.8 points while desired visibility drops by about 6.9, and on GEO-Bench the drops are about 7.2 and 6.3. Answer quality on desired queries stays close to the unpruned GEO baseline.
Critical strengths
- A clear formulation of the problem. Most GEO work only tries to push visibility up for the target queries. Treating off-target leakage as a constrained optimization (keep as much desired visibility as possible, subject to a minimum cut in undesired visibility) gives a clean way to talk about selective visibility control.
- No test-time queries or engine feedback. In a real publishing setting the content is edited long before anyone searches, so a rule that can be applied offline is practical. The learned rule is applied through plain text edits, and answer quality on the desired queries holds up on E-commerce. Note that it still needs an LLM editor call per source.
Critical weaknesses
- The baselines are weak and the method is never ablated. Table 1 only compares OFF-GEO with the original text and with AutoGEO_API, so I can't tell whether a simple alternative would do as well. I'd want to see easy baselines like zero-shot LLM pruning or length/similarity-based filtering. I'd also like to see a multi-query method such as IF-GEO adapted to penalize passages that trigger the non-target query. On top of that, the method has several moving parts: the learned rule, the selection procedure and a prompt-level "orthogonality guard" that throws away edits touching the source's core topic. The authors say the guard complements the rule, but never remove it. Without that, I can't tell whether the retention comes from the learned rule or just from the guard.
- The GEO-Bench margin is thin and the statistics are missing. On GEO-Bench the net gap is only about +0.9 points, and it costs more than a third of GEO's desired-visibility gain (39.8 down to 33.5). The paper doesn't give the number of train or test examples, and Table 1 has no standard deviations or significance tests, so with a stochastic LLM engine I can't tell a gap this small from noise. I'm also unsure about the filtering: sources with zero undesired visibility under AutoGEO_API are dropped. The filter doesn't depend on OFF-GEO, so it isn't unfair to it, but keeping only sources with high baseline leakage can make the measured reductions look bigger than they are because of regression to the mean.
Questions for authors
- How many train and test examples are there for each dataset, and what share of sources is removed by the zero-visibility filter?
- What happens to the results if the orthogonality guard is removed?
- Can you report standard errors or confidence intervals, and a paired significance test, for the GEO-Bench gap in Table 1?
- Can you compare against simple pruning baselines and an adapted IF-GEO?
- The threshold gamma is 0.02, but the observed reductions are 7 to 10 points. Does this constraint ever actually bind during selection?
- Pairs with cosine similarity above 0.6 are excluded, yet that is where passage conflicts should be strongest. How does OFF-GEO do in that range?
- Why is answer quality only evaluated on E-commerce and not on GEO-Bench?