- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses the topic of Generative Engine Optimization (GEO), the task of rewriting documents to improve their citation within LLM generative‑RAG responses. Existing GEO approaches suffer from two drawbacks: 1) insufficient coverage of less prominent information needs. 2) overlook the structure of evidence in the docment. To mitigate both issues, the authors propose StructGEO. It first builds a top‑down intent tree: coarse‑grained parent intents are recursively decomposed into fine‑grained leaf‑level intents, to avoid intent concentration. Then it performs structure‑aware document rewriting: leaf rewriting, section merging, and doucment merging, to produce the complete rewritten article. Experiments on GEO-Bench and E-Commerce show that StructGEO outperforms existing methods, like AutoGEO, and IF-GEO.
Critical strengths
- This paper has two motivations. Both of them are not purely intuitive, and they provide quantitative evidence for query concentration and evidence dispersion in Fig. 2.
- In comparison with other baselines, the method achieves substantial improvements across multiple LLM backbones and two datasets. Therefore, the effectiveness of this method is well‑supported with statistical significance.
- The experiments exhibit sound fairness control, with dedicated analyses covering query budget, LLM‑call budget, and cross‑backbone bias.
Critical weaknesses
- The quality, fidelity and coverage of the intent trees are not directly evaluated. StructGEO fully relies on LLMs to automatically generate parent and child intents, yet the paper lacks either human annotations or automated metrics to measure critical properties: whether the intents are genuinely askable, whether they are grounded in the source document, whether sibling nodes contain redundant intents, and whether key aspects are omitted.
- The systematic evaluation of rewrited documents just relies on a LLM‑judge. Human fact verification are absent. This evaluation can measure superficial quality, yet it cannot reliably prove that rewriting does not introduce hallucinations, distort qualifying conditions, or alter numerical values.
- Hyperparameter sensitivity analysis is inadequate. The intent tree serves as the starting point of the proposed method and determines downstream evidence allocation and rewriting merging. The paper does not provide sufficient ablation studies on intent-tree hyperparameters such as tree depth, number of child nodes, and similarity threshold between evidence and nodes.
Questions for authors
- Can the authors provide a matched flat-intent baseline with the same number of leaf queries, retrieval calls, adaptive-retrieval calls, rewrite calls, and final merge procedure as StructGEO? This is needed to isolate the contribution of the parent-child hierarchy itself.
- Can the authors provide some hyperparameter sensitivity experiments over the number of parent intents, tree depth, number of leaves per parent node, retrieval top-k, and threshold τ?
5 comments
Q1. Contribution of Hierarchical Intent Modeling
To better isolate the contribution of hierarchical intent modeling, we introduced two controlled variants using the StructGEO rewriting framework on E-Commerce (Section 6.3, Fig. 4).
| Method | Word & Pos | Query Coverage (%) | Word & Pos Gain (%) | Coverage Gain (%) |
|---|---|---|---|---|
| Topic Tree | 34.48 | 10.82 | +19.8 | +86.6 |
| Flat Queries | 37.99 | 15.87 | +8.7 | +27.2 |
| StructGEO | 41.30 | 20.19 | - | - |
- Flat Queries: We applied the StructGEO rewriting framework to 15 independently generated IF-GEO queries, treating them as direct children of the root node. StructGEO achieves 8.7% higher Word & Pos and 27.2% higher real user query coverage than this variant.
- Topic Tree: We constructed a three-level hierarchy from document topics rather than user search intents, matching StructGEO's tree configuration. StructGEO achieves 19.8% higher Word & Pos and 86.6% higher query coverage than this variant.
These results suggest that StructGEO's improvements stem not merely from additional rewriting or document formatting, but also from hierarchically modeling user information needs.
W3, Q2. Hyperparameter Sensitivity
We conducted additional sensitivity experiments on five hyperparameters: the number of parent intents, tree depth, number of leaf nodes per parent, retrieval top-k, and evidence similarity threshold τ (Appendix E, Fig. 9). Results are Word & Pos scores on E-Commerce using Gemini-3.1 Flash-Lite.
| Leaves per parent | 1 | 3 | 5 | 7 | 9 | 11 |
|---|---|---|---|---|---|---|
| Word & Pos | 37.76 | 41.22 | 41.51 | 42.80 | 42.20 | 43.15 |
| Tree structure | 1×15 | 3×5 | 5×3 | 3×3×2 |
|---|---|---|---|---|
| Word & Pos | 40.25 | 41.51 | 41.36 | 40.81 |
| Retrieval top-k | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| Word & Pos | 44.31 | 42.48 | 41.51 | 40.39 | 39.18 |
| Threshold τ | 0.2 | 0.4 | 0.6 |
|---|---|---|---|
| Word & Pos | 41.51 | 42.61 | 40.86 |
The results show that StructGEO consistently outperforms the competing GEO methods across all tested configurations, demonstrating its robustness to hyperparameter variations. While performance varies across settings, the default configuration provides substantial visibility gains with a practical optimization budget.
Thank you again for your thoughtful and constructive review. Your valuable feedback has helped us strengthen our work, and we have conducted additional experiments to address your concerns.
As the author-reviewer discussion period approaches its end, we would like to briefly summarize our responses for your consideration.
-
W1. Intent Tree Quality, Fidelity, and Coverage: StructGEO achieves the highest real user query coverage while demonstrating strong source grounding, negligible sibling redundancy, and substantial information preservation.
-
W2. Factual Faithfulness and Human Verification: StructGEO achieves 88.6% faithfulness, and human verification of 200 claims shows that the LLM judge agrees with two annotators at rates of 77.5% and 74.0%, respectively.
-
W3, Q2. Hyperparameter Sensitivity: Our additional sensitivity experiments show that StructGEO consistently outperforms competing GEO methods across all tested hyperparameter settings.
-
Q1. Contribution of Hierarchical Intent Modeling: Using the same rewriting pipeline, StructGEO outperforms Flat Queries and Topic Tree by 8.7% and 19.8%, respectively.
If our responses and additional experiments have sufficiently addressed your concerns, we would sincerely appreciate it if you would consider reassessing your rating. If you have any remaining concerns or questions, we would be happy to address them before the discussion period ends.
Thank you again for your time and thoughtful feedback.
We sincerely thank the reviewer for the thoughtful and constructive feedback. We hope our response alleviates your concerns and further reinforces the key contribution of our approach.
W1. Intent Tree Quality, Fidelity, and Coverage
We appreciate the reviewer's valuable feedback. To address these concerns, we conducted additional evaluations and experiments. Our detailed responses are provided below.
1. Quality (Real User Query Coverage)
To evaluate whether hierarchical intent modeling better captures actual user information needs, we conducted an additional experiment comparing the coverage of real user queries by flat and hierarchical query generation.
We first extract information requirements from each real user query, then evaluate whether each generated query satisfies them. A real user query is fully covered if at least one generated query satisfies all its requirements.
StructGEO achieves relative coverage improvements of 10.83% on GEO-Bench and 27.22% on E-Commerce over Flat Queries.
2. Fidelity (Source Grounding)
To evaluate fidelity (source grounding), we assess whether the generated search intents are grounded in their source documents through semantic similarity. Specifically, we use each intent as a retrieval query to rank five candidate documents, including the original target document, using Qwen3-Embedding-8B. The target document achieves average retrieval ranks of 1.444 on GEO-Bench and 1.538 on E-Commerce, indicating strong semantic relevance between the generated intents and their source documents.
3. Document Coverage
We additionally use an LLM judge using GPT-4o mini to evaluate how well the generated intents cover the original document's content. Specifically, the judge assesses whether information in the original document can meaningfully contribute to answering the generated intents. The resulting coverage rates are 90.05% and 92.07% on GEO-Bench, and 87.01% and 89.09% on E-Commerce, using GPT-5 mini and Gemini-3.1 Flash-Lite, respectively. These results suggest that the generated intents meaningfully cover a substantial proportion of the source content.
4. Sibling Redundancy
We evaluated sibling intent redundancy in intent trees generated by GPT-5 mini, using GPT-4o mini as the LLM judge. Two sibling intents are considered redundant only if they request the same information with identical scope, regardless of differences in wording. The redundancy rates are 0.16% on GEO-Bench and 0.29% on E-Commerce. These results suggest that most sibling intents represent distinct information needs.
5. Coverage of Key Aspects
To evaluate whether key information is preserved in the final rewritten documents, we conducted a claim-level evaluation following FActScore. StructGEO achieves 88.4% preservation. These results suggest that StructGEO largely preserves key source information despite prioritizing user-relevant search intents.
W2. Factual Faithfulness and Human Verification
We appreciate the reviewer's important concern. To complement our LLM-based quality evaluation, we further assessed potential hallucinations. We (1) conducted an LLM-based, claim-level faithfulness evaluation of rewritten documents against their sources and (2) validated the LLM judge's assessments through human annotation.
Faithfulness evaluation. We conducted an atomic claim-level evaluation, as in FActScore [1], on the E-Commerce dataset, using GPT-5 mini as the optimizer and GPT-4o mini as the judge. An LLM extracts atomic claims from each rewrite, and the judge checks whether each claim is supported by the original.
StructGEO achieves the highest faithfulness (88.6%), as each stage is grounded in the original source evidence.
Human verification. To assess the reliability of our automated factual consistency evaluation, two human annotators independently classified 200 claims as either "supported" or "unsupported." The LLM judge achieved agreement rates of 77.5% and 74.0% with the two annotators, respectively, compared to an inter-annotator agreement of 86.5%. These results indicate a degree of consistency between the LLM judge's assessments and human judgments.