- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how structural information should be incorporated into retrieval-augmented generation (RAG) without unnecessarily disrupting reliable semantic rankings. The authors propose TRACER, a selective structural calibration framework that combines candidates from dense and structure-based retrieval and constructs compact candidate, structural, and task-level features.
TRACER employs a task-conditioned multi-head readout to extract candidate-specific structural signals. These signals are subsequently used to adjust a frozen base score through two complementary components: a bounded residual correction and a within-query reordering branch. The training objective combines ranking losses with regularization terms designed to discourage excessive deviations from the base ranking.
Experiments on LongMemEval, 2WikiMultihopQA, and MuSiQue demonstrate improvements in retrieval and downstream QA performance. The authors further provide matched candidate-pool comparisons, component ablations, structural-noise analysis, and runtime measurements.
Critical strengths
S1. A practically relevant formulation of structure-aware retrieval.
The paper addresses an important limitation of existing structure-enhanced RAG systems: structural connectivity does not necessarily imply relevance to the current query. Rather than treating structural scores as uniformly beneficial, TRACER formulates their integration as candidate-specific corrections to a semantic ranking. This perspective is meaningful, particularly for multi-hop retrieval and long-term memory applications where relational evidence may be necessary but structural noise can also be harmful.
S2. Strong empirical results with useful controlled analyses.
The reported retrieval and QA improvements are substantial, particularly on 2WikiMultihopQA. The authors make a valuable effort to separate improvements due to candidate coverage, adapted base scoring, and reranking. On the fixed candidate pool, TRACER improves average NDCG@5 from 71.07 for the matched QuDAR-style fusion control to 79.03. The component ablations and structural corruption experiments also provide useful evidence about the contributions and limitations of the proposed framework.
Critical weaknesses
W1. Insufficient validation of the proposed architecture's necessity.
The ablation results suggest that most improvements come from structural features and within-query reordering, while task-conditioned readout contributes relatively little. Although the matched fusion comparisons are useful, the paper lacks feature-matched supervised reranking baselines, such as MLP or LambdaMART. This makes it difficult to isolate the contribution of the proposed architecture beyond conventional learning-to-rank methods.
W2. Limited evidence for reliable selective calibration.
Although TRACER aims to preserve reliable base rankings, the within-query reordering branch is not explicitly bounded, and its strength is controlled by a global scalar rather than a query-specific gate. The performance degradation under severe structural corruption further raises concerns about robustness. More systematic evidence is needed to demonstrate that TRACER avoids unnecessary or harmful ranking changes.
Questions for authors
Q1. How much of TRACER's improvement is attributable to its proposed architecture rather than structural features and supervised reranking?
Table 5 suggests that structural features and within-query reordering account for most of the performance gains, while task conditioning and other calibration components contribute comparatively smaller improvements. Could the authors include feature-matched supervised learning-to-rank baselines, such as an MLP reranker or LambdaMART, using identical candidate pools, structural features, and training supervision? This would help clarify the incremental contribution of the proposed architecture beyond conventional supervised reranking.
Q2. How effectively does TRACER preserve reliable base rankings while correcting unreliable ones?
Could the authors provide a query-level analysis distinguishing cases where the frozen base ranking already contains all gold evidence within the top five from those where evidence is missing? In particular, how frequently does TRACER degrade an already correct ranking, and how effectively does it repair incorrect rankings with high base-score margins? An ablation of the confidence-weighted regularizer would also help establish its contribution to selective calibration.
Q3. How sensitive is TRACER to candidate-pool size and structural evidence coverage?
The main experiments use top-512 dense and structural candidate lists, with the structural pool achieving 100% gold-evidence coverage on 2WikiMultihopQA. How does TRACER perform with smaller candidate budgets, such as top-128 or top-256, or when structural evidence coverage is reduced? Do its advantages over matched fusion baselines and the relative contributions of its calibration components persist under these conditions?
Q4. To what extent are the dataset-specific feature representations transferable to new structural backends or domains?
Although TRACER uses a shared architecture, the feature-construction functions (F_\phi), (F_\psi), and (F_t) rely on dataset-specific structural statistics. Could the authors clarify which feature families are essential and which require backend-specific engineering? How much feature redesign or retraining would be required when applying TRACER to a new domain with a different structural retrieval pipeline