- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ARMOR improves the query encoder in a telecom RAG system while leaving the document encoder, index, and answer model unchanged. It combines RAG likelihood and InfoNCE, learns a temperature for each loss, and penalizes query embeddings that move away from the original encoder's outputs. The experiments cover three telecom domains, and a capacity argument is used to motivate retriever tuning instead of generator tuning.
Critical strengths
- Updating only the query encoder avoids rebuilding the document index, and holding out source documents gives a useful test of whether the update generalizes (Sections 2.1 and 5.1).
- Table 2 shows that learned temperatures and query distillation work better together than either change alone on ISAC, which gives a concrete reason to study the combination.
Critical weaknesses
- The theory uses dS and dR as both parameter counts and sequence lengths, so I cannot see how it supports the claim that fewer trainable parameters give retriever tuning a better bound (Section 2.2; Appendix E.2).
- The answer-score changes from Base RAG are small (+0.0226, −0.0044, and +0.0025 on ISAC, JCC, and SAGIN), and without uncertainty estimates it is unclear whether these changes would persist across training runs or test samples (Table 1).
Questions for authors
- Are dS and dR parameter counts or sequence lengths, and how does the fourfold parameter reduction relate to the bound after this notation is corrected?
- Could you provide the training settings and uncertainty estimates needed to assess the small answer-score differences?
- How does ARMOR compare with tuned fixed temperatures and generator tuning on the same test documents?