- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper mainly introduces two components. First, Polygraph is a gradient-boosted detector using 16 statistics and the second is Veritas, which is PPO loop using detector as a reward model. The hallucination that no confidence-based detector motivates the changing in the generation policy. In the AI-coaching conversations dataset, the detector accuracy is 85.5% and a reduction from 12.3% to 8.0% in the stated hallucination rate.
Critical strengths
1.The research direction is useful, which is combining factual monitoring and policy adaptation can be used in conversational agent depolyment, instead of relying on improving the generator. 2. The proposed detector implementation is sound, including feature groups in use, and deployment details. 3. Besides the main experiments, this paper also performs feature ablations, difficult-case analysis, and an external FActScore audit to further valid the components of the methods.
Critical weaknesses
-
The hallucination rate is defined as the false-negative rate on the FAIL class from Section 6, and this does not measure how often the generator can cause errors. So reducing this quantity from 12.3% to 8.0% does not demonstrate improved generation correctness. Also, the summation of Hall% and Rec does not equal to 1, given hallucination rate definition, is a confusion.
-
The setting has drawbacks. Only judge-flagged examples receive human review. Unflagged judge errors may therefore enter the reference labels as correct, potentially biasing both training and comparisons against those judges.
-
On experiment perspective, the RL evidence comes from one run and one domain, and the authors does not include baseline comparisons, such as DPO. A controlled PPO comparison with and without the CF penalty is especially important to add.
Questions for authors
In the autonomous retraining mechanism, an unlabeled MMD drift test can detect distribution change, but it does not supply factual labels for retraining the supervised reward model. Could you explain where the label comes from?