- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces ProvenanceLab, a longitudinal, externally auditable corpus of top LLM outputs (Claude, GPT-5.6, and Gemini ) collected across EU AI Act deployment boundary. It shows each record carries cryptographic integrity hashing. They also validate on a controlled watermark on/off calibration experiment and it shows predefined features related to vocabulary diversity and bigram diversity constitute of watermarking, which offers a reproducible "on/off" effect.
Critical strengths
-
Given 2026 amid the implementation of the EU AI Act’s synthetic-content marking requirements, Paper study an pratical time-sensitive problem.
-
ProvenanceLab shows a carefully designed, multi-layered structure, encompassing logs from service providers or the cloud, cryptographic hashes, RFC 3161-compliant timestamps, and "write-once, immutable" storage.
-
It designed the validation experiments to show reproducible on/off effect.
Critical weaknesses
-
Although the watermark is related to EU requirement, but not every country has this requirement. The motivation is not enough.
-
it uses a small open model (Gemma-2B-IT). Not sure scalability. It seems that paper's analysis of commercial models is limited purely to registry and collection. No direct watermark testing was performed on them.
-
Since watermark have been extensive researched, there are many anti-watermark strategy. We don't see many strong baseline.
-
With the addition of watermarking, the diversity of bigrams increases from 0.831 to 0.856; this does not imply that the model’s performance has worsened or its reliability has decreased.
-
For the datasets. We need to know more statistics about prompts, including question category and prompt length.
Questions for authors
-
Given the randomness of LLM, do we need to worry about the gap with and without watermark if the qualities are similar?
-
If EU requirement is a motivation, does it consdier different langugage?