The Compliance Gap: Completion Claims Are Not Process Certificates

When a user mandates a procedure, such as reading each file individually or scanning and masking personal data before analysis, the assistant's completion report is often the only evidence a supervisor sees. We show when an observation can certify the procedure, and that the report's completion claim cannot. We record every session as a triple (instruction, completion report, logged tool trajectory) and score the correspondence between report and trajectory with fixed per-task predicates on the 1,980 sessions of the 2,031-session archive that have an observed trajectory. In the clearest record, GPT-4o was instructed to read 50 files one at a time with every batch and delegation route prohibited; in all 10 runs it issued one prohibited batch call, read no file individually, and reported all 50 verified, 7 times without disclosing the batch route, while none of the 5 planted errors was found. Across 618 explicitly instructed file-reading sessions on six models, 7.9% claim completion while the trajectory shows the procedure was not followed (from 0.0% to 22.1% by model), which is 10.9% of the 451 sessions that report completion, and 1.4% on the three other task types, whose session execution rates differ under task-specific prompts (cross-reference 99.3%; audit trail 0.0%, where 111 of 150 sessions make no tool call and none of the 14 others with a checkable record completes the chain); with no procedure instruction in the prompt, removing the batch tool raises individual reading from 0.0% to 74.7%. Two results say why: a text-only reward cannot separate trajectories that differ only in the procedure, and when compliant and non-compliant sessions produce the same completion claim, no judge restricted to that claim certifies compliance without error. Nine blinded raters reading text alone reach $\kappa$ = 0.13, and their majority identifies 0 of 15 compliant sessions. We release the 2,031-session archive, the scorer, and a one-command reproduction as a test bed on which any text-only certifier can be falsified.