TraceQueue: Source-Grounded Triage Accelerates Paper–Repository Verification
Abstract
Between repository resolution and rerun execution, an auditor must turn one queue entry into a verified follow-up decision. TraceQueue makes this handoff measurable: resolved pairs share an ordered paper-plus-repository evidence frame, a -field disclosure contract, calibrated risk and abstention, field-level source pointers, and a temporal rerun ledger. With GPT-4.1, evidence, temperature, calibration folds, scalar request, and thresholds held fixed, replacing a free-form rationale by the complete contract yields ROC ( CI ), improves ECE by (), and increases explicit abstention by points (); 92.8% of audited Open Structured pointers identify their intended source. The decisive test is a blinded study of 126 pairs: the structured entry reduces median verification time from 17.8 to 13.6 minutes (23.6%, ), lowers false follow-up from 14.6% to 8.8%, and raises verified-issue recall by 20 minutes from 64.9% to 71.6%. Paper and table evidence adds holdout AUC over repository-only features and retains after maturity balancing; in a fresh -pair cohort, full-contract risk has adjusted odds ratio 1.62 for decision-changing verified issues. Cross-team and disjoint reruns preserve the score ordering. These results show that ranking quality alone can conceal a consequential system property: how efficiently a prediction becomes a verified expert decision.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.