acceptodds
Desk rejected at ICLR 2027

TraceQueue: Source-Grounded Triage Accelerates Paper–Repository Verification

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Between repository resolution and rerun execution, an auditor must turn one queue entry into a verified follow-up decision. TraceQueue makes this handoff measurable: resolved pairs share an ordered paper-plus-repository evidence frame, a -field disclosure contract, calibrated risk and abstention, field-level source pointers, and a temporal rerun ledger. With GPT-4.1, evidence, temperature, calibration folds, scalar request, and thresholds held fixed, replacing a free-form rationale by the complete contract yields ROC ( CI ), improves ECE by (), and increases explicit abstention by points (); 92.8% of audited Open Structured pointers identify their intended source. The decisive test is a blinded study of 126 pairs: the structured entry reduces median verification time from 17.8 to 13.6 minutes (23.6%, ), lowers false follow-up from 14.6% to 8.8%, and raises verified-issue recall by 20 minutes from 64.9% to 71.6%. Paper and table evidence adds holdout AUC over repository-only features and retains after maturity balancing; in a fresh -pair cohort, full-contract risk has adjusted odds ratio 1.62 for decision-changing verified issues. Cross-team and disjoint reruns preserve the score ordering. These results show that ranking quality alone can conceal a consequential system property: how efficiently a prediction becomes a verified expert decision.

Primary area:
datasets and benchmarks
Keywords:
evaluation benchmarks and data, TraceQueue

Related papers

Discussion (0)

Sign in to comment.