SimDocBench: Benchmarking the Robustness of Document Parsing via Paper Realistic Simulation

Modern optical character recognition (OCR) and document parsing systems convert document images into machine-readable text and structured content, and now achieve strong performance on clean, scanned documents. However, their robustness to real-world physical capture remains poorly understood, as photographed documents often exhibit diverse geometric, photometric, and environmental variations that are not systematically evaluated by existing benchmarks. To address this gap, we introduce SimDocBench with a Source-Neutral-Perturbed protocol that separates rendering effects from acquisition-induced degradation. Using physically based thin-sheet simulation, path-traced rendering, and camera-effect modeling, SimDocBench constructs 158,496 single-factor renders covering 31 acquisition factors across 5 dimensions, together with Regular and Hard multi-factor settings. Rather than measuring parsing accuracy alone, SimDocBench evaluates three complementary properties of document parsers: accuracy under physical acquisition changes, consistency across content-preserving observations of the same document, and recoverability through controlled input restoration. Extensive experiments reveal that physical robustness is not simply an extension of clean-document accuracy: models with similar clean performance can exhibit substantially different degradation, particularly under challenging acquisition conditions. Moreover, accuracy degradation and prediction consistency capture distinct robustness behaviors, while controlled restoration further reveals heterogeneous failure mechanisms across acquisition factors and parsers. Together, these results show that physical robustness is a distinct and multi-dimensional capability largely invisible to clean-document benchmarks.