A Hybrid Approach for Key-Information Extraction from Structured Documents

Automated extraction of key information from structured document images is important for accounting, document management, and business automation, yet it remains difficult under blur, noise, mixed languages, and variable layouts. Although recent vision-language-model-based solutions have shown strong performance on document-understanding tasks, many of them rely on large models that require expensive hardware or cloud-based inference. In practical settings, however, document data may be sensitive, and some clients do not permit sending files to external servers, making fully local processing on affordable devices a necessary requirement. This work studies that problem through three extraction settings: a classical pipeline based on field detection, OCR, and post-processing; an OCR-free vision-language pipeline that generates JSON directly from the full document; and a hybrid pipeline that combines field localization with vision-language recognition on cropped regions. The proposed workflow includes orientation correction, field detection, recognition, normalization, and validation for producing machine-readable JSON outputs. Experiments were conducted under a unified schema and included comparisons at the detector, OCR, vision-language-model, and end-to-end levels. The results show that the hybrid pipeline achieved the best overall performance, with lower error rates, higher exact-match scores, and fully valid JSON outputs compared with the other evaluated settings. The study also showed that YOLO11s, Tesseract, and Qwen2-VL-2B-Instruct were the strongest components in their respective comparisons. These findings indicate that combining explicit localization with lightweight vision-language recognition on cropped regions provides a more accurate, practical, and resource-efficient solution for structured receipt understanding than either a fully classical or a fully OCR-free approach.