How to Increase Accuracy in Straight-Through Processing (STP) Rate in Document Intelligence Systems

Straight-Through Processing (STP) is the extent to which document-driven processes can be fulfilled end-to-end without human intervention and thus is a clear-cut metric of whether automation is effective in document intelligence systems. Despite the new developments in optical character recognition, natural language processing, and machine learning which have greatly enhanced component-level performance, the actual performance of STP in practice has steadily remained low. This error occurs due to the accrual of errors in the processing pipeline, such as document heterogeneity and semantic ambiguity, layout variability, confidence miscalibration and strict validation rules resulting in unnecessary exceptions. The issue of low STP accuracy in this paper is approached on the system level instead of from the model-centric perspective. It finds a definite research gap in the literature, in which the assessment of improvement usually occurs separately at the extraction or classification point, without providing sufficient consideration to the error propagation along the entire document intelligence pipeline. To fill this gap, the paper suggests an analytical framework that separates STP into the various stages of ingestion, classification, data extraction, validation, and exception handling and explores how design decisions at each phase affect the overall automation performance. The analysis shows that strong improvements in the accuracy of the STP cannot be made by simple improvements in each model but by coordinating improvements which are normalisation of the input quality, use of context to improve classification, use of confidence to extract, adaptive and validation, and exception routing. The results determine that STP accuracy is an ultimate pipeline optimisation issue, which needs holistic design and assessment approaches. This paper adds a systematic background for further studies and realistic system architecture to meet the goal of reliability and high accuracy in straight-through document processing in document intelligence systems.