Optical Character Recognition (OCR) is regularly pitched as a solution that helps businesses achieve 100% straight through processing (STP) for their document workflows. And while this might be true on a technical level, there’s a problem with using straight through processing as a success metric for automation initiatives.
In this post we’ll break down what straight through processing is, and why it’s the wrong metric for measuring automation success. We’ll also provide an alternative metric for measuring the impact your automation initiatives have on your operations.
What is Straight Through Processing?
With regards to document processing, the goal of STP is to process documents with no human intervention. Once documents have been processed by a traditional OCR solution, error correction is handled by dedicated employees. Businesses frequently use STP to automate back-office functions. The goal of using STP is to reduce document processing times, minimize errors, and reduce expenses.
With STP, it’s possible to remove humans from the equation entirely, therefore reaching automation rates of 100%. This can be misleading, however, as automation rate doesn’t account for time spent handling exceptions—more on this later.
Why Business Leaders Think They Want Straight Through Processing
Naturally, the idea that you can be completely “hands off” with document processing is highly appealing. It makes sense to assume that pushing documents through a workflow without manually touching them will save time, and therefore, money.
Because many organizations process thousands of documents per day, manually processing every document can cause massive slowdowns. To combat process bottlenecks, businesses look to STP to keep document-related data accurate, secure, and easily accessible for decision making.
Unfortunately, when it comes to traditional automation initiatives like OCR, measuring straight through processing fails to show how the technology influences business performance.
Why Straight Through Processing Rate is the Wrong Metric
The trouble with using STP to measure OCR performance is that STP ignores key limitations inherent in OCR solutions. We’ve covered OCR at length elsewhere, but among the primary constraints, three are worth mentioning here.
- Lack of Context: Because OCR reads character-by-character, it lacks any sense of real-world context. Messy handwriting, scratched out text, and letters extending beyond boxes are constant realities on documents, but with traditional OCR, they introduce data inaccuracies.
- Complex to Implement & Scale: OCR is built on templates, and when a document layout changes, these templates break. They must be re-designed, creating a resource bottleneck in IT or Engineering. Documents that come in during this “down time” require a human exception path until a new template is created.
- No Error Handling: Traditional OCR is essentially a black-box solution. You can control which documents go in, but once they’re in, there’s no further interaction possible until they come out. When errors occur, they must be corrected later—there’s no way to do so within the solution.
Why do these shortcomings matter? When using STP as the key metric for automation success, there’s no regard for context, scale, or error handling—they’re all sacrificed in order to achieve higher levels of STP. It doesn’t take long to realize that ignoring the limitations of OCR for the sake of greater automation leads to inaccurate data, resource bottlenecks, and loss of revenue.
An Illustration: Where Straight Through Processing Falls Short
Say you’ve identified document processing as an efficiency bottleneck. Your goal is to reduce document processing time by a total of 10 hours per week—you have other mission critical areas that need attention, and you’d like to re-deploy some employees so they can focus on these responsibilities.
To save this time, you implement an OCR solution that delivers 100% STP. Initial results look promising—as expected, document processing is largely automated, leading you to believe that efficiency has dramatically increased.
Weeks later, you learn that it takes 10 employees two hours each day to correct the inaccuracies resulting from documents being forced through an automated queue. That’s half a week of work lost every week just to correct mistakes made by the solution that was supposed to make document processing take less time.
You’re surprised to hear a manager say that the process takes almost as long now as it did before you implemented the OCR solution—employees just don’t trust the application, so they spend extra time checking over documents, and they regularly find additional mistakes. You realize that even though you spent the money on a solution that provides 100% automation, the total time saved is negligible once all the limitations have been accounted for.
Measuring the effectiveness of technology (100% STP in this case) certainly isn’t a bad thing, but it’s far more important to measure and understand the impact a solution has on organizational goals.
According to Gartner® 1 2, “It’s typical to measure technology applications and tools to ensure that they are working as designed. However, this doesn’t reflect whether or not the project is successful. Measuring the impact on processes and the enterprise as a whole is key to the success of automation.”
Measuring What Matters: Average Handling Time
One of the primary goals of any document processing solution is to help businesses save time and effort. To calculate time saved, you wouldn’t look at straight through processing rates—you’d look at how long documents took to process before a solution was implemented, and how much time they take to process after a new solution is introduced. This is the concept of average handling time.
Average handling time is so useful because it takes into account all the limitations of the technology you use. When you’re measuring how long it takes to accomplish something, it doesn’t matter how much of the process is automated—if the total time spent on the entire process isn’t any faster, an STP rate of 100% offers no tangible value toward the goal of saving time.
Furthermore, if you wish to compare document processing solutions, pitting a traditional OCR solution that offers “100% straight through processing” against an intelligent document processing solution (IDP) that uses a human-in-the-loop process to prioritize accuracy (and therefore time savings later), using straight through processing to compare the two won’t result in a fair comparison.
Instead, the best metric to compare the two is average handling time—and the amount of effort needed to produce accurate results that benefit customers.
A Real-World Example
While working with a public sector organization, we found that their legacy OCR tool resulted in a 30% exception rate for a specific document type. To correct these exceptions, it took an average of 15 minutes per document. Extrapolating this time over the year equated to a cost of over $100k—just to fix the errors in this one document type.
Once this organization realized how much revenue was being lost due to the lengthy exception handling process, they decided to implement an IDP solution and chose Hyperscience for its ease-of-use and ability to accurately extract data from complex documents.
With the new ML-led solution in place, processing the same document type with a human-in-the-loop ensured the highest level of accuracy during the extraction, decreasing the costs by over 80% for the year.
While the first solution offered STP, it failed to account for the time needed to correct the errors. Switching to a solution that prioritized accuracy drastically reduced the average handling time of each document.
Prioritize Business Impact Over Technology Performance
Implementing an intelligent automation solution is a step taken to help achieve business goals, not to reach automation milestones. And while OCR can help with this, remember that 100% straight through processing rate isn’t the way to get there. Instead, increases in efficiency are better measured by time saved. Additionally, solutions that take different approaches must be compared on an equal footing. When it comes to evaluating success of document processing solutions, that footing is average document handling time.
To explore how an intelligent document processing solution can help your organization automate more effectively, watch our Hyperscience Platform demo where you’ll see how to classify and extract data across complex, difficult-to-read documents, including handwritten forms, PDFs, images, emails and more.
- Gartner Press Release, “10 Automation Mistakes to Avoid”, April 19, 2022.
- GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
