Where is the point of optimal efficiency and gain in invoice automation?
Lately, I’ve had numerous conversations with customers and peers about the efficiency levels of automation models - and what goal we should set for the automation rate.
Just to clarify, when I refer to a "model," I mean a mathematical representation that simplifies the world into a system of numbers and equations that the computer can understand.
"All models are wrong, but some are useful." This quote often attributed to George Box, resonates with me because it acknowledges that while models may fall short, they can still be helpful.
When evaluating a model, it’s not enough to check how accurate it is. We must also assess whether it overlooks important factors and if there are any associated costs. By "costs," I mean not just monetary value, but also other detriments, such as lead-time, speed, bias, or risk. A useful model produces significant value at a low cost.
When evaluating the performance of an automation setup, it’s essential to realize that, for example, if we have 80% automation rate, it doesn’t necessarily mean the automation fails 20% of the time. The automation may stop (up to) 20% of the time due to controls we’ve put in place. Essentially, we’ve made a deliberate choice to halt the model and let a human take over.
Achieving 100% automation with 100% control is very challenging (and costly) for any process. We can increase automation if we're willing to accept less-than-100% control. How far we go depends on the organization's overall risk appetite.
Before diving into risk-taking, let me return to those efficiency discussions. One recent conversation I had focused on order matching and achieving touchless matching.
Order matching—or any matching really—can be seen as a two-step process. First, there’s a coarse sorting to quickly isolate potential candidates (which reduces the overall search space), followed by a finer, more careful selection and validation from the narrowed down candidates to find the optimal solution.
In the context of our product, Basware Invoice Matching, we typically refer to these two steps as "association" and "matching." Since association is mainly about search efficiency, I’ll focus on the matching step, which is where discussions about touchless matching usually arise.
"How do we fine-tune this to achieve 100% touchless matching?" is a common way to phrase the question. However, this question fails to consider the cost of control.
As mentioned above, the matching step involves candidate selection and validation, and validation is another term for control. So, this question is really asking how much we're willing to relax validation to improve the match rate. The real question is: "How much risk are we willing to accept to achieve 100% touchless matching?"
With automation, we aim to replicate human behavior to save time and money. However, both humans and models make mistakes – mistakes we may not even notice unless we evaluate the results. If we don’t examine the outcomes, we might believe the execution was flawless.
What kind of mistakes can occur in AP invoice processing? We might select an incorrect candidate PO, leading to time loss and additional manual work (updating data and reprocessing invoices). We could have incorrect price data, resulting in increased or shifted costs on the P&L. Or we might have incorrect supplier data, preventing the correct PO from being identified, which again leads to time loss and manual work.
The costs of these mistakes typically involve time and effort to rectify, or the distortion of our financial visibility (unless we spend time and effort to rectify).
Validation helps minimize the risk of such issues. Relaxing validation will lead to more touchless matching, but how do we quantify the increased risk from reducing controls?
Risk assessment typically involves probability and impact. To evaluate the trade-off, we translate risk into a monetary value. Let’s keep it simple and set the probability to 100%, meaning the impact becomes the worst-case cost. If we also assign a monetary value to the benefit (the efficiency gain), we can assess the gain vs. risk trade-off.
One easy-to-understand validation control is amount tolerance—the match is permitted if the amount is within +/-X of the expected value (i.e. if the absolute difference in amounts is lower than or equal to X).
If we increase the value of X, we increase the number of successful matches (great, fewer manual validation cases!) but at the risk of higher-than-anticipated payment (not so great!). Note that "higher-than-anticipated" doesn’t necessarily mean "incorrect"; after further validation, it could turn out that our expected value was incorrect.
Let's run through a quick example for clarity. For simplicity, I’ll use rounded numbers.
In this simplified example, that translates to a 75% touchless match rate, leaving 250 invoices for manual validation. Upon comparing the amounts, we find the distribution is skewed—many of the 250 invoices have small differences, while only a few have large discrepancies.
From the graph, we can see that:
We could increase the amount difference (X) allowed, but doing so increases the total sum of differences (all the differences between invoice amount and expected amount). This total sum of differences is the impact. The impact represents the "higher-than-anticipated" amount we’re willing to accept in exchange for not manually validating all 250 invoices.
We can quantify the impact at different values of X. The table below presents the deviation amounts in the top row, the cumulative frequency (running count) in the second row, and the total cumulative sum of the differences in the third row.
Returning to the earlier question, "How much risk are you willing to accept to achieve 100% touchless matching?" To reach 100% touchless matching, we just need to set X to at least €5000. But that efficiency gain (removing another 250 manual validation cases) comes with the risk of €41067—1.6% of our total spend, which is likely too high. So, we must allow for some manual validation.
Which value of X should we choose?
There's no one-size-fits-all answer for the optimal value of X. It depends on how much risk (or loss of control) you’re willing to accept. In this example, values between 20-200 might be reasonable, depending on your risk appetite and the value of each reduction in manual validation.
So, which value should you choose? The one that best aligns with your CFO’s priorities.
In my experience, CFOs are happiest when they can:
When using automation, we must not overlook the control aspect. Remember that control (validation) and automation (cost reduction) exist in a cost/benefit relationship. Ignoring risks doesn't make them disappear—it just hides the true cost.
Want to talk about world-leading invoice automation? Contact us!
Explore our SmartPDF solution for achieving over 90% automation with balanced validation rules. A typical product where you can achieve well beyond 90% automation by smartly balancing validation rules: SmartPDF.