Horses Not Zebras: The Implications of Prior Probability
By: Josh Johnston, Data Scientist at Kount
You just tested positive for a rare disease. The test is 99% accurate, so you are almost certainly sick, right? Actually, you are almost certainly healthy. Read on to journey through some numbers to see why, and in the process learn how fraud is costing you money, even if your chargeback rate is under control.
99% accuracy seems like a sure thing. If you test 10,000 people, just 100 of their results will be wrong. Now I said the disease is rare, so maybe 10 in 10,000 people have it. Imagine a cross section of the population is tested: 10,000 healthy and 10 sick people. With 99% accuracy, all 10 sick people should be correctly identified. As for the healthy people, 1% will be wrongly told they are sick. 100 healthy people and 10 sick people will receive a positive result. That means 10 times as many healthy people than sick people got the bad news!
The odds are strongly in your favor that you are one of these 100 healthy people rather than one of the 10 unlucky ones. While 99% is high, it isn’t as high as the percentage of healthy people: 99.9%. In fact, we could make a test that just tells everyone they are healthy and it would be right 99.9% of the time. It would also be useless.
The lesson is the importance of prior probability in pattern recognition. When your Data Scientist wants to feel smug, he’ll use the Latin a priori probability. Prior probability means the chance a hypothesis is true before you’ve measured anything. In our example, the prior probability you are healthy is 99.9% before we’ve run the test. After testing positive for disease, your posteriori probability of being healthy is still about 90%.
But artificial intelligence is just the practice of understanding and duplicating the intelligence we humans already have. In other words, the math might be new (and much, much faster), but the concept isn’t. I’m sure you’ve heard the saying: “When you hear hoofbeats, think of horses not zebras.”
The third party fraud attempt rate for many e-commerce retailers is around 1%. Through aggressively reviewing and declining orders, you may reduce this to achieve a chargeback rate of 0.1%, or 10 in 10,000. Like the healthy people in the example above, however, you may be declining ten, one hundred, or even more good orders for every fraudulent one you stop. Even with a 99% accuracy rate, 90% of what you decline may be good orders.
Chargebacks and fees are not the only cost of fraud. Reviews cost time, money, shipping delays, and order abandonment. Declining a legitimate order may lose you the entire future lifetime value of that customer plus the marketing and sales costs it took to bring them to your site.
If you have a 0.5% or lower chargeback rate but your fraud process is less than 99.5% accurate, you are leaving money on the table and your customers in the lurch. You may have a fraud problem without having a chargeback problem.
Kount’s tagline is Boost Sales, Beat Fraud. Beat Fraud is obvious, but why Boost Sales? We help merchants preserve more good orders and allow transactions, like cross border or next day shipping, they may have considered too risky without our protection.
Kount’s Persona Linking provides reviewers much more information than you can collect from the transaction alone. Our rule engine and Persona Score make sure you are only reviewing the trickiest transactions, as customized with our Client Success team to your specific business definitions.
The numbers are on your side, and so is the AI. Don’t let fraud prevent good transactions any longer! Want to know more? Attend “The Truth about Machine Learning in Fraud Prevention” webinar at 11:00 am PST / 2:00 pm EST on Thursday, March 23.