A Criminal Misuse of Statistics?

One of my favorite topics is the very human tendency to misuse and misunderstand the information that is provided by business intelligence systems.

Here’s a link to a TED Talk by Peter Donnelly, an Oxford statistician, pointing out some of the common errors that people make, and the serious consequences that can result.

Here’s a taste of the material:

(1) Imagine you’re sequentially tossing a coin. On average, which of these sequences will take longer to achieve?:

  • Head-Tail-Tail
  • Head-Tail-Head

They have to be the same, right? (hint: wrong)

(2) Imagine you take a medical test for a disease that has a 99% success rate, and you have a positive result. How likely is it that you have the disease? (hint: no, it’s not 99%)

Peter goes on to talk about a case where a pediatrician gave statistically-invalid expert testimony at a trial that resulted in a UK mother being convicted of murdering her children (later thankfully overturned on appeal).

We should all take account of these kinds of frightening tales when implementing business intelligence:

  1. Just providing information doesn’t mean that it will be used effectively, and IT organizations should take responsibility for the end result of BI — better corporate performance — not just think of themselves as tools providers.
  2. Analytic expertise and data expertise is as important as technology expertise, and should be an explicit part of any business intelligence competency center.
  3. Sharing information widely and providing collaboration will make it more likely for mistakes to be spotted and corrected.

Does anybody else have any good stories and examples of data misuse?