This is the start of a hopefully long series of BI FAILs, examples of dysfunctional behavior and problems gathered from conversations with companies implementing business intelligence projects. Certain details (may) have been changed to protect the guilty. 🙂
“We had just implemented a new dashboard system, using data from a corporate warehouse. One key performance indicator had been typically been shown on a standard chart with the y-axis varying between 0 and 100, and the typical score was 95. The new chart engine automatically zoomed in, based on the values, to show the y-axis starting at 90.
Soon after the new system was installed, the team that owned the KPI had a tough quarter, and it dropped to 90.5 for the month. Because of the new charting system, this looked at first glance like a massive drop, rather than a modest 5% setback, and the manager called the dashboard implementation team to complain.
The team refused to make changes, explaining that the system was working as specified, and that a zoomed-in view was more appropriate.
The next day, after a refresh of the underlying data warehouse, controlled by a different (and more malleable?) team, the KPI had mysteriously jumped back up to 95…”
Comments
3 responses to “BI FAIL #1”
I’d say the team that owned the KPI was right. Zooming in is inappropriate behavior, and this example demonstrates why. In fact, most of the intro to stats books I’ve seen describe cutting off the Y axis in that way as a classic technique to mislead people if done deliberately, and a common but inexcusable blunder if done by accident. If the design of the dashboard caused a perfectly modest dip to look like a catastrophic failure, then the dashboard was designed poorly.
[…] posts included the first BI Fails and BI Briefs, including my opinions on BI futures, why it’s important to have more than […]
Timo, this brings to mind the issues I discussed in this post: http://www.bimonkey.com/2009/09/the-dangers-of-the-dashboard/ – basically KPI’s can be false comfort or poor indicators of performance if there is no understanding behind what they mean, as in the case above.