A BI Self-Service Debate: Can Users Be Trusted?

At SAP TechEd in Amsterdam this week, I had a conversation with SAP mentor Steve Rumsby about BI self-service and user trust.

You can get an idea of our discussion below, which echoes some other conversations I’ve had with IT organizations that are worried about potential bad analysis that could result from unfettered access to data. I’m very interested in your feedback: chime in with your thoughts and comments!

steve timo fight

Steve, before we start, what’s your background and role?

I’m the SAP technical manager at the University of Warwick in the UK. I’m formally responsible for our technical architecture, servers, storage, network, plus the development work. We run SAP ERP 6, GRC, solution manager, plus other peripheral systems. I joined the IT department directly after finishing my computer science degree at the university.

So what do you think about BI self service?

I wrote about this on the SAP community network. I can see why people get excited about it, but it makes me feel nervous. I think it’s a double-edged sword. There are lots of good things that you get from it – people able to get information themselves, without having to wait weeks, and so on. But what makes me nervous is people doing that without understanding the nature of the data they are trying to manipulate.

Any examples?

codThere have been some famous ones – for example, one respected UK newspaper reported that there were only a handful of adult cod left in the North Sea.

The BBC did an investigation, and the result was that they were wrong by a factor of 1000000. The misunderstanding was based on comparing life-time information from cod in a different part of the world, and making unwarranted assumptions… if you don’t understand fish, you won’t realize you’ve made a mistake until you publish. If that was the organization’s cash flow, people might not be so happy!

But surely the problem was this was a newspaper, with no expertise — if it was the cash flow, wouldn’t the analysis be done by finance people, who do know what they’re talking about?

Yes, but what gets people really excited is the ability to take data from all different places, and mash it up. So you start with data you do understand, but you mash it with data you don’t necessarily understand, and you get an answer that you believe, but maybe you shouldn’t. How do you know if it’s false?

So is the answer to force business people to always go through IT experts? That has resulted in huge frustration, and business people creating shadow IT organizations…

The answer is that people need to understand what data they can trust and what they can’t. It’s about education and training: as soon as you go and use outside data, you must think harder about the trust-worthiness of the data and the analysis.

But when I look at organizations that are keen on “self-service”, it’s typically not about external data. It’s more about fast-changing or manually sourced local or departmental data doesn’t make financial sense to put in the data warehouse (e.g. for a one-off analysis) — yet some organization’s BI strategy is still focused on the DW, so the users are stuck… what happens then?

In those cases, yes, users should be able to go ahead and do it, but they need to speak to somebody who can help them understand and sense-check the data.

I believe this is one of the critical roles of a BI competency center: it’s not just about owning an infrastructure, but owning the general business problem of helping business people with their analysis, no matter what the tools used…

I agree…

steve timo agree

Further reading:


Posted

in

by

Tags:

Comments

3 responses to “A BI Self-Service Debate: Can Users Be Trusted?”

  1. Pat Hennel Avatar

    ” you start with data you do understand, but you mash it with data you don’t necessarily understand, and you get an answer that you believe, but maybe you shouldn’t”

    I think that’s a very legitimate concern. If you are pulling in multiple sources of data and don’t know how to clean it, or ensure its accuracy, you’re going to come up with results that are no where close to the truth. That could lead to some very costly mistakes.

  2. Simon Carpenter Avatar
    Simon Carpenter

    In a era when we are all striving to improve the customer experience by being responsive (even proactive) through having empowered and informed employees at the coal-face, self-service BI is an obvious capability. However, the best tools and algorithms will mean little without people who have the ability to discern and debate meaning and then act. I think the key is to invest in people and in particular in building competencies around Critical Thinking, the Socratic Method and suchlike in order to maximise the benefits latent in the technology.

  3. Donald MacCormick Avatar

    The answer for the majority (i.e. end-users, who just want the right information to guide their everyday decisions) lies in thinking about the problem a different way.

    Rather than trying to train/force them to use analyst tools, we need to deliver BI which speaks their language. BI which delivers answers GYGWYNGOGO-style*.

    The increasingly natural way to do this is through mobile-delivered, interactive dashboards. Think about the BBC weather site it is a BI interface into a big-big data set, with lots of predictive analysis but can be used by everyone with no training and protects users from themselves (great example further out forecasts are given in less detail).

    This is the bar for next-generation, end-user business intelligence.

    Interestingly SAP has a tool which is supposed to deliver this (i.e. Design Studio), but for some reason it never seems to appear in the marketing. All the focus seems to be on pushing analyst tools (i.e. Visi) which are only really appropriate for a small (if important) minority of users.

    Get In, Get What You Need, Get Out, Get On”