Q&A: Self-Service vs Traditional Business Intelligence

I was recently contacted by Falk Tewes, an  MBA student at New Buckinghamshire University in the UK who is working on a master thesis about ‘Management aspects of a Self-Service BI strategy: the role of BI Governance‘.

With his permission, here are his questions and my answers around self-service BI and governance. If you have your own ideas/answers to these questions, please reach out to Falk, or use the comment section below!

What is your background with BI and self-service BI?

I have worked in business intelligence for the last 25 years or so. I worked for Shell in 1989, and one of my roles was gathering information from the payroll and HR systems and using it to create a series of management reports for executive decision-making. This required a mainframe, a few floppy disks, the one PC the department owned, and some macros on an early copy of Lotus 1-2-3, the first big commercial spreadsheet program. I then joined BusinessObjects in 1991, and have worked for the company ever since (now part of SAP).

Part 1: Definition and expectations of self-service BI

How would you define self-service BI?

There are always two “definitions” when it comes to IT terms, and most people try to mix them up, creating endless confusion.

The first is the business need, e.g. in this case:

• Business people accessing the data they need for decision-making, without having to go to technology experts each time they have a new question.

The second is the technology used to fulfill that business need, e.g.

• Tools that allow people to gather information from multiple sources, analyze it, and share it with others, without having to know the technical protocols required to access the data.

For the purposes of this document, it’s best to focus on the technology, because that is what typically changes first, solving old business needs and creating new ones.

What do you find people misunderstand about self-service BI’s concept?

That it’s a very hard thing to define concretely in terms of technology implementation. The underlying business need is a very broad concept that covers a very wide range of different types of technologies and information uses, and that the distinctions between “reports,” “dashboards”, “data discovery,” etc are blurry — and the need for “business information” covers a lot more than what is stored in traditional databases (documents, external news feeds, etc.)

There’s no truly self-service BI solution. Somebody has always done some work in advance. The data has to be brought together at some point in a way that makes sense, and this has to be done by somebody with expertise in data and technology.

In your opinion, what are the benefits of self-service BI compared to traditional BI and how can it help organizations to meet its strategic and operational business goals?

BusinessObjects was explicitly a “self-service BI” tool in 1991 (using my definitions above), so I’m not sure the comparison with “traditional” is appropriate.

Business needs don’t change so much over time — what does change is the technology used, the data available, and the culture/expertise of information use.

So the question should really be about comparing “new technology possibilities” with “old technology possibilities”

What’s different now is that:

• There is a lot more data available, and a smaller proportion of that data come from corporate processes. You could mash up data with full-client tools like BusinessObjects in the 1990s, but the default assumption was that most of it would come from your corporate systems and data warehouse.
• The underlying data was too complicated for users to access in “raw” form, and a “semantic layer” of some kind was required that gave a business-friendly view of the information. Increased analytic maturity (i.e. people are more used to manipulating data) and better underlying technology platforms (simpler, faster, iterative interactions with data are now possible) have reduced this need (but not eliminated it — indeed, it’s one of the biggest challenges of “governed data discovery”)
• Although the tools were designed for self-service, many deployments of these solutions ended up being “report factories” where the semantic layer was used by IT/business experts to create reports for others. This often reintroduced the bottlenecks and frustrations that self-service BI was supposed to get rid of.

What is your opinion towards visual analytics in the context of self-service BI and how do they differ?

The ability to have fast, interactive feedback to new questions means that the user interface can be improved. Instead of forcing users to ask a question, get a result, and then use that result to make a chart, you can do much more of the interaction on the chart itself. It’s more appropriate for iterative discovery than reporting. It’s also easier and more appealing to use, which is an important consideration in a technology that is still, sadly, seen as a “nice to have” in more organizations — i.e. people are rarely forced to use BI as part of their job.

Part 2: The Role of Governance

How would you define Governance in regard to self-service BI?

My definition of data governance is “stopping people from doing stupid things with data”. Sadly, stupidity is a vast subject.

Data governance includes security, having people agree on common definitions (it’s amazingly hard to define something like “how many employees does the company have?” — and the answer will vary considerably depending on why you are asking the question/what you’re going to do with the data), bad analysis, wasteful recreation of the same analyses, etc.

Self-service BI can exacerbate all these problems by removing the checks and balances on data preparation and use. Without governance you are likely to end up with lots of silos of information, bad analysis, and extra costs.

There’s some nice literature on the problems of spreadsheet governance that is similar to the problems unsupervised self-service BI can lead to.

How would you rate the role of Data Governance for successful self-service BI usage?

First define “success”!

Self-service tools can be very popular with business people that have been frustrated with the red tape and lack of agility associated with traditional IT organizations. To the extent that they can now make better business decisions faster, it’s clearly “successful” — but there can be tradeoffs in terms of long-term costs and complexity.

Data Governance is just part of Governance. What other aspects of Governance would you describe as important to self-service BI and why?

I’m going to take “governance” to mean processes and procedures in general.

The cultural aspects of information use are extremely important for BI— whether people are incented to use information, whether information is hoarded as “power,” who is responsible for information, how it impacts people’s bonuses, how the people involved in providing information to the business are organized, etc.

All of this is vastly more important than the technology itself. Frank Buytendijk has written a series of books that cover many of these people and culture issues in more detail.

Why do you find Governance as important for self-service BI and would you recommend a company without Governance policy to implement a self-service BI strategy?

De facto “governance” always exists, even if it’s not codified. The underlying principles around information culture are more important than having documented procedures, etc. And as noted above “success” depends on what is being measured, but I believe that the benefits of governance policies and procedures clearly outweigh the costs.

Part 3: Success factors for Self-Service BI

Are there any factors that you would consider as critical when developing and implementing a self-service BI strategy?

One of the most interesting things about BI is that it’s ultimately indistinguishable from whatever we mean by “management.” I’m not sure there are any “critical” factors, but there’s a long, long list of best practice.

Some more here on “why BI projects fail” — sadly I never wrote this up as a full blog post

What do organizations have to change to take advantage of empowering their users to gain new insights through self-service BI?

Change the information culture

Part 4: Components of traditional BI usable for SS BI

In what way could self-service BI improve the organizational wide use of BI tools?

Self-service BI typically builds on “traditional BI” — i.e. the trusted data sources (finance, etc.) available through the enterprise data warehouse are a key part of the data analysis done in the “self-service” tools.

What types of components of traditional BI strategies would you consider for using with self-service BI?

There are many benefits to traditional BI. In particular, successful analytics requires at least some key information to be reliable, consistent, and secured. This often-painful process is one of the key areas of success of traditional BI implementations.

Annotations

Is there anything else you would like to mention when talking about self-service BI?

At one level, “business intelligence” is just an artifact of limited technology.

The very first business application ever, in 1951, was used for analytics (for predictive “bakery valuations”). But it quickly became clear that because of technology limitations, it wasn’t economically feasible to do “transactions” (OLTP) and “queries” (OLAP) on the same system. So the data had to be copied to a separate data store for analytics, with “business intelligence” tools for analysts.

But new technology means that we can now go back to the origins of business applications, with “Hybrid Transactional/Analytic Processing” systems

For example, the new SAP  “Simplified Accounting” application uses in-memory technology to store data just once, at the most granular level. Every other financial view needed (consolidations, balance sheet, management accounting, etc.) is calculated on the fly, in real-time. This reduces the data storage needs by up to 98% and vastly reduces the complexity that lead to the poor performance of “traditional BI”

In other words, the fast growth of “self-service BI” (driven by local in-memory tech enhancements) has been because of traditional limitations — and those limitations are falling (because in-memory is now core to the underlying transactional systems)