{"id":12420,"date":"2014-05-19T11:41:02","date_gmt":"2014-05-19T10:41:02","guid":{"rendered":"http:\/\/timoelliott.com\/blog\/?p=6672"},"modified":"2014-05-19T11:41:02","modified_gmt":"2014-05-19T10:41:02","slug":"data-democracy-vs-data-anarchy","status":"publish","type":"post","link":"https:\/\/timoelliott.com\/blog\/2014\/05\/data-democracy-vs-data-anarchy.html","title":{"rendered":"Data Democracy vs. Data Anarchy: Governed Data Discovery"},"content":{"rendered":"

Successful business intelligence has always been about negotiating the right tradeoffs between the needs of individual business people and the needs of the organization as a whole.<\/p>\n

In 2001, Bernard Liautaud<\/a>, founder of self-service BI pioneer BusinessObjects<\/a>\u00a0 wrote a book called eBusiness Intelligence: Turning Information Into Knowledge Into Profit<\/a> that discussed these tradeoffs. He drew the analogy with systems of governments, illustrated by an information governance quadrant that outlined the choices between\u00a0Information Dictatorship<\/em>, Information Anarchy<\/em>, and Information Democracy*<\/em>.<\/p>\n

\"information<\/a><\/p>\n

BI technology has taken a big leap forward since the book came out a decade ago. But the tensions between IT and the business and need for tradeoffs have remained, and we\u2019re now seeing the latest generation of analytics struggling with the same issues.<\/p>\n

Much of the recent growth in the business intelligence market has been in the area of \u201cdata discovery\u201d tools that take advantage of in-memory technology to allow users to quickly mash up data from multiple sources and explore it interactively and visually without having to create new \u201cqueries\u201d for each step.<\/p>\n

These tools are often enthusiastically adopted by business users as a reaction against the red tape and restrictions of central IT organizations, which are perceived as\u00a0information Dictatorships<\/em> where access to data is concentrated in the hands of a few.<\/p>\n

Although this data discovery has empowered business users in new ways, it has also introduced new dangers, in the form of\u00a0Information Anarchy<\/em>\u00a0as defined by Liautaud:<\/p>\n

Information anarchy results from individuals or entire departments taking their informational needs into their own hands. As businesses grew more competitive in past years, departmental managers realized they needed better information to make good business decisions. They realized that whatever information they could procure from the glass house of the IT department would not be adequate.<\/p><\/blockquote>\n

One common side-effect of such tools is new data silos, where each group or department compiles its own figures, resulting in inconsistent data and duplicated efforts:<\/p>\n

Similar to the brief yet short-lived instances of anarchy in world history, the onset of information anarchy is often followed by a brief period of euphoria. Departments that set up their data silos are briefly enthusiastic about their limited solution, until the point in time when someone in senior management receives two reports with inconsistent data from different departments and questions the validity of the information. The uncoordinated silo approach sooner or later will result in data consistency issues, while little security enforcement on data access is feasible.<\/p><\/blockquote>\n

Even worse, \u201cdata fiefdoms\u201d can arise that replace central IT with more local bottlenecks. Some call this creation of a new \u201cinformation aristocracy<\/em>\u201d — and it\u2019s hard to read all the praise for new “Data Scientist” roles without worrying that we are creating a new layer<\/a> between business people and the data they need for decisions. As Liautaud put it a decade ago:<\/p>\n

Executives and other business managers \u201cbeholden to number-crunching wizards specially trained with reporting, analytic, and statistical software installed on their computers. The concept was to expand the use of information to many more business users\u2026 however, very quickly, this technology-schooled cadre of individuals became, without knowing it, information dictators of another sort.<\/p><\/blockquote>\n

Businesses should strive to combine consistent information governance with decentralized access to data and decision-making. Liautaud called this \u201cinformation democracy,\u201d and it\u2019s similar to what Gartner now calls governed data discovery<\/em>: platforms that address both business users’ requirements for ease of use and enterprises’ IT-driven requirements.<\/p>\n

According to Gartner<\/a>, the race is now on between newer data discovery vendors and traditional vendors to provide these platforms. \u00a0The customers of data discovery vendors:<\/p>\n

\u201cwould like to expand use and even standardize on data discovery platforms for their larger enterprise BI deployments but find that in many cases these platforms …lack the necessary enterprise features in relation to governance, administration and scalability, among other things.\u201d<\/p><\/blockquote>\n

And in the meantime:<\/p>\n

\u201cthe current IT-centric BI and analytics platform Leaders, which currently “own” the installed base market share… are trying to change that by focusing their new product investment on business-user-driven data discovery and analysis.\u201d<\/p><\/blockquote>\n

For example, SAP, the clear leader in the business intelligence space<\/a> for many years has introduced the data discovery product\u00a0SAP Lumira<\/a>. A desktop version of the tool allows analysts to mash up data and create visualizations, then easily share them via the existing SAP analytics platform. This allows organization to support a \u201cnetwork of truth<\/a>\u201d or \u201ccollaborative insight<\/a>\u201d without sacrificing either the ease of use needs of the individual or the data governance needs of the organization as a whole.<\/p>\n

Of course, any form of democracy is hard, precisely because it requires constantly updated tradeoffs between different groups. The key is to build a system that everybody respects, even if they don’t always like the result.\u00a0This in turn requires that both IT and business people have to realize that some compromise is necessary. For example:<\/p>\n