![\"thinker-banner\"](\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2012\/11\/thinker-banner.jpg?resize=690%2C310&ssl=1\" "\\"thinker-banner\\"")
<\/p>\n<\/p>\n
Last year, I presented at dozens of conferences, explaining the big changes to BI infrastructures brought about by new technologies<\/a> (in-memory, hadoop, MPP, in-db analytics, real-time, etc.). And this year, I\u2019ve presented dozens of examples of the real-life business changes that have been made possible<\/a> because of these technologies.<\/p>\n But one of the most important business processes in modern organizations is the provision of business intelligence itself.<\/em><\/p>\n I think we haven\u2019t yet talked enough about how the new technologies will disrupt traditional organizations, roles, and processes in analytics. I believe much of the \u201cnext big thing in BI\u201d is actually about getting back to the original ideals of BI \u2013 hence the reference to \u201cnew old BI ideas\u201d.<\/p>\n Here are three big changes I see:<\/p>\n We\u2019ve always known that \u2013 in theory \u2013 we\u2019re supposed to start from the decisions that need to be made and then work backwards. But the reality is that, because of technology limitations, most BI projects are still organized around traditional layered architecture approaches: gather the data available, figure out how to cleanse it and transform it, load it into an analytic structure, then figure out what reports\/dashboards people want to see.<\/p>\n The new technologies promise to get us back to our ideal. Imagine starting your BI project with a blank sheet of paper \u2013 literally. Ask the business people what questions they need answering in order to run their business. Using a collaborative environment (like SAP Jam<\/a>), get them to upload a sketch of the data visualizations they would like to see, imagining that they have access to any data they might want.<\/p>\n Debate, iterate, and prototype with data discovery tools (like SAP Visual Intelligence<\/a>) and temporary or dummy data sets. Once the outlines of an ideal solution are in place, invite the data architects into the same collaboration session, and ask them where the detailed data required is originally<\/em> stored (e.g. in SAP Business Suite<\/a>, in a customer\/partner system via Ariba<\/a>, social media sentiment<\/a>, etc.).<\/p>\n Once you have determined the visualizations you want, and the origins of the data, everything in-between can now \u2013 at least in theory \u2013 become a \u201cblack box.\u201d\u00a0 Thanks to in-memory solutions like SAP HANA, you can now do real-time queries on huge quantities of row-level data. You should no longer have to care what the underlying analytic infrastructures are: the system should be able to determine what storage is appropriate based on the questions being asked. As new visualizations and data sources are required, the system would adapt the storage appropriately, so there\u2019s no longer any tension between \u201csiloed\u201d and \u201centerprise\u201d BI.<\/p>\n The dream isn\u2019t really new, but the tools are now much, much more powerful, and this type of architecture is imaginable for the first time.<\/p>\n Sounds unrealistic, a pipe dream? It\u2019s closer than you think.<\/p>\n The poster children are the brand-new SAP EPM OnDemand<\/a> and SAP Precision Retai<\/a>l solutions. Both take row-level data from core operational systems, and serve up beautiful, interactive, mobile analytics for business people via the cloud. In both cases, they use SAP HANA to deliver great performance. But what\u2019s the underlying analytic structure? You don\u2019t know, and don\u2019t really care!<\/p>\n It\u2019s only a question of time before all of our analytic systems work this way.<\/p>\n One of the recurring causes of BI failure is analytic systems that are too static. For ERP, it makes sense to have big projects to put in place coordinated systems to gather and process consistent operational data. These systems are not likely to need to change very often (although as the world changes, more flexibility is increasingly required, and this would be one of the big advantages of an in-memory ERP system such as Business Suite on HANA<\/a>).<\/p>\n But BI is different. If it\u2019s not changing, it\u2019s not working. It\u2019s like a bicycle \u2013 if it\u2019s not moving forward, it\u2019s falling over.<\/p>\n This is because accessing information is, on its own, a useless activity. Unless somebody is looking at those reports and dashboards, and making changes to business process, there\u2019s no benefit to the business. And if changes to business processes are being made, these should, in turn, result in changes to the information needed to run the business. Business analytics teams should be constantly running after the increasingly-sophisticated information needs of business users and executives.<\/p>\n If you are in charge of a business intelligence deployment, and people have been looking at the same reports for over a year, and have made no requests for changes, it\u2019s time to investigate \u2013 somebody is not doing their job. Either the reports are being ignored, or their aren\u2019t being acted upon.<\/p>\n So BI systems have to change over time. But building business analytics system for change has been hard.<\/p>\n To explain why, first note that there is a very distinct BI lifecycle for information needs. It starts off with new business initiatives and information sources. By definition, there\u2019s no big infrastructure to support these initiatives, so they make do with the best tools available, typically manual data scrapes and spreadsheet analysis.<\/p>\n Over time, the initiatives prove their worth, and need to be operationalized with more powerful tools. This is where departmental BI initiatives typically start, but there\u2019s usually wrenching change between the old spreadsheet-based approach and the new systems.<\/p>\n Progress continues, and then the data also needs to be integrated with corporate data and made consistent so that it can be used across the company. So there\u2019s an initiative to integrate it into the enterprise data warehouse, typically with more wrenching change.<\/p>\n A concrete example of this would be the recent rise in social media analysis. It started off with a series of small, completely separate tools to allow marketing teams to do some basic analysis of twitter, facebook, and other social tools. Now most larger organizations are starting to take a more strategic approach with tools that give an overview of social data from multiple different systems, and integrate it with the rest of the marketing analytics. And big consumer retailers such as General Mills have already started integrating the data into their corporate data warehouses<\/a>.<\/p>\n Today, BI is like a car with five gears, but no clutch. Each of the gears represents a different level and sophistication of analysis, from first gear (spreadsheets) to fifth gear (a full enterprise-wide data warehouse). Over time, we\u2019d like the car do be able to move from one gear to another, but without a clutch, it\u2019s a very painful process that generates the business equivalent of loud grinding noises. (This analogy may also illustrate why so many companies have failed with enterprise data warehouse projects in the past \u2013 the only way to get a car into fifth gear is to be running downhill or with a very strong wind behind you.)<\/p>\n In the future, BI has to be more like an automatic car. We want to press on the accelerator and smoothly increase speed. The gears will still be there, but handled for us more automatically.<\/p>\n New BI technology and tools enable this. Data discovery tools such as SAP Visual Intelligence<\/a> provide \u201csecond gear\u201d \u2013 they can take existing spreadsheet data and other sources and produce business-friendly visualizations in a more robust way. In particular, the move to 64 bit technology and lower memory costs has enabled laptop computers to perform powerful analysis without requiring extensive corporate IT support.<\/p>\n To move to third gear, it should be easy to take the data sets and visualizations and share them, via a \u201csandbox\u201d on a corporate-wide infrastructure, to mobile devices. This retains the benefits of business unit autonomy \u2013 they get to completely control the analysis being done \u2013 while IT provides a robust, scalable, cost-effective system.<\/p>\n This process is made much easier by in-memory systems like SAP HANA. Before, it was hard to imagine a business unit looking after its data, because of the extensive skills needed to optimize the data for analytics use. Now, people can upload detailed data directly into memory and carry out powerful shared analysis with much less expertise required.<\/p>\n Fourth gear is when business units realize that their solution needs to be able to access and share data with corporate systems. If this data is already being stored on the central infrastructure, it\u2019s much easier to integrate, and experts in the IT department can monitor what is being done around the company, ensuring that there isn\u2019t needless duplication of information, and proposing new complimentary data to existing solutions.<\/p>\n Finally, fifth gear involves the data set becoming a fully-fledged corporate resource as part of the official enterprise data warehouse, with corresponding guarantees on quality and timeliness.<\/p>\n The changes described above don\u2019t help with the most intractable problems in BI, which are to do with people, culture, and organization.<\/p>\n People are undoubtedly the most powerful \u201ctechnology\u201d in the business analytics solution. Only people can take information, make sense of it, and actually change something. We only use the word \u201cdecision\u201d when it requires a person to make sense of an ambiguous set of data.<\/p>\n Computers cannot, and never will, make \u201cdecisions\u201d \u2013 as soon as the data is clear, and a computer can make a choice for us, we no longer use that word, and it simply becomes a feature of our applications. Examples include things like the pricing of airline seats, or the routing of goods in supply chains. People used to do these things, and they called it \u201cdeciding\u201d.<\/p>\n Now people \u201cdecide\u201d the algorithms, and the computers do the rest. The history of computing is the story of how, over time, computers consistently take over low-level \u201cdecisions\u201d while we move on to the next level of complexity and ambiguity (which is also why business people will NEVER be happy with their information systems).<\/p>\n All too often, people are considered passive \u201cusers\u201d of BI, whereas they should be considered the heart of the system. Every time somebody touches information, and choosing what to take pay attention to, and what actions to take, they are adding value, and this value needs to be captured and made part of the overall solution.<\/p>\n People are a powerful technology, but at the same time, one of the painful, under-acknowledged truths in the industry is that even with \u201cperfect\u201d BI tools, many people could and would still make bad decisions.<\/p>\n This is because, as a host of recent popular books have shown, human beings are meat-based machines with a host of psychological characteristics that can get in the way of using data as effectively as possible. Situations I seen in most real-life business analytics environments include lack of analysis skills, lack of motivation, bad incentives and KPI definitions, and misaligned organizational support.<\/p>\n Clearly, in-memory technology will not help here. But enterprise social collaboration can.<\/p>\n I believe the best way to avoid the problems created by people is to add more people. A classic example is the elementary difference between correlation and causation. The fact that chocolate consumption and earning Nobel prizes is correlated<\/a> does NOT mean that chocolate makes you smarter.<\/p>\n I believe the best way to avoid bad information analysis is to share that analysis more widely and more transparently, and let people point out any problems with the data (think of it as equivalent to the peer reviews done in the scientific community) .<\/p>\nUpside-down, black-box BI<\/h3>\n
Support for seamless, dynamic BI lifecycles.<\/h3>\n
Putting people back into BI<\/h3>\n