A New Data Architecture for Faster, Better BI: SAP BusinessObjects Explorer

Today at the SAPPHIRE user conference in Orlando, SAP announced it would “revolutionize BI with a new solution that delivers insight at the speed of thought”. The new package is called SAP BusinessObjects Explorer Accelerated, and it combines the search and visualization capabilities of SAP BusinessObjects Polestar (now renamed SAP BusinessObjects Explorer) with SAP Business Warehouse Accelerator (BWA).

Photo: SAP CEO Léo Apotheker showing that even a CEO can use the new SAP BusinessObjects Explorer (with a little help from Ian Kimball)

Implementing business intelligence always involves some level of tradeoff between query speed, ease of use, data volumes, and accuracy. For example, it’s perfectly possible to do BI directly against transactional applications, but strict limits are required to stop system failure, and business users are likely to want more than can be made available. So some sort of dedicated analytic storage is always required, but that in turn adds other problems such as data latency.

Determining the right BI data architecture for an organization is a complex and difficult process. For example, the analyst group Forrester even provides their subscribers with a “BI Data Architecture Decision Tool” that requires a 1 to 5 response to 20 different questions to come up with a recommendation – and that’s just with the default weights to each question.

Unfortunately, when faced with this complexity, some IT organizations end up selecting what’s easiest and cheapest to maintain, rather than what’s best for the business users. This helps in part to explain why the average proportion of BI users in organizations remains a relatively dismal 8.2%, according to Nigel Pendse’s latest OLAP Report excerpt (registration required).

To try and minimize the tradeoffs, there’s been a recent surge in interest in data access and storage technologies including column databases, data warehouse appliances, in-memory processing, and complex event processing. And vendors are partnering to provide more seamless solutions – for example, SAP and Teradata recently announced a new partnership to support Business Warehouse on the Teradata database (SAP BusinessObjects has worked closely with Teradata for many years).

The new SAP BusinessObjects Explorer Accelerated combines the best of the latest knowledge in data modeling, hardware and in-memory processing, and user interfaces in order to minimize the traditional tradeoffs and barriers to wide-scale BI deployments.

Innovative Architecture

Explorer Accelerated combines the best of the new technologies, with TREX in-memory processing and a blade server architecture provided by SAP’s hardware partners to index data from a BW enterprise data warehouse. Then users can access that data using the intuitive Polestar interface.

The product unveiled today is designed specifically for BW, but future versions will be available for other environments.

The Need for Speed

One of the key requirements for effective business intelligence has always been fast response times. Unfortunately, few users are happy about what they have available today. According to the latest OLAP report extract:

For the fifth year in a row, slow query performance was again the problem generating the most user complaints. But although 19 percent of users saw this as one of the most serious problems, only 11.7 percent of consultants and 11.2 percent of vendors agreed.

A few years ago, Nigel Pendse tried to calculate the tradeoff between query speed and business value, in an article in DM Review entitled “The Effect of the BI Application on Business Performance.”

He first calculated at composite business benefit index (BBI) (based on KPIs such as saved costs, better business decisions, etc.), and then charted it for various characteristics, including the typical query time. As Nigel says in his article:

[this chart] is remarkable. Most people would expect that query performances of under one second were hardly different from query performances between one and five seconds, and there would be no impact on the overall business benefits achieved – but there actually is a measurable impact on business performance as query performance declines.

When compared to other factors, variations in query performance have more effect on the BBI than any other. If there is one aspect of an application that you want to optimize, choose this one.

SAP BusinessObjects Explorer uses hardware acceleration and in-memory processing to produce the kind of sub-second query response times that make such a big difference to the overall business benefits of BI.

Intuitive Experience

Hundreds of articles have been written over the last twenty years about making BI easier to use, especially for “casual users” who need access to information to do their jobs, but don’t want to use complex query tools. And tools have improved, but ease-of-use issues are still typically near the top of barriers to wider adoption (see diagram).

It’s always been possible to provide predefined interfaces such as executive dashboards or operational reports that have exactly what people want. But these can be expensive to build and maintain and what people need inevitably changes over time as problems get fixed and processes are improved. The ideal interface would still be flexible enough to let people browse through information, but without requiring any specific set up of hierarchies or windows.

This is the where Explorer is such a breakthrough. It uses an attractive, Adobe-Flex-based interface that provides an experience much more like a typical Google search, and requires no training to use.

Search. Users can enter any keywords they are interested in, and Explorer will instantly find the most relevant data from across all data sources available in the BW warehouse.

Faceted navigation. Explorer provides “contextual exploration” – as you browse through data, explorer automatically provides the most relevant details, without having to understand anything about the underlying data model.

For example, if I’m looking at fridges, Explorer may display “volume” as the next most relevant characteristic, but if I look at TVs, it may be “screen size”. It’s important to note that these suggestions are derived from the data itself, and can easily be overridden as the user looks for data.

Automated chart generation. Explorer automatically picks the best chart format for the data available, and colors and transitions are used to help the users’ eyes follow the data they are interested in.

Easy sharing. Insights can easily be shared with others, by sending URL links or diagrams, or exporting to other systems.

Serious Data Loads

The volume of data available for analysis continues to explode – just three years worth of transaction detail for a national chain adds up to billions of transaction records. Because of the server blades, Explorer provides consistent and predictable response times, typically five times as fast as highly-tuned queries and up to 100 times faster than non-tuned queries. Combined with BW, this means you can access information throughout the organization, quickly see the big picture and connections, and still be able to navigate to the detailed information.

Conclusion

Above all, SAP BusinessObjects Explorer is defined by the tradeoffs you don’t have to make. For the first time, a BI solution is available where you don’t have to understand the data structure, you don’t need training, you aren’t constrained by what data you can have access to, and you have sub-second query response time. And all this without needing to create any specific dashboard interfaces.

With the new exploratory capabilities, people can answer questions they didn’t even know they had. As you explore data, your questions evolve quickly and SAP BusinessObjects Explorer keeps pace with the answers. Response from early customers and beta testers has been impressive:

“If you know how to use a computer you can use Explorer.”

“This is a perfect tool for our managers who don’t know what they are looking for.”

“Finding information is as easy as searching the Internet.”

“The point and click interface allows users to pick what they want – like you would do on any website.”

I’ll post more information and resources about Explorer in the coming posts…