The real trend this year is not the technology. It’s about helping business people make better decisions, and actually change the way companies do business. Analytics has always been about transforming business, but the recent huge changes in analytic technology have created interesting new opportunities for business innovation.

Most organizations are now starting to understand the technical opportunities, but many struggle to apply those new opportunities to their business processes. This blog post attempts to explain what’s going on in the analytics market and give concrete examples of how other companies have implemented the new technologies in “game-changing” ways (sorry kittens).

Wrenching Change and A Foggy Outlook

The chart below illustrates the wrenching effects of recent financial problems on the world gross domestic product: companies today have to be ready to react to unprecedentedly fast changes to their economic environment.

And the economic environment is fraught with extreme uncertainty. This year, the people who run the world will change, and so will many of the policies of the countries they manage. Financial markets have still not completely stabilized, notably with the future of the Euro still not assured.

Companies have reacted to this uncertainty by slashing costs and accumulating cash, and now need to start investing that cash into future development. Since interest rates are low and the business outlook is still uncertain, many of them are using the money for new technology that can help them prepare for the future.

In particular, companies want better visibility about what’s going on in their market, and increased organizational agility in order to be able to deal with change fast. It’s like driving in the fog without a map – in order to survive, you should invest in better visibility, brakes, and steering to be able to spot and avoid fast-moving objects looming out of the fog.

Analytics provides these capabilities: business intelligence to peer into the road ahead, risk-management to provide fast alerts to new obstacles, and flexible financial planning systems to help swerve around them.

Analytics: Hotter Than Ever

Companies are investing heavily in analytics:

• Analytics is the #1 top technology priority for both CIOs and CFOs, according to Gartner
• Nucleus Research recently released a report showing that organizations get $10.66 of value for every $1 invested in analytics
• IDC has increased growth forecasts faced with stronger-than-expected figures for recent years
• IDC analyst Dan Vesset: “After three decades, the business analytics market is finally reaching the mainstream” and “There are few growth inhibitors in the foreseeable future”
• At the recent BI Gartner Summit in London, Gartner’s Dan Sommer announced an early estimate of 10%+ growth in analytics during 2011, outpacing general IT growth.

Fast-Moving Technology

Analytics technology has been changing fast. On the back end, new technologies have come together to provide what Gartner calls “extreme data performance”. These include in-memory, column data stores, in-database calculations, massively parallel architectures, complex event processing, Big Data / NoSQL / Hadoop, and cloud architectures.

The combination of these technologies provides a opportunity to access massive amounts of a greater variety of data, faster, and more flexibly. The key opportunity is that these new platforms “collapse the stack” so that organizations can implement and update analytic projects much faster than ever before.

And on the front end, various technologies are coming together to provide unprecedented levels of context-based “actionable insights”, including self-service data discovery, advanced visualization including maps, mobile analytics, predictive analytics, collaborative decision-support. They help provide more action-oriented interfaces optimized for the context of the users, both inside and outside the organization.

These technology advances are clearly important, and we’re going to continue to see great improvements this year. The new opportunities have reached a tipping point similar to the rise of digital cameras vs. analog photography – and you don’t want to leave it too late to make the change, like Kodak, which recently filed for bankruptcy protection!

However, the real opportunity is using these new possibilities not only to improve analytics but fundamentally rethink key business processes.

High Resolution Management

University researchers have pointed out that today’s management techniques are based on the limitations of information scarcity:

“How many times has someone in your company uttered, “We don’t have that level of accuracy in the information, so we have to make aggregated estimates”? Under the current paradigm, it is sometimes impossible to drill down and understand what is happening at a highly detailed level.”

They coined the term “High Resolution Management” to describe what becomes possible with the new technology opportunities:

“We contend that these technologies will change drastically how management makes decisions. Why? Because with access to the finest granularities of information, management will be able to move freely from macro to micro levels and will be able to measure, plan and act accordingly. With increased resolution come more options to drill down, eliminate inefficiencies and cut costs.”

Lets take a look at three different types of High Resolution Management opportunity, letting companies:

• Remove bottlenecks
• Rethink business
• Flip business models

Remove Bottlenecks

Better technology always means business opportunity, but the new analytic platforms are rapidly eliminating some of the key bottlenecks that have prevented organizations from getting value from their data:

Faster, more flexible data access. Companies like Red Bull have been able to speed up and simplify their data warehousing environments. Using the HANA in-memory database, the company can now load detailed data twenty-five times faster into their data warehouse, and they were able to eliminate several levels of data staging, increasing the flexibility of the solution.

Data volumes and complexity. Companies like Colgate-Palmolive, Provimi, and Danone have long had access to vast amounts of detailed data about their production facilities and sales channels – but the quantity of data meant that they were unable to run full analytics in a reasonable time frame. That has now changed. For example, according to Colgate-Palmolive CIO Tom Greene:

“We will be able to run analytics at a local level on specific brands and locations, and at the lowest level of detail in real time”

And Danone can now measure the carbon emissions of 35,000 different products, with new systems that:

“collect, measure, and analyze data across the entire product life-cycle, from sourcing through production, transport, retail, distribution, consumption, and end of cycle”

New forms of data: ‘unstructured’ data such as text has long been difficult to effectively analyze and incorporate into mainstream corporate analytics. The new systems make it much easier for companies like Medtronic to access and analyze the large amount of complaints and feedback data they receive about their products, combine it with other data sources, and provide it to business users with dynamic interfaces:

New interfaces and users. Companies like Altron have been able to get the data to their users where they needed it. As Debra-Lynn Marais, Group Information Manager explains:

“The days of our users and execs being in the office have gone. They work from home or on the road. We had to develop a solution that gets information out to where our people are. Everything we do is mobile first. In addition, it’s less cumbersome and cheaper to buy and use a tablet than any other form.”

Rethink Business

Many companies are going beyond “just” improving their existing analytic capabilities, using analytics in new ways to change the way they do business. Instead of analytics being something that is used to monitor and eventually improve a business process, analytics is becoming a more fundamental part of the business process itself.

Proactive Analytics. Instead of using analytics only to assess previous performance, companies are using the new capabilities to get data fast enough to make a real difference. For example, online grocer Fresh Direct, instead of just understanding what problems happened yesterday, can now understand what problems will happen in the next few hours, so they can actually fix them before a customer is impacted:

“FreshDirect has an operations center that manages its fleet of delivery trucks. In a large metropolitan area like New York, traffic doesn’t always flow predictably. A traditional approach to BI would be to print a report showing the level of on-time deliveries (OTDs) the day before and then ask the transportation department what went wrong for the orders that were delivered late. FreshDirect uses analytics in a more impactful way.”

“The company monitors the delivery rate of every truck and enters that data into the BI system on an ongoing basis. Every hour, it uses the previous hour’s data to predict how many deliveries will be on-time in the next hour. If the predicted OTD rate is below FreshDirect’s target, the company sends out an auxiliary truck or trucks to help make deliveries. The company holds 10 trucks in reserve for just this purpose.”

Integrated Risk Assessments. Among other products, publisher Houghton Mifflin Harcourt produces educational books. Schools pass orders in June or July after the end of the school year, and then expect delivery for the start of the next school year in September. Getting books printed during the summer is expensive, as many publishers compete for the limited supply of printers available.

To avoid these extra costs, Houghton Mifflin Harcourt uses using sophisticated, risk-based forecasting. The company prints books in January or February, when printing is much cheaper. In order to minimize of excess inventory, it has carefully analyzed all the causes of previous forecasts, and now takes account of all the different things that influence book obsolescence.

Before, the buying team just ordered based on the volume forecast from sales. Now they have much greater context for their decisions. For example, if there’s a vote coming up on schools funding that may result in the canceling of a math adoption program for the year, they can decide to hold back on those purchases until the outlook is clearer. The fast, more accurate forecasting mechanism has saved them tens of millions of dollars, and they have more of the products their customers want.

New Customer Services. International grocery chain Casino is rolling out a new mobile shopping application for its customers. It provides data from its enterprise systems directly to its customers, resulting in increased shopping convenience and increased customer loyalty.

German healthcare provider AOK (“the good health organization”) is committed to helping its members avoid illnesses in the first place. It is planning to introduce a new, market-differentiating service: personalized healthcare advice for each customer, with tools that:

“Conduct real-time analyses of the tremendous amounts of medical data we receive, recognize potential health risks, assemble various preventive care programs and respond to those risks appropriately and ahead of time.”

As an added bonus, they also believe that this tailored prevention program will result in significant cost reductions by preventing expensive unneeded treatments.

BC Hydro is saving $70 million dollars a year through the installation of new smart electricity meters, using SAP systems, and offering new services to commercial customers based on the new data possibilities. Companies like Centrica are planning to use SAP’s Smart Data Analytics, giving them deep understanding into consumer consumption.

Flip Business Models

The really interesting opportunity for businesses is where companies have managed to use analytics to fundamentally flip the way their businesses work: instead of analytics being part of a process, it “becomes the business model”.

Tim Ferriss, author of the 4-hour workweek, is an interesting example of this. He didn’t do what most authors do: write a book, and then figure out how to publicize it. He used an analytics-first approach: he bought Google Ads, with mockups of book covers, with a variety of titles of books that he might be interested in writing – and then wrote the book that got the most clickthroughs! This is one step beyond using analytics such as focus-groups, which are typically there to validate existing products. The next generation of products and services are being created “on the fly” based on an analysis-first approach.

The clothing brand Zara shook up fashion retailing with “analytics first” – instead of having a designer creating clothes and then trying to sell them six months later, they realized new manufacturing techniques meant they could create clothes “in the moment”. They could observe what people were wearing in the street, quickly make small batches of variations on that theme, and get them into the stores. If they sold well they made more, if they didn’t sell they discounted quickly. Instead of a season-oriented, “batch” business, they switched to a flow-oriented business, using new technology capabilities.

The new analytic platforms mean that this analytics-first approach is available to many more businesses than in the past. For example, T-Mobile is in the process of transforming the way they attract customers. Instead of laboriously creating a range of rate plans, promoting them, and analyzing the results, they now use analytics to automatically create hundreds of more complex, personalized rate plans. They then throw them out into the market, monitor in real time, and quickly cull any that aren’t successful. It’s a way of doing business that would have been inconceivable in the past, and a lot more common in the future.

Conclusion

2012 is the year to rethink your analytic technology to take account of new opportunities:

• On the back end, for extreme data performance
• On the front end, for actionable insights

And it’s time to rethink your business:

• Remove today’s bottlenecks to successful analytics caused by data volumes, data variety, or data access
• Rethink business processes by embedding real-time decisions
• Create new products and services that could only exist because of today’s analytic power

Organizations are using this technology to change the way they do business. If you run an analytics project, you are in the forefront of these changes – it’s your job to help explain to the rest of the business how these technologies should be changing their existing processes. Good luck!

___________________________________________________

If you’re interested in slides that go along with this article, please see this post about the recent Gartner BI Summit in London that includes a download of my presentation at the conference.

As always, it was a huge pleasure to catch up with customers, colleagues, analysts, partners, and competitors at one of Europe’s largest BI conferences. It was a great show overall, and I came away even more optimistic about analytics for the coming year.

Here are Jason Rose and I having an off-the-cuff discussion of what we found interesting about the latest Gartner BI Summit 2012 in London this week:

And here are some longer thoughts about the conference compared to previous years:

The band played on: Gartner analysts Nigel Rayner and Andreas Bitterer performed the opening live show, loosely based on Tubeway Army’s “Are Friends Electric”, to tie in with the opening keynote analogy of “information as electricity” (I’m a fan of ‘80s electonica, but it might have been more fun to have something by AC/DC with Nigel as Angus Young… )

It’s no longer Business Intelligence! Gartner has bowed down to the market trend and dropped the unwieldy category name “Business Intelligence and Performance Management” in favor of the simpler umbrella term “Business Analytics”, following in the footsteps of other analysts (e.g. IDC) and vendors (SAS, SAP, etc.). There was also a shift from BI Competency Centers (BICCs) to Business Analytic Teams (BATs) – apparently because the word “center” (a) doesn’t really represent the reality of diversified real-world organizational structures and (b) has a negative connotation in the US (think of your experiences with shared service centers).

Was it just a simple name change? There was a half-hearted attempt in some sessions to associate Business Analytics with “more than BI”, emphasizing that business change must be the result, not just reports. This is of course absolutely true and essential – but this was always part of their previous category definition. It was clearly a recent change: all the conference content reflected the old naming.

Should we care about this change? No! Here’s an blog I wrote a while ago on business intelligence vs business analytics, with the conclusion: “everybody has an opinion, nobody knows, and you shouldn’t care”. In particular, if you need to continue to call it “business intelligence” to communicate with somebody who is comfortable with that term, then you should continue using it!

Analytics is hot! The conference was literally packed, with people uncomfortably stuffed into a slightly too-small venue. Every indicator pointed towards analytics having another banner year – it’s back to the #1 technology priority for CIOs, it’s estimated to be growing at 10%, faster than overall IT spending, and the number of users is set to rise to 50% by 2014. Next year’s Gartner BI Summit (BA Summit?) is going to be in Barcelona, and that decisions is apparently at least in part because of the need for extra space.

Less technology, more business, more success. Last year’s opening keynote presentation was about the “four Vs”: volume, variety, velocity and validity, and talked a fair amount about technology. This year the emphasis had very clearly moved to business value – emphasizing the “why” rather than the “how” – analytics has to support business decision making and result in business innovation. I attended several excellent sessions on how to make analytics sessions more successful – for example, emphasizing that 20-30% of your time should be spent on “marketing” your analytic solution.

Tell stories, and make a difference. Out of the nominees for the Gartner BI Excellence awards, Medway Youth Trust shone out. They would have been favorites for the award anyway, because they are doing social work for the community, but there were three other other good reasons they won over the other nominees:

1. While all the nominees had clearly done a great job of successfully implementing business intelligence in their organizations, Medway had the best case for both “technical innovation” (using text analytics to get value from unstructured information) and “business innovation” (the data really make a difference to their organization, by allowing them to focus their limited resources on the key business goal)
2. As a small organization, they showed that you don’t need to have a big budget or a big team of people in order for analytics to make a difference.
3. They told their story better. In particular, a Spanish insurance company did a slick presentation about the corporate benefits of their standardized BI efforts, and quoted some impressive figures, but there wasn’t really a concrete example of how they had really made a difference that the audience could connect with.

Overall, it reminded me how vital it is to have real stories to tell people when trying to sell the benefits of your project – ROI isn’t enough: it has to be about people, and (surprising) business change.

Validation of current trends. Analytic technology trends were covered in in-depth sessions:

• Gartner called in-memory a “strategic imperative”, and advised that organizations should look at in-memory as “a quantum leap in their computing strategy” because “dramatically faster data access can profoundly change the nature of some applications”
• Mobile business intelligence is clearly now a given in the market – for example, all the participants of the vendor panel agreed that mobile BI is not going to be a long-term differentiator (although the underlying mobile device management certainly still might be).
• Cloud BI is still something for the future for most attendees – for many, only when the underlying operational systems are themselves running in the cloud.
• Sessions on social networking analytics and operational analytics were no longer marginal, with full crowds.
• The topic of “big data” – or “extreme data” as Gartner prefers to call it – is embedded in the new notion of a “logical data warehouse” that is poised to replace today’s more monolithic structures. One analyst mentioned that big Gartner customers were ripping up their current data warehousing plans and adapting them to the new technology possibilities. A session on big data by Roxanne Edjlali (formerly with Business Objects) was well-attended and well-received.

Big data not big enough? Overall, I don’t think Gartner had quite taken enough account of the appetite for more information about big data topics such as Hadoop, data science, etc. Every session with big data in the title was completely packed, and there didn’t seem to be many people from those communities at the show. I hope that Gartner’s conference team targets the big data constituency more aggressively next year — it would be a shame if people with the same underlying goals (turning information into business innovation) end up going to different conferences just because of some differences in the technologies they use (big data conferences are booming).

Fail in the right direction. Tim Harford, the Undercover Economist, was the guest keynote speaker. His presentation was very entertaining, but in general only tangentially related to analytics. The overall theme did resonate, however: that nobody has all the answers, and that it’s only through being humble about your knowledge that you have a change to succeed. The key is to “fail in the right direction”: make experiments, iterate, and learn from experience in order to move ever closer to better solutions.

Overall theme: going with the flow? This wasn’t really mentioned at the show, but if I had to pick one overall theme, it would be the move from batch-based BI to a greater appreciation of information flow, at every level of implementing systems and consuming information. New data warehouse technologies allow organizations to gather and structure information faster (and this is important: Bill Hostmann estimated that fully 70% of the requirements of a BI project change in the first year alone). Data discovery tools allow business people to iteratively structure and access new information in new ways. And businesses are realizing that analytics isn’t just something that you use to improve business processes: it can and should be part of the business processes themselves.

My presentation at the conference, “Business In the Moment, From Reactive to Proactive” was along the same lines – while there has been lots of technology change over the coming year, many organizations are still struggling to turn that new technology into business innovation opportunities. I talked about the big changes in the technology landscape and gave examples of organizations that had used these technologies to transform the way they did business, through removing business bottlenecks, rethinking business processes, or flipping business models to an “analytics first” approach.

You can download the slides in Microsoft Powerpoint or Adobe PDF format, and I’ll explain the main themes in a separate post. I look forward to Barcelona next year!

I presented at my first conference of the year last week, the Heliview Business Intelligence and Data Warehousing 2012 conference in ‘s Hertogenbosch, Netherlands.

The first keynote, by Erasmus scholar Dr. Roeland Dietvorst, was about “Performance Management in the Brain”, and a subject close to my heart: the (severe) limitations we evolved apes have when trying to make rational decisions. He illustrated his point with examples of great research that show that:

• Including “irrelevant” product choices can make big changes to preferences;
• Our emotional state has a big influence on what choices we make;
• Men shown pictures of attractive women tend to make worse financial decisions;
• etc

He also reviewed some of his own research, scanning the brains of sales people to see if there was a correlation between brain activity, competence in understanding other people’s state of mind, and selling skills. He finished by underlining that we have “two brains,” and that business intelligence can help us move decision making to our more rational, less emotional side.

I did the other morning keynote on “Real time enterprise from theory to practice”:

“The amount of information we face is growing by the day. Both volume and type of information. IT organizations not only incorporate customer or product data, but new sources such as location data and data streams from social media play a significant role. In addition, we want information available in the context in which we operate, and preferably independent of the site or device. To make this possible innovative solutions are needed that can work with large volumes of data. In-memory computing may by some be considered as not yet available but the reality is different. There are several examples where this is being successfully implemented and numerous organizations are achieving demonstrable benefit. This presentation shows that the real-time revolution has already started and how it’s being practiced today.”

The presentation reviews the state of the BI market and technology, and ends with some examples of companies using information to change the way they do business. As usual, here’s a copy of the slides, in Adobe PDF and Powerpoint PPTX format.

I’ll be in London next week presenting at the Gartner BI Summit 2012 — hope to see you there!

Update: here’s a nice write-up from the Dutch team about the event.

Mobile BI has been around for a long time. Starting in the late-1990s, the first SMS-enabled telephones became mainstream in Europe, with basic broadcasting of the latest figures available in your BI system (or email, fax, pager, etc.). By the end of the decade, the first telephones with WAP browsers were used to provide interactive BI, quickly followed by connected PDAs with basic HTML browsers.

Here’s what SAP BusinessObjects looked like on the Nokia 7110 in 1999, on a Compaq PDA running Windows Pocket IE, an AvantGo PDA, and a Japanese DoCoMo i-mode phone in 2001:

The arrival of all these new mobile devices was supposed to usher in a new dawn of mobile analytics. Here’s a slide from a presentation a decade ago by then-marketing-VP Dave Kellogg, including the heady prediction that “5-25% of companies indicated they already provide or will provide wireless access to BI within 6-12 months”.

Business Objects launched a big initiative to go after the mobile market, and managed to sell projects to customers including JP Morgan (Palm Pilots) and Zurich Insurance (a mobile extranet for risk managers).

But, clearly, the market didn’t take off – PDAs became more widely used, and phones got better, but they weren’t used much for BI. The user interfaces were too clunky and connection speeds were too slow. Interest in mobile BI did grow slowly over the decade, notably as RIM blackberry devices became ubiquitous, but it took the wide availability of 3G wireless and modern smartphones/tablets to provide truly usable interfaces.

Finally, a decade and a half after the first tentative steps, everybody seems to agree that this is the year that mobile BI will really take off.

2012 Is the Year of Mobile BI

Here’s a taste of the mountain of research data that’s been generated about mobile BI in the last few months:

• Boris Evelson of Forrester says mobile BI will go mainstream this year. “One needs to make decisions when and where they need to be made. Not ‘when I get back to the office,’ which may be too late.” He also says that “professionals must start evaluating and prototyping mobile BI platforms and applications to make sure that all key business processes and relevant information are available to knowledge workers wherever they are.”
• According to Gartner, “by 2013, 33 percent of BI functionality will be consumed via handheld devices.”
• A survey by analyst Howard Dresner indicates that BI has already become the third most in-demand enterprise mobile application, behind only email and personal information management apps such as calendars, and 68 percent of those surveyed rated mobile BI as “critical” or “very important,” up from 52 percent a year earlier.
• A recent survey by Information Week showed that 25 percent of organizations are planning to implement some form of BI this year.
• Sixty-one percent of the participants in a TDWI survey said that they expect users to spend more time accessing BI from mobile devices in the next 12 months.
• Business Intelligence and mobile are the top two technology priorities for CIOs in 2012.

Barriers to deployment

Not every organization is moving forward with mobile BI. Here are the main concerns:

• Security and administration: Organizations are concerned about data getting outside the organization, and the administration overhead generated by managing BI on mobile devices. A mobile device management (MDM) platform (not to be confused with the other MDM) like Sybase Afaria is key.
• Expectations-setting: An easy interface doesn’t mean that the data users want is readily available. New opportunities mean new requirements. Having the right data foundations in the first place, with a robust, standard BI platform in place makes it easier to react fast to user expectations.
• Platform choices: This is perhaps the biggest factor delaying widespread deployment. Just like the operating system wars of last century (remember IBM OS/2?), there’s no one obvious platform to standardize on for rolling out mobile applications. There are three main strategies, all with pros and cons:
  • Native applications—mobile applications written directly for iOs or Android. The advantage is optimal ease-of-use and access to the capabilities of the native device. The disadvantage is the cost and complexity of supporting multiple platforms and different user interfaces.
  • HTML—accessing mobile BI through a Web browser. The advantage is that you don’t care what device is being used to access the data – at least in theory. In reality, the disadvantage is that browser-based interfaces are generally far behind what’s possible using the native features. There are high hopes that the proposed HTML5 standard will help – but it hasn’tt yet reached maturity.
  • Hybrid solution—mobile enterprise platforms such as Sybase Unwired Platform. You create applications once, and then generate different versions of them optimized for different mobile platforms, including HTML. It’s an insurance measure against the turbulent real-life world of changing mobile platforms, but there’s some upfront investment.

An example of today’s mobile BI solutions -  SAP BusinessObjects Mobile showing the integrated Google maps pioneered by the Innovation Center.

Of course, most of us just want all these options to be available as part of the standard business intelligence platform – and that’s getting closer…

Next Steps

Gartner predicts that “by 2013, 15% of BI deployments will combine BI, collaboration and social software into decision-making environments”. In other words, mobile BI will become part of an “orchestrated” experience that combines accessing data with acting on it, and we’re starting to see this in the form of mobile medical analytics for doctors or mobile beauty advisor applications.

Interestingly, one brand new area of opportunities is the integration of mobile with voice-controlled interfaces such as Apple’s Siri. Business Objects was WAY ahead of the curve with this one, with the project codenamed “Ariel”. It sadly didn’t take off, but anybody who saw it demoed will have fond memories…

Here’s a recording of a World Tour keynote presentation (it happens to be from Belgrade, Serbia, but I did similar presentations in several different regions). It covers how companies can build competitive advantage through customer intimacy, product leadership, category renewal, and operational excellence, using examples from SAP’s customers.

You can download the slides in either PowerPoint or pdf format.

Here’s a video from this years SAPPHIRE NOW in Madrid, featuring Mimi Speir, Steve Lucas, and Andrew Murray giving an overview of SAP BusinessObjects mobile directions:

Highlights include:

• New SAP BusinessObjects experience mobile application, based on the Exploration Views functionality from the BusinessObjects innovation center, with demo data from Experience.SAP.com
• Easily take a mobile analysis and create a StreamWork activity for collaborative decision-making
• Easily “mobilizing” your existing BusinessObjects reports
• Support for each level of the “mobile needs hierarchy”, including fully customized mobile applications

• Demonstration of integration with RightHemisphere, a recent SAP acquisition that allows companies to synchronize visual and business data, combining a camera and schematic view:

Overlaying colors by temperature from data in the ERP system:

Finding a required part replacement using geolocation data:

Sybase (“an SAP Company”) recently published an analytics guide called “Intelligence for Everyone”.

The first section called “The Big Lie about Big Data” includes the following:

Why “big lie?” Because, as the article points with lots of historical examples, there’s always been a “crisis” in data storage, but as data volumes have risen technology has always evolved to deal with it, and that’s as true today as it has been in the past:

• 1956 Transaction data volumes overloaded current memory systems, leading to IBM’s creation of the first hard drive (costing $10,000 per Mb)
• 1970 Alvin Toffler’s book Future Shock popularizes the phrase “information overload”
• 1986 Technology critic Theodore Roszak:  “An excess of information may actually crowd out ideas, leaving the mind… distracted by sterile, disconnected facts, lost among shapeless heaps of data.”
• 1990 An IEEE conference features a session “Crisis in Mass Storage.”
• 1995 A Montreal data mining conference talks about the “data firehose phenomenon” swamping users

Interestingly enough, the guide manages to talk a lot about Big Data without mentioning what many people associate the term with: open source technologies such as Hadoop/Map Reduce. This is all the stranger because the latest version of Sybase IQ 15.4 includes a native MapReduce API and Hadoop integration, getting closer to the ideal architectures of the future that take the best of both worlds.

The guide also slides over the advantages of in-memory storage such as SAP HANA, noting just that “64-bit systems with their larger RAM space make this technology more attractive, if more expensive”

For more on SAP HANA and Sybase IQ, here’s analyst group Ovum’s take: “SAP’s HANA and Sybase IQ are separate but complementary“

The guide concludes that:

“Big Data is not to be feared. It’s to be exploited. The analytics industry today has no excuses when it comes to working with Big Data. It has no excuses when it comes to scaling their analytics data warehouse to include thousands of users. It has no excuses when it comes to applying analytics to variable data types from every imaginable source, including, for example, the vast unstructured information from social media sites.”

I was honored to speak at this years UK & Ireland SAP User Group Conference held in Birmingham, UK. The first main day saw great keynotes from the new SAP UK Managing Director, Steve Winter; Analyst Ray Wang; SAP’s CIO, Oliver Bussmann; and Olympic Sprinter Steve Cram. You can read a full review by Christoph Ziedler on SAP.info, and see the various videos on YouTube starting with the introduction by Alan Bowling, Chairman of UKISUG.

Here’s a selection of photos I took during the sessions:

And there are many, many more photos on the UKISUG Flickr Stream.

I gave the keynote on the second main day. Here’s the video taken by the organizers, including the slides:

You can download my slides in pdf format or in the original PowerPoint format.

Here’s a chart from a recent study by the Economist Intelligence Unit called “Big data: Harnessing a game-changing asset”, showing that only 9% of respondents believe that they use data worse than their competitors.

Let’s face it, they’re probably in denial. It’s often called “Illusory Superiority” or the “Lake Wobegon Effect” (a radio show where “all the children are above average”), and it’s a real force in the analytics market.

Because better data is often considered a “nice to have”, or “important but not urgent”, companies often only invest in better business intelligence when confronted with evidence that their competitors have the edge (this applies to BI vendors, too: long-clamored-for features mysteriously get prioritized the instant another vendor provides it).

This chart shows that companies are probably underestimating their competitors ability to use data – and therefore that they should probably be investing more in business intelligence…

Key texts:

• “Face-to-face conferences through television will be held coast-to-coast, and intricate calculations of quotas or sales by territories will be turned out at the touch of an assistant’s finger. Records will appear as if by magic from files automatically operated in the electronic age ahead.”
• “Television/telephone devices will eliminate distances”
• “All figuring will be done by miraculous electronic devices”.

Sure sounds like today’s analytics to me!