The real trend this year is not the technology. It’s about helping business people make better decisions, and actually change the way companies do business. Analytics has always been about transforming business, but the recent huge changes in analytic technology have created interesting new opportunities for business innovation.

Most organizations are now starting to understand the technical opportunities, but many struggle to apply those new opportunities to their business processes. This blog post attempts to explain what’s going on in the analytics market and give concrete examples of how other companies have implemented the new technologies in “game-changing” ways (sorry kittens).

Wrenching Change and A Foggy Outlook

The chart below illustrates the wrenching effects of recent financial problems on the world gross domestic product: companies today have to be ready to react to unprecedentedly fast changes to their economic environment.

And the economic environment is fraught with extreme uncertainty. This year, the people who run the world will change, and so will many of the policies of the countries they manage. Financial markets have still not completely stabilized, notably with the future of the Euro still not assured.

Companies have reacted to this uncertainty by slashing costs and accumulating cash, and now need to start investing that cash into future development. Since interest rates are low and the business outlook is still uncertain, many of them are using the money for new technology that can help them prepare for the future.

In particular, companies want better visibility about what’s going on in their market, and increased organizational agility in order to be able to deal with change fast. It’s like driving in the fog without a map – in order to survive, you should invest in better visibility, brakes, and steering to be able to spot and avoid fast-moving objects looming out of the fog.

Analytics provides these capabilities: business intelligence to peer into the road ahead, risk-management to provide fast alerts to new obstacles, and flexible financial planning systems to help swerve around them.

Analytics: Hotter Than Ever

Companies are investing heavily in analytics:

• Analytics is the #1 top technology priority for both CIOs and CFOs, according to Gartner
• Nucleus Research recently released a report showing that organizations get $10.66 of value for every $1 invested in analytics
• IDC has increased growth forecasts faced with stronger-than-expected figures for recent years
• IDC analyst Dan Vesset: “After three decades, the business analytics market is finally reaching the mainstream” and “There are few growth inhibitors in the foreseeable future”
• At the recent BI Gartner Summit in London, Gartner’s Dan Sommer announced an early estimate of 10%+ growth in analytics during 2011, outpacing general IT growth.

Fast-Moving Technology

Analytics technology has been changing fast. On the back end, new technologies have come together to provide what Gartner calls “extreme data performance”. These include in-memory, column data stores, in-database calculations, massively parallel architectures, complex event processing, Big Data / NoSQL / Hadoop, and cloud architectures.

The combination of these technologies provides a opportunity to access massive amounts of a greater variety of data, faster, and more flexibly. The key opportunity is that these new platforms “collapse the stack” so that organizations can implement and update analytic projects much faster than ever before.

And on the front end, various technologies are coming together to provide unprecedented levels of context-based “actionable insights”, including self-service data discovery, advanced visualization including maps, mobile analytics, predictive analytics, collaborative decision-support. They help provide more action-oriented interfaces optimized for the context of the users, both inside and outside the organization.

These technology advances are clearly important, and we’re going to continue to see great improvements this year. The new opportunities have reached a tipping point similar to the rise of digital cameras vs. analog photography – and you don’t want to leave it too late to make the change, like Kodak, which recently filed for bankruptcy protection!

However, the real opportunity is using these new possibilities not only to improve analytics but fundamentally rethink key business processes.

High Resolution Management

University researchers have pointed out that today’s management techniques are based on the limitations of information scarcity:

“How many times has someone in your company uttered, “We don’t have that level of accuracy in the information, so we have to make aggregated estimates”? Under the current paradigm, it is sometimes impossible to drill down and understand what is happening at a highly detailed level.”

They coined the term “High Resolution Management” to describe what becomes possible with the new technology opportunities:

“We contend that these technologies will change drastically how management makes decisions. Why? Because with access to the finest granularities of information, management will be able to move freely from macro to micro levels and will be able to measure, plan and act accordingly. With increased resolution come more options to drill down, eliminate inefficiencies and cut costs.”

Lets take a look at three different types of High Resolution Management opportunity, letting companies:

• Remove bottlenecks
• Rethink business
• Flip business models

Remove Bottlenecks

Better technology always means business opportunity, but the new analytic platforms are rapidly eliminating some of the key bottlenecks that have prevented organizations from getting value from their data:

Faster, more flexible data access. Companies like Red Bull have been able to speed up and simplify their data warehousing environments. Using the HANA in-memory database, the company can now load detailed data twenty-five times faster into their data warehouse, and they were able to eliminate several levels of data staging, increasing the flexibility of the solution.

Data volumes and complexity. Companies like Colgate-Palmolive, Provimi, and Danone have long had access to vast amounts of detailed data about their production facilities and sales channels – but the quantity of data meant that they were unable to run full analytics in a reasonable time frame. That has now changed. For example, according to Colgate-Palmolive CIO Tom Greene:

“We will be able to run analytics at a local level on specific brands and locations, and at the lowest level of detail in real time”

And Danone can now measure the carbon emissions of 35,000 different products, with new systems that:

“collect, measure, and analyze data across the entire product life-cycle, from sourcing through production, transport, retail, distribution, consumption, and end of cycle”

New forms of data: ‘unstructured’ data such as text has long been difficult to effectively analyze and incorporate into mainstream corporate analytics. The new systems make it much easier for companies like Medtronic to access and analyze the large amount of complaints and feedback data they receive about their products, combine it with other data sources, and provide it to business users with dynamic interfaces:

New interfaces and users. Companies like Altron have been able to get the data to their users where they needed it. As Debra-Lynn Marais, Group Information Manager explains:

“The days of our users and execs being in the office have gone. They work from home or on the road. We had to develop a solution that gets information out to where our people are. Everything we do is mobile first. In addition, it’s less cumbersome and cheaper to buy and use a tablet than any other form.”

Rethink Business

Many companies are going beyond “just” improving their existing analytic capabilities, using analytics in new ways to change the way they do business. Instead of analytics being something that is used to monitor and eventually improve a business process, analytics is becoming a more fundamental part of the business process itself.

Proactive Analytics. Instead of using analytics only to assess previous performance, companies are using the new capabilities to get data fast enough to make a real difference. For example, online grocer Fresh Direct, instead of just understanding what problems happened yesterday, can now understand what problems will happen in the next few hours, so they can actually fix them before a customer is impacted:

“FreshDirect has an operations center that manages its fleet of delivery trucks. In a large metropolitan area like New York, traffic doesn’t always flow predictably. A traditional approach to BI would be to print a report showing the level of on-time deliveries (OTDs) the day before and then ask the transportation department what went wrong for the orders that were delivered late. FreshDirect uses analytics in a more impactful way.”

“The company monitors the delivery rate of every truck and enters that data into the BI system on an ongoing basis. Every hour, it uses the previous hour’s data to predict how many deliveries will be on-time in the next hour. If the predicted OTD rate is below FreshDirect’s target, the company sends out an auxiliary truck or trucks to help make deliveries. The company holds 10 trucks in reserve for just this purpose.”

Integrated Risk Assessments. Among other products, publisher Houghton Mifflin Harcourt produces educational books. Schools pass orders in June or July after the end of the school year, and then expect delivery for the start of the next school year in September. Getting books printed during the summer is expensive, as many publishers compete for the limited supply of printers available.

To avoid these extra costs, Houghton Mifflin Harcourt uses using sophisticated, risk-based forecasting. The company prints books in January or February, when printing is much cheaper. In order to minimize of excess inventory, it has carefully analyzed all the causes of previous forecasts, and now takes account of all the different things that influence book obsolescence.

Before, the buying team just ordered based on the volume forecast from sales. Now they have much greater context for their decisions. For example, if there’s a vote coming up on schools funding that may result in the canceling of a math adoption program for the year, they can decide to hold back on those purchases until the outlook is clearer. The fast, more accurate forecasting mechanism has saved them tens of millions of dollars, and they have more of the products their customers want.

New Customer Services. International grocery chain Casino is rolling out a new mobile shopping application for its customers. It provides data from its enterprise systems directly to its customers, resulting in increased shopping convenience and increased customer loyalty.

German healthcare provider AOK (“the good health organization”) is committed to helping its members avoid illnesses in the first place. It is planning to introduce a new, market-differentiating service: personalized healthcare advice for each customer, with tools that:

“Conduct real-time analyses of the tremendous amounts of medical data we receive, recognize potential health risks, assemble various preventive care programs and respond to those risks appropriately and ahead of time.”

As an added bonus, they also believe that this tailored prevention program will result in significant cost reductions by preventing expensive unneeded treatments.

BC Hydro is saving $70 million dollars a year through the installation of new smart electricity meters, using SAP systems, and offering new services to commercial customers based on the new data possibilities. Companies like Centrica are planning to use SAP’s Smart Data Analytics, giving them deep understanding into consumer consumption.

Flip Business Models

The really interesting opportunity for businesses is where companies have managed to use analytics to fundamentally flip the way their businesses work: instead of analytics being part of a process, it “becomes the business model”.

Tim Ferriss, author of the 4-hour workweek, is an interesting example of this. He didn’t do what most authors do: write a book, and then figure out how to publicize it. He used an analytics-first approach: he bought Google Ads, with mockups of book covers, with a variety of titles of books that he might be interested in writing – and then wrote the book that got the most clickthroughs! This is one step beyond using analytics such as focus-groups, which are typically there to validate existing products. The next generation of products and services are being created “on the fly” based on an analysis-first approach.

The clothing brand Zara shook up fashion retailing with “analytics first” – instead of having a designer creating clothes and then trying to sell them six months later, they realized new manufacturing techniques meant they could create clothes “in the moment”. They could observe what people were wearing in the street, quickly make small batches of variations on that theme, and get them into the stores. If they sold well they made more, if they didn’t sell they discounted quickly. Instead of a season-oriented, “batch” business, they switched to a flow-oriented business, using new technology capabilities.

The new analytic platforms mean that this analytics-first approach is available to many more businesses than in the past. For example, T-Mobile is in the process of transforming the way they attract customers. Instead of laboriously creating a range of rate plans, promoting them, and analyzing the results, they now use analytics to automatically create hundreds of more complex, personalized rate plans. They then throw them out into the market, monitor in real time, and quickly cull any that aren’t successful. It’s a way of doing business that would have been inconceivable in the past, and a lot more common in the future.

Conclusion

2012 is the year to rethink your analytic technology to take account of new opportunities:

• On the back end, for extreme data performance
• On the front end, for actionable insights

And it’s time to rethink your business:

• Remove today’s bottlenecks to successful analytics caused by data volumes, data variety, or data access
• Rethink business processes by embedding real-time decisions
• Create new products and services that could only exist because of today’s analytic power

Organizations are using this technology to change the way they do business. If you run an analytics project, you are in the forefront of these changes – it’s your job to help explain to the rest of the business how these technologies should be changing their existing processes. Good luck!

___________________________________________________

If you’re interested in slides that go along with this article, please see this post about the recent Gartner BI Summit in London that includes a download of my presentation at the conference.

Marcus Borba:

• BI Predictions for 2009
• More BI Predictions for 2009

Vincent McBurney: 100 BI Predictions for 2009: Cloud is Hot, SOA is Cold and MDM is Irrelevant

Vincent put together a great graphic showing the "weight" of different predictions (although I suspect Stephen Few would have a field day with it)

Jorgen Heizenberg, Business Intelligence Predictions 2009: The paradox between demand and supply.

1. Rationalization
2. Mastering & Governing Core Data
3. Architecture The Other Way
4. Operational Excellence
5. Customer Centric
6. Using The Data
7. Overcoming Differences
8. Organizational Transformation
9. Need For Cooperation

Mike Ferguson: Some BI Ideas for 2009

• Integration of BI with Information Management infrastructure for trusted data
• Integration of BI with Performance Management software to roll up metrics into higher level KPIs
• Capturing of additional insight from unstructured content (e.g. customer emails) and from external information on the internet (e.g. about market intelligence and about what people are saying about your products and services)
• Event driven and on-demand Operational BI – a hugely exciting area for 2009 to continuously monitor operations and deliver right -time BI in the context of process activities for continuous business optimisation
• Integration of Enterprise Search with BI to open up broader access to intelligence via a search interface
• Exploitation of appliances for lower total cost of ownership on specific workloads
• Integration of BI with social software and collaboration workspaces to facilitate sharing and exploitation of knowledge in a collaborative environment. This is particularly relevant for those of you wishing to exploit products IBM Lotus Quickr quickplaces as well as Microsoft SharePoint workspaces. Integrating BI here will become increasingly important in 2009.

Craig Shiff: What’s Ahead for Performance Management in 2009?

• "2009 should be another great year for business performance management, the established vendors, consultants and, most importantly, the end users of performance management and business intelligence solutions."
• Companies will continue to invest in performance management.

"While only 9% said the economy is causing them to reduce their focus on BPM, more than 51% of the respondents so far agree that BPM is more critical than ever and are increasing their efforts in that area (the remainder say their BPM plans have not been impacted one way or the other by the current economy)."

• BPM will expand to become the key front-office decision system company-wide
• IFRS will drive more interest in BPM’s financial consolidation capabilities
• Some smaller vendors will disappear

Industry Predictions

Looking forward: Trends in IT management and budgeting for 2009

Economic downturn will hurt IT hiring in 2009
Survey: Economy puts nonessential IT projects on back burner
Shifting IT business models in time of economic crisis
IT and the recession: Focus on business strategy, smaller projects
Adjusting your IT budget in a volatile economy
Gartner: Restructuring top concern for CEOs in 2009
How the SEC’s proposed IFRS will affect your accounting systems

ITSinsider

2009 Enterprise 2.0 prognostications

JackBe: Web 2.0 What’s Out and What’s In

• Out: "Faster, Better, Cheaper", In: "Cheaper, Agile, Faster" (in that order)
• Out: Business Analysts, In: Mashup Analyst
• Out: SOA by IT for IT, In: SOA by IT for the Business
• Out: Service-Oriented Data, In: Decision-Oriented Data
• Out: Better Business Intelligence, In: Lesser Business Intelligence
• Out: SOA, In: SOA
• Out: SOA-in-house, In: SOA-on-the-cloud
• Out: Dashboards, In: Mashboards
• Out: Emailing Excel Spreadsheets, In: Mashing Excel Data
• Out: Silo Bashing, In: Silo Loving
• Out: Aligning Business and IT, In: Buying IT
• Out: SOA Architects, In: SOA Social Workers
• Out: Salesforce.com as a SaaS, In: Salesforce.com as a Business Portal

I’ve also added a couple of business and industry trend overviews that mention business intelligence. I’m sure I’ve missed some others — please don’t hesitate to comment…

2009 Business Intelligence Trends

Shawn Rogers: 2009 Business Intelligence Technology Poll

Take the Poll! See the latest results by clicking on the chart, or going here

Experts forecast business intelligence market trends for 2009

Wayne Eckerson

• Analytic database platforms go mainstream
• Open source BI gets evaluated
• Packaged analytic applications gain traction
• Software as a Service (SaaS) picks up in the midmarket
• Next-generation dashboards emerge.
• Analytical literacy improves
• More analytical sandboxes come to the fore
• BI goes green
• Advanced visualization corrals BI
• Event-driven analytic platforms hit the scene

James G. Kobielus

• BI moves into the cloud
• BI adopting Web 2.0 development paradigm
• BI growing more federated
• BI evolving into advanced analytic applications

Gartner Inc.  (various analysts)

• By 2012, business units will control at least 40% of the total budget for BI
• Through 2012, more than 35% of the top 5,000 global companies will regularly fail to make insightful decisions about significant changes in their business and markets
• By 2010, 20% of organizations will have an industry-specific analytic application delivered via SaaS as a standard component of their BI portfolio
• In 2009, collaborative decision making will emerge as a new product category that combines social software with BI platform capabilities
• By 2012, one-third of analytic applications applied to business processes will be delivered through large-grained application mashups

Rich Sherman: Business Intelligence & Data Warehousing Trends: 9 for ’09

1.    Economic concerns distorts IT budget decision-making
2.    Business intelligence expands
3.    IT expands its BI vendor shortlist
4.    Data integration  continues healthy growth
5.    Data integration breaks out from the “Magic Quadrant”
6.    Incremental beats out big bang projects
7.    “Good enough” replaces best-in-class
8.    SMB BI demands outstrip a constrained adoption
9.    Industry consolidation continues

David Stodder: Nine BI Megatrends for 2009

Megatrend 1: The Impact of Open Source

Megatrends 2, 3 & 4: BI Tool Innovation

• BI becomes less isolated
• Users demand a richer experience
• BI will focus on relationships

Megatrend 5: Business Modeling Meets MDM

Megatrends 6, 7 & 8: Breaking the BI/DW Mold

• MapReduce meets large-scale data analysis
• Column-oriented databases take aim at performance woes
• Event processing opens new analytical possibilities

Megatrend 9: Too Big to Fail

Ken Rudin: What’s in store for Business Intelligence in 2009?

• Cloud computing will cause a shift in the BI balance of power from IT to business users.
• Simplicity will be the driving mantra for both consumers and vendors of BI.
• The continued drive for simplicity will cause a shift towards prebuilt analytic solutions with best practices built in, and away from generic toolsets.
• Data interpretation will become a significant challenge for new BI users.

James Taylor: Predictions for 2009 (Enterprise Decision Management)

• Cloud computing will impact decision management
• More use of analytics by systems rather than people
• More focus on rules from application and platform vendors 
• More business rule vendors
• More rules in Business Process Management
• Business rules to decision management 
• Pre-built decisioning components 
• Simulation and scenario management 
• More business user control

Ted Cuzzillo: BI Transformation in 2009

• The few big tools will start giving way to many small tools.
• Business users will take more of BI back from analysts.
• Analytics will gain new importance.
• BI’s focus will sharpen on the human factor.
• BI will surge in the mid-market.

Colin White: BI Predictions for 2009: What Ever It Takes to Get the Job Done

• Quick and low-cost approaches will have most impact in 2009
• Open source software, BI software-as-a-service, low-cost application appliances, search, the integration of BI with collaborative and social computing software, rich internet applications, web syndication, and data and presentation mashups
• Line-of-business IT rather than the enterprise IT
• May lead to anarchy and islands of data and software

Neil Raden: Surround the Warehouse: Prediction for 2009

."..a move toward a broader data warehouse concept that includes data sources that are only connected to the data warehouse through some sort of metadata"

Oco Predicts 2009

• Growth of Complementary BI
• SaaS Goes Upmarket
• Information Silos Will Tumble
• Collaborative BI Throughout the Extended Enterprise
• Corporate Hot Spots Will Drive BI

Business and Industry Trends

The McKinsey Quarterly: Eight business technology trends to watch

Managing relationships

1. Distributing cocreation

2. Using consumers as innovators

3. Tapping into a world of talent

4. Extracting more value from interactions

Managing capital and assets

5. Expanding the frontiers of automation

6. Unbundling production from delivery

Leveraging information in new ways

7. Putting more science into management

8. Making businesses from information

7 Experts Paint Enterprise IT Landscape for 2009

Dana Gardner, Briefing Direct:

1. Shadow IT
2. Cut Costs.
3. High-Scale Business Intelligence
4. No Stomach for Upgrades
5. Social Data-CRM Mashups

Jim Kobieius, Forrester:

1. Obama
2. Cloud Computing
3. Recession
4. Governance, Risk and Compliance (GRC)
5. Social Networking

Tony Baer, Ovum:

1. Cost Savings
2. Low Cost or No Cost IT
3. Managed Clouds
4. IT Service Management
5. GRC

Brad Shimmin, Current Analysis:

1. Collaborative Social Networks
2. Cloud Software
3. Enterprise Oligarchy Models
4. Blended Internal and External Communities
5. Virtual Worlds Gain Foothold

Joe McKendrick, independent analyst

1. It’s the Economy.
2. IT Can’t Cut Too Much More
3. Enterprise 2.0
4. Cloud Economics
5. Low-Cost Methods to Reach Markets

Dave Linthicum, Linthicum Group

1. Cloud Computing Matures
2. Open Cloud Services
3. Some Cloud Social Connections
4. Rogue Clouds and PaaS.
5. SOA Gets Cloudy

Mike Meehan, Current Analysis

1. Take My Hardware, Please.
2. Tough License Negotiations
3. Easier Integration
4. Smooth SOA
5. Telecom Realignment

JP Morgenthal, Burton Group

1. Business Process Focus.
2. Social Networking Backlash
3. Era of Anti-IT
4. Millennial Workforce Shifts
5. Digital Rights Management Changes

IDC Predictions 2009: An Economic Pressure Cooker Will Accelerate the IT Industry Transformation

• Global IT growth will be cut in half
• Emerging markets and small businesses spending will slow significantly
• The IT industry’s expansion to "the cloud" will accelerate
• The struggling offline economy will drive more shoppers to the online economy
• The telecom industry will consolidate, and expand, in 2009
• It will be a grim year for mobile gadgets.
• The crumbling of the "business/personal" wall in IT will accelerate
• The reinvention of information access and analysis will accelerate in 2009
• Green technologies will have a good year, disguised as "cost cutting"
• Government initiatives in 2009 will catalyze massive IT investments and industry growth 

or "Reasons To Be Cheerful if Your Job Involves BI"

The Economy is Awful

Around the world, governments have drastically slashed their growth estimates for 2009.

And according to a recent IDC survey, more than half of us believe that the current economic crisis will last at least a year.

Faced with all the mayhem, it’s not surprising that today’s New York Times Op-ed includes the phrase "Let’s not mince words: this looks an awful lot like the beginning of a second Great Depression."

Some forecasters see a fast recovery, but this may say more about newspaper performance incentives than the state of the economy (counter-intuitive headlines drive readership, so it was just about inevitable that such an article would be written, no matter how few optimists). And depressions often have short periods of optimism before settling in for long-term misery (sometimes poetically called the "dead cat bounce").

So is 2009 going to be an terrible year for us all? The answer is clearly no: the misery isn’t spread equally. You probably wouldn’t want to work in the finance or auto sectors in 2009, but IT — and particularly business intelligence — looks to be one of the best possible sectors to ride out the crisis.

IT Spending is Still Increasing

IT spending growth expectations have been slashed as CEOs demand cutbacks — but they remain positive. According to IDC, worldwide IT spending is expected to grow 2.6% in 2009. Much of this will be driven by emerging economies (Central & Eastern Europe, Middle East & Africa, and Latin America), but even the established markets of the US and Western Europe will see positive growth.

Gartner research agrees and predicts that even in a worst case scenario, IT spending will increase 2.3% in 2009, providing a better outlook than the spending cuts seen during the dot com bust.

Within IT spending, hardware and services are expected to be the worst hit, with negative growth. Software spending, on the other hand, is still predicted to grow at over 4% world-wide.

These numbers are backed up by a survey from the Society of Information Management (SIM) carried out in November that showed that only 19% of CIOs and IT executives expect to cut their budgets in 2009.

Why this optimism? It’s partly because IT spending is only loosely connected to the current economy. It’s driven much more by the underlying infrastructure cycles, and some big drivers of IT spending show no sign of slowing down: commerce and customer service via the Internet; interactive communication devices; and increasing IT investment in small and medium-sized businesses.

Compared to previous recessions, there’s also more awareness that scrimping on IT can have dire consequences down the road. Good times means that there’s enough to go around for everybody. Tough economic conditions, perversely, provide the best opportunity to innovate and grow market share significantly, as the less-prepared fall by the wayside. The Economist, November 2008:

"Paradoxically, a recession can be a fantastic time to launch innovations. For one thing, tougher times can make consumers reconsider many of their purchasing decisions, leaving them open to trying something new. For another, a less crowded marketplace makes it easier—and cheaper—to create awareness of a new offering."

Business Intelligence Spending Even More Robust

According to Gartner, Business intelligence has been the number one technology for the last three years running, and a SIM survey of CIOs and IT executives showed business intelligence as the #2 most important "application and technology issue" for 2009, after anti-virus protection.

IDC, in a market study published in November 2008, predicted BI that spending will grow at a rate of almost 10% over the next 5 years:

"The business analytics software market continued to grow despite acquisitions and mature product offerings… We expect a slowdown in growth in 2009, but the fundamental drivers of adoption remain strong and will help the market to recover to previously forecast growth rates in 2010."

Why is business intelligence such a bright spot? It’s because BI is more important than ever in tough environments — as organizations make cuts, executives will only get once chance to make the right strategic decisions — as the old phrase has it: "measure twice, cut once".

Rumored to be one of the top searched-for terms on their web site, Gartner picked business intelligence as a top strategic technology for 2009:

"Business Intelligence (BI), the top technology priority in Gartner’s 2008 CIO survey, can have a direct positive impact on a company’s business performance, dramatically improving its ability to accomplish its mission by making smarter decisions at every level of the business from corporate strategy to operational processes. BI is particularly strategic because it is directed toward business managers and knowledge workers who make up the pool of thinkers and decision makers that are tasked with running, growing and transforming the business. Tools that let these users make faster, better and more-informed decisions are particularly valuable in a difficult business environment."  (emphasis added)

In an article called "Business Intelligence: A Bull in a Bear Economy", Madan Sheina of Ovum calls BI "recession proof technology":

"BI has proved to be an unusual segment of enterprise softwa
re in that it never seems to have a down-cycle. That’s because BI is a Janus-faced technology. If the economy is doing well then BI and data warehousing helps companies optimize their operations and grasp new and lucrative business opportunities before their competitors do. However, if the economy isn’t doing so well then BI becomes an effective cost-savings tool, allowing companies to squeeze greater cost efficiencies from their existing processes and resources, and identify and mitigate business risk." (emphasis added)

A Computerworld research white paper called "BI: Proven Tools for Competitive Advantage in Uncertain Times" reports (emphasis added):

"Turbulent economic times can impact expenditures and technology deployment plans. However, 42% of respondents expected their overall expenditures for BI tools and solutions to increase from 2008 to 2009. Even with the current state of the U.S. and world economies, respondents said they are extremely or very likely to invest or expand investments in BI in the areas of production reporting (48%), spreadsheets (47%) and ad hoc queries (45%)." (emphasis added)

Which Vendors Will Do Best?

It’s not clear who will benefit most from the BI spending growth in 2009.

On the one hand, people typically turn to established leaders in tough times. On the other hand, companies are reviewing their spending priorities, which may lead to opportunities for perceived lower-cost options such as open-source BI, or SaaS BI vendors (whether they actually turn out to be lower-cost is a longer conversation — and note that the leading vendors also provide SaaS BI options).

One study, at least, seems to show that SAP may be in a better position than its direct competitors:

"Companies seem more willing to spend on SAP products than those of other vendors, according to a recent UBS survey of the spending priorities of 100 CIOs. CIOs expected IT spending in general to decline about 2% over the previous year, according to the report. But when asked about demand for products from specific vendors, the CIOs expected to see spending increases in 2009 for SAP products but not for products from IBM, Oracle and Microsoft." (emphasis added)

Conclusion

So despite a weak economy, IT spending growth remains positive. Within IT, software spending is the healthiest. And within software spending, BI is a top priority.

So if your job involves business intelligence, you should bless your good fortunes in 2009, as you should can expect a (relatively) good year — because BI really does transform the way the world works.

First, I note that I didn’t really propose anything radical, and most of them are impossible to actually measure, but here goes my subjective assessment — always interested in your disagreements…

1. The BI World Divides into Three. Certainly seems like it to me, but unfortunately, I don’t know of any official figures that would back up my hunch that the remaining independents have been losing market share (IDC only recently came out with the 2007 numbers, which showed the independents staying stable/growing slightly from 2005 to 2007). And I was clearly wrong in predicting the acquisitions would continue in 2008.

2. Technology Goes up Another Level of Abstraction. Not yet. In fact, because of overlapping products, many conversations have gone down a level into the weeds of exactly how which interfaces connects to which data sources. (Although companies are certainly more interested in buying big bundles of everything they need, see next point)

3. A Surge in Standardization. A huge surge in interest, but relatively little actual standardization this year (the big deals are, of course, traditionally in 4Q).

4. An Increase in Innovation. Again, not yet — I think it’s happening, but it’s not yet public.

5. BI success. Awareness of BI among senior leaders has risen massively, but this hasn’t yet translated into greater project success.

So, overall: I still believe these are the underlying trends, but I forgot that there’s a big gap between trends and real-life deployments.

1. The BI World Divides into Three

The BI market will now be split into three different types: “megavendors” (disclosure: ich arbeite jetzt für einen Mega-Vendor), “independents”, and “open source”.

• The open source vendors are already in a separate-but-linked world from other BI vendors, and a new split will open up between the megavendors (SAP/Business Objects, Oracle/Hyperion, IBM/Cognos, Microsoft, and others who may enter, e.g. HP) and the remaining independents (SAS, Microstrategy, etc.). For example, this week, in response to customer requests, Gartner set up a special megavendor comparison session as part of their European BI Summit.
• Each of the markets will be driven by different dynamics, and will compete only indirectly.
• The megavendors will divide up most of the enterprise market, and try to extend further into the mid-market, and the independents will grab niches where they can (although in a growing market, there’s plenty of room for everybody to be successful)
• The independents will struggle to get the attention they feel they deserve, and sales may suffer because of it — but not yet enough to overcome Dr. Goodnight’s and Mr. Saylor’s reluctance to join other megavendor stacks.
• The megavendors will continue to reach into the pool of independents to extend their existing offers.

2. Technology Goes up Another Level of Abstraction

Remember the days when reporting, querying & OLAP were different products that came from different vendors? Humans hate complexity: as overlapping markets converge, expect to see simplification and clustering in order to keep up — e.g. products and purchasing decisions at the level of “Enterprise Information Management” or “BI Platform”, rather than separate decisions for data integration, data quality, etc.  Expect to see other vendors follow SAP’s lead on focusing on a smaller number of higher-level product bundles.

3. A Surge in Standardization

There aren’t many excuses left for organizations not to start standardizing / rationalizing their BI platform.

• Market maturity means that there’s lots of BI floating around in organizations and enough money is being spent to make it onto the radar of corporate purchasing departments (Gartner just announced that BI is the #1 priority for CIOs this year, for the third year running, and the trend is accelerating).
• Market consolidation has removed most of the fear, uncertainty and doubt that may have held back large organizations from choosing a single preferred vendor (If you have chosen SAP as your applications standard and you have a mix of Business Objects and Cognos, which way are you going to go?)
• Technology acquisition has meant the market-leading platforms are able to cover enough of the diverse user needs for standardization to be feasible
• The transition from vendors’ “old” to “new” platforms is now complete, with most customers on, or in the process of moving to, the new platforms, and no “legacy” features stopping them from increasing their deployments

4. An Increase in Innovation

This one’s more controversial, but I believe you get innovation when things get shaken up, and there’s been more of that in the last few months than there has in the last few years.

• Mixing teams. Yes, the larger vendors will need to spend time integrating their purchases, but at the same time, there will be lots of innovation from bringing together teams and initiatives in new ways (e.g. what happens if you mashed up, say, Polestar and T-Rex?, or Cognos’ tools with IBM’s ManyEyes?)
• Bigger ecosystems. Because of acquisitions at every level, technology that was niche last year will now be exposed to vastly increased audience of solution developers (e.g. Inxight’s text analytics technology).
• Differentiate or die. Faced with the megavendors and standardization, the smaller vendors will have to differentiate even more to make the sale (either of their products to customers, or of their company to a megavendor).

5. BI Success

I think it will be a golden year for BI where it really counts: successfully implemented BI projects. The technology is mature–what has held back wider, more strategic deployments has been a lack of business awareness, especially from senior executives. This will now change and the benefits will apply to every aspect of the industry.

• Vendors: I’ve already heard several stories of sales teams, used to dealing with say, “Senior BI Team Leaders”, being summoned to talk to the CIO to explain what the new market consolidation means in terms of opportunities for their organization.
• Customers: BI and Performance Management will play an increasingly large role in the standard McKinsey / Bain / Accenture etc. management presentations, making it easier to get funding and executive support for projects.