The real trend this year is not the technology. It’s about helping business people make better decisions, and actually change the way companies do business. Analytics has always been about transforming business, but the recent huge changes in analytic technology have created interesting new opportunities for business innovation.

Most organizations are now starting to understand the technical opportunities, but many struggle to apply those new opportunities to their business processes. This blog post attempts to explain what’s going on in the analytics market and give concrete examples of how other companies have implemented the new technologies in “game-changing” ways (sorry kittens).

Wrenching Change and A Foggy Outlook

The chart below illustrates the wrenching effects of recent financial problems on the world gross domestic product: companies today have to be ready to react to unprecedentedly fast changes to their economic environment.

And the economic environment is fraught with extreme uncertainty. This year, the people who run the world will change, and so will many of the policies of the countries they manage. Financial markets have still not completely stabilized, notably with the future of the Euro still not assured.

Companies have reacted to this uncertainty by slashing costs and accumulating cash, and now need to start investing that cash into future development. Since interest rates are low and the business outlook is still uncertain, many of them are using the money for new technology that can help them prepare for the future.

In particular, companies want better visibility about what’s going on in their market, and increased organizational agility in order to be able to deal with change fast. It’s like driving in the fog without a map – in order to survive, you should invest in better visibility, brakes, and steering to be able to spot and avoid fast-moving objects looming out of the fog.

Analytics provides these capabilities: business intelligence to peer into the road ahead, risk-management to provide fast alerts to new obstacles, and flexible financial planning systems to help swerve around them.

Analytics: Hotter Than Ever

Companies are investing heavily in analytics:

• Analytics is the #1 top technology priority for both CIOs and CFOs, according to Gartner
• Nucleus Research recently released a report showing that organizations get $10.66 of value for every $1 invested in analytics
• IDC has increased growth forecasts faced with stronger-than-expected figures for recent years
• IDC analyst Dan Vesset: “After three decades, the business analytics market is finally reaching the mainstream” and “There are few growth inhibitors in the foreseeable future”
• At the recent BI Gartner Summit in London, Gartner’s Dan Sommer announced an early estimate of 10%+ growth in analytics during 2011, outpacing general IT growth.

Fast-Moving Technology

Analytics technology has been changing fast. On the back end, new technologies have come together to provide what Gartner calls “extreme data performance”. These include in-memory, column data stores, in-database calculations, massively parallel architectures, complex event processing, Big Data / NoSQL / Hadoop, and cloud architectures.

The combination of these technologies provides a opportunity to access massive amounts of a greater variety of data, faster, and more flexibly. The key opportunity is that these new platforms “collapse the stack” so that organizations can implement and update analytic projects much faster than ever before.

And on the front end, various technologies are coming together to provide unprecedented levels of context-based “actionable insights”, including self-service data discovery, advanced visualization including maps, mobile analytics, predictive analytics, collaborative decision-support. They help provide more action-oriented interfaces optimized for the context of the users, both inside and outside the organization.

These technology advances are clearly important, and we’re going to continue to see great improvements this year. The new opportunities have reached a tipping point similar to the rise of digital cameras vs. analog photography – and you don’t want to leave it too late to make the change, like Kodak, which recently filed for bankruptcy protection!

However, the real opportunity is using these new possibilities not only to improve analytics but fundamentally rethink key business processes.

High Resolution Management

University researchers have pointed out that today’s management techniques are based on the limitations of information scarcity:

“How many times has someone in your company uttered, “We don’t have that level of accuracy in the information, so we have to make aggregated estimates”? Under the current paradigm, it is sometimes impossible to drill down and understand what is happening at a highly detailed level.”

They coined the term “High Resolution Management” to describe what becomes possible with the new technology opportunities:

“We contend that these technologies will change drastically how management makes decisions. Why? Because with access to the finest granularities of information, management will be able to move freely from macro to micro levels and will be able to measure, plan and act accordingly. With increased resolution come more options to drill down, eliminate inefficiencies and cut costs.”

Lets take a look at three different types of High Resolution Management opportunity, letting companies:

• Remove bottlenecks
• Rethink business
• Flip business models

Remove Bottlenecks

Better technology always means business opportunity, but the new analytic platforms are rapidly eliminating some of the key bottlenecks that have prevented organizations from getting value from their data:

Faster, more flexible data access. Companies like Red Bull have been able to speed up and simplify their data warehousing environments. Using the HANA in-memory database, the company can now load detailed data twenty-five times faster into their data warehouse, and they were able to eliminate several levels of data staging, increasing the flexibility of the solution.

Data volumes and complexity. Companies like Colgate-Palmolive, Provimi, and Danone have long had access to vast amounts of detailed data about their production facilities and sales channels – but the quantity of data meant that they were unable to run full analytics in a reasonable time frame. That has now changed. For example, according to Colgate-Palmolive CIO Tom Greene:

“We will be able to run analytics at a local level on specific brands and locations, and at the lowest level of detail in real time”

And Danone can now measure the carbon emissions of 35,000 different products, with new systems that:

“collect, measure, and analyze data across the entire product life-cycle, from sourcing through production, transport, retail, distribution, consumption, and end of cycle”

New forms of data: ‘unstructured’ data such as text has long been difficult to effectively analyze and incorporate into mainstream corporate analytics. The new systems make it much easier for companies like Medtronic to access and analyze the large amount of complaints and feedback data they receive about their products, combine it with other data sources, and provide it to business users with dynamic interfaces:

New interfaces and users. Companies like Altron have been able to get the data to their users where they needed it. As Debra-Lynn Marais, Group Information Manager explains:

“The days of our users and execs being in the office have gone. They work from home or on the road. We had to develop a solution that gets information out to where our people are. Everything we do is mobile first. In addition, it’s less cumbersome and cheaper to buy and use a tablet than any other form.”

Rethink Business

Many companies are going beyond “just” improving their existing analytic capabilities, using analytics in new ways to change the way they do business. Instead of analytics being something that is used to monitor and eventually improve a business process, analytics is becoming a more fundamental part of the business process itself.

Proactive Analytics. Instead of using analytics only to assess previous performance, companies are using the new capabilities to get data fast enough to make a real difference. For example, online grocer Fresh Direct, instead of just understanding what problems happened yesterday, can now understand what problems will happen in the next few hours, so they can actually fix them before a customer is impacted:

“FreshDirect has an operations center that manages its fleet of delivery trucks. In a large metropolitan area like New York, traffic doesn’t always flow predictably. A traditional approach to BI would be to print a report showing the level of on-time deliveries (OTDs) the day before and then ask the transportation department what went wrong for the orders that were delivered late. FreshDirect uses analytics in a more impactful way.”

“The company monitors the delivery rate of every truck and enters that data into the BI system on an ongoing basis. Every hour, it uses the previous hour’s data to predict how many deliveries will be on-time in the next hour. If the predicted OTD rate is below FreshDirect’s target, the company sends out an auxiliary truck or trucks to help make deliveries. The company holds 10 trucks in reserve for just this purpose.”

Integrated Risk Assessments. Among other products, publisher Houghton Mifflin Harcourt produces educational books. Schools pass orders in June or July after the end of the school year, and then expect delivery for the start of the next school year in September. Getting books printed during the summer is expensive, as many publishers compete for the limited supply of printers available.

To avoid these extra costs, Houghton Mifflin Harcourt uses using sophisticated, risk-based forecasting. The company prints books in January or February, when printing is much cheaper. In order to minimize of excess inventory, it has carefully analyzed all the causes of previous forecasts, and now takes account of all the different things that influence book obsolescence.

Before, the buying team just ordered based on the volume forecast from sales. Now they have much greater context for their decisions. For example, if there’s a vote coming up on schools funding that may result in the canceling of a math adoption program for the year, they can decide to hold back on those purchases until the outlook is clearer. The fast, more accurate forecasting mechanism has saved them tens of millions of dollars, and they have more of the products their customers want.

New Customer Services. International grocery chain Casino is rolling out a new mobile shopping application for its customers. It provides data from its enterprise systems directly to its customers, resulting in increased shopping convenience and increased customer loyalty.

German healthcare provider AOK (“the good health organization”) is committed to helping its members avoid illnesses in the first place. It is planning to introduce a new, market-differentiating service: personalized healthcare advice for each customer, with tools that:

“Conduct real-time analyses of the tremendous amounts of medical data we receive, recognize potential health risks, assemble various preventive care programs and respond to those risks appropriately and ahead of time.”

As an added bonus, they also believe that this tailored prevention program will result in significant cost reductions by preventing expensive unneeded treatments.

BC Hydro is saving $70 million dollars a year through the installation of new smart electricity meters, using SAP systems, and offering new services to commercial customers based on the new data possibilities. Companies like Centrica are planning to use SAP’s Smart Data Analytics, giving them deep understanding into consumer consumption.

Flip Business Models

The really interesting opportunity for businesses is where companies have managed to use analytics to fundamentally flip the way their businesses work: instead of analytics being part of a process, it “becomes the business model”.

Tim Ferriss, author of the 4-hour workweek, is an interesting example of this. He didn’t do what most authors do: write a book, and then figure out how to publicize it. He used an analytics-first approach: he bought Google Ads, with mockups of book covers, with a variety of titles of books that he might be interested in writing – and then wrote the book that got the most clickthroughs! This is one step beyond using analytics such as focus-groups, which are typically there to validate existing products. The next generation of products and services are being created “on the fly” based on an analysis-first approach.

The clothing brand Zara shook up fashion retailing with “analytics first” – instead of having a designer creating clothes and then trying to sell them six months later, they realized new manufacturing techniques meant they could create clothes “in the moment”. They could observe what people were wearing in the street, quickly make small batches of variations on that theme, and get them into the stores. If they sold well they made more, if they didn’t sell they discounted quickly. Instead of a season-oriented, “batch” business, they switched to a flow-oriented business, using new technology capabilities.

The new analytic platforms mean that this analytics-first approach is available to many more businesses than in the past. For example, T-Mobile is in the process of transforming the way they attract customers. Instead of laboriously creating a range of rate plans, promoting them, and analyzing the results, they now use analytics to automatically create hundreds of more complex, personalized rate plans. They then throw them out into the market, monitor in real time, and quickly cull any that aren’t successful. It’s a way of doing business that would have been inconceivable in the past, and a lot more common in the future.

Conclusion

2012 is the year to rethink your analytic technology to take account of new opportunities:

• On the back end, for extreme data performance
• On the front end, for actionable insights

And it’s time to rethink your business:

• Remove today’s bottlenecks to successful analytics caused by data volumes, data variety, or data access
• Rethink business processes by embedding real-time decisions
• Create new products and services that could only exist because of today’s analytic power

Organizations are using this technology to change the way they do business. If you run an analytics project, you are in the forefront of these changes – it’s your job to help explain to the rest of the business how these technologies should be changing their existing processes. Good luck!

___________________________________________________

If you’re interested in slides that go along with this article, please see this post about the recent Gartner BI Summit in London that includes a download of my presentation at the conference.

Next Wednesday, February 15th, from 3pm to  4.30pm PST, I will be moderating a panel on the theme of Products+Social=Better Products as part of Social Media Day at SAP Palo Alto (and Online).

We’ll discussing how to make better products thanks to social techniques. Please join us! You can register here and the conference hashtags are #SMW12 and #SAP.

Panel Session Outline

Here’s the outline of the panel session:

• 3:00-3:15 Key themes and Introductions:
• 3:15-3:35 Improving products through social techniques:
• 3:35-3:55 Creating social-enabled products:
• 3:55-4:15 Creating new products on top of social networks:
• 4:15-4:30 Wrap-up and final thoughts

These times are just a guideline – we’ll go where the conversation takes us. We will be encouraging the audience (both local and online) to tweet questions and comments, and I will pass them on to the panelists.

Here are some more details about the panelists and the show:

Meet the Panelists

In alphabetic order, here are the participants of the panel – click on their name to see their profile on twitter.

	David Brockington is a solution manager responsible for SAP’s new collaboration solution, SAP StreamWork. He has been an evangelist and thought leader within SAP on social business topics. Previously, he spent 10 years in various product management roles helping grow SAP Business Objects Enterprise from its very first iteration to the number one market leading business intelligence solution.
	Timo Elliott is a 20-year veteran of SAP BusinessObjects, and has spent the last quarter-century working with customers around the world on information strategy. He works closely with SAP research and innovation centers around the world to evangelize new technology prototypes.His popular Business Analyticsblog tracks innovation in analytics and social media, including topics such as augmented corporate reality, collaborative decision-making, and social network analysis.His PowerPoint Twitter Tools lets presenters see and react to tweets in real time, embedded directly within their slides. Elliott presents regularly to IT and business audiences at international conferences, on subjects such as why BI projects fail and what to do about it, and the intersection of BI and enterprise 2.0
	Ross Mayfield is the Vice President of Business Development at SlideShare, developing strategic partnerships and helping organizations engage with the world’s largest professional sharing community.He is also currently Chairman & Co-founder of Socialtext, the Enterprise Social Software pioneer, and was the founding CEO. Previously, Mayfield Co-founded and served as President of RateXchange, a publicly traded B2B commodity exchange for telecommunications.Mayfield is a former advisor to the Office of the President of Estonia and began his career in the non-profit sector. He holds a B.A. in Political Science from the University of California at Los Angeles and completed the Management Development for Entrepreneurs (MDE) program of the Anderson School of Business
	Kuhan Milroy is Director of Social Business Innovation, SAP Community Network at SAP. By bringing the social element to innovation at SAP, Kuhan is responsible for transforming the way SAP and customers, partners, and consumers innovate together to help shape the future of SAP’s products and solutions and more importantly the businesses, industries and markets the products serve. He manages initiatives surrounding the community and bringing a social approach to how SAP innovates, and researches how SAP can expand beyond small group to mass users for ideas and innovation.
	Lisa Joy Rosner is the Chief Marketing Officer at NetBase, where she works with the largest brands in the world—including Coca-Cola, Kraft and HP—as they change their approach to understanding consumers’ desires.Ms. Rosner is a tenured marketing executive in Silicon Valley, who is known for her creative and educational marketing approach. Before joining NetBase, she was an e-commerce expert, having served as vice president of Worldwide Marketing at BroadVision Inc. and vice president of Marketing at MyBuys, where she worked with such companies as Sears, Wal-Mart and Circuit City.Prior to that, Ms. Rosner worked in the data warehouse market at Brio, DecisionPoint and Oracle. Lisa Joy has served on the marketing advisory board for the Silicon Valley Red Cross and the Content Committee of Shop.org, the AMA and Benchmark. She was named the Gold winner of the Great Minds Award by the ARF in 2011. Ms. Rosner received a bachelor’s degree (sum cum laude) in English literature from the University of Maryland. She is a member of Phi Beta Kappa
	Nicholas Webb is one of the countries top thought leaders in the area of innovation. His books include The Innovation Playbook, The Digital Innovation Playbook, and The Innovation Superstar Workbook.As a Senior Partner at Lassen Innovation he helps some of the best companies in the world drive innovation through improved customer insights and true innovation best practice.

As always, it was a huge pleasure to catch up with customers, colleagues, analysts, partners, and competitors at one of Europe’s largest BI conferences. It was a great show overall, and I came away even more optimistic about analytics for the coming year.

Here are Jason Rose and I having an off-the-cuff discussion of what we found interesting about the latest Gartner BI Summit 2012 in London this week:

And here are some longer thoughts about the conference compared to previous years:

The band played on: Gartner analysts Nigel Rayner and Andreas Bitterer performed the opening live show, loosely based on Tubeway Army’s “Are Friends Electric”, to tie in with the opening keynote analogy of “information as electricity” (I’m a fan of ‘80s electonica, but it might have been more fun to have something by AC/DC with Nigel as Angus Young… )

It’s no longer Business Intelligence! Gartner has bowed down to the market trend and dropped the unwieldy category name “Business Intelligence and Performance Management” in favor of the simpler umbrella term “Business Analytics”, following in the footsteps of other analysts (e.g. IDC) and vendors (SAS, SAP, etc.). There was also a shift from BI Competency Centers (BICCs) to Business Analytic Teams (BATs) – apparently because the word “center” (a) doesn’t really represent the reality of diversified real-world organizational structures and (b) has a negative connotation in the US (think of your experiences with shared service centers).

Was it just a simple name change? There was a half-hearted attempt in some sessions to associate Business Analytics with “more than BI”, emphasizing that business change must be the result, not just reports. This is of course absolutely true and essential – but this was always part of their previous category definition. It was clearly a recent change: all the conference content reflected the old naming.

Should we care about this change? No! Here’s an blog I wrote a while ago on business intelligence vs business analytics, with the conclusion: “everybody has an opinion, nobody knows, and you shouldn’t care”. In particular, if you need to continue to call it “business intelligence” to communicate with somebody who is comfortable with that term, then you should continue using it!

Analytics is hot! The conference was literally packed, with people uncomfortably stuffed into a slightly too-small venue. Every indicator pointed towards analytics having another banner year – it’s back to the #1 technology priority for CIOs, it’s estimated to be growing at 10%, faster than overall IT spending, and the number of users is set to rise to 50% by 2014. Next year’s Gartner BI Summit (BA Summit?) is going to be in Barcelona, and that decisions is apparently at least in part because of the need for extra space.

Less technology, more business, more success. Last year’s opening keynote presentation was about the “four Vs”: volume, variety, velocity and validity, and talked a fair amount about technology. This year the emphasis had very clearly moved to business value – emphasizing the “why” rather than the “how” – analytics has to support business decision making and result in business innovation. I attended several excellent sessions on how to make analytics sessions more successful – for example, emphasizing that 20-30% of your time should be spent on “marketing” your analytic solution.

Tell stories, and make a difference. Out of the nominees for the Gartner BI Excellence awards, Medway Youth Trust shone out. They would have been favorites for the award anyway, because they are doing social work for the community, but there were three other other good reasons they won over the other nominees:

1. While all the nominees had clearly done a great job of successfully implementing business intelligence in their organizations, Medway had the best case for both “technical innovation” (using text analytics to get value from unstructured information) and “business innovation” (the data really make a difference to their organization, by allowing them to focus their limited resources on the key business goal)
2. As a small organization, they showed that you don’t need to have a big budget or a big team of people in order for analytics to make a difference.
3. They told their story better. In particular, a Spanish insurance company did a slick presentation about the corporate benefits of their standardized BI efforts, and quoted some impressive figures, but there wasn’t really a concrete example of how they had really made a difference that the audience could connect with.

Overall, it reminded me how vital it is to have real stories to tell people when trying to sell the benefits of your project – ROI isn’t enough: it has to be about people, and (surprising) business change.

Validation of current trends. Analytic technology trends were covered in in-depth sessions:

• Gartner called in-memory a “strategic imperative”, and advised that organizations should look at in-memory as “a quantum leap in their computing strategy” because “dramatically faster data access can profoundly change the nature of some applications”
• Mobile business intelligence is clearly now a given in the market – for example, all the participants of the vendor panel agreed that mobile BI is not going to be a long-term differentiator (although the underlying mobile device management certainly still might be).
• Cloud BI is still something for the future for most attendees – for many, only when the underlying operational systems are themselves running in the cloud.
• Sessions on social networking analytics and operational analytics were no longer marginal, with full crowds.
• The topic of “big data” – or “extreme data” as Gartner prefers to call it – is embedded in the new notion of a “logical data warehouse” that is poised to replace today’s more monolithic structures. One analyst mentioned that big Gartner customers were ripping up their current data warehousing plans and adapting them to the new technology possibilities. A session on big data by Roxanne Edjlali (formerly with Business Objects) was well-attended and well-received.

Big data not big enough? Overall, I don’t think Gartner had quite taken enough account of the appetite for more information about big data topics such as Hadoop, data science, etc. Every session with big data in the title was completely packed, and there didn’t seem to be many people from those communities at the show. I hope that Gartner’s conference team targets the big data constituency more aggressively next year — it would be a shame if people with the same underlying goals (turning information into business innovation) end up going to different conferences just because of some differences in the technologies they use (big data conferences are booming).

Fail in the right direction. Tim Harford, the Undercover Economist, was the guest keynote speaker. His presentation was very entertaining, but in general only tangentially related to analytics. The overall theme did resonate, however: that nobody has all the answers, and that it’s only through being humble about your knowledge that you have a change to succeed. The key is to “fail in the right direction”: make experiments, iterate, and learn from experience in order to move ever closer to better solutions.

Overall theme: going with the flow? This wasn’t really mentioned at the show, but if I had to pick one overall theme, it would be the move from batch-based BI to a greater appreciation of information flow, at every level of implementing systems and consuming information. New data warehouse technologies allow organizations to gather and structure information faster (and this is important: Bill Hostmann estimated that fully 70% of the requirements of a BI project change in the first year alone). Data discovery tools allow business people to iteratively structure and access new information in new ways. And businesses are realizing that analytics isn’t just something that you use to improve business processes: it can and should be part of the business processes themselves.

My presentation at the conference, “Business In the Moment, From Reactive to Proactive” was along the same lines – while there has been lots of technology change over the coming year, many organizations are still struggling to turn that new technology into business innovation opportunities. I talked about the big changes in the technology landscape and gave examples of organizations that had used these technologies to transform the way they did business, through removing business bottlenecks, rethinking business processes, or flipping business models to an “analytics first” approach.

You can download the slides in Microsoft Powerpoint or Adobe PDF format, and I’ll explain the main themes in a separate post. I look forward to Barcelona next year!

As part of this year’s Social Media Week, I’ll be hosting one of the sessions at a free one-day event held at SAP Palo Alto on February 15th. It’s entitled “Social Business: Designing Social into Products”, and I will be accompanied by a star-studded lineup of guests: meet the panelists. Please join us by registering for the session or join via online streaming on the SAP Facebook page, and the hashtag for the event is #SMW12

In this post, I’d like to share some ideas about why I believe that there are now fantastic new opportunities to improve all kinds of products using social media techniques.

PLEASE GIVE YOUR FEEDBACK. This blog post and the session I’ll be hosting are also “products”, in a sense, and I’d like to use the techniques outlined below to make them as good as possible: please post your feedback and any answers you have to the questions below, or suggestions for great panelists, and I’ll use that feedback to improve the session above!

Introduction

I believe that we’re still in the era of a “horseless carriage” version of social media: we added a motor, but kept the rest of the carriage.

In other words, while we understand how the new technologies work, we’re still tending to bolt the new social techniques onto our existing processes, rather than fundamentally rethinking those processes in the light of the new opportunities.

This tends to be true of all new technologies, of course. But for some reason there seems to be a bigger gap than usual – perhaps it is because the “trees” are so obvious (social media analytics, enterprise collaboration, etc.) that they tend to obscure the “wood”: the opportunity to sweep away existing bottlenecks in our business processes.

Social media is too often a marginal activity that people are happy to leave up to a dedicated team elsewhere in the organization, rather than embedded in everything we do. This post looks in particular at how social media techniques can be applied to the process of product creation.

Social / Product Trends

Why introduce something new? Why can’t we just keep doing what we’re doing today? Let’s start with some of the background trends:

Transparency: Whether you have a great product or an awful one, prospective customers can get the information they need directly from unbiased peers. This means that traditional product sales and marketing is being marginalized, and that core product quality becomes even more fundamental.

A great product – one that customers are delighted to own and use, and talk about to other people – can now take off at lighting speed, with almost no promotional cost. And news about product problems or poor service can spread even faster. The good news is that product creators can communicate with their customers more cheaply than ever: we may be naked, but we have a megaphone. It has to be used wisely. Honesty and credibility are essential values when talking about your products to the market.

Direct Contact With Customers: In large product organizations, there’s often a big communications gap between the people creating products and the people using them. The creators and designers of software, for example, have typically had to rely on other people to research the needs of potential users. The researchers then pass on that information in the form of “consumer profiles”, “personas”, or “ethnographic research”, which is used as a basis for creation. Something often gets lost in the translation, but is often seen as the only feasible way to operate.

Software engineers (for example), frustrated by this limited visibility, complain how hard it is to get access to customers, who are often protectively fenced off by sales teams (perhaps worried that developers might let too much of the truth slip out about product bugs or delays).

The advances of social media means all this can now change: vast numbers of potential users are only a few mouse-clicks away, participating in industry forums, complaining about alternative products, or talking about their favorite features.

Network Leverage: There are now socially-enabled running shoes, socially-enabled cameras, socially-enabled toys, and socially-enabled enterprise software. Almost any product can now be “social”, and hence experience network effects that may outweigh the other product features.

Extended Ecosystems:By embedding more use of social techniques into product creation and selling, we’re inevitably creating more complex, interactive networks of ecosystems around our products, with customers, partners, suppliers of social networking, etc.

How do “Social” and “Product” Interact?

I believe there are three main ways in which we can create new or better products through social media techniques. It’s clear, however, that there is still a lot to learn before these techniques become commonplace — in each section, I’ve added some of the questions I believe need to be addressed: again, any feedback you have is more than welcome!

Social-improved products

First and most obviously we can use social media to improve the way we create existing products. New techniques include:

Social Research. It’s now easy to find data about new opportunities, such as customers complaining about business problems or competitor products. And it’s easy to get customer feedback on problems with our own products. Given the potential for better products, I believe we should be investing extensively in these new areas. Questions:

• What valuable data is available now that wasn’t before?
• What are the costs and opportunities associated with these new techniques?
• How much time should product creators spend communicating with communities vs. creating products?
• How do systems have to change to ensure that this type of research is consistently integrated into the product creation process?
• What are the real-life limitations of such research? i.e. what kinds of important data can we not get with these processes?
• What social research tools can we build into the product experience? (e.g. making it easy to invite others when using the product, and tracking success, or an online game maker tracking the price of different virtual weapons)

Ideation. One of the most painful parts of any product creation process is prioritization – we can never make a “perfect” product. There will always be some compromise in terms of functionality or cost. New ideation platforms, such as SAP’s Idea Place offer an opportunity to ask customers and potential customers to give their feedback directly on possible new features and what compromises to make.

These opportunities are not limited to software or technical products – consumer goods companies can run surveys on online forums, authors can ask online discussion boards for plot ideas for their next book, etc. This gets us closer to “crowdsourcing” the creation and improvement of products. Questions:

• When is ideation not appropriate?
• What types of products and features work best for ideation?
• How can we motivate our customers and prospects to participate?
• What level of transparency is appropriate?
• How do we handle rejection of non-chosen products and features?
• What are the dangers of competitors seeing the data, or gaming the results?

Social Prototyping. Basic ideation isn’t enough. I’m sure we can all think of an experience where we didn’t realize we wanted or needed a particular product until we tried it out. Product designers, after all, can have great ideas of their own, based on their deep market knowledge. One key problem today is that somebody in a company may what they believe is a fantastic idea for a new and different product. But in order to pursue the product, they need resources and permission of several layers of management.

Those managers may not have any real frame of reference to determine if the new product is a real opportunity or not, and may not be incented to take any risks. This can result in some combination of dissatisfied product creators (if the idea is rejected), wasted time (slow decision-making at each level ), or wasted money (if the idea is accepted, but the product fails). But using social media, it’s now much easier to create fast prototypes (mockups, concept version, wireframes, etc.), and then make them available to customers for testing and feedback.

The benefit is that it’s much clearer whether a product really does appeal to customers or not, helping the prioritization process. The car industry has long done this with “concept cars”, and SAP has tested these techniques with through its SAP Research Prototyping group. Ideas such as integration with Google Maps were shown to be extremely popular (and so were rushed into production) while some ideas weren’t interesting (and the person proposing the new feature had a learning experience). Questions:

• How can we introduce more extensive prototyping and social feedback into our product creation processes?
• How do we decide if a prototype is successful enough to productize?
• Are there any other benefits to this type of process? (marketing, thought leadership, etc.?)
• Does this approach cost more or less than existing methods?

Social-Enabled Products

We can integrate social media into products to improve their usefulness or effectiveness. Games you can play with other people in your social network are more interesting that games you play on your own. Our devices are increasingly wired to be able to share information – you can buy applications and shoes that share information socially on platforms such as RunKeeper. Runners can use the social-enabled devices to share data with a coach, boast of their achievements, embarrass themselves into improving their times, or let relatives track where they are during a marathon. And if you’re logged into Facebook when you visit the site, it will tell you which of your friends are already using the products.

Hybrid cars can keep track of your fuel consumption, so you can compete with your friends about who is the most sustainable driver. Restaurant guides can give us information based on the ratings given by our friends and other restaurants we’ve visited on foursquare or “liked” on Facebook. Enterprise software vendors can build collaboration into existing business applications, letting people apply social media techniques to supply chain collaboration or track the progress of sales deals. Even Lego is becoming social.

• What are your favorite examples of social-enabled products?
• How important is social enablement compared to other features of a product?
• Do product creators have to be aware of new power-players in the social eco-system?
• What other products should include social but don’t today?
• What about the limits of social privacy when using such products?

New Products On Top of Social

There are opportunities to create new products “on top of” social networks, or as by-products of them. Companies such as LinkedIn have been able to create new “products” based on the data gathered in their networks, such as “Talent Match” or “Jobs You May Be Interested In”. New tools could help improve the success or failure of a big merger by analyzing the different social networks within the two organizations over time. Companies could develop more sophisticated “friends and family” offers for their products. Car-sharing services could leverage social networks to improve usage rates.

• What are some other good examples of leveraging social networks to create new products and services?
• Is this something that the rest of us even need to think about?

Conclusion

We’re a long way from “build it and they will come”, but we’re not yet at “come build it with us”. I look forward to your feedback!

Interested in Social Media? (and everybody should be, because these techniques are changing EVERY business process, not just marketing). Come and join us!

As part of this year’s Social Media Week, SAP Palo Alto will be hosting a free, day-long non-SAP-focused event on Wednesday, February 15th with a glittering array of industry experts discussing a variety of key social topics:

Just some of the participants: Maggie Fox, CEO of Social Media Group | Ray Wang, CEO of Constellations Research|Jonathan Becher, Chief Marketing Officer of SAP | Barbara Holzapfel, SVP & Managing Director of SAP | Rachel Happe of The Community Roundtable | Srini Tanikella, SCN community member |Kimarie Mathews of Wells Fargo | Deirdre Walsh of Jive Software | Brian Ellefritz, SAP Social Media Strategic Services | Lisa Joy Rosner, Chief Marketing Officer at NetBase |Dave Brockington, Product Marketing & Strategy, StreamWork | Me!

You are invited to attend on-site if you’re in Silicon Valley, but the sessions will also be available through online streaming on the SAP Facebook page.

I’ll be hosting the last session of the day, on Social Media and Products – more details about that session in a separate blog post: Social+Product=Better Products.

Please join us! Click on the links below to find out more and to sign up to attend one or more of the sessions (sorry, to attend the whole day, you have to sign up four times…)

SAP Social Media Day, Wednesday, February 15th

Location: 3410 Hillview Avenue, Building 1 Café, Palo Alto

Hashtags #SMW12 #SAP, and find event-related blogs on the SAP Community Network and SocialMediaWeek.org.

I presented at my first conference of the year last week, the Heliview Business Intelligence and Data Warehousing 2012 conference in ‘s Hertogenbosch, Netherlands.

The first keynote, by Erasmus scholar Dr. Roeland Dietvorst, was about “Performance Management in the Brain”, and a subject close to my heart: the (severe) limitations we evolved apes have when trying to make rational decisions. He illustrated his point with examples of great research that show that:

• Including “irrelevant” product choices can make big changes to preferences;
• Our emotional state has a big influence on what choices we make;
• Men shown pictures of attractive women tend to make worse financial decisions;
• etc

He also reviewed some of his own research, scanning the brains of sales people to see if there was a correlation between brain activity, competence in understanding other people’s state of mind, and selling skills. He finished by underlining that we have “two brains,” and that business intelligence can help us move decision making to our more rational, less emotional side.

I did the other morning keynote on “Real time enterprise from theory to practice”:

“The amount of information we face is growing by the day. Both volume and type of information. IT organizations not only incorporate customer or product data, but new sources such as location data and data streams from social media play a significant role. In addition, we want information available in the context in which we operate, and preferably independent of the site or device. To make this possible innovative solutions are needed that can work with large volumes of data. In-memory computing may by some be considered as not yet available but the reality is different. There are several examples where this is being successfully implemented and numerous organizations are achieving demonstrable benefit. This presentation shows that the real-time revolution has already started and how it’s being practiced today.”

The presentation reviews the state of the BI market and technology, and ends with some examples of companies using information to change the way they do business. As usual, here’s a copy of the slides, in Adobe PDF and Powerpoint PPTX format.

I’ll be in London next week presenting at the Gartner BI Summit 2012 — hope to see you there!

Update: here’s a nice write-up from the Dutch team about the event.

Last year, Ajay Ohri of the DecisionStats web site asked me to predict the top three analytic trends for 2011. He recently challenged me to score how well I did – here’s a copy of the post he put on his site, with the predictions and some comments on what actually happened during the year:

(1) Analytics, reinvented. New DW techniques make it possible to do sub-second, interactive analytics directly against row-level operational data. Now BI processes and interfaces need to be rethought and redesigned to make best use of this — notably by blurring the distinctions between the “design” and “consumption” phases of BI.

Score: 10. I spent most of 2011 talking about this theme at various conferences: how existing BI technology is rapidly becoming obsolete and how the changes are akin to the move from film to digital photography. Technology that has been around for many years (in-memory, column stores, datawarehouse appliances, etc.) came together to create exciting new opportunities and even generally-skeptical industry analysts put out press releases such as “Gartner Says Data Warehousing Reaching Its Most Significant Inflection Point Since Its Inception.” Some of the smaller BI vendors had been pushing in-memory analytics for years, but the general market started paying more attention when megavendors like SAP painted a long-term vision of in-memory becoming a core platform for applications, not just analytics. Database leader Oracle was forced to upgrade their in-memory messaging from “It’s a complete fantasy” to “we have that too”.

(2) Corporate and personal BI come together. The ability to mix corporate and personal data for quick, pragmatic analysis is a common business need. The typical solution to the problem — extracting and combining the data into a local data store (either Excel or a departmental data mart) — pleases users, but introduces duplication and extra costs and makes a mockery of information governance. 2011 will see the rise of systems that let individuals and departments load their data into personal spaces in the corporate environment, allowing pragmatic analytic flexibility without compromising security and governance.

Score: 6. The number of departmental “data discovery” initiatives continued to rise through 2011, but new tools do make it easier for business people to upload and manipulate their own information while using the corporate standards. 2012 will see more development of “enterprise data discovery” interfaces for casual users.

(3) The next generation of business applications. Where are the business applications designed to support what people really do all day, such as implementing this year’s strategy, launching new products, or acquiring another company? 2011 will see the first prototypes of people-focused, flexible, information-centric, and collaborative applications, bringing together the best of business intelligence, “enterprise 2.0”, and existing operational applications.

Score: 6. We didn’t see many of these in the traditional enterprise BI landscape, but 2011 did see the rise of sophisticated, user-centric mobile applications that combine data from corporate systems with GPS mapping and the ability to “take action”, such as mobile medical analytics for doctors or mobile beauty advisor applications, and collaborative BI started becoming a standard part of enterprise platforms.

And one that should happen, but probably won’t: (4) Intelligence = Information + PEOPLE. Successful analytics isn’t about technology — it’s about people, process, and culture. The biggest trend in 2011 should be organizations spending the majority of their efforts on user adoption rather than technical implementation.

Unsurprisingly, there was still high demand for presentations on why BI projects fail and how to implement BI competency centers.  The new architectures probably resulted in even more emphasis on technology than ever, while business peoples’ expectations skyrocketed, fueled by advances in the consumer world. The result was maybe even more dissatisfaction in the past, but we can hope that the benefits of the new architectures should start becoming clearer during 2012.

What surprised me the most:

The rapid enterprise rise of Hadoop / NoSQL. The potential of these technologies has always been impressive, but I was surprised just how quickly they have been used to address real-life business problems (beyond the “big web” companies where they originated), and how quickly it is becoming part of mainstream enterprise analytic architectures (e.g. Sybase IQ 15.4 includes native MapReduce APIs, Hadoop integration and federation, etc.)

Prediction for 2012 

I hope to do a longer post on this, but here’s my initial take:

As I sat down to gather my thoughts about BI in 2012, I quickly came up with the same long laundry list of BI topics as everybody else: in-memory, mobile, predictive, social, collaborative decision-making, data discovery, real-time, etc. etc.

All of these things are clearly important, and we’re going to continue to see great improvements this year. But I think that the real “next big thing” in BI is what I’m seeing when I talk to customers: they’re using these new opportunities not only to “improve analytics” but also fundamentally rethink some of their key business processes.

Instead of analytics being something that is used to monitor and eventually improve a business process, analytics is becoming a more fundamental part of the business process itself. One example is a large telco company that has transformed the way they attract customers: instead of laboriously creating a range of rate plans, promoting them, and analyzing the results, they now use analytics to automatically create hundreds of more complex, personalized rate plans. They then throw them out into the market, monitor in real time, and quickly cull any that aren’t successful. This kind of “analytics first” transformation has happened to other industries in the past, but the new technologies are helping more  industries and companies to do business in new ways that would have been inconceivable in the past.

I look forward to talking more about these themes at upcoming conferences this year, starting with Gartner Business Intelligence Summit 2012 in the UK next Monday — and hope to see you at one of them!

Mobile BI has been around for a long time. Starting in the late-1990s, the first SMS-enabled telephones became mainstream in Europe, with basic broadcasting of the latest figures available in your BI system (or email, fax, pager, etc.). By the end of the decade, the first telephones with WAP browsers were used to provide interactive BI, quickly followed by connected PDAs with basic HTML browsers.

Here’s what SAP BusinessObjects looked like on the Nokia 7110 in 1999, on a Compaq PDA running Windows Pocket IE, an AvantGo PDA, and a Japanese DoCoMo i-mode phone in 2001:

The arrival of all these new mobile devices was supposed to usher in a new dawn of mobile analytics. Here’s a slide from a presentation a decade ago by then-marketing-VP Dave Kellogg, including the heady prediction that “5-25% of companies indicated they already provide or will provide wireless access to BI within 6-12 months”.

Business Objects launched a big initiative to go after the mobile market, and managed to sell projects to customers including JP Morgan (Palm Pilots) and Zurich Insurance (a mobile extranet for risk managers).

But, clearly, the market didn’t take off – PDAs became more widely used, and phones got better, but they weren’t used much for BI. The user interfaces were too clunky and connection speeds were too slow. Interest in mobile BI did grow slowly over the decade, notably as RIM blackberry devices became ubiquitous, but it took the wide availability of 3G wireless and modern smartphones/tablets to provide truly usable interfaces.

Finally, a decade and a half after the first tentative steps, everybody seems to agree that this is the year that mobile BI will really take off.

2012 Is the Year of Mobile BI

Here’s a taste of the mountain of research data that’s been generated about mobile BI in the last few months:

• Boris Evelson of Forrester says mobile BI will go mainstream this year. “One needs to make decisions when and where they need to be made. Not ‘when I get back to the office,’ which may be too late.” He also says that “professionals must start evaluating and prototyping mobile BI platforms and applications to make sure that all key business processes and relevant information are available to knowledge workers wherever they are.”
• According to Gartner, “by 2013, 33 percent of BI functionality will be consumed via handheld devices.”
• A survey by analyst Howard Dresner indicates that BI has already become the third most in-demand enterprise mobile application, behind only email and personal information management apps such as calendars, and 68 percent of those surveyed rated mobile BI as “critical” or “very important,” up from 52 percent a year earlier.
• A recent survey by Information Week showed that 25 percent of organizations are planning to implement some form of BI this year.
• Sixty-one percent of the participants in a TDWI survey said that they expect users to spend more time accessing BI from mobile devices in the next 12 months.
• Business Intelligence and mobile are the top two technology priorities for CIOs in 2012.

Barriers to deployment

Not every organization is moving forward with mobile BI. Here are the main concerns:

• Security and administration: Organizations are concerned about data getting outside the organization, and the administration overhead generated by managing BI on mobile devices. A mobile device management (MDM) platform (not to be confused with the other MDM) like Sybase Afaria is key.
• Expectations-setting: An easy interface doesn’t mean that the data users want is readily available. New opportunities mean new requirements. Having the right data foundations in the first place, with a robust, standard BI platform in place makes it easier to react fast to user expectations.
• Platform choices: This is perhaps the biggest factor delaying widespread deployment. Just like the operating system wars of last century (remember IBM OS/2?), there’s no one obvious platform to standardize on for rolling out mobile applications. There are three main strategies, all with pros and cons:
  • Native applications—mobile applications written directly for iOs or Android. The advantage is optimal ease-of-use and access to the capabilities of the native device. The disadvantage is the cost and complexity of supporting multiple platforms and different user interfaces.
  • HTML—accessing mobile BI through a Web browser. The advantage is that you don’t care what device is being used to access the data – at least in theory. In reality, the disadvantage is that browser-based interfaces are generally far behind what’s possible using the native features. There are high hopes that the proposed HTML5 standard will help – but it hasn’tt yet reached maturity.
  • Hybrid solution—mobile enterprise platforms such as Sybase Unwired Platform. You create applications once, and then generate different versions of them optimized for different mobile platforms, including HTML. It’s an insurance measure against the turbulent real-life world of changing mobile platforms, but there’s some upfront investment.

An example of today’s mobile BI solutions -  SAP BusinessObjects Mobile showing the integrated Google maps pioneered by the Innovation Center.

Of course, most of us just want all these options to be available as part of the standard business intelligence platform – and that’s getting closer…

Next Steps

Gartner predicts that “by 2013, 15% of BI deployments will combine BI, collaboration and social software into decision-making environments”. In other words, mobile BI will become part of an “orchestrated” experience that combines accessing data with acting on it, and we’re starting to see this in the form of mobile medical analytics for doctors or mobile beauty advisor applications.

Interestingly, one brand new area of opportunities is the integration of mobile with voice-controlled interfaces such as Apple’s Siri. Business Objects was WAY ahead of the curve with this one, with the project codenamed “Ariel”. It sadly didn’t take off, but anybody who saw it demoed will have fond memories…

Here’s a recording of a World Tour keynote presentation (it happens to be from Belgrade, Serbia, but I did similar presentations in several different regions). It covers how companies can build competitive advantage through customer intimacy, product leadership, category renewal, and operational excellence, using examples from SAP’s customers.

You can download the slides in either PowerPoint or pdf format.

In a previous post, I talked about the “illusory superiority” effect, and how it blinds people to the fact that, on average, it’s unlikely that they use data better than their competitors.

Guess what? It turns out that it applies to governance, risk, and compliance, too. Here are some figures from another Economist Intelligence Unit Survey, “Ascending the Maturity Curve, Effective Management of Enterprise Risk and Compliance”:

We can see that of those who haven’t experienced failures, only 1% believe that they are worse than average – and even among companies that have experienced failures, fully 87% believe that they are as least as good as their peers.  Unless the Economist has stumbled across a particularly great group of companies to study, it seems clear that most organizations are overestimating the quality of their GRC practices, and hence underestimating the real risks they are running…

There’s also data in the report that seems to indicate that the finance function is the mostly likely to be blindsided – as you can see in the chart below, they are far more likely to say that there was no significant risk or compliance failure in the past three years. Since this is not a group known for their exuberant optimism, it’s likely that they simply didn’t know about the risks run by the other teams…

In conclusion, if you’re in the finance function, and responsible for your GRC practices, it’s likely that you should be investing more than you are today. For more information, check out SAP’s GRC products, and Norman Mark’s blog on Governance, Risk Management, and Internal Audit.