The real trend this year is not the technology. It’s about helping business people make better decisions, and actually change the way companies do business. Analytics has always been about transforming business, but the recent huge changes in analytic technology have created interesting new opportunities for business innovation.

Most organizations are now starting to understand the technical opportunities, but many struggle to apply those new opportunities to their business processes. This blog post attempts to explain what’s going on in the analytics market and give concrete examples of how other companies have implemented the new technologies in “game-changing” ways (sorry kittens).

Wrenching Change and A Foggy Outlook

The chart below illustrates the wrenching effects of recent financial problems on the world gross domestic product: companies today have to be ready to react to unprecedentedly fast changes to their economic environment.

And the economic environment is fraught with extreme uncertainty. This year, the people who run the world will change, and so will many of the policies of the countries they manage. Financial markets have still not completely stabilized, notably with the future of the Euro still not assured.

Companies have reacted to this uncertainty by slashing costs and accumulating cash, and now need to start investing that cash into future development. Since interest rates are low and the business outlook is still uncertain, many of them are using the money for new technology that can help them prepare for the future.

In particular, companies want better visibility about what’s going on in their market, and increased organizational agility in order to be able to deal with change fast. It’s like driving in the fog without a map – in order to survive, you should invest in better visibility, brakes, and steering to be able to spot and avoid fast-moving objects looming out of the fog.

Analytics provides these capabilities: business intelligence to peer into the road ahead, risk-management to provide fast alerts to new obstacles, and flexible financial planning systems to help swerve around them.

Analytics: Hotter Than Ever

Companies are investing heavily in analytics:

• Analytics is the #1 top technology priority for both CIOs and CFOs, according to Gartner
• Nucleus Research recently released a report showing that organizations get $10.66 of value for every $1 invested in analytics
• IDC has increased growth forecasts faced with stronger-than-expected figures for recent years
• IDC analyst Dan Vesset: “After three decades, the business analytics market is finally reaching the mainstream” and “There are few growth inhibitors in the foreseeable future”
• At the recent BI Gartner Summit in London, Gartner’s Dan Sommer announced an early estimate of 10%+ growth in analytics during 2011, outpacing general IT growth.

Fast-Moving Technology

Analytics technology has been changing fast. On the back end, new technologies have come together to provide what Gartner calls “extreme data performance”. These include in-memory, column data stores, in-database calculations, massively parallel architectures, complex event processing, Big Data / NoSQL / Hadoop, and cloud architectures.

The combination of these technologies provides a opportunity to access massive amounts of a greater variety of data, faster, and more flexibly. The key opportunity is that these new platforms “collapse the stack” so that organizations can implement and update analytic projects much faster than ever before.

And on the front end, various technologies are coming together to provide unprecedented levels of context-based “actionable insights”, including self-service data discovery, advanced visualization including maps, mobile analytics, predictive analytics, collaborative decision-support. They help provide more action-oriented interfaces optimized for the context of the users, both inside and outside the organization.

These technology advances are clearly important, and we’re going to continue to see great improvements this year. The new opportunities have reached a tipping point similar to the rise of digital cameras vs. analog photography – and you don’t want to leave it too late to make the change, like Kodak, which recently filed for bankruptcy protection!

However, the real opportunity is using these new possibilities not only to improve analytics but fundamentally rethink key business processes.

High Resolution Management

University researchers have pointed out that today’s management techniques are based on the limitations of information scarcity:

“How many times has someone in your company uttered, “We don’t have that level of accuracy in the information, so we have to make aggregated estimates”? Under the current paradigm, it is sometimes impossible to drill down and understand what is happening at a highly detailed level.”

They coined the term “High Resolution Management” to describe what becomes possible with the new technology opportunities:

“We contend that these technologies will change drastically how management makes decisions. Why? Because with access to the finest granularities of information, management will be able to move freely from macro to micro levels and will be able to measure, plan and act accordingly. With increased resolution come more options to drill down, eliminate inefficiencies and cut costs.”

Lets take a look at three different types of High Resolution Management opportunity, letting companies:

• Remove bottlenecks
• Rethink business
• Flip business models

Remove Bottlenecks

Better technology always means business opportunity, but the new analytic platforms are rapidly eliminating some of the key bottlenecks that have prevented organizations from getting value from their data:

Faster, more flexible data access. Companies like Red Bull have been able to speed up and simplify their data warehousing environments. Using the HANA in-memory database, the company can now load detailed data twenty-five times faster into their data warehouse, and they were able to eliminate several levels of data staging, increasing the flexibility of the solution.

Data volumes and complexity. Companies like Colgate-Palmolive, Provimi, and Danone have long had access to vast amounts of detailed data about their production facilities and sales channels – but the quantity of data meant that they were unable to run full analytics in a reasonable time frame. That has now changed. For example, according to Colgate-Palmolive CIO Tom Greene:

“We will be able to run analytics at a local level on specific brands and locations, and at the lowest level of detail in real time”

And Danone can now measure the carbon emissions of 35,000 different products, with new systems that:

“collect, measure, and analyze data across the entire product life-cycle, from sourcing through production, transport, retail, distribution, consumption, and end of cycle”

New forms of data: ‘unstructured’ data such as text has long been difficult to effectively analyze and incorporate into mainstream corporate analytics. The new systems make it much easier for companies like Medtronic to access and analyze the large amount of complaints and feedback data they receive about their products, combine it with other data sources, and provide it to business users with dynamic interfaces:

New interfaces and users. Companies like Altron have been able to get the data to their users where they needed it. As Debra-Lynn Marais, Group Information Manager explains:

“The days of our users and execs being in the office have gone. They work from home or on the road. We had to develop a solution that gets information out to where our people are. Everything we do is mobile first. In addition, it’s less cumbersome and cheaper to buy and use a tablet than any other form.”

Rethink Business

Many companies are going beyond “just” improving their existing analytic capabilities, using analytics in new ways to change the way they do business. Instead of analytics being something that is used to monitor and eventually improve a business process, analytics is becoming a more fundamental part of the business process itself.

Proactive Analytics. Instead of using analytics only to assess previous performance, companies are using the new capabilities to get data fast enough to make a real difference. For example, online grocer Fresh Direct, instead of just understanding what problems happened yesterday, can now understand what problems will happen in the next few hours, so they can actually fix them before a customer is impacted:

“FreshDirect has an operations center that manages its fleet of delivery trucks. In a large metropolitan area like New York, traffic doesn’t always flow predictably. A traditional approach to BI would be to print a report showing the level of on-time deliveries (OTDs) the day before and then ask the transportation department what went wrong for the orders that were delivered late. FreshDirect uses analytics in a more impactful way.”

“The company monitors the delivery rate of every truck and enters that data into the BI system on an ongoing basis. Every hour, it uses the previous hour’s data to predict how many deliveries will be on-time in the next hour. If the predicted OTD rate is below FreshDirect’s target, the company sends out an auxiliary truck or trucks to help make deliveries. The company holds 10 trucks in reserve for just this purpose.”

Integrated Risk Assessments. Among other products, publisher Houghton Mifflin Harcourt produces educational books. Schools pass orders in June or July after the end of the school year, and then expect delivery for the start of the next school year in September. Getting books printed during the summer is expensive, as many publishers compete for the limited supply of printers available.

To avoid these extra costs, Houghton Mifflin Harcourt uses using sophisticated, risk-based forecasting. The company prints books in January or February, when printing is much cheaper. In order to minimize of excess inventory, it has carefully analyzed all the causes of previous forecasts, and now takes account of all the different things that influence book obsolescence.

Before, the buying team just ordered based on the volume forecast from sales. Now they have much greater context for their decisions. For example, if there’s a vote coming up on schools funding that may result in the canceling of a math adoption program for the year, they can decide to hold back on those purchases until the outlook is clearer. The fast, more accurate forecasting mechanism has saved them tens of millions of dollars, and they have more of the products their customers want.

New Customer Services. International grocery chain Casino is rolling out a new mobile shopping application for its customers. It provides data from its enterprise systems directly to its customers, resulting in increased shopping convenience and increased customer loyalty.

German healthcare provider AOK (“the good health organization”) is committed to helping its members avoid illnesses in the first place. It is planning to introduce a new, market-differentiating service: personalized healthcare advice for each customer, with tools that:

“Conduct real-time analyses of the tremendous amounts of medical data we receive, recognize potential health risks, assemble various preventive care programs and respond to those risks appropriately and ahead of time.”

As an added bonus, they also believe that this tailored prevention program will result in significant cost reductions by preventing expensive unneeded treatments.

BC Hydro is saving $70 million dollars a year through the installation of new smart electricity meters, using SAP systems, and offering new services to commercial customers based on the new data possibilities. Companies like Centrica are planning to use SAP’s Smart Data Analytics, giving them deep understanding into consumer consumption.

Flip Business Models

The really interesting opportunity for businesses is where companies have managed to use analytics to fundamentally flip the way their businesses work: instead of analytics being part of a process, it “becomes the business model”.

Tim Ferriss, author of the 4-hour workweek, is an interesting example of this. He didn’t do what most authors do: write a book, and then figure out how to publicize it. He used an analytics-first approach: he bought Google Ads, with mockups of book covers, with a variety of titles of books that he might be interested in writing – and then wrote the book that got the most clickthroughs! This is one step beyond using analytics such as focus-groups, which are typically there to validate existing products. The next generation of products and services are being created “on the fly” based on an analysis-first approach.

The clothing brand Zara shook up fashion retailing with “analytics first” – instead of having a designer creating clothes and then trying to sell them six months later, they realized new manufacturing techniques meant they could create clothes “in the moment”. They could observe what people were wearing in the street, quickly make small batches of variations on that theme, and get them into the stores. If they sold well they made more, if they didn’t sell they discounted quickly. Instead of a season-oriented, “batch” business, they switched to a flow-oriented business, using new technology capabilities.

The new analytic platforms mean that this analytics-first approach is available to many more businesses than in the past. For example, T-Mobile is in the process of transforming the way they attract customers. Instead of laboriously creating a range of rate plans, promoting them, and analyzing the results, they now use analytics to automatically create hundreds of more complex, personalized rate plans. They then throw them out into the market, monitor in real time, and quickly cull any that aren’t successful. It’s a way of doing business that would have been inconceivable in the past, and a lot more common in the future.

Conclusion

2012 is the year to rethink your analytic technology to take account of new opportunities:

• On the back end, for extreme data performance
• On the front end, for actionable insights

And it’s time to rethink your business:

• Remove today’s bottlenecks to successful analytics caused by data volumes, data variety, or data access
• Rethink business processes by embedding real-time decisions
• Create new products and services that could only exist because of today’s analytic power

Organizations are using this technology to change the way they do business. If you run an analytics project, you are in the forefront of these changes – it’s your job to help explain to the rest of the business how these technologies should be changing their existing processes. Good luck!

___________________________________________________

If you’re interested in slides that go along with this article, please see this post about the recent Gartner BI Summit in London that includes a download of my presentation at the conference.

Next Wednesday, February 15th, from 3pm to  4.30pm PST, I will be moderating a panel on the theme of Products+Social=Better Products as part of Social Media Day at SAP Palo Alto (and Online).

We’ll discussing how to make better products thanks to social techniques. Please join us! You can register here and the conference hashtags are #SMW12 and #SAP.

Panel Session Outline

Here’s the outline of the panel session:

• 3:00-3:15 Key themes and Introductions:
• 3:15-3:35 Improving products through social techniques:
• 3:35-3:55 Creating social-enabled products:
• 3:55-4:15 Creating new products on top of social networks:
• 4:15-4:30 Wrap-up and final thoughts

These times are just a guideline – we’ll go where the conversation takes us. We will be encouraging the audience (both local and online) to tweet questions and comments, and I will pass them on to the panelists.

Here are some more details about the panelists and the show:

Meet the Panelists

In alphabetic order, here are the participants of the panel – click on their name to see their profile on twitter.

	David Brockington is a solution manager responsible for SAP’s new collaboration solution, SAP StreamWork. He has been an evangelist and thought leader within SAP on social business topics. Previously, he spent 10 years in various product management roles helping grow SAP Business Objects Enterprise from its very first iteration to the number one market leading business intelligence solution.
	Timo Elliott is a 20-year veteran of SAP BusinessObjects, and has spent the last quarter-century working with customers around the world on information strategy. He works closely with SAP research and innovation centers around the world to evangelize new technology prototypes.His popular Business Analyticsblog tracks innovation in analytics and social media, including topics such as augmented corporate reality, collaborative decision-making, and social network analysis.His PowerPoint Twitter Tools lets presenters see and react to tweets in real time, embedded directly within their slides. Elliott presents regularly to IT and business audiences at international conferences, on subjects such as why BI projects fail and what to do about it, and the intersection of BI and enterprise 2.0
	Ross Mayfield is the Vice President of Business Development at SlideShare, developing strategic partnerships and helping organizations engage with the world’s largest professional sharing community.He is also currently Chairman & Co-founder of Socialtext, the Enterprise Social Software pioneer, and was the founding CEO. Previously, Mayfield Co-founded and served as President of RateXchange, a publicly traded B2B commodity exchange for telecommunications.Mayfield is a former advisor to the Office of the President of Estonia and began his career in the non-profit sector. He holds a B.A. in Political Science from the University of California at Los Angeles and completed the Management Development for Entrepreneurs (MDE) program of the Anderson School of Business
	Kuhan Milroy is Director of Social Business Innovation, SAP Community Network at SAP. By bringing the social element to innovation at SAP, Kuhan is responsible for transforming the way SAP and customers, partners, and consumers innovate together to help shape the future of SAP’s products and solutions and more importantly the businesses, industries and markets the products serve. He manages initiatives surrounding the community and bringing a social approach to how SAP innovates, and researches how SAP can expand beyond small group to mass users for ideas and innovation.
	Lisa Joy Rosner is the Chief Marketing Officer at NetBase, where she works with the largest brands in the world—including Coca-Cola, Kraft and HP—as they change their approach to understanding consumers’ desires.Ms. Rosner is a tenured marketing executive in Silicon Valley, who is known for her creative and educational marketing approach. Before joining NetBase, she was an e-commerce expert, having served as vice president of Worldwide Marketing at BroadVision Inc. and vice president of Marketing at MyBuys, where she worked with such companies as Sears, Wal-Mart and Circuit City.Prior to that, Ms. Rosner worked in the data warehouse market at Brio, DecisionPoint and Oracle. Lisa Joy has served on the marketing advisory board for the Silicon Valley Red Cross and the Content Committee of Shop.org, the AMA and Benchmark. She was named the Gold winner of the Great Minds Award by the ARF in 2011. Ms. Rosner received a bachelor’s degree (sum cum laude) in English literature from the University of Maryland. She is a member of Phi Beta Kappa
	Nicholas Webb is one of the countries top thought leaders in the area of innovation. His books include The Innovation Playbook, The Digital Innovation Playbook, and The Innovation Superstar Workbook.As a Senior Partner at Lassen Innovation he helps some of the best companies in the world drive innovation through improved customer insights and true innovation best practice.

Here’s a recording of a World Tour keynote presentation (it happens to be from Belgrade, Serbia, but I did similar presentations in several different regions). It covers how companies can build competitive advantage through customer intimacy, product leadership, category renewal, and operational excellence, using examples from SAP’s customers.

You can download the slides in either PowerPoint or pdf format.

In a previous post, I talked about the “illusory superiority” effect, and how it blinds people to the fact that, on average, it’s unlikely that they use data better than their competitors.

Guess what? It turns out that it applies to governance, risk, and compliance, too. Here are some figures from another Economist Intelligence Unit Survey, “Ascending the Maturity Curve, Effective Management of Enterprise Risk and Compliance”:

We can see that of those who haven’t experienced failures, only 1% believe that they are worse than average – and even among companies that have experienced failures, fully 87% believe that they are as least as good as their peers.  Unless the Economist has stumbled across a particularly great group of companies to study, it seems clear that most organizations are overestimating the quality of their GRC practices, and hence underestimating the real risks they are running…

There’s also data in the report that seems to indicate that the finance function is the mostly likely to be blindsided – as you can see in the chart below, they are far more likely to say that there was no significant risk or compliance failure in the past three years. Since this is not a group known for their exuberant optimism, it’s likely that they simply didn’t know about the risks run by the other teams…

In conclusion, if you’re in the finance function, and responsible for your GRC practices, it’s likely that you should be investing more than you are today. For more information, check out SAP’s GRC products, and Norman Mark’s blog on Governance, Risk Management, and Internal Audit.

Here’s a video from this years SAPPHIRE NOW in Madrid, featuring Mimi Speir, Steve Lucas, and Andrew Murray giving an overview of SAP BusinessObjects mobile directions:

Highlights include:

• New SAP BusinessObjects experience mobile application, based on the Exploration Views functionality from the BusinessObjects innovation center, with demo data from Experience.SAP.com
• Easily take a mobile analysis and create a StreamWork activity for collaborative decision-making
• Easily “mobilizing” your existing BusinessObjects reports
• Support for each level of the “mobile needs hierarchy”, including fully customized mobile applications

• Demonstration of integration with RightHemisphere, a recent SAP acquisition that allows companies to synchronize visual and business data, combining a camera and schematic view:

Overlaying colors by temperature from data in the ERP system:

Finding a required part replacement using geolocation data:

I’ll be an official “social ambassador” for analytics at SAPPHIRE NOW from Madrid next week. This blog attempts to summarize the main analytics-related activities during the two days of SAPPHIRE NOW – but note that this year, SAPPHIRE NOW attendees also have access to all SAP TechEd sessions that are going on at the same time, making it even harder than usual to choose from the dizzying array of possible sessions (the SAP TechEd sessions are technology-focused, and include more networking sessions with IT peers, while the SAPPHIRE NOW sessions focus on the business aspects and include more customer sessions).

For the rest of this post, I’ll focus on SAPPHIRE NOW. There are four main campuses, staffed by SAP experts available to answer any questions you may have:

For this post, I’ll concentrate on activities in the Business Analytics Campus, but note that if you’re interested in mobile BI, analytics integrated with SAP, or the latest in in-memory systems, you may also want to check out the contents of the other three campus areas (as an example, “How Pfizer Uses Mobile Business Analytics from SAP” is on Wednesday afternoon). There are also analytics-related, “TED-like” sessions in the Spotlight theater (which will be recorded and available after the event)

Business Analytics Campus

There are an amazing 87 analytics-related, information-packed sessions scheduled over the two days – use the personal agenda builder to pick which ones you’d like to attend, and leave time to visit the analytics discussion tables (meet and discuss with SAP experts), the demo pods (working versions of the latest analytics software, including HANA, BusinessObjects 4.0, and EPM 10), and the exhibitor area.

Personally, I’ll be concentrating on the customer presentations and panel sessions, so here’s a recap of them below – to see the scheduled times, see the full list of sessions later in this post.

Customer Presentations

Bowers and Wilkins: Transform how you do business by “mobilizing” decision making. Mobile BI is the new reality for companies who want to transform how they make decisions today.  Learn how customers are making fact-based decisions wherever they are with built-for-mobile BI.  Discover how you can impact your business’ performance with reporting, analysis, dashboards and visualization delivered real-time to any mobile device.

Novartis: Our journey to Integrated Business Intelligence. Learn how Novartis Pharma is consolidating its BI environment worldwide from seven instances of SAP NetWeaver Business Warehouse (BW) and transitioning to one single source of truth, introducing SAP BusinessObjects BI 4.0 and SAP BusinessObjects Planning and Consolidation 7.5 to enable a complete new level in business decision making capabilities.

Altron: Meet Growing Information Demands with SAP BusinessObjects 4.0 Innovations. Due to business conditions and increasing demands for information access, BI teams now face steeper requirements for information integrity, burgeoning data volumes, and demand for analytics across lines of business. Meanwhile end users clamor for simpler, faster mobile access to information. Hear how innovations in SAP BusinessObjects 4.0 can help.

Charite: Turn “Big Data” into Business Value with Real-Time BI. Increasing expectations of BI users for access to big data, coupled with rapid response times and rising need for intuitive end-user experience, put pressure on BI professionals. Hear how Charite, a large German university hospital supports its operations and research demands by leveraging in-memory technology with SAP BusinessObjects solutions.

All for One Drives Rapid Decisions and Project Delivery with SAP StreamWork. Learn about All for One Midmarket AG, a one-stop shop for outsourcing services and application management. Hear how the company increased customer interaction, streamlined its project management and delivery, and enabled faster, improved decision making with the cloud-based social collaboration solution, the SAP StreamWork application.

Tecan: Automating Internal Controls with SAP BusinessObjects GRC Solution. Hear how Tecan, a leader in life sciences supply, has implemented an automated internal controls system with SAP BusinessObjects governance, risk, and compliance (GRC) solutions. Learn how Tecan has revolutionized manual processes and achieved stronger and more reliable controls with the SAP BusinessObjects Process Control 10.0 application.

Why Medtronic Chose SAP HANA and SAP NetWeaver BW to Stay Competitive. Hear Medtronic Inc., a global leader in medical technology, explain its recent deployment of the SAP HANA platform and the SAP NetWeaver Business Warehouse (SAP NetWeaver BW) component. Learn how SAP HANA allows the company to better analyze business operations, lower total cost of ownership, and speed time to market. – Implementation/Technology Partner: Cisco

How Pfizer Uses Mobile Business Analytics from SAP. This discussion will help you learn firsthand how Pfizer Inc. embraced mobile analytics as part of its enterprise strategy. Hear about what the company learned and where it’s going. (Mobility Campus)

BASF: Secure Supply Chain with SAP BusinessObjects Global Trade Services. BASF runs the SAP BusinessObjects Global Trade Services application, including global trade compliance and trade preference management. The company recently implemented new features, including Excise Movement and Control System and integration to the SAP Event Management application, to proactively monitor and manage a first-class supply chain.

Integrated Financial Reporting and Planning with SAP Software at Statoil. Hear about Statoil’s successful integrated financial reporting and group planning. The company will share its implementation experiences with business intelligence and the SAP BusinessObjects Planning and Consolidation application, which helped increase quality and efficiency in producing internal and external financial reports.

Why AOK Chose SAP HANA for Real-Time Analytics. Hear why Allgemeinen Orts Krankenkassen (AOK), the largest health insurance organization in Germany, chose the SAP HANA platform for real-time analytics and data warehousing. Learn about AOK’s short- and long-term use cases for SAP HANA, including plans to supercharge the organization’s landscape for the SAP NetWeaver Business Warehouse component. – Implementation/Technology Partner: Dell

Get the Inside Story on Deloitte’s Implementation of SAP HANA. Find out how key stakeholder support was gained and how a large amount of structured and unstructured data was deployed at Deloitte. See the impact the SAP HANA platform has on the company’s front-end reports, including a reduction in data load time from 48 hours to 12 minutes and report run time from 30 minutes to milliseconds. – Implementation/Technology Partner: Deloitte

Logista: Managing Access Risk with SAP BusinessObjects Access Control. Hear how Logista, a leading logistics group, uses the SAP BusinessObjects Access Control application to address compliance challenges, reduce access-related audit findings, and achieve compliant role provisioning. Hear how, with an automated solution, Logista increases business ownership while reducing the cost and effort of access risk management.

Panel Discussions

BI Customer Panel: Featuring Shell and Celesio, and hosted by Adam Binnie (GM & Global VP Business Intelligence Solutions). How do you get the most from your business intelligence solutions? What should you look for when finding the right BI solutions? Many factors come into play regardless what industry you are from. Hear from companies explain how the SAP BusinessObjects solutions helped their users make smarter and faster decisions.

BI Product Panel: Where BI Takes Us Today: Featuring Adam Binnie (GM & Global VP Business Intelligence Solutions), Timo Elliott (Analytics Evangelist) and Thierry Donnadieu (Head of Business Analytics EMEA consulting). As businesses strive to find efficient ways to deliver attractive offerings to customers, how can yours do the same? BI solutions help businesses identify profitable products, services, processes and even customers. What BI innovations are SAP delivering that makes companies successful? Hear from SAP product leaders to learn how BI can help your business reach new heights.

Data Warehousing Customer Panel: Providing Instant Access to Big Data: Customers discuss experiences leveraging data warehousing solutions from SAP to solve challenges stemming from managing and providing real-time access to large volumes of data. Understand the role that can be played by the SAP HANA platform; the SAP NetWeaver Business Warehouse component, which is powered by SAP HANA; and Sybase IQ software.

Information Governance Customer Panel: We Deliver Quality and Trusted Information to Our Business: Find Out How. With corporate decisions relying on the quality of data, and the volume of data doubling every 18 months, it’s vital for companies to firm up their corporate information governance. But who does, and why do they do it? Hear from Vodafone Group plc, Boehringer Ingelheim GmbH, and Alliander about how they deliver functional data management models. – Software Solution Partner: OpenText

EPM Product Panel: Helping Ensure Quick Time to Value with SAP BusinessObjects EPM Solutions. Implementing solutions that enable better control of growth and profitability remains a priority. But today implementations need to help ensure rapid time to value. Hear project managers who have led successful implementations of SAP BusinessObjects enterprise performance management (EPM) solutions discuss their approaches and experiences.

HANA Product Panel: Getting Started with an SAP HANA Platform Implementation. Join the SAP business analytics services team and our customers to get an overview of the latest and greatest in the world of the SAP HANA platform. Learn how SAP HANA has already helped numerous customers. This session will provide you with a clear understanding of the solution, road map, deployment options, and considerations.

Innovation Panel: Innovative Use of Business Analytics for Your Industry and Line of Business. Hear how SAP customers are benefitting from key innovations in business analytics – such as the SAP HANA platform, mobility, cloud services, and collaboration – for solving critical business issues. Hear them discuss their business challenges and their experience in selecting, implementing, and using business analytics to derive business value.

BI Value Panel: How We Drive Value from Business Intelligence. How do you get the most from your business intelligence (BI) solutions? What should you look for when trying to find the right BI solutions? Many factors come into play regardless of what industry you are from. Hear from three companies about how SAP BusinessObjects solutions helped their users make smarter and faster decisions.

Midmarket BI Panel: Unleashing the Power of Business Analytics for Midsize Companies. Join this panel of midsize companies to learn their real-life stories. Hear what they did to deploy SAP BusinessObjects analytic applications, and learn how they are using the solutions to gain actionable insights – including best practices – that otherwise have been invisible to them.

Full Schedule

Here’s the full schedule of sessions that touch on Business Analytics over the two days of SAPPHIRE NOW, grouped by time slot, so you can more easily see the painful choices involved. Use the agenda builders for SAP TechEd and SAPPHIRE NOW to see more about the sessions, and to create your own list of activities that you can then email or download.  I’ve put in bold the ones I hope to attend myself – see you there!

Tuesday 9th November

SAP TechEd starts a day earlier than SAPPHIRE NOW, packed with detailed, technical information about analytics, and a keynote from Vishal Sikka, SAP Executive Board Member leading technology and innovation– see the SAP TechEd agenda for more information (Vishal presents to the SAPPHIRE NOW audience on Thursday morning).

Wednesday 10th November

Wed., 09:00 a.m. – 10:30 a.m. General Session – SAP Executive Keynote Speaker: Jim Hagemann Snabe, Co-CEO of SAP

• Wed., 11:00 a.m. – 11:45 a.m.,B31336 Strategies for Enterprise Business Intelligence Success: Do You Have One?, 45-Minute Microforum, Business Analytics Microforum 303
• Wed., 11:00 a.m. – 11:45 a.m.,B31377 What’s New in SAP BusinessObjects Planning and Consolidation 10.0, 45-Minute Microforum, Business Analytics Microforum 320
• Wed., 11:00 a.m. – 11:45 a.m.,B31601 Drive Rapid Decisions with Collaborative BI Using SAP StreamWork, 45-Minute Microforum, Business Analytics Microforum 313
• Wed., 11:00 a.m. – 11:45 a.m.,B32755 Using Business Analytics in Your Finance Department, 45-Minute Microforum, Business Analytics Microforum 300
• Wed., 11:00 a.m. – 11:45 a.m.,B32762 Next-Generation Analytic Applications for Industry and Line of Business, 45-Minute Microforum, Business Analytics Microforum 325
• Wed., 11:00 a.m. – 11:30 a.m.,B31335 Where BI Takes Us Today, 30-Minute Panel Discussion, Business Analytics Theater 1
• Wed., 11:00 a.m. – 11:30 a.m.,B31392 How Do Your GRC Programs Measure Up?, 30-Minute Panel Discussion, Business Analytics Theater 2

Lunch

• Wed., 01:00 p.m. – 01:45 p.m.,B31337 Embedding Analytics into SAP Business Suite Applications, 45-Minute Microforum, Business Analytics Microforum 303
• Wed., 01:00 p.m. – 01:45 p.m.,B31367 Understanding the Road Map for SAP HANA with SAP NetWeaver BW, 45-Minute Microforum, Business Analytics Microforum 310
• Wed., 01:00 p.m. – 01:45 p.m.,B31376 What’s New in Treasury and Risk Management, 45-Minute Microforum, Business Analytics Microforum 320
• Wed., 01:00 p.m. – 01:45 p.m.,B31600 Deliver Your Collaboration Strategy with SAP StreamWork, 45-Minute Microforum, Business Analytics Microforum 313
• Wed., 01:00 p.m. – 01:45 p.m.,B32753 Budgeting, Planning, and Consolidation for SMEs, 45-Minute Microforum, Business Analytics Microforum 300
• Wed., 01:00 p.m. – 01:45 p.m.,B32781 Increase Competitiveness Through Better Customer Insight, 45-Minute Microforum, Business Analytics Microforum 325
• Wed., 01:00 p.m. – 01:20 p.m.,B31332 Meet Growing Information Demands with SAP BusinessObjects BI Suite, 20-Minute Presentation, Business Analytics Theater 1
• Wed., 01:00 p.m. – 01:20 p.m.,B31390 Tecan: Automating Internal Controls with SAP BusinessObjects GRC Solutions, 20-Minute Presentation, Business Analytics Theater 2

• Wed., 01:30 p.m. – 01:50 p.m.,B31349 Information Governance from the Ground Up: How to Make It Real, 20-Minute Presentation, Business Analytics Theater 1
• Wed., 01:30 p.m. – 01:50 p.m.,B31598 All for One Drives Rapid Decisions and Project Delivery with SAP StreamWork, 20-Minute Presentation, Business Analytics Theater 2

• Wed., 02:00 p.m. – 02:45 p.m.,B31338 Transition from SAP Business Explorer to SAP BusinessObjects BI Solutions, 45-Minute Microforum, Business Analytics Microforum 303
• Wed., 02:00 p.m. – 02:45 p.m.,B31353 Why Information Governance Is Important Now, 45-Minute Microforum, Business Analytics Microforum 310
• Wed., 02:00 p.m. – 02:45 p.m.,B31380 What’s New in SAP BusinessObjects Strategy Management 10.0, 45-Minute Microforum, Business Analytics Microforum 320
• Wed., 02:00 p.m. – 02:45 p.m.,B31397 Managing Risk in Today’s Business Environments, 45-Minute Microforum, Business Analytics Microforum 313
• Wed., 02:00 p.m. – 02:45 p.m.,B32747 Making Business Intelligence Work for You, 45-Minute Microforum, Business Analytics Microforum 300
• Wed., 02:00 p.m. – 02:45 p.m.,B32760 Analytics in Corporate Functions: Know, Decide, and Act With Confidence, 45-Minute Microforum, Business Analytics Microforum 325
• Wed., 02:00 p.m. – 02:20 p.m.,B32750 Release 4.0 of SAP BusinessObjects Edge Solutions Give SMEs the “Edge”, 20-Minute Presentation, Business Analytics Theater 2
• Wed., 02:00 p.m. – 02:20 p.m.,B34204 Outinnovate Your Competition with Business Analytics Solutions from SAP, 20-Minute Presentation, Business Analytics Theater 1

• Wed., 02:30 p.m. – 02:50 p.m.,B31374 Release 10.0 of SAP BusinessObjects EPM Solutions, 20-Minute Presentation, Business Analytics Theater 1
• Wed., 02:30 p.m. – 02:50 p.m.,B33948 How SAP HANA Makes Businesses Run in Real Time, 20-Minute Presentation, Business Analytics Theater 2

• Wed., 03:00 p.m. – 03:45 p.m.,B31343 Explore Data Anywhere with SAP BusinessObjects Explorer, 45-Minute Microforum, Business Analytics Microforum 303
• Wed., 03:00 p.m. – 03:45 p.m.,B31354 Managing the Quality of Your Data from Within SAP Business Suite, 45-Minute Microforum, Business Analytics Microforum 310
• Wed., 03:00 p.m. – 03:45 p.m.,B31379 What’s New in Consolidation Solutions from SAP, 45-Minute Microforum, Business Analytics Microforum 320
• Wed., 03:00 p.m. – 03:45 p.m.,B31393 Today’s User-Access Balancing Act: Risk Versus Cost of Managing Access, 45-Minute Microforum, Business Analytics Microforum 313
• Wed., 03:00 p.m. – 03:45 p.m.,B32770 Big Data: Big In-Memory Computing Opportunities, 45-Minute Microforum, Business Analytics Microforum 300
• Wed., 03:00 p.m. – 03:45 p.m.,B32782 Advanced Analytics and Performance Management in the Public Sector, 45-Minute Microforum, Business Analytics Microforum 325
• Wed., 03:00 p.m. – 03:30 p.m.,B31364 Providing Instant Access to Big Data, 30-Minute Panel Discussion, Business Analytics Theater 1
• Wed., 03:00 p.m. – 03:30 p.m.,B32765 A New Day in the Life of Your Organization: How Business Analytics Can Help, 30-Minute Panel Discussion, Business Analytics Theater 2
• Wed., 03:00 p.m. – 03:45 p.m.,M24847 How Pfizer Uses Mobile Business Analytics from SAP, 45-Minute Microforum, Mobility Microforum 201

• Wed., 04:00 p.m. – 04:45 p.m.,B31341 Manage More with the SAP BusinessObjects BI Platform, 45-Minute Microforum, Business Analytics Microforum 303
• Wed., 04:00 p.m. – 04:45 p.m.,B31357 How Does EIM Fit Within Your Enterprise Architecture?, 45-Minute Microforum, Business Analytics Microforum 310
• Wed., 04:00 p.m. – 04:45 p.m.,B31378 What’s New in SAP BusinessObjects Profitability and Cost Management 10.0, 45-Minute Microforum, Business Analytics Microforum 320
• Wed., 04:00 p.m. – 04:45 p.m.,B31398 Integrating Identity Management with Access Risk Management Solutions, 45-Minute Microforum, Business Analytics Microforum 313
• Wed., 04:00 p.m. – 04:45 p.m.,B32748 Why Data Strategy Is Critical for Implementation of BI and SAP HANA, 45-Minute Microforum, Business Analytics Microforum 300
• Wed., 04:00 p.m. – 04:45 p.m.,B32780 Using Business Analytics to Manage Risk for Your Industry, 45-Minute Microforum, Business Analytics Microforum 325
• Wed., 04:00 p.m. – 04:20 p.m.,B31362 Why Medtronic Chose SAP HANA and SAP NetWeaver BW to Stay Competitive, 20-Minute Presentation, Business Analytics Theater 1*

* [I’ll be presenting “The Future of Work” in the Spotlight theatre: it will be recorded and available for viewing after SAPPHIRE

• Wed., 04:30 p.m. – 04:50 p.m.,B33927 BASF: Secure Supply Chain with SAP BusinessObjects Global Trade Services, 20-Minute Presentation, Business Analytics Theater 1
• Wed., 05:00 p.m. – 05:20 p.m.,B31331 Novartis: Our Journey to Integrated Business Intelligence, 20-Minute Presentation, Business Analytics Theater 1
• Wed., 05:30 p.m. – 05:50 p.m.,B31333 Turn “Big Data” into Business Value with Real-Time BI, 20-Minute Presentation, Business Analytics Theater 2
• Wed., 05:30 p.m. – 05:50 p.m.,B32745 Combining the Power of SAP HANA with Predictive Analytics, 20-Minute Presentation, Business Analytics Theater 1

Day 2 Thursday 11th November

9:00 am to 10:30 am Keynote presentation – Vishal Sikka, SAP Executive Board Member leading technology and innovation

• Thu., 11:00 a.m. – 11:45 a.m.,B31340 Gain Business Insight with SAP BusinessObjects Web Intelligence, 45-Minute Microforum, Business Analytics Microforum 303
• Thu., 11:00 a.m. – 11:45 a.m.,B31358 Enterprise Information Management with the Right Data Migration Strategy, 45-Minute Microforum, Business Analytics Microforum 310
• Thu., 11:00 a.m. – 11:45 a.m.,B31382 Understanding SAP Solutions for Cost and Profitability Analysis, 45-Minute Microforum, Business Analytics Microforum 320
• Thu., 11:00 a.m. – 11:45 a.m.,B31395 Navigating Your Risk, Compliance, and Controls Programs, 45-Minute Microforum, Business Analytics Microforum 313
• Thu., 11:00 a.m. – 11:45 a.m.,B32759 Learn How Collaboration and Analytics Are Changing Supply Chain Management, 45-Minute Microforum, Business Analytics Microforum 325
• Thu., 11:00 a.m. – 11:45 a.m.,B32771 Future of Analytics: Are You Ready for In-Memory Computing and Mobility?, 45-Minute Microforum, Business Analytics Microforum 300
• Thu., 11:00 a.m. – 11:30 a.m.,B31352 We Deliver Quality and Trusted Information to Our Business: Find Out How, 30-Minute Panel Discussion, Business Analytics Theater 2
• Thu., 11:00 a.m. – 11:30 a.m.,B31375 Helping Ensure Quick Time to Value with SAP BusinessObjects EPM Solutions, 30-Minute Panel Discussion, Business Analytics Theater 1

Lunch

• Thu., 01:00 p.m. – 01:45 p.m.,B31339 Information Access with the Semantic Layer of SAP BusinessObjects BI Suite, 45-Minute Microforum, Business Analytics Microforum 303
• Thu., 01:00 p.m. – 01:45 p.m.,B31366 Value Drivers: Understanding Business Benefits Derived from SAP HANA, 45-Minute Microforum, Business Analytics Microforum 310
• Thu., 01:00 p.m. – 01:45 p.m.,B31383 The Financial Closing Cockpit, 45-Minute Microforum, Business Analytics Microforum 320
• Thu., 01:00 p.m. – 01:45 p.m.,B31394 The Next Step in Your Control and Compliance Programs: Automated Controls, 45-Minute Microforum, Business Analytics Microforum 313
• Thu., 01:00 p.m. – 01:45 p.m.,B32754 SAP BusinessObjects BI Suite: Complementary Solutions to SAP Software, 45-Minute Microforum, Business Analytics Microforum 300
• Thu., 01:00 p.m. – 01:45 p.m.,B32763 Stay Competitive 24×7 with Analytics on Your Mobile, 45-Minute Microforum, Business Analytics Microforum 325
• Thu., 01:00 p.m. – 01:20 p.m.,B31371 Integrated Financial Reporting and Planning with SAP Software at Statoil, 20-Minute Presentation, Business Analytics Theater 1
• Thu., 01:00 p.m. – 01:20 p.m.,B32751 Helping Midsize Companies Plan, Budget, and Forecast Amid Constant Change, 20-Minute Presentation, Business Analytics Theater 2

• Thu., 01:30 p.m. – 01:50 p.m.,B31330 Transform How You Do Business by Mobilizing Decision Making, 20-Minute Presentation, Business Analytics Theater 1
• Thu., 01:30 p.m. – 01:50 p.m.,B32776 Get the Inside Story on Deloitte’s Implementation of SAP HANA, 20-Minute Presentation, Business Analytics Theater 2

• Thu., 02:00 p.m. – 02:45 p.m.,B31342 Maximize Analysis Potential with SAP BusinessObjects Analysis Software, 45-Minute Microforum, Business Analytics Microforum 303
• Thu., 02:00 p.m. – 02:45 p.m.,B31365 SAP NetWeaver Business Warehouse 7.3: Update and Impact of SAP HANA, 45-Minute Microforum, Business Analytics Microforum 310
• Thu., 02:00 p.m. – 02:45 p.m.,B31381 Going the Last Mile of Finance with New Disclosure Management Software, 45-Minute Microforum, Business Analytics Microforum 320
• Thu., 02:00 p.m. – 02:45 p.m.,B31396 What’s New in SAP BusinessObjects Global Trade Services 10.0, 45-Minute Microforum, Business Analytics Microforum 313
• Thu., 02:00 p.m. – 02:45 p.m.,B32749 Forecast Accuracy: Maximizing the Usefulness of Business Planning, 45-Minute Microforum, Business Analytics Microforum 300
• Thu., 02:00 p.m. – 02:45 p.m.,B32784 Predictive Analytics and SAP In-Memory Computing for CP and Retail, 45-Minute Microforum, Business Analytics Microforum 325
• Thu., 02:00 p.m. – 02:30 p.m.,B32746 Getting Started with an SAP HANA Platform Implementation, 30-Minute Panel Discussion, Business Analytics Theater 2
• Thu., 02:00 p.m. – 02:30 p.m.,B32779 Innovative Use of Business Analytics for Your Industry and Line of Business, 30-Minute Panel Discussion, Business Analytics Theater 1

• Thu., 03:00 p.m. – 03:45 p.m.,B31344 Empower Your Workforce with Mobile Analytics, 45-Minute Microforum, Business Analytics Microforum 303
• Thu., 03:00 p.m. – 03:45 p.m.,B31368 Leveraging Sybase IQ for Better Business Performance, 45-Minute Microforum, Business Analytics Microforum 310
• Thu., 03:00 p.m. – 03:45 p.m.,B32769 Analytics on the Go: What’s Your Mobile Strategy?, 45-Minute Microforum, Business Analytics Microforum 300
• Thu., 03:00 p.m. – 03:45 p.m.,B32783 What Does Predictive Analytics Mean for Product and Service Creation?, 45-Minute Microforum, Business Analytics Microforum 325
• Thu., 03:00 p.m. – 03:20 p.m.,B31363 Why AOK Chose SAP HANA for Real-Time Analytics, 20-Minute Presentation, Business Analytics Theater 1
• Thu., 03:00 p.m. – 03:20 p.m.,B32778 Next-Generation Analytic Apps for Your Industry and Line of Business, 20-Minute Presentation, Business Analytics Theater 2

• Thu., 03:30 p.m. – 03:50 p.m.,B32758 Business Analytics for Your Operations, Procurement, and Supply Chain, 20-Minute Presentation, Business Analytics Theater 2
• Thu., 03:30 p.m. – 03:50 p.m.,B33779 Logista: Managing Access Risk with SAP BusinessObjects Access Control, 20-Minute Presentation, Business Analytics Theater 1

• Thu., 04:00 p.m. – 04:45 p.m.,B31356 Driving Business Value with Holistic Lifecycle Management of Information, 45-Minute Microforum, Business Analytics Microforum 310
• Thu., 04:00 p.m. – 04:45 p.m.,B32761 Drive Effective Sales and Marketing Decisions with Analytics from SAP CRM, 45-Minute Microforum, Business Analytics Microforum 325
• Thu., 04:00 p.m. – 04:20 p.m.,B31350 Empower Business Users to Bring Data Quality to the Next Level, 20-Minute Presentation, Business Analytics Theater 1
• Thu., 04:00 p.m. – 04:20 p.m.,B32757 Improve Profitability by Leveraging Better Business Analytics for CRM, 20-Minute Presentation, Business Analytics Theater 2

• Thu., 04:30 p.m. – 04:50 p.m.,B31351 Best Practices for a Successful Enterprise Information Management Project, 20-Minute Presentation, Business Analytics Theater 2

• Thu., 05:00 p.m. – 05:30 p.m.,B31334 How We Drive Value from Business Intelligence, 30-Minute Panel Discussion, Business Analytics Theater 1
• Thu., 05:00 p.m. – 05:30 p.m.,B32752 Unleashing the Power of Business Analytics for Midsize Companies, 30-Minute Panel Discussion, Business Analytics Theater 2

Finally, here’s a quick guide to getting the most out of the event from a social point of view:

______________________________________________

Photo of the Feria de Madrid by marcp_dmoz on Flickr

It’s a stormy and sunny day here in Orlando, Florida, for the  2011 ASUG SAP BusinessObjects User Conference in the Dolphin Hotel (ASUG members and conference attendees can see videos and download conference materials at http://www.asugonline.com/)

Bridgette Chambers, CEO of the Americas’ SAP Users’ Group (ASUG) (@bchambersASUG on Twitter), kicked things to the packed room off by reminiscing about how much relations have improved since the SAP and BusinessObjects user groups came together for the first time at the ASUG BusinessObjects conference four years ago.

She cheered up many in the crowd by announcing the extension of the ASUG Valueship for another year (complimentary membership for North American BusinessObjects customers).

She explained that next year’s joint SAP BusinessObjects User Conference and SAP Analytics conference will again be in Orlando, 9th-12th September (book the date!), and urged everybody to take a more active part in the user group.

Steve Lucas, Global GM of SAP Business Analytics delivered the main keynote (@nstevenlucas on Twitter). After an intro worthy of a standup comedian (explaining that the “get 20%” signs on stage were just props, not a promise of future discounts: “SAP is still SAP”) , he explained that membership of the usergroup leads to concrete benefits: “I talk a lot to customers, and I can tell you that BusinessObjects customers are happier when they’re a member of ASUG”.

He went on to emphasize that SAP had delivered on the promises from last year’s user conference: BI 4.0, SAP HANA, BI Mobile, and Line of Business solutions all went production in the course of the year — and gave out his email (steve.lucas@sap.com) to anybody who needed his help.

The focus of the presentation was BI 4.0. Steve: “What’s New in BI 4.0? Everything!”

He then went on to elaborate the key themes of the release:

Openness

“When did being open get passé?!” asked Steve: “Open is still cool. Being open is powerful.” He explained that the ability to access ANY data is one of the key strengths of BusinessObjects: “We are freakin’ open. We can report off of smoke signals.”

He illustrated the point showing off the real-time integration capabilities with a retail demo. “Here’s the ‘traditional’ dashboard that shows you your sales by store, that looks so good that your CEO would love it, even if he didn’t understand it”:

“But let’s see what you can do with real real-time data”. As he grabbed a pair of jeans, a security camera tracked the movement and added extrapolated information such as an estimate of gender and age (this is a real retail solution from a partner).

As Steve chose a pair of Jeans, a “conversion” was registered in the dashboard at the top of the screen:

Geographic location

Steve was joined up on stage by Jason Rose, VP of BI solution marketing (@rosejason on Twitter),, showing a demo of the integration between SAP and Google Maps: “not just what and when: where!”. Jason did a great demo of the third-party GMaps Plugin for Dashboards by Ryan Goodman of Centigon Solutions that allows you to draw areas on the screen that you’d like to analyze further.

Social

Jason then did a demonstration of sentiment analysis on a twitter stream, using Text Analysis and Dashboards: “social data shouldn’t be yet another silo with a specific solution” (you can take a look at the technology yourself at the experience SAP web site).

Mobile

Stephanie Buscemi, Group VP of Marketing, Business Analytics, did a world premiere demo of the next version of SAP BusinessObjects Explorer including offline explorationand the new “Exploration Views” prototyped by the SAP BusinessObjects Innovation center (available first half of next year). Stephanie also showed the latest BusinessObjects Mobile BI product that allows you to access Web Intelligence reports on an iPad.

Hana

Fred Samson came up on stage to demonstrate affinity analysis using SAP Dashboards on top of a HANA datasource — real-time access to 3 Terabytes of retail data, with sub-second response times.

Collaborative

My personal favorite of the show: Jack Miller, GVP for SAP Streamwork and BI OnDemand talked about collaborative business intelligence, with BI information included in a StreamWork “Activity”, and StreamWork feeds appearing directly within the BI portal.

Steve finished up with a summary of why he believes SAP BusinessObjects provides the most complete suite of analytic products available today on the market

There are more photos available on flickr, and a recording of the full keynote will be available within 24 hours on the http://www.asugonline.com/ web site.

I presented yesterday at the SAP Sweden user conference “Impuls” at the green-foliage-decor Stockholmsmässan. Per Högberg of the SAPSA Board kicked things off with a welcome speech.

José Duarte of SAP gave an overview of the key SAP directions:

Michel Putnik of Microsoft and Jo Weilbach of SAP gave an overview of the Microsoft/SAP partnership, and key directions with the updated and improved Duet software.

I gave a short keynote and a track session, focusing on SAP HANA and the in-memory revolution. As usual, here are the slides:

• Keynote: Adobe Acrobat PDF or Microsoft PowerPoint
• Track Session: Adobe Acrobat PDF or Microsoft PowerPoint

Finally, one great anecdote during customer meetings: an executive came to visit the sales office, and noticed that the walls were covered with vacation planning: “I want to see your sales up there, not your holidays!”. The vacation chart was replaced with several large-screen TVs and Xcelsius dashboards (SAP Dashboard Design), and the staff are now able to see their sales compared to other groups — including clapping when a big deal bumps up the charts…

Here’s one real-life example of the limits in quantitative thinking observed in an average company, shared with me by Michael Thompson. He recently worked with Dan Rosenberg’s UX team on a SAP research project that isolated over 100 distinct, real-life uses of business analytics across dozens of roles and business functions.

A retail analyst was attempting to figure out the success of product promotions on the “sales” page of a web site. After gathering the data, he used Excel to create the table below. He compared the increase in each product sold with and without the promotions, and then tried to calculate the average percentage increase across the different products. The result was that “the promotions increased sales by 151% on average”.

This result was then used to create a rule-of-thumb of how much extra inventory the company should have on hand at the start of a promotion: “roughly double”. This lowball figure, it was felt, would generally provide enough stock for promotions, while reducing the chances that the company would be left with too much stock at the end of the promotion.

But there are a whole bunch of problems with this analysis. The most basic mathematical problem is that, while there are exceptions, calculating the average of percentages almost always gives an unwanted answer.

A better way to calculate the success of the promotions is to compare the average sales of all the products before and after a promotion (see table below). This shows that the promotions increased the average quantity of products sold by only 71% – considerably less than double.

* note that it would be it would be easier to compare the % increase in totals items sold, which would come out to the same result, but I’ve used the averages to make the comparison between the two methods as obvious as possible.

So if the company did use the proposed rule of thumb they might have a nasty shock. Instead of an expected “lowball” inventory that would be used up by the promotions, they should instead expect to have an extra 29% of the original items left over at the end.

And beyond the numbers, there’s a lot of unanswered questions about the analysis that call into question its usefulness:

• Is units sold the best measure? Wouldn’t it be better to work with revenue or profit?
• There’s huge variability between the different categories of product. Is the notion of an average even meaningful? Maybe categories of products could be created for more meaningful analysis?
• Do these percentages stay reliable over successive promotions, or are they also very variable?
• Is it in fact better to have too little stock than too much? By how much?
• Might external variables or seasonal variations have lead to the observed results, rather than the promotion? (e.g. maybe it was the 4th of July, and charcoal sales would have soared anyway)

In general, people are tempted to throw up their hands at this point and say “well, we have to make some sort of prediction for inventory levels – isn’t this better than nothing?” And the answer is yes (as long as you get the basic math right!), but somebody in the company should be aware of how limited the analysis currently is, and be constantly trying to improve it. Unfortunately, as in this case, real-world business people often rely on basic, faulty analysis, and lack the curiosity (or incentive?) to push it any further.

Given human nature, it’s probable that this example is not an isolated case. If you really want to improve your company’s return on investment in analytics, you may want to consider investing in more training rather than yet another technology solution.

Many thanks to William Gardella and others for the content below:

Traditional enterprise data warehousing and Hadoop/Big Data are like apples and oranges – the well-known and trusted approach being challenged by a zesty newcomer (sweet oranges were introduced to Europe sometime in the 16th century). Is there room for both? How will these two very different approaches co-exist?

This post is an attempt to summarize the current state of play with Hadoop, “Big Data” and Enterprise BI, and what it means to existing users of enterprise business intelligence. See the list of articles at the end of the post for more detailed materials.

What is Hadoop?

Hadoop is open-source software that enables reliable, scalable, distributed computing on clusters of inexpensive servers. It is:

• Reliable: The software is fault tolerant, it expects and handles hardware and software failures
• Scalable: Designed for massive scale of processors, memory, and local attached storage
• Distributed: Handles replication. Offers massively parallel programming model, MapReduce

Hadoop is designed to process terabytes and even petabytes of unstructured and structured data. It breaks large workloads into smaller data blocks that are distributed across a cluster of commodity hardware for faster processing. And it’s part of a larger framework of related technologies:

• HDFS: Hadoop Distributed File System
• HBase: Column oriented, non-relational, schema-less, distributed database modeled after Google’s BigTable. Promises “Random, real-time read/write access to Big Data”
• Hive: Data warehouse system that provides SQL interface. Data structure can be projected ad hoc onto unstructured underlying data
• Pig: A platform for manipulating and analyzing large data sets. High level language for analysts
• ZooKeeper: a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services

Image: William Gardella

Are Companies Adopting Hadoop?

Yes. According to a recent Ventana survey:

• More than one-half (54%) of organizations surveyed are using or considering Hadoop for large-scale data processing needs
• More than twice as many Hadoop users report being able to create new products and services and enjoy costs savings beyond those using other platforms; over 82% benefit from faster analyses and better utilization of computing resources
• 87% of Hadoop users are performing or planning new types of analyses with large scale data
• 94% of Hadoop users perform analytics on large volumes of data not possible before; 88% analyze data in greater detail; while 82% can now retain more of their data
• Organizations use Hadoop in particular to work with unstructured data such as logs and event data (63%)
• More than two-thirds of Hadoop users perform advanced analysis — data mining or algorithm development and testing

How is it Being Use in Relation to Traditional BI and EDW?

Currently, Hadoop has carved out a clear niche next to conventional systems. Hadoop is good at handling batch processing of large sets of unstructured data, reliably, and at low cost. It does, however, require scarce engineering expertise, real-time analysis is challenging, and it much less mature than traditional approaches. As a result, Hadoop is not typically being used for analyzing conventional structured data such as transaction data, customer information and call records, where traditional RDBMS tools are still better adapted:

“Hadoop is real, but it’s still quite immature. On the “real” side, Hadoop has already been adopted by many companies for extremely scalable analytics in the cloud. On the “immature” side, Hadoop is not ready for broader deployment in enterprise data analytics environments…” James Kobelius, Forrester Research.

To considerably over-simplify: if we consider what’s called the 3 ‘V’s of the data challenge: “Volume, Velocity, and Variety” (and there’s a fourth, Validity), then traditional data warehousing is great at Volume and Velocity (especially with the new analytic architectures), while Hadoop is good at Volume and Variety.

Today, Hadoop is being used as a:

• Staging layer: The most common use of Hadoop in enterprise environments is as “Hadoop ETL” — preprocessing, filtering, and transforming vast quantities of semi-structured and unstructured data for loading into a data warehouse.
• Event analytics layer: large-scale log processing of event data: call records, behavioral analysis, social network analysis, clickstream data, etc.
• Content analytics layer: next-best action, customer experience optimization, social media analytics. MapReduce provides the abstraction layer for integrating content analytics with more traditional forms of advanced analysis.

Most existing vendors in the data warehousing space have announced integrations between their products and Hadoop/MapReduce, or their intention to provide them – for example, SAP has announced that they intend to implement MapReduce in the next version of Sybase IQ.

What Does The Future Look Like?

It’s clear that Hadoop will become a key part of future enterprise data warehouse architectures:

“The bottom line is that Hadoop is the future of the cloud EDW, and its footprint in companies’ core EDW architectures is likely to keep growing throughout this decade. “ James Kobelius, Forrester Research

But (despite some of the almost religious fervor of its backers) Hadoop is unlikely to supplant the role of traditional data warehouse and business intelligence:

“There are places for the traditional things associated with high-quality, high-reliability data in data warehouses, and then there’s the other thing that gets us to the extreme edge when we want to look at data in the raw form”  Yvonne Genovese, Gartner Inc.

Companies will continue to use conventional BI for mainstream business users to do ad hoc queries and reports, but they will supplement that effort with a big-data analytics environment optimized to handle a torrent of unstructured data – which, of course, has been part of the goal of enterprise data warehousing for a long time.

Hadoop is particularly useful when:

• Complex information processing is needed
• Unstructured data needs to be turned into structured data
• Queries can’t be reasonably expressed using SQL
• Heavily recursive algorithms
• Complex but parallelizable algorithms needed, such as geo-spatial analysis or genome sequencing
• Machine learning
• Data sets are too large to fit into database RAM, discs, or require too many cores (10’s of TB up to PB)
• Data value does not justify expense of constant real-time availability, such as archives or special interest info, which can be moved to Hadoop and remain available at lower cost
• Results are not needed in real time
• Fault tolerance is critical
• Significant custom coding would be required to handle job scheduling

Does Hadoop and Big Data Solve All Our Data Problems?

Hadoop provides a new, complementary approach to traditional data warehousing that helps deliver on some of the most difficult challenges of enterprise data warehouses. Of course, it’s not a panacea, but by making it easier to gather and analyze data, it may help move the spotlight away from the technology towards the more important limitations on today’s business intelligence efforts: information culture and the limited ability of many people to actually use information to make the right decisions.

References / Suggested Reading

• Hadoop Goes Mainstream for Big BI Tasks
• Hadoop: Is It Soup Yet?
• Hadoop: What Is It Good For? Absolutely . . . Something
• Hadoop: What Are These Big Bad Insights That Need All This Nouveau Stuff?
• Hadoop: Future Of Enterprise Data Warehousing? Are You Kidding?
• Hadoop: When Will The Inevitable Backlash Begin?
• Hadoop finds niche alongside conventional database systems
• ‘Big data’ analytics fulfilling the promise of predictive BI
• Big data: The next frontier for innovation, competition, and productivity