Photo Essay Part 1: Analytics at SAPPHIRE NOW Pre-Show

It was a hectic SAPPHIRE NOW event in Orlando last week! In this series of posts, I’d like to give you a flavor of what it’s like to attend the event from an analytics point of view — and I’ve included links to some of the great analytics content that came out of the show.

Here’s part one: the run up to the show…

I was a “social media ambassador” for analytics for the conference – my job was to keep track of everything about business intelligence, analytics, and big data at the event. After entertaining myself with my introductory animated gif above, I prepared a pre-show reading list of the useful blogs and links.

I set about preparing my SAPPHIRE NOW survival gear – mostly lots of camera gear and electronics but also my “HANA HANA HANA…” t-shirt that I thought might come in handy…

 

Unfortunately, just before I was due to leave, I managed smash the screen of my iPhone screen – no phone, email or instagram during the show? Gulp…  Luckily, I managed to find a place that offered an (expensively) same-day screen replacement service, and I was back in action (phone shown here with an appropriate “cloud” reflection):

 

On the Friday before the conference, SAP announced the new SAP Lumira Cloud product (in beta), and had me busy explaining the renaming of SAP Visual Intelligence.

Early on Monday morning, I started the 15+ hours of travel it took to get to Orlando – hey! what’s that view from my airline window?!

The travel was wonderfully uneventful, for once, and I just had time for a quick shower before heading off to the Human Face of Big Data networking evening event sponsored by Intel.

The inimitable Steve Lucas (now in charge of both big data and analytics at SAP) opened up the evening with his “ah-hana” moment: some personal thoughts on how big data affects his life as a type 1 diabetic.

And author Rick Smolen gave some great examples of how big data will affect all of our lives in the future.

And Intel were on hand to explain their vision and the SAP partnership around Intel’s Hadoop distribution.

Lucky attendees walked away with a signed copy of Rick’s book, which is suitably… big! I strongly recommend it for anybody who wants to get a feel for what is becoming possible with analytics (or get the iPad app: it’s a selection of the better stories, and it’s much cheaper and easier to lift!)

Here’s a Vine recording of some of the pages:

In between networking and catching up with old friends, I had the opportunity to interview several attendees on camera to get their thoughts on what big data means to them (video in preparation!):

Time to head back to the hotel through the walkways that snake around the conference center and the closest hotels:

…and that was the end of Day 0! More soon.

Share and Enjoy

Customized Voice of The Customer Analytics using SAP HANA

At SAPPHIRE NOW and ASUG 2013,  Mantis Technology Group CEO Doug Turner and Senior Consultant Jim Egan gave a presentation explaining how they provide their customers with sophisticated, customizable “voice of the customer” solutions, based on SAP’s powerful text analysis technology.

Their product, Mantis Pulse Analytics, uses a combination of SAP BusinessObjects technologies to gather information from social feeds, analyze sentiment, and combine it with other corporate metrics to make the data actionable.

Voice of the Customer Analysis

There are lots of different sentiment analysis tools on the market, but many of them provide only shallow classification based on the presence of words like “bad” or “good” in a tweet.

Others, like SAP Social Media Analytics by NetBase, are very powerful and easy to use, but are not primarily designed to flexibly combine information from both internal and external data sources. Mantis uses SAP’s text analysis functionality to combine sentiment data with other information from the web, or from internal systems, to help make the data more actionable.

Originally acquired from a company called Inxight, SAP’s out-of-the box “entity extraction” technology breaks documents down into sentences, then phrases, then concepts, and then sentiment analysis is applied to these concepts.

Here’s an example analysis on customer survey data from a cruise ship company:

Different social sources present a spectrum of different opportunities. Emoticon and acronym-filled teenage tweets may produce little useful sentiment information for a brand, while Facebook posts or entries on a customer feedback page are more likely to provide useful information, and a full customer survey (“please tell us what else we can do…)” will likely result in highly qualified, very clean information. Individual tweets can be processed very quickly, while entire documents naturally take much longer.

Using Sentiment Data to Improve Business

The number one questions Mantis is asked is “what do I do with sentiment data?” — and the answer, of course, depends on the organization. It could be improving brand loyalty, customer service perception, or increase sales – the goal is to improve the metrics organizations currently use and take better advantage of information that is currently being ignored.

Customers are now starting to expect the same level of response from a tweet or a post to a Facebook page as a call to your customer service center. If the tweet is ignored, it’s like not picking up the call, with negative consequences to your brand image. The good news is that the most important signals are likely to be the clearest. If, as a consumer, I decide to purposely leverage social media to get the attention of a brand, then I’m likely to use direct, easily understood language.

Once the relevant information has been processed, it can be presented to customer service representatives in priority order:

The Mantis Pulse Analytics system lets users reply directly on social media, or integrate it into existing email-based service ticket systems:

Over time, analysis has become more sophisticated and customized to individual needs. Data is collected from more channels including custom sources such as web surveys and information about particularly influential social media users. Existing CRM customer records can be expanded to include social media profile information.

Time-based correlation analysis allows the comparison of sentiment with things like sales promotions. This helps make the information more actionable. The example below shows sentiment combined with Google Analytics web metrics to show the correlation and success of a marketing campaign by different social channels.

Other examples of Mantis Pulse reporting:

 

The Challenges of Sentiment Analysis

Examples of problems and issues encountered by Mantis (if you’re interested in the subject, you should read articles by Seth Grimes of Alta Plana, and see presentations from the Sentiment Analysis Symposium he organizes each year)

• If somebody says “I hated the install of the new version, but loved the new features,” the sentiments don’t cancel each other out, but have to be separated and ranked separately for the top positive and negative lists.
• A long list of problems followed by a single “resolved” may badly underestimate the real sentiment
• A consumer might wax poetically and enthusiastically about chocolate in general for five sentences, but then say “but I hate M&Ms” in two different ways. In this case, it would probably be more useful to calculate the ratio of positive to negative sentiment as 1 to 1 rather than 5 to 2.
• Many words can be interpreted as either a verb or a noun, and quality will be negatively effected by poor sentence structure, punctuation, or run-on sentences that combine several different unrelated concepts.
• Not all clear brand sentiment is very actionable. For example, one of Mantis’ customers had predictable spikes of drunken “I love Bud beer!” tweets on Friday and Saturday nights.
• The band “Big Bad Voodoo Daddy” generated negative sentiment, as did the tag line of a clothing campaign “dress irresponsibly”, so those are “tokenized” to take them out of the equation. In addition, specific fields and language  may be required for certain industries – a cruise ship is interested in recording the name of a ship, the cabin number, etc.
• Phrases had to be added that weren’t recognized such as “abend” (from “abnormal end” or crash)
• Some expletives have to be classed as positive sentiment, in phrases such as “just bought a guitar, it’s f****g awesome”
• Call centers exist in order to deal with problems, so rather than just classifying “problem” as negative, it’s important to distinguish between phrases such as “you solved my problem” and “I have a new problem.”

Mantis found that tuning the system for different customer meant adding specific custom dictionaries to the system. Rather than modifying the underlying text analysis “CGUL” files – which are the same for all the Mantis Pulse Analytics customers, and can generate unwanted side effects, since rules are based on other rules — Mantis used SAP Data Services to make appropriate upfront changes to the data.

In general, it’s important to realize that sentiment analysis can never be an exact science, given the complexity of human language – and much language is fundamentally ambiguous, with different people classifying sentiment in different ways. Manual evaluation of data samples by Mantis and their customers indicates an accuracy rate of around 80% compared to human interpretation of sentiment.

In particular, humans have the benefit of a greater amount of “context knowledge” that can be discerned just from the text itself  – which means there will always be false positives and negatives. This means that the results are much, much better than ignoring the data entirely, and can be used in aggregate to detect important trends, but have to be used with caution when deciding, for example, whether a customer service representative should keep his or her job.

The Underlying Technology

Mantis Pulse Analytics has existed for several years, in both on-premise and Amazon-cloud-based versions. Custom tools are used to collect the data, using standard social APIs and code to crawl specialized forms and blogs. Once the data is collected, it was put into a MySQL holding area, then the sentiment was extracted using SAP BusinessObjects Data Services Text Data Processing to create a data mart, with reporting using SAP BusinessObjects.

Mantis Pulse Analytics was rumored one of the largest BusinessObjects text analytics installations ever, with a Text Analytics Farm of 20-25 concurrent versions of the text analytics engine.

 

Customers wanted ever-more powerful analysis, ever faster, and Mantis turned to SAP HANA One running in the Amazon cloud, which now provides the same text analysis engines, but built directly into the platform, and processed in-memory.  The result is much faster, with fewer moving parts.

The data is collected directly in a SAP HANA table, and a full text index created, without any need for rollups or aggregate tables. Reporting is unchanged – except that sentiment analysis against million of documents now running in seconds.

Currently, all customer data is processed using the same configuration – in the future, the company may move to using multiple tables, each with their own customizable table index.

The SAP HANA-based solution currently provides a little less visibility into the text processing process than the previous solution, but this is more than made up for by the dramatic advantages when it comes to the reporting and analysis – “It’s coming to the state where the processing is at the speed of thought” says Doug Turner. “Our customers are trying to get to the point where they can respond to a tweet before a customer leaves a store”

If you’d like to know more about the Mantis Pulse Analytics solution, you can follow @mantispulse on twitter, or visit the web site.

There’s also a version of the solution designed specifically for sports organizations called Mantis Pulse Athletics, designed to “monitor, measure, and visualize social media and to reduce the risk of regulatory and organizational compliance issues”

Share and Enjoy

SAPPHIRE NOW: The Human Face of Big Data

SAPPHIRE NOW in Orlando was packed with examples of how Big Data affects all our lives.

SAP Executive Steve Lucas was the host of an Intel-sponsored Big Data special interest group meeting at the conference. He kicked off the session by explaining that he considers himself to be an example of the human face of new data technologies. As a type 1 Diabetic, he wears a device that measures his blood sugar once a second, and communicates it to a hand-held device that he uses to constantly monitor his diet and take control of his health:

“For a diabetic, you literally are the living embodiment of data because you spend most of your spare time during your day checking and monitoring your blood sugar number.”

Rick Smolen, a former Time Life journalist and photographer, spoke at the event and during a panel the next day.

After working on various projects including the very successful “Day in the Life” series of books, Rick’s most recent book (and accompanying interactive iPad application) is called The Human Face of Big Data, and it’s full of compelling stories of how new data technologies are making a difference.

Rick explained that Marissa Meyer, now CEO of Yahoo, helped draw him to the project when she described technology as “helping the planet develop a nervous system.” In other words, by aggregating all of the sensors, searches and transactions in the world, it gives us a sense of what’s happening in the planet in real time.

In his forward to the book, Rick says:

“On the following pages, as you meet the men, women, and children whose lives are being transformed by this data revolution, I suspect you will come to the same conclusion that I have: Big Data may well turn out to be the most powerful tool set the human race has ever had to address the widespread challenges facing our species and our planet”

Rick believes that big data will change our lives – and not just in the business world. “Healthcare is at the top of the list” he explained. Today, as drug companies do clinical trials, they may find that it can help 99% of patients, but harm a small percentage, so could never get approved by the FDA. But the price of genome decoding is plummeting, holding out the promise of personalized medicine, prescriptions based on what is effective for your particular DNA. “Today, the cures exist, but they are sitting on the shelves” said Rick “today, antibiotics is one size fits all. The whole work is going to change thanks to medicine designed to treat the individual.”

Sensors will also result in new opportunities. “For example, 14 seconds before the recent massive earthquake in Japan, every bullet train and every factory came to a halt” said Rick. “Because the country had prepared a hardwired sensor system to detect the wave that comes before the violent one”.  That required a huge upfront capital investment. In silicon valley, a group of researchers realized that every modern laptop has sensors to protect their hard drives: if you drop the computer, it lifts the head off the platter to try to save the spinning disk. So they created the Quake Catcher Network, that allows crowd-sourced earthquake detection around the world. You download the free software to your computer and let it run in the background (it can be your default screensaver, for example). Then the software aggregates all the data and if one person’s laptop sensor is set off by, for example, a heavy truck going by, nothing happens, but if all the sensors in a 30 mile radius go off, then it’s an earthquake…

Sensors are also behind a new “magic carpet”: “My mother is 90 years old, and my father passed away six years ago. Six months ago, my mother fell, and then she fell another time, and the third time, nobody found her for five hours. We asked her to move in with us, but she didn’t want to. We hired people to live with her in shifts, and she hated it. Now GE and Intel are introducing products aimed at aging at home, and one of the prototypes is a carpet filled with sensors. Over time, it creates a baseline knowledge of ‘normal behavior’ – she walks on the carpet at 9:30 am, and here’s her gait – over time it can predict muscle weakness, and changes to normal patterns and tweet me to “call mom!”

Rick believes one of the biggest opportunities is making better use of previously ignored “dark data”. “For years, meteorologists have had to filter out ‘bioclutter’ from Doppler radar weather systems – the “noise” generated by flocks of birds or bats. But when bird researchers realized they had 15 years of invaluable data on migration patterns they were delighted!”

But Rick also cautioned that there will be new challenges. For example, today, data is typically owned by governments and businesses, not individuals, and there needs to be more thinking about how to make sure that powerful data is not misused.

For more fantastic examples and stories about Big Data is affecting people’s lives, visit http://bigdata.saphana.com/

Share and Enjoy

New SAP Lumira: See The Light

As announced last Friday, SAP Visual Intelligence has been renamed to SAP Lumira (you can find the announcement and full FAQ here)

Why Did We Change Another Perfectly Good BI Product Name?

We know that this decision is likely to be met by some eye-rolling from our faithful customers who are still recovering from previous name changes.

The bottom line: we know that there are downsides to such a change, and despite the inevitable disruption it may cause, we feel it’s the right choice. Here are some of the key reasons that tipped the balance:

Family Ties. As we announced earlier this week, the next version of SAP Visual Intelligence (sorry, SAP Lumira) will offer tight integration with SAP’s next-generation cloud BI platform, SAP Lumira Cloud. We feel that it’s more appropriate to use a completely new name for a completely new “network of truth” approach to business intelligence and analytics (oh – and “SAP Visual Intelligence Cloud” is clunky).

Searching and Sharing. SAP Lumira is a more human-friendly yet Google-ready name. As with SAP HANA (which trips off the tongue more easily than “SAP’s new platform for real-time analytics and applications”), a distinctive name makes it easier to find relevant materials (it’s one of the reasons we took the space out of “Business Objects” many years ago). And with community becoming increasingly important, this change should ultimately make it easier to find the information you need to deploy and use the products successfully (but alas, no, this doesn’t mean that Xcelsius is coming back, even if the logic is the same.)

Destined for Greatness. Although BI has been around for a long time, we believe that analytic users are still a minority. SAP Lumira aims to change that by making it easier than ever to get powerful, self-service business intelligence into the hands of everyone. We have big plans for the product that we can’t quite talk about yet, but we believe it is destined for greatness, and it’s less painful to make the switch now than later.

 

SAPLumira New Features

• Enhanced data manipulation capabilities, with better merge functionality, replay manipulation steps & improved formula editor
• Extended acquisition to more data sources & support different data formats during acquisition
• Empower end users to customize charts , add conditional formatting and create simple user defined views without scripting or coding
• Annotate and share your results at a click of a button on-premise or on the cloud.

Please Give Feedback and Spread the Word

If you’re reading this, you’re probably one of the people that know and use our products, and we’re very grateful. We know that changing names is disruptive, not only for you but for the business people in your organization that use our products to get insight. We’re eager to help you make the change as smooth as possible, so please let us know what we can do to help by taking part in the discussions at the SCN area. And we’d like your help to get the word out about the changes to the rest of the world (#SAPVisi is dead! Long live #SAPLumira!).

Data Geek Challenge 2.0

Looking to do something with your new product? At SAPPHIRE NOW, we’re launching the next round of the Data Geek Challenge  at www.sap.com/datageek. To enter, create an interesting view of data using SAP Lumira desktop and your data or data we’ve provided and share with the SAP Data Geek (datageek@sap.com) via SAP Lumira Cloud.

Share and Enjoy

SAP BusinessObjects BI 4.1 Release to Customers

Just before the SAPPHIRE NOW conference, SAP announced the Release to Customer (RTC) of SAP BusinessObjects BI 4.1, Edge BI 4.1 and Crystal Server 2013.

What’s New in BI 4.1

BI 4.1 is one suite for all insight, one place for all information and on standard for enterprise BI. Here are the highlights of the new release – or you can skip to the end of the post to get a detailed technical presentation of the new features.

The features in this latest release include:

• Unified and highly personalized platform with improved interoperability and usability across the BI suite
• Powerful visualizations that empower end users with industry focused visualizations and spatial analytics
• Big Data ready with support for Amazon Elastic MapReduce and Hadoop Hive. Improved access to Oracle’s Exadata, OLAP and Essbase. BI 4.1 also includes enhanced workflow capabilities in conjunction with SAP HANA and BW
• Professional grade BI platform for any size business, any type of deployment and for any role across the organization

BI 4.1 includes features for all five areas of SAP BI innovation

One Suite for All Insight

 

 

One Place for All Information

One Standard for Enterprise BI

Three key types of information access: self-service, dashboards and apps, and reporting

New Self-Service Features

New Dashboarding Features

New Reporting Features

New Platform Features

What’s different about this release?

Summary of Key Messages for Business Objects 4.1

One strategy for enterprise BI: one suite for all insight, one place for all information, one standard for enterprise BI.

One Suite For All Insight

Unified and highly personalized

• Improved interoperability and usability across the BI suite
• Deliver engaging information to users when and where they need it
• Create custom experiences with embedded analytic extensions

Powerful visualizations

• Monitor your business with specialized charting and geospatial analytics
• Empower end users with industry focused visualizations and spatial analytics
• Detect outliers and discover areas to optimize your business

Faster more accurate decisions

• Create and consume BI content from any mobile device
• Empower teams to deliver collaborative decisions
• Simplified deployment to accelerate time to value

One Place For All Information

Big data ready

• Support for Amazon Elastic MapReduce and Hadoop Hive
• Improved access to Oracle Exadata, Oracle OLAP and Oracle Essbase
• Enhanced workflow capabilities in conjunction with SAP HANA and BW

Scale and power

• Immediate response time regardless of data volume
• Improved usability to build pervasive BI applications
• Integrate with SAP monitoring, lifecycle management, search, and user management

Speed to deliver

• Enhanced design experience with new creation wizard and integrated data federation
• Improved Web Intelligence report design performance against BW
• Automated universe deployment and lifecycle management

One Standard For Enterprise BI

 

Professional grade BI platform

• For any size business, small to large
• For any type of deployment, on-premise or on-demand
• For anyone across the organization, individuals, teams, or department

Quality focused

• Increased efficiency to enhance decision making capability and productivity
• Non-disruptive production delivery and information governance across the organization
• Simplified architecture for increased IT responsiveness

Superior breadth of ecosystem

• Open and agnostic suite of products that can leverage existing BI investments
• Supported by global best practices for enhanced adoption
• Enable speedy deployment with shortened time to value

Key Links

• Getting Started with SAP BusinessObjects BI Solutions
• BI4 Upgrade Resources
• How to Select the Right BI Tool for Your Environment
• Sizing and Deploying SAP BusinessObjects BI4
• eLearning product tutorials
• Road Maps on SAP Service Marketplace
• SAP’s Release Strategy for Major Releases of SAP BusinessObjects BI

Full, Detailed Technical Feature Documentation/Presentation

Here’s the official feature-level what’s new manual for BusinessObjects 4.1, and a presentation by Ty Miller, delivered at the ASUG2013 / SAPPHIRENOW Conference

Share and Enjoy

Next Page »