{"id":12322,"date":"2013-05-20T16:59:25","date_gmt":"2013-05-20T15:59:25","guid":{"rendered":"http:\/\/timoelliott.com\/blog\/?p=5006"},"modified":"2013-05-20T16:59:25","modified_gmt":"2013-05-20T15:59:25","slug":"customized-voice-of-the-customer-analytics-using-sap-hana","status":"publish","type":"post","link":"https:\/\/timoelliott.com\/blog\/2013\/05\/customized-voice-of-the-customer-analytics-using-sap-hana.html","title":{"rendered":"Customized Voice of The Customer Analytics using SAP HANA"},"content":{"rendered":"<p>At <a href=\"http:\/\/sapphirenow.com\" target=\"_blank\">SAPPHIRE NOW and ASUG 2013<\/a>,\u00a0 <a href=\"http:\/\/www.mantis-tgi.com\/\" target=\"_blank\">Mantis Technology Group<\/a> CEO <a href=\"http:\/\/www.linkedin.com\/in\/dougturner\" target=\"_blank\">Doug Turner<\/a> and Senior Consultant <a href=\"http:\/\/www.linkedin.com\/in\/jimpegan\" target=\"_blank\">Jim Egan<\/a> gave a presentation explaining how they provide their customers with sophisticated, customizable \u201cvoice of the customer\u201d solutions, based on SAP\u2019s powerful text analysis technology.<\/p>\n<p>Their product, <a href=\"http:\/\/pulse.mantis-tgi.com\/Account\/LogOn\" target=\"_blank\">Mantis Pulse Analytics<\/a>, uses a combination of SAP BusinessObjects technologies to gather information from social feeds, analyze sentiment, and combine it with other corporate metrics to make the data actionable.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;\" title=\"image\" alt=\"image\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/image1.png?resize=640%2C203&#038;ssl=1\" width=\"640\" height=\"203\" border=\"0\" \/><\/p>\n<h3>Voice of the Customer Analysis<\/h3>\n<p>There are lots of different sentiment analysis tools on the market, but many of them provide only shallow classification based on the presence of words like \u201cbad\u201d or \u201cgood\u201d in a tweet.<\/p>\n<p>Others, like <a href=\"http:\/\/www54.sap.com\/pc\/tech\/cloud\/software\/social-media-analytics\/index.html\" target=\"_blank\">SAP Social Media Analytics by NetBase<\/a>, are very powerful and easy to use, but are not primarily designed to flexibly combine information from both internal and external data sources. Mantis uses SAP\u2019s text analysis functionality to combine sentiment data with other information from the web, or from internal systems, to help make the data more actionable.<\/p>\n<p>Originally acquired from a company called Inxight, SAP\u2019s out-of-the box \u201centity extraction\u201d technology breaks documents down into sentences, then phrases, then concepts, and then sentiment analysis is applied to these concepts.<\/p>\n<p>Here\u2019s an example analysis on customer survey data from a cruise ship company:<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;\" title=\"image\" alt=\"image\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/image2.png?resize=640%2C325&#038;ssl=1\" width=\"640\" height=\"325\" border=\"0\" \/><\/p>\n<p>Different social sources present a spectrum of different opportunities. Emoticon and acronym-filled teenage tweets may produce little useful sentiment information for a brand, while Facebook posts or entries on a customer feedback page are more likely to provide useful information, and a full customer survey (\u201cplease tell us what else we can do\u2026)\u201d will likely result in highly qualified, very clean information. Individual tweets can be processed very quickly, while entire documents naturally take much longer.<\/p>\n<h3>Using Sentiment Data to Improve Business<\/h3>\n<p>The number one questions Mantis is asked is \u201cwhat do I do with sentiment data?\u201d &#8212; and the answer, of course, depends on the organization. It could be improving brand loyalty, customer service perception, or increase sales \u2013 the goal is to improve the metrics organizations currently use and take better advantage of information that is currently being ignored.<\/p>\n<p>Customers are now starting to expect the same level of response from a tweet or a post to a Facebook page as a call to your customer service center. If the tweet is ignored, it\u2019s like not picking up the call, with negative consequences to your brand image. The good news is that the most important signals are likely to be the clearest. If, as a consumer, I decide to purposely leverage social media to get the attention of a brand, then I\u2019m likely to use direct, easily understood language.<\/p>\n<p>Once the relevant information has been processed, it can be presented to customer service representatives in priority order:<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;\" title=\"image\" alt=\"image\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/image3.png?resize=640%2C363&#038;ssl=1\" width=\"640\" height=\"363\" border=\"0\" \/><\/p>\n<p>The Mantis Pulse Analytics system lets users reply directly on social media, or integrate it into existing email-based service ticket systems:<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;\" title=\"image\" alt=\"image\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/image4.png?resize=640%2C469&#038;ssl=1\" width=\"640\" height=\"469\" border=\"0\" \/><\/p>\n<p>Over time, analysis has become more sophisticated and customized to individual needs. Data is collected from more channels including custom sources such as web surveys and information about particularly influential social media users. Existing CRM customer records can be expanded to include social media profile information.<\/p>\n<p>Time-based correlation analysis allows the comparison of sentiment with things like sales promotions. This helps make the information more actionable. The example below shows sentiment combined with Google Analytics web metrics to show the correlation and success of a marketing campaign by different social channels.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;\" title=\"image\" alt=\"image\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/image20.png?resize=640%2C392&#038;ssl=1\" width=\"640\" height=\"392\" border=\"0\" \/><\/p>\n<p>Other examples of Mantis Pulse reporting:<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;\" title=\"products_pulse_tile4\" alt=\"products_pulse_tile4\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/products_pulse_tile4.jpg?resize=640%2C562&#038;ssl=1\" width=\"640\" height=\"562\" border=\"0\" \/><\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;\" title=\"products_pulse_tile3\" alt=\"products_pulse_tile3\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/products_pulse_tile3.jpg?resize=640%2C562&#038;ssl=1\" width=\"640\" height=\"562\" border=\"0\" \/><\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;\" title=\"products_pulse_tile2\" alt=\"products_pulse_tile2\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/products_pulse_tile2.jpg?resize=640%2C550&#038;ssl=1\" width=\"640\" height=\"550\" border=\"0\" \/><\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;\" title=\"products_pulse_tile1\" alt=\"products_pulse_tile1\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/products_pulse_tile1.jpg?resize=640%2C552&#038;ssl=1\" width=\"640\" height=\"552\" border=\"0\" \/><\/p>\n<p>&nbsp;<\/p>\n<h3>The Challenges of Sentiment Analysis<\/h3>\n<p>Examples of problems and issues encountered by Mantis (if you\u2019re interested in the subject, you should read articles by <a href=\"http:\/\/altaplana.com\/grimes.html\" target=\"_blank\">Seth Grimes of Alta Plana<\/a>, and see presentations from the <a href=\"http:\/\/sentimentsymposium.com\/\" target=\"_blank\">Sentiment Analysis Symposium<\/a> he organizes each year)<\/p>\n<ul>\n<li>If somebody says \u201cI hated the install of the new version, but loved the new features,\u201d the sentiments don\u2019t cancel each other out, but have to be separated and ranked separately for the top positive and negative lists.<\/li>\n<li>A long list of problems followed by a single \u201cresolved\u201d may badly underestimate the real sentiment<\/li>\n<li>A consumer might wax poetically and enthusiastically about chocolate in general for five sentences, but then say \u201cbut I hate M&amp;Ms\u201d in two different ways. In this case, it would probably be more useful to calculate the ratio of positive to negative sentiment as 1 to 1 rather than 5 to 2.<\/li>\n<li>Many words can be interpreted as either a verb or a noun, and quality will be negatively effected by poor sentence structure, punctuation, or run-on sentences that combine several different unrelated concepts.<\/li>\n<li>Not all clear brand sentiment is very actionable. For example, one of Mantis&#8217; customers had predictable spikes of drunken \u201cI love Bud beer!\u201d tweets on Friday and Saturday nights.<\/li>\n<li>The band &#8220;<a href=\"http:\/\/www.bbvd.com\/\" target=\"_blank\">Big Bad Voodoo Daddy<\/a>&#8221; generated negative sentiment, as did the tag line of a clothing campaign \u201c<a href=\"https:\/\/vimeo.com\/23694105\" target=\"_blank\">dress irresponsibly<\/a>\u201d, so those are \u201ctokenized\u201d to take them out of the equation. In addition, specific fields and language\u00a0 may be required for certain industries \u2013 a cruise ship is interested in recording the name of a ship, the cabin number, etc.<\/li>\n<li>Phrases had to be added that weren\u2019t recognized such as \u201cabend\u201d (from \u201cabnormal end\u201d or crash)<\/li>\n<li>Some expletives have to be classed as positive sentiment, in phrases such as \u201cjust bought a guitar, it\u2019s f****g awesome\u201d<\/li>\n<li>Call centers exist in order to deal with problems, so rather than just classifying \u201cproblem\u201d as negative, it\u2019s important to distinguish between phrases such as \u201cyou solved my problem\u201d and \u201cI have a new problem.\u201d<\/li>\n<\/ul>\n<p>Mantis found that tuning the system for different customer meant adding specific custom dictionaries to the system. Rather than modifying the underlying text analysis \u201c<a href=\"http:\/\/wiki.sdn.sap.com\/wiki\/display\/EIM\/CGUL+Tips+and+Tricks+for+Entity+Extraction\" target=\"_blank\">CGUL<\/a>\u201d files \u2013 which are the same for all the Mantis Pulse Analytics customers, and can generate unwanted side effects, since rules are based on other rules &#8212; Mantis used <a href=\"http:\/\/www54.sap.com\/pc\/tech\/enterprise-information-management\/software\/data-services\/index.html\" target=\"_blank\">SAP Data Services<\/a> to make appropriate upfront changes to the data.<\/p>\n<p>In general, it\u2019s important to realize that sentiment analysis can never be an exact science, given the complexity of human language \u2013 and much language is fundamentally ambiguous, with different people classifying sentiment in different ways. Manual evaluation of data samples by Mantis and their customers indicates an accuracy rate of around 80% compared to human interpretation of sentiment.<\/p>\n<p>In particular, humans have the benefit of a greater amount of \u201ccontext knowledge\u201d that can be discerned just from the text itself\u00a0 \u2013 which means there will always be false positives and negatives. This means that the results are much, much better than ignoring the data entirely, and can be used in aggregate to detect important trends, but have to be used with caution when deciding, for example, whether a customer service representative should keep his or her job.<\/p>\n<h3>The Underlying Technology<\/h3>\n<p>Mantis Pulse Analytics has existed for several years, in both on-premise and Amazon-cloud-based versions. Custom tools are used to collect the data, using standard social APIs and code to crawl specialized forms and blogs. Once the data is collected, it was put into a MySQL holding area, then the sentiment was extracted using SAP BusinessObjects Data Services Text Data Processing to create a data mart, with reporting using SAP BusinessObjects.<\/p>\n<p>Mantis Pulse Analytics was rumored one of the largest BusinessObjects text analytics installations ever, with a Text Analytics Farm of 20-25 concurrent versions of the text analytics engine.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;\" title=\"image\" alt=\"image\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/image5.png?resize=640%2C407&#038;ssl=1\" width=\"640\" height=\"407\" border=\"0\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Customers wanted ever-more powerful analysis, ever faster, and Mantis turned to <a href=\"http:\/\/www.saphana.com\/community\/learn\/cloud-info\" target=\"_blank\">SAP HANA One<\/a> running in the Amazon cloud, which now provides the same text analysis engines, but built directly into the platform, and processed in-memory.\u00a0 The result is much faster, with fewer moving parts.<\/p>\n<p>The data is collected directly in a <a href=\"http:\/\/www.saphana.com\/\" target=\"_blank\">SAP HANA<\/a> table, and a full text index created, without any need for rollups or aggregate tables. Reporting is unchanged \u2013 except that sentiment analysis against million of documents now running in seconds.<\/p>\n<p>Currently, all customer data is processed using the same configuration \u2013 in the future, the company may move to using multiple tables, each with their own customizable table index.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border-width: 0px;\" title=\"hanatextanalyticsmantis\" alt=\"hanatextanalyticsmantis\" src=\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/hanatextanalyticsmantis.jpg?resize=640%2C392&#038;ssl=1\" width=\"640\" height=\"392\" border=\"0\" \/><\/p>\n<p>The SAP HANA-based solution currently provides a little less visibility into the text processing process than the previous solution, but this is more than made up for by the dramatic advantages when it comes to the reporting and analysis \u2013 \u201cIt\u2019s coming to the state where the processing is at the speed of thought\u201d says Doug Turner. <strong>\u201cOur customers are trying to get to the point where they can respond to a tweet <em>before<\/em> a customer leaves a store\u201d<\/strong><\/p>\n<p>If you\u2019d like to know more about the Mantis Pulse Analytics solution, you can follow <a href=\"http:\/\/twitter.com\/mantispulse\" target=\"_blank\">@mantispulse<\/a> on twitter, or <a href=\"http:\/\/www.mantis-tgi.com\/#\/products\" target=\"_blank\">visit the web site<\/a>.<\/p>\n<p>There\u2019s also a version of the solution designed specifically for sports organizations called <a href=\"http:\/\/www.mantis-tgi.com\/#\/products_at\" target=\"_blank\">Mantis Pulse Athletics<\/a>, designed to \u201cmonitor, measure, and visualize social media and to reduce the risk of regulatory and organizational compliance issues\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Mantis Pulse Analytics is a powerful, customizable, cloud-based voice of the customer solution that uses SAP technology. <\/p>\n","protected":false},"author":2,"featured_media":4994,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[2,3,10],"tags":[100,213,324,560,911,916,931,960,961,984,986,1063,1064,1065,1066,1067,1125,1126],"class_list":["post-12322","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-best-practice","category-bi-20","category-products","tag-analytics","tag-businessobjects","tag-dashboards","tag-hana","tag-sap","tag-sap-hana","tag-saphana","tag-sentiment","tag-sentiment-analysis","tag-social","tag-social-analytics","tag-text","tag-text-analysis","tag-text-analytics","tag-text-processing","tag-textanalytics","tag-voc","tag-voice-of-the-customer"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/05\/mantis-pulse-banner-1.jpg?fit=690%2C310&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p3X9RF-3cK","_links":{"self":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts\/12322","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/comments?post=12322"}],"version-history":[{"count":0,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts\/12322\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/media\/4994"}],"wp:attachment":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/media?parent=12322"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/categories?post=12322"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/tags?post=12322"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}