![\"hanadoop-example\"](\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/11\/hanadoop-example.jpg?resize=690%2C310&ssl=1\" "\\"hanadoop-example\\"")
<\/p>\n<\/p>\n
An example use case of Hadoop and in-memory systems extracted from the CIO guide to using Hadoop and SAP systems<\/a>.\u00a0<\/em><\/p>\n Earlier this year, I attended the Hadoop Summit in Europe<\/a>, sponsored by Hortonworks. There were many excellent presentations at the conference, but the divide between \u201cold\u201d and \u201cnew\u201d analytics was very clear. There were relatively few \u201ctraditional\u201d companies presenting sessions, and those that were seemed faintly embarrassed to mention that they still had data warehouses.<\/p>\n The Hadoop use cases discussed were mainly new, standalone systems rather than integrations with with more traditional systems or analytic architectures. There were two notable exceptions.<\/p>\n The bank needed a single data platform that could provide 360-degree views of clients, operations and products. To provide this, the team had been struggling with a complex, \u201cbrittle\u201d architecture based on over 150 source systems, 900 ETL jobs, 3 data warehouses, and 15 data marts.<\/p>\n The resulting system was expensive, and too slow to meet the business needs: it took months or years to make changes. The team concluded that they needed a different way of doing things, one that would support more agile, parallel streams of development, without being disruptive.<\/p>\n HSBC decided to try using Hadoop, with the work done in Gaungzhou, China. The project was a big success:<\/p>\n At the same time, however, Anderson explained how his analytics needs were maybe a little different from more traditional data warehousing. The focus of the project was fast-moving, \u201cagile information\u201d typically requiring several different iterations of analysis — and he explained that other parts of the business such as the retail banking division, did not have the same \u201cfunky needs.\u201d<\/p>\n He admitted that\u00a0 HSBC “genuinely doesn’t know yet how the new architecture will combine with existing data warehouses in other regions”<\/p>\n The second notable presentation was Deutsche Telekom\u2019s J\u00fcrgen Urbanksi on how to determine the right technical solutions for different types of enterprise data usage. He presented an overall view of different architectures and suggested questions that should be asked of the business in order to determine which technology was the best fit.<\/p>\n Hadoop was generally positioned as the \u201cbetter\u201d choice, although some of the comparisons with in-memory systems already seemed out of date (e.g. see slide below), and there was little discussion of how to integrate transaction systems (other than as simple data sources).<\/p>\n Integrating Hadoop With Existing Systems<\/span><\/p>\n The unspoken assumption of many of the delegates seemed to be that it was just a question of time before Hadoop gained the extra features that would enable it to take over all enterprise needs. Some seemed almost proud to ignore existing enterprise data architectures and any best practice learned over the previous decades (information governance springs to mind \u2013 this is still a very new concept for many organizations using Hadoop).<\/p>\n Many users of enterprise systems, on the other hand, seemed to have decided that Hadoop that only applies to web companies, or is restricted to refining semi-structured data before putting it into a “normal” data warehouse.<\/p>\n I believe that Hadoop is an incredible opportunity for most enterprises, both large and small. But I also believe that the big changes in enterprise architecture driven by in-memory systems, and the need for analytics close to transactions, mean that the ultimate best practice architecture will be one based on a combination of existing approaches, not just Hadoop alone.<\/p>\n With this in mind, SAP will redistribute and support the Intel Distribution Apache Hadoop<\/a> and the Hortonworks Data Platform<\/a> — InformationWeek journalist Doug Henschen<\/a> explains the background to the deals in his article SAP Expands Big Data Push<\/a>.<\/p>\n As part of the series of SAP \u201cBig Data\u201d discussions, Hortonworks CTO Ari Zilka explained why he felt that combining Hadoop and enterprise systems was the best of both worlds<\/a>.\u00a0 And there\u2019s more information about how real-life organizations are using Hadoop in their organization, in organizations as diverse as football<\/a> and genetics<\/a>, visit the SAP Big Data<\/a> website.<\/p>\n For more detailed technical information about Hadoop can be integrated with traditional information architectures, check out the CIO Guide on Big Data: How to Use Hadoop With Your SAP Software Landscape<\/a>.<\/p>\n I\u2019m looking forward to next year\u2019s Hadoop Summit Europe in April<\/a>, and thoroughly recommend you attend. It\u2019s a great venue and a great crowd. And please vote for my presentation on \u201cReal-Life Examples of Using Hadoop to Drive Business Innovation<\/a>\u201d!<\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" Enterprises should be more interested in Hadoop — and the Hadoop community should be more interested in helping them integrate Hadoop into their existing architectures. <\/p>\n","protected":false},"author":2,"featured_media":5881,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[14],"tags":[27,55,93,173,431,556,557,558,560,576,625,911],"class_list":["post-12366","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-thoughts","tag-bigdata","tag-55","tag-amsterdam","tag-big-data","tag-enterprise","tag-hadoop","tag-hadoop-summit","tag-hadoopsummit","tag-hana","tag-hortonworks","tag-intel","tag-sap"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/11\/hanadoop-example-1.jpg?fit=690%2C310&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p3X9RF-3ds","_links":{"self":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts\/12366","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/comments?post=12366"}],"version-history":[{"count":0,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts\/12366\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/media\/5881"}],"wp:attachment":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/media?parent=12366"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/categories?post=12366"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/tags?post=12366"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}It\u2019s Time For Two Worlds To Come Together<\/h3>\n
![\"alasdair_anderson\"](\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/11\/alasdair_anderson.jpg?resize=150%2C191&ssl=1\" "\\"alasdair_anderson\\"")
The first was by Alasdair Anderson<\/a>, Global Head of Architecture for HSBC Global Banking and Markets, who presented on the theme of \u201cEnterprise Integration of Disruptive Technologies.\u201d<\/p>\n![\"hsbc\"](\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/11\/hsbc.jpg?resize=690%2C393&ssl=1\" "\\"hsbc\\"")
<\/p>\n\n
![\"deutsche](\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/11\/deutsche-telekom-data-storage.jpg?resize=690%2C514&ssl=1\" "\\"deutsche")
<\/p>\n![\"hadoop-logo\"](\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2013\/11\/hadoop-logo.jpg?resize=300%2C71&ssl=1\" "\\"hadoop-logo\\""){kind=logo}
SAP has teamed up<\/a> with major Hadoop providers to combine\u00a0the speed of in-memory computing with the storage power and flexibility of Hadoop.<\/p>\nHadoop Summit Europe 2014<\/h3>\n