Interview: Big Data Trends

At the UK SAP Innovation Forum on March 11th, I’ll be hosting the keynote discussion panel on The Business of Business Innovation where we’ll be discussing how organizations can turn new technology into real business change. My guests will include Irfan Khan, SVP & GM of SAP’s Global Big Data Organization. I caught up with Irfan to discuss the latest trends in Big Data.

What’s your background?

I originally started off in hardcore R&D engineering for Sybase developing data management and middleware software. After a decade or so I moved to the role of head of technology evangelism, and became the Chief Technology Office of Sybase in 2008. After the acquisition of Sybase by SAP I became part of the database and technology organization, looking after the go-to-market strategy for the database business. Then in June 2013 I took on the broader “big data” portfolio, including our team of 100 or so data scientists, the role of defining our big data platform and components, and defining some of the applications on top of the platform. Finally, in January of this year, I took on the role of General Manager of our global big data organization. We see databases, data warehousing, and analytics converging to form a solid foundation on which we can provide applications and a hosted application development infrastructure in the cloud.

What are some of the key database trends?

First, the movement to in-memory platforms, as evidenced by everybody trying to catch up with SAP HANA. IBM, Oracle, and Microsoft are all adding in-memory options on top of their existing databases.

Second, the rise of popularity and importance of data scientists. Organizations need to recruit chief data officers who can really make sense of data, not just use it as a historical cold store.

Third, the simplification of data architectures. Traditionally, you would have to have very complex landscapes and many different products, with separate operational and analytic environments and data movement and data duplication between the two. We can now build applications on a consolidated, simplified architecture that supports multiple workloads.

[Gartner calls this “HTAP” and says that:

“Hybrid transactional/analytical processing will empower application leaders to innovate via greater situation awareness and improved business agility. This will entail an upheaval in the established architectures, technologies and skills driven by use of in-memory computing technologies as enablers” ]

Fourth, the rise of SQL on Hadoop. Traditional systems and Hadoop are reaching out to each other. In the future, existing IT teams will be able to easily create applications that use the power of a distributed platform like Hadoop, but without having to adopt new and different skills that are not readily available readily in the market today. Today, Hadoop is more than just a cold store for data, but it’s not an agile real-time platform either. Its power is that it is distributed in nature, with different compute nodes interoperating to solve broader problems.

What does the future hold?

Traditional systems have to embrace what Hadoop provides and create a powerful, highly distributed, elastic platform that allows you to store petabytes of data in a reliable and efficient way. This means integrating with Hadoop, but also ensuring key functionality is available through traditional platforms. For example, we already provide in-database MapReduce. Several vendors today have collections of individually strong components, but the key is to provide a consolidated and simplified view, rather than putting the effort on the shoulders of the customers. You can read more details about how we will be bringing together technologies like SAP HANA, IQ, ASE, ESP, replication, and data services in the roadmap published on the SAP service marketplace.

Why should people to come to the conference?

There’s a lot of change and hype in the market, and we want to help people understand the choices and make the right decisions. If you want to get a better insight into what SAP customers are doing, what they can do, and what would define success in the future, you need to come to our conference!

Thanks for your time!

Read more about Big Data topics on Irfan’s blog, the SAP Community Network, and SAPBigData.com.

Image created from source pictures by Leo Reynolds on Flickr.