What Is Big Data Discovery?

big data discovery

According to Gartner, “Big Data Discovery” is the next big trend in analytics.

It’s the logical combination of three of the hottest trends of the last few years in analytics: Big Data, Data Discovery, and Data Science.

Each of these areas has seen explosive growth, but there are clear upsides and downsides to each. For example, Data Discovery excels in ease of use, but allows only limited depth of exploration, while Data Science provides powerful analysis but is slow, complex, and difficult to implement.

big data discovery graphic
Source: Gartner. Big Data Discovery is the combination of Big Data, Data Science, and Data Discovery.

Since the disadvantages of the three technologies map to nicely to the advantages of the others, they are now starting to blend, and Gartner believes Big Data Discovery will be a distinct new market category by 2017.

gartner strategic planning assumption

The emerging Big Data Discovery tools will be simpler to use than data science products and accessible to a wider ranger of users, with more powerful manipulation of a wider variety of data sources.

According to Gartner Analyst Joao Tapadinhas, these tools will be used by new “Citizen Data Scientists” who marry the skills of traditional business analysts with some of the expertise of expert statisticians.

citizen data scientists

These users would not replace existing data scientists, but complement their limited availability to expand the use of these powerful new technologies to more business opportunities.

new business opportunities

The marketing users are Vodafone are a good example of this new role in action: they were able to use data science to create new product offers, without requiring the services of the company’s dedicated team of statisticians.

SAP is an example of a vendor who has been working to converge the three different areas.

SAP Big Data Discovery

The company provides a fast, in-memory platform called SAP HANA that includes various Big Data analytic engines, including predictive analytics, and links to open source platforms such as Hadoop and Spark.

The SAP Lumira data discovery product runs on top of the platform to provide self-service data manipulation and visualization, and integrates tightly with the latest SAP Predictive Analytics 2.0 product. The latter combines a traditional advanced analytics workbench with the famously easy-to-use automated data preparation and mining of the SAP Infinite Insight product.

In addition, SAP is taking things to the next level. After carrying out co-innovation projects with customers, the company has created new packaged business applications such as SAP Predictive Maintenance. The application uses combines sensor data access, embedded data science algorithms, and traditional business measures to create new best-practice business processes.

What do you think? Will Big Data Discovery take over from existing approaches, or add to them? (and will it just become “Data Discovery” over time?)



6 responses to “What Is Big Data Discovery?”

  1. […] Data discovery (you can find an interesting article here) is now ready to be implemented. We all used 2015 to understand and practice how to connect to Big […]

  2. […] Get closer to the vision of “Big Data Discovery” […]

  3. […] also excited by Big Data Discovery, which brings together three existing technologies: Big Data, Data Science and Data Discovery.  […]

  4. […] Get closer to the vision of “Big Data Discovery” […]

  5. Pat Hennel Avatar

    “These users would not replace existing data scientists, but complement their limited availability to expand the use of these powerful new technologies to more business opportunities.”

    As adoption of big data and business intelligence continues to grow, more employees are utilizing data beyond just data scientists and analysts. I agree that new users aren’t going to replace data scientists – but rather employees will soon incorporate data analysis into their day-to-day operations.

  6. Troy Thurston Avatar
    Troy Thurston

    Although I believe that “Big Data” will someday just be “Data” (the TB and PB of today will become the MB and GB of tomorrow), there’s no denying the challenges of data discovery and data science with the 3 V’s of big data now. I personally like SAP’s focus in addressing these challenges with the integration of HANA, Predictive Analysis, and Lumira.