Is big data doomed?

Mar 13th, 2017

Gary Allemann, MD at Master Data Management

By Gary Allemann, Managing Director, Master Data Management

Big data has been relatively slow to take off in South Africa. Despite the hype, many organisations are only now beginning to experiment with technologies such as Hadoop. Big data is often still seen as the domain of the IT department, or the Chief Data Officer, and real successes have been few and far between.

Does this mean that the hype about big data will never deliver in practise?

The reality, however, is not that the importance of Big Data has been overstated but rather that organisations that have failed to focus their implementations correctly are not following best practices to ensure value and Return on Investment (ROI).

In a world where information is currency, data should always be a priority for business, and the ability to understand and gain insight from data is invaluable. This not only helps organisations understand how the business is being run, but also informs how the business can be run better and more efficiently in the future.

Despite the clear evidence of the need for enhanced data and analytics, many enterprises end up with what they see as failed solution implementations. In many cases this can be linked to the lack of a legitimate big data use case.

Approach Big Data from the right perspective

Many organisations approach Big Data implementations from a technology perspective, identifying a solution and trying to make it fit their business rather than understanding what their business needs and finding a suitable solution.

In order to ensure Big Data delivers as expected, it is essential to understand a number of different elements before attempting to implement a solution to fit. These include identifying which area of the business you wish to analyse data from, which data sources are most relevant and who is able to access this data.

It is also important to get a rough idea of the quality and structure of the data to be analysed, and understand how to mitigate and resolve any issues with data quality.

Finally, it is critical to understand what type of insight you are hoping to derive from the data, as well as which insights will add business value, as this will help to focus the project.

Plan before implementation

Before you involve the data scientists, ingest any data, or push any buttons, it is important to answer questions such as:

  • What data domain or business areas do you wish to analyse?
  • What data sources will be most relevant to your analysis?
  • How will you get access to that data?
  • Are you allowed to use the data for the purpose identified?
  • Do you understand the data quality? If not, are you able to get a preview?
  • Can you identify the subject-matter expert who can help you assess data quality and give you pointers on how to mitigate and resolve data quality issues?
  • Do you know what insights you’re hoping to derive form the data?
  • How will these insights support the business goals?

One of the main reasons that Big Data projects fail to deliver as expected, is because organisations do not answer these data governance questions or follow the necessary steps to plan the project before implementing a tool.

Conducting the planning phase up front increases efficiency and helps to ensure that outcomes are seen to deliver value.