It didn’t take long for the first personal computers to run out of storage space. Today, even the most basic smartphone is much more powerful, with way more capacity than those early models. The term Big Data was coined during discussions around that time on how to harness and store the vast amount of information that was being produced daily – an amount which, compared to today’s output each second, is minuscule. It is estimated that by 2025, around 460 exabytes of data will be created each day, which would bring the total existing data to 175 zettabytes – to put those numbers into perspective, if you tried to download all of that with today’s average broadband, it would take around 1.8 billion years…
Industries across almost all sectors use and benefit from data in some way. It goes far beyond the high-profile negative examples of Facebook harvesting our details to serve us personalised ads or a breach by an airline losing our email addresses. The last couple of years have seen exposure data help tackle a global pandemic and genome databases used to fast-track the development of vaccines. Unparalleled benefits to society, such as the worldwide attempts to transition to cleaner energy, would not be possible without the analysing of data. But the problem is, how do we keep up with the amount being produced?
Organisations have been collecting data for some time now, but because of the amount created, the benefits are becoming harder to uncover, with a recent survey even going so far as to claim that data was at the tipping point of being more of a burden than a benefit, with employees finding their work lives becoming harder, not easier. The answer lies in an effective data strategy that can discern which data is valuable and then convert it into actionable insights to allow a business to make more informed, faster decisions. Companies with a data-driven strategy are widely believed to outperform their competitors, with Netflix, for example, claiming it saves $1bn per year on customer retention through successful data analysis. Though a significant number of companies had already embarked upon their digital transformation journeys beforehand, the global pandemic single-handedly forced the issue for the vast majority of those that hadn’t yet started. Although this was a positive in many ways, many of these journeys are still underway and already more than half of organisations believe that they have too much data to know what to do with, with the concern that many are not meeting compliance requirements because of their handling of it. While machine learning tools can help detect and discard anomaly data for many, it seems that a number of businesses feel like they are drowning.
One of the issues is unstructured data (rather than structured) which already makes up 80% of enterprise data and is growing year on year. While there are applications that can digest structured data, the analytics tools for disseminating unstructured data are still in their early stages, which means organisations are leaving a lot of data untouched and potentially going to waste. This has led to the creation of data lakes – storage facilities that can hold huge amounts of raw unstructured data as well as structured – as a secure, futureproof space that can be mined as and when the tools become available. As the technology progresses, accessing vast amounts of flat file unstructured data for trends and information will be incredibly valuable for the forward-thinking businesses that are adopting data lakes.
The last Dell Technologies Digital Transformation Index produced some fascinating findings that support this anecdotal, overwhelming production of data. The report warned that the overload of data was the third highest barrier to digital transformation, but that 71% of businesses were still collecting data faster than they could analyse it. And while 64% of businesses surveyed claimed to be data-driven in their approach, only 23% prioritised its use across their business. The Dell Index was a timely reminder about digital transformation best practice for everyone. A successful digital transformation needs its foundations in a data strategy, with goals in place for analysing data, leveraging the findings and using the insight effectively for decision making. If you are in the midst of a digital transformation, or need advice on how to take it to the next stage, contact us today.
