Dark data: Organizations are collecting a lot of data, but are they using it? Source: Shutterstock

Dark data: Organizations are collecting a lot of data, but are they using it? Source: Shutterstock

Do businesses really make use of all the data they collect?

THERE’s a lot of data in your business. No need for a spoiler alert, you already knew that part.

The problem is, a huge amount of the information locked up inside any given organization is never managed or acted upon… and this is due to a number of reasons. 

We could call this information ‘dark’ data. 

It’s information that is lost because it is too granular and minute (log file data would be a good example), or because it is essentially unstructured (voice and video files along with email are always a good example). 

Or it could be because there is no management tool in place to capture and categorize the data itself… so all that information risks ending up being disconnected and therefore having no value.

The information age, right?

If this is the information age and some of our workplace systems are creating data-centric information streams that we are failing to capture, then surely we should step back and ask why we are performing those work actions in the first place, right?

According to Stewart Bond, Research Director for data integration and integrity software and Chandana Gopal, research director for business analytics solutions from IDC, “[The] vast majority of data that is generated today is lost. In fact, only about 2.5 percent of all data is actually analyzed.

“The biggest challenge to unlocking the potential that is hidden within data is that it is complicated, siloed and distributed. To be effective, decision-makers need to have access to the right data at the right time and with context.”

The question is then, how do we counter the issue of dark data and disconnected information? We have three possible answers: 

  • DataOps 
  • No-code software platforms
  • Collaboration tools

# 1 | DataOps

DataOps is a collaborative data management practice designed to help users realize the full potential of the data they create at every level. 

DataOps involves the creation of a central data hub, repository, and management area to gather, organize and then distribute data so that data analytics can be more widely democratized across an entire organization.

Using DataOps, we can ensure that data is stored at the lowest possible cost at the right service level for the business case it relates to. We can also use DataOps to make sure that our data is searchable, accessible and appropriately governed. In this way, we can move towards more actionable insights and the full economic value of the data is captured.

# 2 | No-code software platforms

A further step towards triaging the hemorrhaging state of enterprise data comes in the shape of no-code software platforms. 

Simpler than low-code platforms (which exist as template-based programmer shortcuts and still require software engineering expertise), no-code software platforms can be used by management staff to create applications through nothing more than drag-and-drop controls.

This is a key route for information capture because ‘non-techie’ businesspeople can use no-code to build forms-based applications designed to question users, customers, prospects (everybody basically) and start capturing information that would have previously remained untapped.

# 3 | Collaboration tools

Almost everybody uses some form of software-based collaboration tool already. This is of course because almost everybody uses email, text messages or some combination of the two.

Many go further and use Skype, Slack, Microsoft Teams, Google Hangouts, Trello, Flock or one of the other popular solutions offered in this space. 

Across the spectrum of collaboration platforms currently vying for widespread user adoption, there are some defining differences. 

Some present themselves as highly-collaborative discussion-based tools with ‘permanent chat’ and options for both text and video meetings. Some present themselves as more project management-centered, with options for multiple users to work on documents together.

Others still aim to provide particularly closely-managed user support and maintenance options, or extensions to mobile usage. So while it’s still a broad church in the collaboration tools space, the key point is that all of these products do one thing, which is to contain, collate, and corral our data… and that all helps to counter the dark data that we know is driving the information age at a lower rate of cadence than it could.

No cure-all for data use

There are limits to this discussion i.e. even taken as a triumvirate of innovation, no combination of DataOps, no-code and collaboration can immediately start to capture all an organization’s dark and hemorrhaged data. 

But let’s be more positive, if the analysts at IDC are right and only about 2.5 percent of all corporate data is actually analyzed, then surely we can do better and push that percentage into double figures and beyond.


This article was previously featured in our sister publication TechHQ.