Huge bookshelves in Starfield Library, South Korea. Source: Shutterstock

The relevance of big data and the importance of real-time access

Since data has been established as the new oil, companies have been aggressively investing in ways to collect data from their own workflows and processes as well as along the value chain, right up to the end user or customer.

This ‘big data’ is phenomenal and helps organizations change their understanding of the world around them when compared to how they operated in the past.

For example, in the past, inventory was a function of sales forecasts which themselves were calculated based on last year’s sales and adjusted for rise or fall in demand. With big data, organizations can factor in other metrics that impact sales, making their forecast more reasonable.

So, there’s no doubt that collecting data is helpful. Neglecting to collect or procure data can be quite a disadvantage to most organizations, especially because their competitors have access to it and are constantly looking for creative ways to exploit it for growth.

Truth be told, as a result of technologies such as 5G, the internet of things (IoT), edge computing, and so on, companies using big data to their advantage are doing so by tapping into it and analyzing it in real-time.

In the retail industry, for example, smart shelves can help store managers ensure that customers always find the right items, in the right spot, and at the right time. But if consumer packaged goods (CPG) companies tap into this data, they can get more intelligent with their distribution to ensure deliveries are prioritized to stores where demand is high.

The next victories with big data will come from those using it in real time

Using big data in real time creates specific advantages – but the uses cases must be thought through.

In the case of the retail industry partnering with CPG players, while sensors on smart shelves collect plenty of data, CPG companies could create a smart workflow with edge computing architecture processing all data to make sure only relevant data is passed on to supply chain managers empowered to handle logistics on the ground.

Collecting big data has its advantages, but there’s a cost associated with storing it all (even if it is only archived). Further, if not used quickly, or in real-time, it might not be very useful at all.

CPG players collecting data from smart shelves in retail stores get the most out of it only when they process it in real-time.

If they process it later, it could help identify trends in demand that correlate with other factors such as season, affluency in a neighborhood, and so on, but is unlikely to have a direct and immediate impact on sales.

Further, not every bit of the data might be useful, so organizations that plan to continue collecting data for the foreseeable future need to come up with a plan to assess their data streams and understand which sets need to be saved for the future and which have a short lifespan and must be analyzed immediately and deleted.

Real-time big data analytics, in most cases, isn’t hard. It’s limited by the imagination of business leaders and has already provided outsized returns for those willing to go beyond their comfort zone.