Is your ecommerce site equipped for big data?

Big data today is practically synonymous with ecommerce. Large companies like Amazon, eBay, Disney and Netflix are continuously expanding the amount of data they are able to capture and capitalize on new ways to track, analyze and use that data to improve customer satisfaction and to increase revenue. Of course, storing and analyzing ever-increasing volumes of data from more and more sources doesn’t come without some pain points. If your company’s growing pains are overshadowing the benefits of using big data, it may be time to consider making some changes to your platform.

Big data — Three types

All of the types of data that are accessible to an ecommerce business can be categorized into three types: transactional data from your own website, click-stream data from other sources on the internet as and unstructured data that comes from online video streaming and voice calls. 

Transaction data

Transaction data comes from exchanges between the customer and the ecommerce company. Traditionally, this data is tracked through the website itself, using cookies on the customers’ web browsers, so the data can be tracked and accessed whenever the customer returns to the website to personalize their experience.

However, this data can also come from software, like customer relationship programs, which track things like support calls or complaints, as well as data that comes from other websites or platforms. It’s not unheard of for this to involve billions of data points from millions of customers from hundreds of online stores and thousands of SKUs.

Click-stream data

This data comes from sources like online advertisements, social media platforms like Facebook and Twitter, blogs and comments. This data is used to predict customers’ interests and tastes, not only in what products they may be interested in, but which advertisements to serve them while they are online. Netflix, for example, gathers and analyzes over 1 billion data points from movie reviews and movies that are “liked” on social media to provide recommendations to its customers.

Video and voice data

Using image analysis software, big ecommerce companies can collect the unstructured live data that comes from video. Netflix, for example, uses video data to determine customer experiences, preferences, and to predict viewing habits. A range of large companies use data from customer service calls, not just to monitor likes and dislikes, but even to determine preferences based on language patterns, choice of words and use of slang.

Today’s pain points in big data management

As our access to data and the amount of data available both increase, new levels of complexity have come into play. More data from different sources have increased options but have also introduced a new level of complexity into the equation.

Data quality

Data needs to be accurate, timely and standardized before it can be used for analysis. When data is coming from multiple sources, this can be challenging. The question becomes how it is best to store this data for retrieval.

Storing data

Until recently this was often done with data warehousing, but a growing number of the top ecommerce businesses are no longer relying on this model alone and are augmenting it with data lakes. Data lakes are able to store unstructured data from a variety of different sources, like mobile apps, social media, IoT devices and proprietary automated analytics applications, where it can be accessed as and how needed, for analysis or for use by AI.