9 types of Consumer Product data to consider for predictive analytics
It’s no surprise that the benefits resulting from digital transformation programs have been demonstrated by many companies in the last few years. Companies that are more “digitally-enabled” have systematically performed above average when compared to their competitors. They’ve been able to take advantage of new optimization capabilities to drive positive results. Digital transformation and optimization are the key components of RGM and analytics are the key result.
With the understanding that analytics are critical, many companies still struggle to obtain data, keep clean data and consolidate it from multiple sources. Data analytics is at the top of the agenda for many companies, but most have encountered significant challenges in moving forward at the right pace or in achieving their expectations. Data is repeatedly pointed to as one of the most difficult areas to master.
What’s good for the goose is good for the gander, or is it?
Which data we are talking about? Does all data have the same importance? You need to determine what data is needed to perform analytics. In other words, what should be analyzed.
Internal Data: The first set of data is the internal data. All financial information (prices, discounts, rebates, COGS, sales incentives, shipment volumes), marketing and promotional calendars (with all information that is related to mechanics, duration, depth, distribution) and logistics data (inventory, lead time, product availability) which has the biggest impact downstream. All internal data is the foundation of effective analysis.
Consumption Volume: The second group of data is syndicated data reflecting POS consumption. Most commonly, this includes consumption volume, shelf prices and distribution. It can be purchased from companies such as IRI, Nielsen, Retail Link and others and typically provides Point of Sale data by week and at the product level.
Because everyone cannot afford consumption data and you really need both internal and consumption data at a bare minimum to start talking about optimization, deriving consumption data from shipment data is an option. But this can work only for very specific subcategories (e.g., highly perishable products) with a direct to delivery distribution model. Although you can leverage shipment data to optimize, most of the time, shipment is not enough to run even an entry level project for price and promotional effectiveness.
With internal and consumption data, you will have a solid starting point to run predictive analysis for promotions and pricing so that you can start identifying places to recover margins or to improve effectiveness.
Category Data: You are not alone in the market. While it is expensive and a lot to ingest, you can also purchase category data on your competitors. This is especially important if you are brand number two. In fact, you may have to have this data to understand brand loyalty and its impact to your promotions. Competitor promotional calendars and price points are important to understand.
Retail Execution Data: Another precious source of knowledge comes from retail execution data which is collected by retail execution companies or brokers in-store by retail execution teams. In this case, store checks are conducted and include pictures of shelves, price tags, position and promotions to track inventory and whether or not promotions were on deal. Better than syndicated data provided monthly, retail execution data can be provided weekly or with greater frequency. This knowledge will provide additional KPIs like out of stocks, number of facing and more accurate information about the competition.Category and Retail Execution Data can provide two additional layers of data and benefits. First, it will help you with prescriptive optimization so you can decide what the best promotions are to run in terms of depth and frequency. For example, running a 2 for 5 vs. a 2 for 6. This way you don’t have to spend more money on trade than necessary. And second, you can be a better partner to your retailer as a credible advisor in your joint business plan with category data.
Consumer Data: Another data source is related to consumer data which includes focus groups, sampling, surveys and more as additional ways to collect data. Often used for one shot analysis, this data once incorporated in models, improves the level of accuracy of some parameters or KPIs as cannibalizations, incrementality and price sensitivity.
Weather Data: A causal factor in your model, weather data is another element which can bring more accuracy both in the evaluation of the past and future forecasts. For example, people tend to buy non-perishables in the event of a hurricane because you don’t need electricity to keep these products fresh. But the more granularity that weather data provides is really only relevant in specific categories. In most cases, with the understanding of seasonality you already know enough to understand your base volume.
Shopper Data: Another expensive form of data that provides a level of precision and detail that includes KPIs is shopper data. Retailers with shopper card programs can easily obtain valuable consumer information and in some cases, this data is sold to manufacturers providing buying patterns plus demographic information which is valuable in North America. Direct shopper data is available to companies that have a significant Direct to Consumer channel.
Web Data: One additional group of data is categorized by its digital rather than physical source. Web data is all data which can be data scraped from the web, enabling additional use cases that are aligned with information around consumer sentiments or competitor initiatives. Embedded in the forecast information for next month’s competitor promotions, you’ll find powerful data if you can assume that the behavior will be the same as the previous year. The more important warning here is that managing this source of information (big data, crawling tools and harmonization) will increase the complexity of the architectural landscape.
Macroeconomic Data: Finally, because of its volatile nature due to events like inflation, macroeconomic data or variables, may only sometimes be included in forecasting models.
A gaggle can get quite noisy so how do you bring all together?
With 9 types of data, there’s certainly not a lack of data sources available. The challenge is knowing what type you need, ingesting it and most importantly, using it to optimize for RGM. How specifically should you handle this wealth of information and how can you obtain actionable insight from it.
It is also true that the situation is very different depending on your region. Access to syndicated data, level of collaboration in data sharing between manufacturers and retailers, and the cost of data are all elements that can be very different across regions. But more and more, the constraints related to availability is replaced by the challenge on how to put it into an end-to-end data strategy with the ability to:
- Put infrastructure and architecture landscape in place
- Prioritize business goals and related domains
- Clarify accountability and responsibility defined across the entire company
- Acquire the skills and talent development needed
We’ll go deeper on these four points in a follow-up blog.
Remember that any journey requires several intermediate steps. And each of these steps can bring additional value to your organization’s RGM success.