9 types of Consumer Product data to consider for predictive analytics

Internal Data: The first set of data is the internal data. All financial information (prices, discounts, rebates, COGS, sales incentives, shipment volumes), marketing and promotional calendars (with all information that is related to mechanics, duration, depth, distribution) and logistics data (inventory, lead time, product availability) which has the biggest impact downstream. All internal data is the foundation of effective analysis.

Consumption Volume: The second group of data is syndicated data reflecting POS consumption. Most commonly, this includes consumption volume, shelf prices and distribution. It can be purchased from companies such as IRI, Nielsen, Retail Link and others and typically provides Point of Sale data by week and at the product level.

Because everyone cannot afford consumption data and you really need both internal and consumption data at a bare minimum to start talking about optimization, deriving consumption data from shipment data is an option. But this can work only for very specific subcategories (e.g., highly perishable products) with a direct to delivery distribution model. Although you can leverage shipment data to optimize, most of the time, shipment is not enough to run even an entry level project for price and promotional effectiveness.

With internal and consumption data, you will have a solid starting point to run predictive analysis for promotions and pricing so that you can start identifying places to recover margins or to improve effectiveness.

Category Data: You are not alone in the market. While it is expensive and a lot to ingest, you can also purchase category data on your competitors. This is especially important if you are brand number two. In fact, you may have to have this data to understand brand loyalty and its impact to your promotions. Competitor promotional calendars and price points are important to understand.

Retail Execution Data: Another precious source of knowledge comes from retail execution data which is collected by retail execution companies or brokers in-store by retail execution teams. In this case, store checks are conducted and include pictures of shelves, price tags, position and promotions to track inventory and whether or not promotions were on deal. Better than syndicated data provided monthly, retail execution data can be provided weekly or with greater frequency. This knowledge will provide additional KPIs like out of stocks, number of facing and more accurate information about the competition.Category and Retail Execution Data can provide two additional layers of data and benefits. First, it will help you with prescriptive optimization so you can decide what the best promotions are to run in terms of depth and frequency. For example, running a 2 for 5 vs. a 2 for 6. This way you don’t have to spend more money on trade than necessary. And second, you can be a better partner to your retailer as a credible advisor in your joint business plan with category data.‌

Consumer Data: Another data source is related to consumer data which includes focus groups, sampling, surveys and more as additional ways to collect data. Often used for one shot analysis, this data once incorporated in models, improves the level of accuracy of some parameters or KPIs as cannibalizations, incrementality and price sensitivity.

Weather Data: A causal factor in your model, weather data is another element which can bring more accuracy both in the evaluation of the past and future forecasts. For example, people tend to buy non-perishables in the event of a hurricane because you don’t need electricity to keep these products fresh. But the more granularity that weather data provides is really only relevant in specific categories. In most cases, with the understanding of seasonality you already know enough to understand your base volume.

Shopper Data: Another expensive form of data that provides a level of precision and detail that includes KPIs is shopper data. Retailers with shopper card programs can easily obtain valuable consumer information and in some cases, this data is sold to manufacturers providing buying patterns plus demographic information which is valuable in North America. Direct shopper data is available to companies that have a significant Direct to Consumer channel.

Web Data: One additional group of data is categorized by its digital rather than physical source. Web data is all data which can be data scraped from the web, enabling additional use cases that are aligned with information around consumer sentiments or competitor initiatives. Embedded in the forecast information for next month’s competitor promotions, you’ll find powerful data if you can assume that the behavior will be the same as the previous year. The more important warning here is that managing this source of information (big data, crawling tools and harmonization) will increase the complexity of the architectural landscape.

Macroeconomic Data: Finally, because of its volatile nature due to events like inflation, macroeconomic data or variables, may only sometimes be included in forecasting models.