The Quandary of Measurement
I hate to say it. But there is NO perfect way to measure with 100% accuracy the impact of most marketing efforts and checkoff programs are no exception. Sure, it is a lot easier to measure the success of a marketing effort when you have hard retail sales data but our business by and large does not work that way, so we use the best measurement tools we have. We report on product awareness and shifts in attitudes. We measure whether consumers like, share and engage with our social content. And, we will use an econometric evaluation which the USDA finds to be the best tool for evaluating impact.
Most econometric evaluations work with several years of data to model the best estimate of advertising impact over time and, therefore, are reported several years after the start of a campaign. However, given the availability of monthly data vs annual data required for our model, we were able to begin building our model earlier than most checkoffs and put our first “toe in the water” on reporting campaign impact this spring.
A preliminary study of the Paper and Packaging Board’s (P+PB) advertising campaign suggests it helped generate in excess of 500,000 tons of consumption in 2016. Charged with stemming the decline in paper, the advertising campaign helped mitigate the loss of a minimum of 210,000 tons in paper consumption. The campaign is also charged with growing demand on the packaging side, where it helped create in excess of 290,000 tons of packaging consumption
Over their 50-year history, checkoffs, as well as the USDA, have found econometric modeling to be the most rigorous evaluation method for these programs. We’ve worked with Harry Kaiser, PhD, a preeminent, independent economist from Cornell and an expert on checkoff program evaluations, to create an econometric model that takes into consideration outside variables, without advertising, that impact demand for forest products. Then, the model looks at how the addition of advertising affects the outcome. The model the campaign is using is the result of 18 months of deliberation, informed by our companies and our Metrics Working Group. The models are “counterfactual,” which means they look at historical consumption data and try to account for the drivers behind that data including the absence of or addition of advertising.
Might there be other drivers of demand outside of what we are accounting for? Possibly, but we’ll continue to kick the tires on the model to make sure we have not left something truly important and macroscopic out; however, I can say, without a doubt, this is one of the best tools at our disposal. The econometric model allows us to isolate the other major drivers of consumption, so that we can report out the incremental impact of the advertising.
As many of you have heard me state over the last year, there are many kinds of metrics (or our equivalent of leading economic indicators) we look at to ensure the most robust measurement and accountability to our companies. It is not perfect but we are working to give the industry the most rigorous evaluation we can. Because that is what our companies deserve.
Here’s to two years under our belt and our first look at the potential impact of collective spending to maintain and grow forest product usage.