In the last post, we discussed the importance of counting lift, rather than total revenue, when calculating ROI of a marketing campaign. Using total revenue will wildly inflate any ROI measurement. The advantage is that it is easy to calculate — we can see exactly who redeemed a specific offer, sum up everyone’s revenue on the redemption day, de-layer that revenue total if necessary, and just like that, we have a revenue total. On the other hand, pinning down lift, which is essentially an imaginary number — what would revenue have been if we didn’t run that promotion? — can often be difficult.
One excellent way, that most marketers are very familiar with, is to run a control group within each campaign. In the last post, we used a coffee maker giveaway as an example. 2000 players redeemed and generated $120,000 in total revenue. Let’s assume 15,000 players were qualified to receive this offer, and we held out 10%, or 1,500, as our control group. So 2,000 out of 13,500 redeemed, or 14.8%. What if we also found out that 180 members of our control group also visited that day — that’s a 12% “visit” rate without this incentive. In the example, we also stated that the 2,000 redeemers lost $120,000 in total ($60 average). Now let’s also assume that our 180 visitors from the control group lost $12,600 — that’s a $70 average. So the offer group had a higher redemption rate, but the control group had a higher average loss. If we extrapolate our 1,500 player control group to match the size of the offer group (multiply by 9), they would have lost $113,400. Now the lift generated by the promotion is pretty small — just $6,600. Now if we subtract the cost of the coffee makers (plus printing & postage, labor, miscellaneous costs), there’s a good chance that we’re actually underwater on this promotion.
Having a control group is terrific, but it’s a luxury we’re not always granted. What to do without one? One option is to review the visitation pattern of the invited group over the past few weeks. We can check how many visits were made by the 15,000 invitees and how much revenue they tended to generate. If we find out that over the past 60 days (x 15,000 players = 900,000 possible visits), this group of players made 99,000 total visits and lost $66 per visit on average, then it’s easy to calculate a typical visitation rate of 11% (99,000 / 900,000). More importantly, we can see that on a random day, the invite group generates $7.26 revenue per person (99,000 * 66 / 900,000), versus $8.00 (120,000 / 15,000) on the promotion day. This would imply that the lift from the promotion is $0.74 * 15,000 or $11,100.
Another statistic we can leverage is the non-redemption rate. In the example promotion, 2,000 out of 15,000 invitees redeemed. What if we also found out that 700 of the 15,000 visited and played on the giveaway day, but did not redeem the offer? This is a very useful piece of information, because it’s proof that some of those redemption trips would have occurred without the coffee maker incentive. Knowing this, we have to acknowledge that a chunk of the redeemers picked up a free gift while they were already here, but the decision to visit was not caused by the promotion. Exactly how many, we don’t know for sure. However, knowing that lots of players who were on-property, and eligible to redeem an offer didn’t actually do so, indicates that the gift or promotion wasn’t all that enticing, and therefore it is essential to greatly discount the revenue total, to account for redeemers whose trip was not incented by the promotion.
Finally, the most generic and simplest way of all — compare the day’s total revenue with the same weekday over the last 4 to 12 weeks. If the coffee maker promotion was on a Tuesday, and that specific Tuesday had 5% higher revenue than the average over the last 8 Tuesdays, then it’s fair to infer that 5% is your lift. Of course, you have to use common sense here, if you did another impactful promotion on the same day, if you had one “whale” who was personally responsible for a large portion of that lift on the promotion day, if weather affected some of the comparison days, etc., then obviously you need to make adjustments.
Whichever way you go, however, the key is to measure or estimate lift rather than total revenue attributable to the promotion. Marketers may not like it because it will certainly lower (by a lot) your ROI calculations, but any other way is simply fooling yourself, your marketers, and your executive team about the impact of your promotions.
