Cheat sheet · Oferta no WhatsApp
Measuring the impact
The return rate alone is misleading. What counts is how much extra the message brought in beyond what would have come anyway, in customers and in money.
Two questions, in this order
| Question | Formula |
|---|---|
| 1 · Extra return | rate among those who got it − control rate |
| 2 · Extra revenue | (revenue per customer among those who got it − control's) × customers who got it |
| Profit | extra revenue × margin − cost of the messages |
Example
| With message | Control | |
|---|---|---|
| Customers | 1,000 | 200 |
| Came back within 15 days | 110 (11%) | 20 (10%) |
| Revenue per customer | R$ 9.90 | R$ 8.50 |
Extra return: 1 point, about 10 customers, not 110. Extra revenue: R$ 1.40 × 1,000 = R$ 1,400. At these sizes, 1 point could still be chance.
Rules
- Draw the control group at random before sending: ~10%, at least 30 to 50 customers.
- Same window: 15 days from sending, for both groups.
- Count every Goomer order in the window, with or without the coupon, at the amount paid.
- Small difference with a small group: repeat and add up the results before drawing conclusions.
- No control group? Compare with similar customers over a period with no sends. That's an estimate.
Sources: Sahni, Zou & Chintagunta, 2017 (≈90% of the effect outside the coupon); Anderson & Simester, 2004; Kohavi, Tang & Xu, 2020. Lesson: 5 · What the message added. Tool: the calculator in lesson 5.
Training