What won't your churn model tell you?

How can you effectively address churn?

• 🛠️ Improve your product constantly
• 🤝 Enhance your Customer Success Management (CSM)
• 🎉 Get better at customer onboarding to make the first experiences count
• 💡 Focus on elevating the overall customer experience
• 📊 Leverage analytics
  ‍

Obviously, I will focus on the last one! One of the approaches is leveraging historical data about your customers and, by building a model, transforming them into valuable signal which you can use to estimate the probability that your customer will leave you. Having that, you can engage with these customers to address the issues they may be facing and change their decision. You can re-run your model and refresh churn probabilities for your customer base on a regular basis (e.g. monthly).

Seems like we leveraged analytics to address our business problem! Cool! But there are two caveats to be considered when productionizing it. Let's explore them:

Alignment Problem

There's a misalignment between the goal of the model and our business goal. The model aims to minimize the error in predicting who will churn, while our goal is to maximize profit - in this context, by reducing churn with minimal effort. So, is information on all correctly predicted churners useful to us? Yes, but only to some extent. 🤔

Let’s analyze scenarios of churn when targeted (e.g., by marketing) and not:

• Do Not Disturb: Customers who will churn if targeted and will not if left alone. When identified as potential churners, targeting them with marketing activities can lead them to resign. 🚫
• Lost Causes: Customers who will churn regardless of our actions. They have a high probability of being identified as potential churners, but any marketing action is a waste of effort. 💔
• Sure Things: Customers who will not churn, targeted or not. They have a low probability of being marked as churn, but any marketing activity is a waste since they would stay anyway. ✅
• Persuadables: Customers who will not churn when targeted and will churn when not. They are the only group for which an optimal strategy involves action, as acting on them can yield real ROI. 🎲

‍

‍

The problem is our churn model does not predict them directly.

Feedback Loop

Another point of contention with churn models is their susceptibility to feedback loops, especially when models are periodically updated. This can lead to two problematic scenarios:

• Scenario 1: Models may incorrectly learn to focus on "Lost Causes", neglecting customers whose churn decisions are more nuanced and potentially reversible. 🔁
• Scenario 2: Customers who did not churn due to successful intervention are mislabeled as naturally loyal, skewing future predictions and potentially overlooking similar "Persuadables". 🔄

Despite these challenges, churn models remain invaluable tools in our retention strategy toolkit. Identifying and addressing these issues could enhance their performance. Potential improvement? Better distinguishing "Persuadables". How do we identify them? 🎯

Instead of focusing on making the prediction of who will churn most accurate, focus on modeling the treatment effect with greater accuracy. Why? 🤔

‍

You will be able to identify customers for which your actions will have greater impact, which allows you to optimize your efforts and maximize ROI. 💡

What do we need to do this:

• Data about our users: similar to what's needed for churn model
• Target metric: information whether given user churned, this is also analogue to regular churn model
• Treatment data: promotions, emails, calls, essentially any flag indicating some sort of action you made to specific customer. This data should be collected in A/B test manner - on two randomly selected groups where in one of them all customers are “treated” and second one where none of the customer gets the treatment.

Now, when you have the necessary data, you can start modeling. There are many approaches to uplift modeling; in this example I will walk you through the most intuitive one, called T-Learner, which stands for “two learners”.

How does it work?

We train two models:

• One on treated users
• The other on not treated users

Then, we use both models to infer on future users. The delta between predicted metrics for them is the predicted uplift. Users with the highest predicted uplift are the ones we should approach.

In our example, those would be users for whom a given treatment is predicted to have the biggest effect on churn probability → Persuadables.

Let’s wrap up

• Unlike churn models that predict who will leave, uplift models predict the impact of our actions. Will offering a discount or a new feature make our at-risk subscribers stay? Uplift modeling gives us that insight. 🔍
• Implementing uplift modeling means we can focus our efforts more effectively. Instead of blanket promotions, we target subscribers who are most influenced by certain actions. 🎯
• Uplift models have more use cases, especially in e-commerce, you can model almost any business KPI (revenues, conversion) versus any action and the potential impact it can have for a given customer on that metric. 🚀

More resources

[1] https://www.priceintelligently.com/hubfs/Price-Intelligently-SaaS-Pricing-Strategy.pdf

[2] https://github.com/uber/causalml

[3] Devriendt, F., Berrevoets, J., & Verbeke, W. (2019). Why you should stop predicting customer churn and start using uplift models.

‍