09-25, 11:20–11:55 (Europe/Amsterdam), Nebula
This talk dives into the challenge of measuring the causal impact of app installs on customer loyalty and value, a question at the heart of data-driven marketing. While randomized controlled trials are the gold standard, they’re rarely feasible in this context. Instead, we’ll explore how observational causal inference methods can be thoughtfully applied to estimate incremental value with careful consideration of confounding, selection, and measurement biases.
This session is designed for data scientists, marketing analysts, and applied researchers with a working knowledge of statistics and causal inference concepts. We’ll keep the tone practical and informative, focusing on real-world challenges and solutions rather than heavy mathematical derivations.
Attendees will learn:
* How to design robust observational studies for business impact
* Strategies for covariate selection and bias mitigation
* The use of multiple statistical and design-based causal inference approaches
* Methods for validating and refuting causal claims in the absence of true randomization
We’ll share actionable insights, code snippets, and a GitHub repository with example workflows so you can apply these techniques in your own organization. By the end of the talk, you’ll be equipped to design more transparent and credible causal studies-and make better decisions about where to invest your marketing dollars.
Requirements: A basic understanding of causal inference and Python is recommended. Materials and relevant links will be shared during the session
For most companies one of the top marketing priorities is to increase the Customer Loyalty towards their brand and products and therefore increase the lifetime value from their customers. One of the ways that companies try to achieve this is to onboard customers onto their app. While mobile App users tend to have higher engagement and retention rates, often generating more profits through direct purchases, the challenge lies in determining if this is due to app installs or if high value customers are more likely to install the app. The ideal way to measure the causal impact of an app install would be to set up a randomized controlled trial, but this is impractical as we cannot force users into treatment (install the app) and control (do not install) arm. Therefore, we leveraged observational causal inference methods to understand the incremental value of an app install, where we compared app users to non app users. The biggest challenge of using observational causal inference for business decision making is to demonstrate stronger evidence for causal conclusions and address biases from confounding, selection and measurement, which can result in an underestimate or overestimate of the effect of interest. Our work aims to address these concerns by exploring covariate selection strategies, different statistical approaches, refutation methods and design-based approaches. The estimated causal effects from this study helped the business to identify the most valuable install acquisition channels and optimize the app marketing spend to drive more incremental CLV. This work can be considered part of wider efforts to improve the transparency and robustness of observational causal inference studies by thoughtful application of multiple approaches (statistical and design based) each with their own strengths and weaknesses. Our next step is to implement a comprehensive sensitivity analysis to further strengthen our causal claims.