1046: Weighing a Feather: How to Design A/B Tests That Work, with Billur Engin
In Episode #1046, Billur Engin (who leads data science and analytics at Prolific) joins Jon Krohn to explain how to design experiments that actually teach you something. Too many A/B tests, she says, are like bathroom scales trying to weigh a feather the effect is real, but the system was never built to detect it. In her first-ever podcast appearance, Billur walks through why complexity has to earn its place in your models, the biggest myth about A/B testing, the AA test she runs to catch broken measurement, three ways to get at causality when you can’t run a controlled experiment, and why causal inference matters more than ever now that prediction has become cheap. Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn: (01:18) What Prolific does and why representative data matters (05:19) Why complexity has to earn its place (07:29) The biggest myth about A/B testing (11:34) Why causal inference matters more than ever (14:39) Getting at causality when you can’t run an experiment
You May Also Like

Ungovernable Misfits
UNGOVERNABLE

TechCrunch Startup News
TechCrunch

Rodrigo Arnedo Noticias BTC
Rodrigo Arnedo Noticias BTC

First Ring Daily
Paul Thurrott and Brad Sams

Mark Tilbury
Mark Tilbury Fan

AI Unraveled: Latest AI News & Trends, ChatGPT, Gemini, DeepSeek, Gen AI, LLMs, Agents, Ethics, Bias
Etienne Noumen

Engadget News + Next
Engadget

Tech Update | BNR
BNR Nieuwsradio

Jak Piggott
Jak Piggott

ADSP: Algorithms + Data Structures = Programs
Conor Hoekstra, Bryce Adelstein Lelbach & Ben Deane
Community discussion