If you’re testing a horse against a car, you don’t need an A/B test.
The single biggest source of value from data often comes from A/B tests. However, if an A/B test works well you don't need statistics to tell if it worked. Therefore what's the point in analysing data!? If all the analysis gets you is incremental change, is that good enough? Can we not do better than that?
Maybe data tells you why things have gone wrong? This is the field of causal analysis. Or maybe data can give us idea about what to try in the first place? This is the role of nascent generative models like GPT-3 and DALLE.
An interesting, if slightly rambling start from Taylor.