Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Hi

Have you looked into these two?

- Trustworthy Online Controlled Experiments by Kohavi, Tang, and Xu

- Statistical Methods in Online A/B Testing by Georgi Georgiev

Recommended by stats stackexchange (https://stats.stackexchange.com/questions/546617/how-can-i-l...)

There's a bunch of other books/courses/videos on o'reilly.

Another potential way to approach this learning goal is to look at Evan's tools (https://www.evanmiller.org/ab-testing/) and go into each one and then look at the JS code for running the tools online.

See if you can go through and comment/write out your thoughts on why it's written that way. of course, you'll have to know some JS for that, but it might be helpful to go through a file like (https://www.evanmiller.org/ab-testing/sample-size.js) and figure out what math is being done.



PS - if you are looking for more of the academic side (cutting edge, much harder statistics), you can start to look at recent work people are doing with A/B tests like this paper -> https://arxiv.org/abs/2002.05670


Even more!

Have you seen this video - https://www.nber.org/lecture/2024-methods-lecture-susan-athe...

Might be interesting to you.


I’ll second Trustworthy Online Controlled Experiments. Fantastic read and Ron Kohavi is worth a follow on LinkedIn as he’s quite active there and usually sharing some interesting insights (or politely pointing out poor practices).


speaking of Georgi Georgiev, I can’t recommend enough his AB testing tools at https://www.analytics-toolkit.com

being able to tell when an experiment has entered the Zone of Futility has been super valuable.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: