Datalad tool
The Peeking Simulator
Run a few hundred A/B tests where the variant does absolutely nothing, and watch how many you would have called a winner. Look once at the end and it is about one in twenty, exactly as promised. Check every day and stop when it goes green, and it is far more.
1
Set up the experiment
These are the settings of a single A/B test. The simulator then runs that same test many times over, with fresh random data each time.
Per cent
Split evenly
The planned length
Change it to reshuffle
[…] zero Choose how often you check, and how long it runs Fixed seed, so the run is reproducible Open the simulator Free · runs in your browser · no […]
[…] length Change how often you sample the running result Fixed random seed, so a run is reproducible Open the simulator Free · runs in your browser · no […]
[…] Sample the statistic once, or repeatedly as it runs Fixed seed, so you can rerun the same draw Open the simulator Free · runs in your browser · no […]