Datalad tool

Concurrent Test Collision Checker

Running several experiments at once is normally fine, and the reasons it is fine are not the ones people give. This works out which of your tests actually collide, how many users land in both, and which of the overlaps genuinely threaten the result rather than merely looking untidy.

1

What is running

Per cent relative
2

Every pair

Two tests only interfere if they overlap on all three of time, audience and surface. Overlapping on traffic alone is not interference, it is just two tests running.

3

What to do about it

View Comments (1)

Leave a Reply

Subscribe to My Newsletter

Subscribe to my email newsletter to get the latest posts delivered right to your email. Pure inspiration, zero spam.

Discover more from Discuss Data Science, Machine Learning and Analytics

Subscribe now to keep reading and get access to the full archive.

Continue reading