TransWikia.com

How can an A/B test show significant result without enough data

Cross Validated Asked by Jonas Palačionis on January 11, 2021

I am running an A/B test and to check if I have enough data I use this calculator which tells me that it does not have the required sample size per variant, but does tell me that variant 1 is significantly worse with p = 0.0387.

I assume that you can’t tell if A or B is significantly better or worse if you did not reach the required sample size per variant. Isn’t these two metrics linked to each other?

enter image description here

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP