Are you really sure about your A/B tests?

Check if your conversion has really changed!

Lots of services out there promise you to increase your store’s conversion rate.

If you observe an increase or decrease in your conversion, it does not automatically means that your conversion really changed.

It is crucial that you keep in mind that you need a lot of traffic to be sure that your conversion really changed, especially if your conversion is low.

We made this tool to make sure that your A/B tests have a real impact on your conversion rate.

Before change

10 %

After change

10 %

(p = )
Powered by PricingBot


Who are you?

We are the team behind PricingBot. This competitor monitoring software allows you to monitor your competitors' prices. It allows you to understand your competitor's pricing strategy and adapt yours.

If I read that there is no significant difference, should I rollback my change?

Not necessarily, what it means is that you don't have enough data to make a decision. If you observe a 1 point increase in conversion over your next 100 customers you can't be sure of anything. On the contrary, if you observe a 2 points increase in conversion over your nest 1400 customers, you are now able to make a decision. This tool is here to help you make them.

I don't get it, my conversion went from 1% to 2%, it nearly doubled and your tool tells me that there is no significant difference?

Yes, we know it can be a little bit confusing. Let us put is this way, our tool only tells you that you can't be absolutely sure that your conversion actually increased. It means that you need to do more sales, with the same conversion rate, to actually be sure that there is a change.

How much should I wait before results become significant ?

This is a tricky question and this is exactly why we made this tool. How much should you wait before being sure that you observe a significant difference depends on two things. First it depends on the difference between your two conversions, the greater the gap, the faster you can be sure that there is a gap. Secondly it depends on the number of visitors you used to compute your conversions.

Can I read more about this somewhere ?

Sure, this tool is actually a simpler version of this one. It leverages statistics and probabilty to ensure you that you make the right decisions. If you really like mathematics, we suggest you read this wiki article and this one.