JWDT/Daily/2 — Introduction to A/B Testing and Google Optimize.
Just a quick one today, as I’ve only recently started using Google Optimize.
If you’ve never heard of A/B Testing, the basic concept is this:
You have 2 versions of a web page, with a small difference. It might be a different word choice, or a different coloured button. Maybe you’re testing out a new shiny plugin or something. In any case, there’s two versions of the page:
A — The original version.
B — The changed version.
When someone visits your website, they either get A or B. Let’s say it’s an even split, half get A, half get B.
Whilst they’re using your website, they will either do the thing you want them to do — such as buy your product, or book a call with you, whatever the goal of the website is — or they won’t.
If, during the testing, 70% of your sales come from the people who got version B, and only 30% from A, then you know that whatever you changed has increased your sales.
In other words:
A got 50% of the traffic and 30% of the sales.
B got 50% of the traffic and 70% of the sales.
That data suggests if you stopped doing the A/B testing, and only used version B, you would get an increase in sales (remember, A in this example was the original page before testing started).
Installing Optimize was as simple as editing my Google Analytics code, and adding a line in provided when I set up the tests.
What I found most impressive about it was how easy it was to use.
With the Chrome plugin I was able to edit elements on my website, save the changes, and then add that as a test page. Google then (presumably, not looked into how it works) runs some JS, imported through the analytics loader, to make the changes to the page’s HTML/CSS.
A few important points to remember are:
- Only change one thing at a time. If you change multiple things, it’s difficult to tell which of them actually helped (or hindered) your goal. You can, however, try a completely different page, but then do further testing to find out which parts of it were actually helping. This can be good, for example, in testing whether mobile optimised, or simplified landing pages, help.
- Don’t stress over small data sets. If only 10 people visit your site whilst the testing is going on, the results are probably not worth much. In most cases that’s too small a sample set to mean anything. A good benchmark is probably around 500–1000 visits for a small website, or a few months if you get about 10 visits/day.
- Record everything. Google Optimize seems to do a pretty good job of this, it asks you to write down your goals for the testing, what you have changed, and what you expect. Make sure you know what the difference is that you’re testing, and record the results. They will be handy when you compare the results with other tests.