JWDT/Daily/2 — Introduction to A/B Testing and Google Optimize.

  1. Only change one thing at a time. If you change multiple things, it’s difficult to tell which of them actually helped (or hindered) your goal. You can, however, try a completely different page, but then do further testing to find out which parts of it were actually helping. This can be good, for example, in testing whether mobile optimised, or simplified landing pages, help.
  2. Don’t stress over small data sets. If only 10 people visit your site whilst the testing is going on, the results are probably not worth much. In most cases that’s too small a sample set to mean anything. A good benchmark is probably around 500–1000 visits for a small website, or a few months if you get about 10 visits/day.
  3. Record everything. Google Optimize seems to do a pretty good job of this, it asks you to write down your goals for the testing, what you have changed, and what you expect. Make sure you know what the difference is that you’re testing, and record the results. They will be handy when you compare the results with other tests.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store