Multi-Variate Landing Page Testing for Beginners
That title is a trick, I’m mostly a beginner myself when it comes to multivariate – it was basically beyond my reach before Google decided to eviscerate the proletariat and release Google Website Optimizer tool (GWO) – killing offer all competitors with another amazing free app. Love to hate those guys. I’d like this to be a follow up to my previous post where I tried to show the essentials of photoshop for beginners, specifically with respect to creating basic layouts for landing pages. It’s really intended for people who need to create (actually be the creative person behind) the landing page elements themselves, even if they can’t do the final design.
Via the process described in that post you can start throwing together different landing page layouts with photoshop. In this post I’d like to talk about how the different elements you’re creating on those pages can be swapped out with multi-variate testing software, and you can easily do more than just test one page against the other (that would be A/B testing), you can try combinations of different elements and work towards one, superior landing page.
So the meat of things here is the process I go through to define a multivariate plan:
Step 1 is to design the layout of the landing page. If you want to test multiple layouts, I would suggest designing two layouts with the same message, offers and calls to action, then doing A/B testing between the pages. Multivariate works best if it is kept as simple as possible, and one aspect of simplicity I like to employ is to try and test elements against one another, while controlling for the effect of the layout simply by keeping it static. You might already have a landing page that you want to improve. Either way, step 2 is the same, so let’s say you’ve got a page like this: Step 2 is to break the page up into different “elements” that might be worth testing. You have to use some intuition here, any localized thing is fair game: images, paragraphs of text, calls to action. The level of granularity that you go to is up to you, but be warned: The more elements you want to test on one page, the longer it will take to get any meaningful results. If you have loads of traffic (tens of thousands of people a day) then you’ll be able to test a lot of elements in a fair amount of time, but if you only get a few hundred people a day to the page you’re testing, it could take months before you get results. The lesson? Keep it simple stupid! GWO allows eight sections on a page to be tested. That’s not very much I hear you say, but noooo, each of those sections can have 127 different variations tested, with GWO maxing out at 10,000 combinations. What counts as a combination? Everything. Let’s do the math: if you have two headlines on a page, plus two images, plus two calls to action, and for each of these headlines, images and calls to action you want to test three variations, to calculate variations you multiply the number of variations per section by one another, three sections of three variations each is:
(3 x 3 x 3), that’s 27 combinations.
And with that lesson learned, let’s patently ignore it and cut a page up into loads of elements: The above step I do in tandem with the copywriter and designer, and I like to have the developer who is going to cut the page into HTML to be present also. The copywriter (slash marketer of the product, hopefully) is the one who will really know if a textual element is in need of testing. Some text elements are obvious test candidates, like headlines and calls to action, others not so obvious, like fine print or sidebar content etc. Sit down with the copywriter and determine which elements on the page deserve to be tested. Then send them off with the task of coming up with as many different versions of each textual element that they think is worth including in a test. A lot of copywriters love this, because while they’re used to having to make a judgement call between two pieces of text based on intuition of what may work better, they now have the option of writing each element a few ways and simply going with whatever works best. This can be really freeing for a writer. The same is true for the designer, who hopefully will welcome the idea of trying a few different images and learning from what works best. Most of the time we don’t spend a lot of time testing small graphical elements and instead concentrate on the larger or more obvious ones. This is the appropriate place to start, but if you’re further down the multivariate testing road, you might start to get a little more granular with the level of graphical element you test. For example, in the above image I place the “submit” button within the form “element'” if after you’ve tested for a while and are confident in some of the other elements on the page, you may wish to run a new test that makes the “submit” button an element unto itself, then you could test different text like “click here” or ‘order now’ or things like the color, size or placement of the button. The reason I like to have the HTML programmer in the room while doing all of this, is because they are going to have to cut the page up, and to do so properly for GWO, they have to understand something: each of the elements that we want to be a “swappable” thing in GWO, absolutely HAS TO BE a contiguous (uninterrupted) block of HTML code. In order to ensure this is the case, the HTML programmer has to know which sections of the page are designated as an “element”. Overall the process is simple (why did I even split it into steps if it’s only two steps? Don’t ask me, I just write here), all you need to start is a best guess at a landing page. From there, sit down with your designers and marketers, and be sure to include the HTML coder in the process at some point – they’ll have to add the GWO codes (or code for whichever multivariate software you’re using) to each element, so they need to understand the process as a whole. So no excuses, it’s simple, it’s free, and it will make you more money, by definition, so go get testing!