Multivariate Testing – The new way in Search – Or ancient technique?
When I started research for my blog post I was planning to write about women in SEO. But then I thought “Do I really want to re-chew the fact that there aren’t enough females in search? And that you’ve got a better chance to be chatted up at a search conference than actually taken seriously.” No not really. So my next thought was why not talk about multivariate and A/B testing! Well since I was a little girl I loved anything to do with numbers, math was my favourite at school. Plus I loved being right, so learning tools and techniques which would let me test an idea and allow me to prove that I was right – has always been up my street.
Although people nowadays think that A/B and multivariate testing were born out of the genius of some SEO person’s mind – I am sorry to disappoint, but they have been around longer than most of us lovely people. But what is multivariate testing? What does it do? And what to look out for?
Multivariate where it comes from
Multivariate testing has its roots in statistics. The name gives away what multivariate testing is used for. It allows you to simultaneously observe and analyse more than one outcome variable. Before it entered the world of search – it made quite a big fuss in the world of Quality management. This is particularly around the topic of DOE (Design of Experiment) in SixSigma, where quality managers would try to systematically determine relationships between different factors affecting a process and/or the outcome of the process. Why would they do that? Well, to “optimise” (little SEO wink) the inputs with the aim of optimising the output. Somehow I suddenly feel that SEO might have pinched some of their terminology from statistics, did you ever hear about canonical correlation (another multivariate technique – looking at how similar 2 things are)? Hmmm… sounds very much like canonicalization to me!
Multivariate in Search
Unless you have been living on the moon for the last few years, you will have realised that multivariate testing and A/B testing have nested themselves into the world of SEO. It seemed like overnight all SEO agencies updated their websites to ensure they offered CRO (conversion rate optimisation). Nevertheless, it seems here to stay and is slowly changing the face of SEO. Imagine going into a client meeting and making a recommendation to implement something on the website. Back in the old days you would sit there hoping that it makes a positive impact – over an undisclosed period of time (3 to 6 months for example). Now having tools like multivariate testing you can “prove” that it will make a positive impact (or negative – I’d recommend to move on to the next idea if so).
Not too long ago I read an article on how companies are not interested in “big data “anymore. They want “big ideas” which can be proven to work. Well, what they really mean is big ideas that will turn into big revenue. Clients are changing, they are becoming a little more savvy and the first thing you hear in a meeting is not “I want to be position 1 for this keyword” anymore, it is more along the lines of “how much return on investment can I expect?”
However, multivariate testing might be a tested statistical technique, but not yet a well-established SEO-Tool. There are still a lot of Digital Marketers out there that do not totally understand that “with great power comes great responsibility!” What I mean by that? Testing can help you come up with those great ideas that will make a difference and help increase conversions (ROI). BUT, testing could also have a detrimental effect on your clients SEO performance (if you are not totally sure about what you are doing).
What to look out for
When starting with any kind of website testing, be it A/B or multivariate, you need to keep 2 important points in mind that could make your test go sour:
1. Good old duplicate content!
You may wonder why I am bringing this up, but just sit back and give it a thought. We are creating multiple variations of a webpage. If we are playing around with the design of some images, or buttons then the majority of the page content will become duplicated across the variations. Now, when you are doing simple A/B testing the variation could start being crawled and indexed, but if you doing Multivariate testing you might have hundreds of different content URLs for all your possible combinations – big duplicate content issue!
2. Where did my Link-Equity go?
Well let’s say we live in a perfect SEO world. Someone lands on our variation URL and loves it -loves it that much that he/she might tweet or include it in his/her next blog post. Outcome – backlinks are suddenly created – which would be good if the variation URL didn’t just stop after a winner is chosen. Google will only accredit that link equity if your pages resolve to a 301 re-direct.
But even worse - let’s say friends of the original people who loved your variation click on the link and want to see for themselves. They will be served with 404 errors because sadly those pages do not exist anymore. The love might stop just here – no matter how amazing your 404 pages look!
Do not despair! You can avoid them quite easily. A good starting point is to add to all your variation URLs a noindex/nofollow/noarchive tag – just to ensure Google does not get hold of them. It might be also a good idea to put your variation on a dedicated directory and ensure to exclude it with a robot.txt.
To avoid losing any link equity or annoying people with a 404, put permanent 301’s in place pointing to your winner, once you are done testing.
Well, that’s all from me and multivariate testing. If you rather liked my initial idea about women in SEO or how to successfully chat them up at conferences (don’t get excited it was a joke!) then just check out those SEO Chicks.
Have fun testing and increasing that conversion rate ($$$).