When my daughter was in a car seat as an infant and we drove our car through town in the Summer, other drivers would roll up their windows because of the piercing noise of her screams. Well-fed, changed, and gently coddled in the softest blanket, her protests mocked it all.
I took to keeping a set of squishy earplugs in the ashtray for tricky city traffic. In a three hour drive to visit grandparents, we’d endure 2:50 of this. One upside was our confidence we’d never hit a deer because of the high-pitched ultrasonic waves sent ahead of the vehicle.
Then, in the last 10 minutes, drenched in sweat, she fell into blissful sleep. Cheerful and welcoming, my parents would probably wonder why we had odd twitches and complained of ringing in our ears for hours. “Why is she so sweaty?” they’d say, “don’t you guys turn on the A/C when you’re driving?”
The bold segue I’ll make here is that of constraints. In split testing, we must constrain variables while we test others. We must be rigid in the control groupings and how we use results. To many clients (and designers) this is just like being strapped down for three hours.
While protests may not come in the form of blood-curdling squalls, they come in other ways. It’s so hard to deploy split testing without there being a complete and total understanding of the protocols and methodologies involved. And that is incredibly difficult for a consultant to achieve.
Thank goodness for PPC. With PPC, marketing consultants like myself can design landing page variations and produce some evidence that we actually can improve the performance of the site. That evidence then serves to fuel discussions of wider tests. Without PPC, split testing simply wouldn’t happen in my world.
The other day, someone said something to me that drove home the difficulty in using split testing in a smaller project situation. “You mean you’re going to design the same page more than once?” they said. While experience will help us get closer with an initial test group, we simply cannot hit it first pass. It’s sad because many of the companies who would be best served by split testing are the ones most resistant to it.
After this, there is another level of misunderstanding. That split testing is a design-time task. “Once you’ve got the split testing done…” started an email I got a couple of weeks ago. I am a believer that as long as the split tests are resulting in a business gain (profits, branding, long term growth) minus the costs of my doing the tests, you should continue doing it. Split testing is like printing money when done right. You don’t stop the presses.
I’m busy developing materials to address some of these issues, and at the same time working on my certification in such testing. For this reason, I’m also working on a 12 week split testing plan for clients – using Google Calendar for the first time! This will let clients know what’s happening in the testing cycle.
I’m going to be looking at some new software for split testing use in the next 4-6 weeks and putting into action on some sites I’m operating.
I’m reviewing the following
…and some other scripts out there as well as some “by hand” methodologies I might use.
Those with opinions I welcome your comments!