Better Living Through A/B Testing

Filed Under Mindjet

dan_siroker_header
Austin Walne

by
April 24, 2013

The Bad News: Your website sucks.

It has a carefully designed layout, attractive stock photography, and maybe even an “explainer” video. You toiled for weeks to write perfect marketing copy and add eye-catching images, only to be stuck with a conversion rate somewhere between awful and abysmal.

The Good News: Anyone can make massive improvements to their website using cold, hard data in the form of A/B & multivariate testing.

A/B What?

What’s an A/B test you might ask? It is the fastest method to either prove or disprove your latest brilliant idea for improving your website or application. You give visitors two different versions of otherwise identical content (A & B), and measure which one results in greater conversion of your desired outcome (purchase, signup, etc.).

A_B_Test_Diagram

(photo credit Optimizely)

Last month, Dan Siroker, co-founder of Optimizely and previously Director of Analytics for Obama for America during the ’08 election, spoke to a crowd of over 200 entreprenuers and marketers at the San Francisco Agile Marketing Meetup (Mindjet is a co-sponsor). His talk was titled: “Lessons learned from 177,295 A/B & multi-variate tests”

Dan Siroker Agile Meetup

(Photo credit @webgeek_)

Dan focused on 6 key learnings:

1. Define quantifiable success metrics.

The first lesson matches the opening of the Agile Marketing Manifesto: “validated learning over opinions and conventions.” Truth is found by questioning the status quo and testing your hypothesis by providing your visitors an alternative. Defining quantified success metrics allow you to find where you should focus your energy when A/B testing.

2. Explore before you refine.

It’s common for people to dip a toe in the water by making minor changes to their site experience. They (mistakenly) assume their website is already pretty good. Instead, you should start with radical changes; and then refine from there.

Refinement vs Exploration(Photo credit Intercom)

3. Less is more. Reduce choices.

ZAGG changed the main image on their product page and experienced a 40% lift in revenue. The Clinton/Bush Haiti Fund did the same and saw an +10.97% increase in dollars raised per pageview. Reduce fields. Increase conversion.

4. Words matter. Focus on your call to action.

Designing your call to action with your desired result can make a big impact on conversion rates. LiveChat changed their call to action from “Free Trial” to “Try it Free.” That simple change increased the trial signup rate by 15%. On the flip side, ABC family tested “full episodes” vs “watch episodes” and found that “full episodes” had 10% more clicks.

5. Fail fast.

Don’t spend weeks writing a 20 page test plan for the next 6 months. Start with an obvious place like the homepage and the call to action most relevant to your business. Don’t over-think it in the beginning.

6. Start today.

This advice is worth exactly $0 until you started testing on your website. Get started now.

Watch the full video of Dan’s talk below:

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>