Launchpad — Introducing Automated A/B Testing

And how we used it to promote Launchpad

Update: This feature is no longer available.
You may use Anima 3 for High-fidelity Prototypes, or HTML Export

☝️ What is A/B Testing

Landing pages are a special type of web pages that have a specific goal. Whether it’s subscriptions to a mailing list or gathering contact information for sales lead, they all focus on a single metric. That metric is called Conversion. A conversion occurs when a person visit the landing pages and submits their information via a form.

A/B Testing is a way to present different landing page variations to different visitors and by tracking the conversion performance of each variation, determine which one is the best.

Different variations can differ in various visual elements as well as textual, such as:

• Different colors for buttons.
• Different tag lines for the product.
• Different pictures around the page.

For example, what text would be better for a Call to Action — “Get Started” or “Start Free”? Green button or Red button? The only way to find out is through A/B Testing.

Here we have 2 different variants to the same landing page | Download Sketch file

🤖 Introducing Automated A/B Testing

Launchpad is a great tool to easily publish landing pages.

As a designer, one can easily design a landing page, and using Launchpad create a form to ask for visitor's contact information.

Today we are proud to introduce Automated A/B Testing for Launchpad.

In contrast to usual A/B Testing, our new Automated A/B Testing goes a step further. Once the most converting variant has been determined, it will then be promoted to be served to all visitors.

Steps needed to run A/B Tests

🐶 How We Promoted Launchpad using Launchpad

In the spirit of dogfooding, we’ve launched a campaign to promote Launchpad using our own Automated A/B Tests.

We’ve purchased the domain www.sketchlandingpages.com and prepared 5 variations to a landing page.

You can check them out. Each one is slightly different.

When you visit http://www.sketchlandingpages.com/, a random variant is assigned to you and from then on you’ll only see that one.

Here are the steps to launch your own Automated A/B Tested campaign:

1. Prepare Variants
2. Publish
3. Drive Traffic
4. Sit Back and Relax (while we do the rest)

🐪 🐫 Prepare Variants

In order to use Automated A/B Testing, all you need to do is follow these steps:

1. Design your landing page in 2 or more variants.
2. Let Launchpad know which web pages are variants of the landing page versus the ones that are not.

1) The 2 pages on the left are variants of the landing page, while the white page on the right is the post-submission page. | Download Sketch file

2) In the Web Dashboard, mark the Variants.

This is done so our servers will now which of the web pages participate in the experiment and should be served to visitors as the first page they see.

🚀 Publish

There are 2 requirements for performing an Automated A/B Test with Launchpad:

1. Setup your website identifier (slug)
2. Setup a custom domain

• Setting up a custom domain is only available for paid plans.

Once you set these up, whenever you publish your new Landing page — Launchpad automatically embeds code that tracks visits and conversions. You don’t have to do anything. Then, after a “Conversion Window” of a minimum of 1,000 visitors, the winning page is determined and promoted. Once a winner is promoted, it will be served to all visitors.

If you want to try for yourself how different visitors see different variants, simply open http://www.sketchlandingpages.com/ in multiple Incognito/Private tabs. Since Private Browsing doesn’t store cookies, every time you visit the page you’ll be randomly assigned one of the 5 variants.

🚗 Drive Traffic

It’s important to drive traffic to the landing page.

For the test to have meaningful results, we need to have statistically significant number of Visitors.

The minimum number of visitors should be 1000 over the period of up to a 2 weeks. Of course the sooner the better since then we can determine the Winner earlier and get even more Conversions.

In this example, Variant named “Landing4” (in gray) is the winner since it has the highest Conversion Rate. | Automated A/B Testing

⚙️ Under The Hood

While Automated A/B Testing hides all the details so you don’t have to deal with it, if you’re still interested in knowing what is going on under the hood, then here it goes:

1. As soon as you publish your landing page with all its Variants, we start tracking Page Views and Conversions.
2. When someone visits the Landing page, we roll a dice and randomly pick a variant to present them. We also store that variant identifier in their cookies so that in future visits they will always see the same one.
3. We wait until 1,000 unique visitors visit the site, as the Variants race each other to get as many Conversions as possible (aka Conversion Rate).
4. Once 1,000 visitors have visited the site, we count how many Conversions each Variant achieved and determine the winner.
5. Now that the Winning Variant is chosen, we promote it and from now on all visitors will only see that Variant.

We hope you enjoy creating awesome websites with Launchpad. If you want to share what you’ve created join us at https://www.facebook.com/groups/animaapp/