A/B Testing Is NOT for Beginners - How the Pros Do It

[ashammi228]

A/B testing is a powerful tool. It can be used to improve various marketing indicators, from the conversion of a single advertisement to the economic indicators of the business as a whole. We asked experts in the field of experiments Vitaly Cheremisinov, Valery Babushkin and Viktor Ryndin about how to achieve really cool results.

Read in this material:

what is important to consider for a split test to be correct;
how to generate hypotheses for experiments;
what tools and statistical methods are best to use;
What are some life hacks for more effective hypothesis testing?
And let us also remind you that the results of A/Btelegram cambodia testing are best tracked using end-to-end analytics . This is full-fledged statistics that take into account all channels, attributions, and types of requests - including calls, thanks to the call tracking system .


Victor Ryndin, CEO of the integrated digital marketing agency WeMakeFab



— What do you need to know to ensure that A/B testing goes well?



1. The less traffic you have during testing, the more likely it is to detect a statistical error rather than draw far-reaching conclusions about the effectiveness of a particular technique. At least 3,000 unique users should participate in testing the hypothesis.

2. Don't rush to conclusions. Two weeks is the minimum period for A/B testing, after which you can expect statistically reliable data.

3. The most common mistake is when testing is done on different audiences, with different creatives and with different landing pages. There should be only one unknown! Want to test different types of blocks on a landing page? Create absolutely identical conditions in everything else: use the same creatives in ads that are seen by a similar audience coming from the same advertising source.

4. Test several hypotheses simultaneously. Everyone is used to testing only 2 landing page variants, although within one test you can add 3 or 4 variants, significantly saving time. The main thing is to make sure that the minimum number of visitors is enough for each of the variants.

5. Test the entire length of the funnel. What's the point if a landing page converts 3x more than usual, but those leads don't make sales?

— Use a specific case to show what conclusions can be drawn based on split tests.

Hypothesis: clients interested in accounting services pay attention only to the price. You can abandon the longread with information about the company in favor of a simple landing page with a calculator of the cost of services.




A

B

Visitors

0:41

0:34

Target action

19

7

Conversion

1.30%

0.45%

The hypothesis was confirmed: on the short version of the landing page, users were less confused and more likely to submit a request.

It is especially important to conduct comparative tests after updating the site design. Even if it seems to you that the new version is obviously better than the old one. For example, due to the site redesign, the appearance of the application form with a discount offer completely changed. Visually, the new version turned out to be more attractive, but in fact, the conversion rate for filling it out fell from 1.51 (option B) to 0.51% (option A)!