Hi and welcome to this quick introduction 
to A/B testing. So what is A/B testing?
At a high level, A/B Testing is a statistical 
way of comparing two or more versions,
such as Version A or Version B.
to determine not only which version performs better
but also to understand if a difference 
between two versions is statistically significant.
So why do businesses conduct A/B tests?
This is the way businesses are run these days
and they have to take a data-driven approach.
A common dilemma that companies
face is that they think they understand
the customer, but in reality customers
would behave much differently than you
would think consciously or
subconsciously. Users don't often even
know why they make the choices they make,
they just do. But when running an
experiment or an A/B test, you might find
out otherwise and the results can often
be very humbling and customers can
behave much differently than you would
think so it's best to conduct tests
rather than relying on intuition.
So let's visualize. For example, in marketing,
or a web design, you might be comparing
two different landing pages with each
other or two different newsletters let's
say you take the layout of the page you
move the content body to the right now
versus the left or maybe you change the
call-to-action from green to blue or
or your newsletter subject line has the
word "promotion" in Version A and the word
"free" in Version B in order for A/B
testing to work, you must call out your
criteria for success before you begin
your test. What is your hypothesis or
rather what do you think will happen by
changing to Version B. Maybe you're
hoping to increase conversion rate or
newsletter signups or increase opens
call out your criteria for success ahead
of time. Also, you will want to make sure
that you split your traffic into two it
doesn't have to be 50/50 but you will
want to figure out what is the minimum
number of people I need to run my A/B test
on to achieve statistically
significant results you can do this with
multiple versions such as two buttons
that are blue and two that are orange
one blue and one orange button say RSVP
and another blue and orange button say
sign up this would be called a multivariate test or a
full factorial test since you are
comparing different factors. So what are
some factors we can test on when
conducting an A/B test? Changing the
layout of the page and shifting where
certain items are such as moving the
content body to the right, the navigation
to the left, or the call to action near
the bottom you can change the call to
action such as changing the color or the
text or where the call-to-action is
located on a landing page or email.
You can compare two different images with
each other to see if one has a higher
conversion rate or a higher
click-through rate. And what about on the
back-end suppose the UX and the UI is
the same but you update your
machine learning algorithm to update the
recommendations that are shown to people
but what happens if something is broken
or funky or the data is messy and the
quality is off or there's too much noise
maybe there's a sampling problem and you
don't randomize correctly it could be a
one to two percent impact but you should
make sure that your A/B test is being
conducted properly first by setting up
an A/A test. Thanks for watching, give us
a like if you found this useful or you
can check out our other videos at Data
Science Dojo tutorials.
