Podcast: SL@P 005 - Introduction to A/B Testing for Lawyers
This article discusses A/B testing and the methodology of comparing two versions of the same webpage or document to test and determine which version performs better for its given task. A/B testing is essentially the process of creating two variants to test a hypothesis.
At its core, A/B testing allows your law firm to create two variants of a webpage, for example, and to divide the traffic and visitors equally to those two variants in order to determine which variant performs better.
Here is a good example. Let’s pretend for a moment that you created two pages: one page with a blue background (Page A) and another page with a red background (Page B). Now, let’s assume that 100 website visitors came to the site, and half of the visitors got directed to Page A, with the blue background, and the other half saw Page B, with the red background.
In this example, Page A converted 5 of the 50 visitors through the process of filling out a contact form. The 5 out of 50 completed forms represents a 10 percent conversion rate. However, Page B, with its different colored background, managed to convert 10 of its 50 visitors, which is an amazing 20 percent conversion rate.
Our starting hypothesis in the above example is: “Will the color of the background cause the page to perform better?” The answer from the limited site visitors is a resounding “Yes.” Now, let’s assuming that instead of each site receiving 50 visitors each, the sites received 50,000 visitors each and still maintained the same conversion percentage. It would be convincing evidence that red backgrounds for this particular website drastically outperform blue backgrounds.
While this is a basic example of A/B testing, the concept holds true for each component of your law firm’s website. However, A/B testing is not limited to just web designs and can be used in each and every component of your legal practice. My firm, Wallace Pierce Law, has A/B testing intake procedures, negotiating strategies, demand packages, client disbursements, advertising materials and a variety of other in-office procedures.
A/B testing is an approach to ensure that you are constantly challenging why your law firm does something a certain way by gathering results and data to make an informed decision about the matter.
The A/B Testing Process
Collect Data for Your Control Group – Collect enough information to get a baseline so that you may determine what you should begin optimizing.
Determine Your Goals – Determine what results you would like to see for your given data set. Determine whether your goal is possible given the data.
Generate a Hypothesis – Develop an idea for what might help your data reach the desired goal. An example of a hypothesis would be: “If I change the background color to red, I will see an increase in conversions.” Another example would be: “If we offer consults after 6 p.m., we will likely see more client consults.”
Create the Variant – Create the variant that permits you to test your hypothesis. The variant should be designed in such a way that it helps you test the hypothesis and assists you in reaching the goal.
Run the Experiment – Run the experiment for the requisite period of time to ensure that you are giving the test the opportunity to gather enough data to be helpful. If you run the experiment for too short a period of time, the data may not be a large enough sample size to make the necessary determinations.
Finally, make sure that your variants are similar enough that you can attribute any increases to what you are actually testing. This means that you want to make subtle changes and test things like headline text and background images, and then slowly make improvements. If you determine that red backgrounds are better, make the next A/B test both contain red backgrounds but of different shades of red. The possibilities are endless.
Remember the golden rule of any A/B testing is that there is never a final destination to this testing journey. Test, test, and test again! And then, test again!
Jared Pierce hung his own shingle right out of law school and has spent every minute since then discovering the joys and difficulties of chasing success. Anyone who has ever met Jared will tell you h