Having spent the past three weeks setting up campaign experiments and analysing their impact, I can feel myself slowly morphing into a bit of a number bod so I thought I would now share my findings from the first wave of my campaign testing.
For those that haven’t come across the ‘Experiment Tab’ under ‘Campaign Settings’ within Google AdWords yet, I would urge you to take a look as not only is it fascinating, but it can really make a huge difference to your keyword bidding strategy and the ROI achieved.
What Are AdWords Campaign Experiments?
Google AdWords Campaign Experiments give advertisers the ability to run split tests to see what impact changes to bid price, match type and placement strategy have on overall campaign performance. The main advantage of running a controlled test via Campaign Experiments is that you are able to analyse and understand the impact that a change in strategy would have on campaign performance, before rolling it out to the rest of your account.
This makes campaign experiments a very attractive, low-risk option for advertisers wishing to get more from their paid search activity. In other words, it is a no-brainer in my opinion!
What I really like about campaign experiments is that the tests run concurrently, with the AdWords system indicating a level of statistical significance, thus removing the possibility of campaign improvements happening by chance. The fact that the tests run at the same time also allows for outside market influences to be taken into consideration, helping to ensure a statistically fair test.
For example, if a travel company increased bids for top performing keywords outside of using ACE, they may not be able to achieve a high confidence interval that raising bids had a positive (or negative) impact on sales and ROI. This is because outside influences could affect the test and make it difficult to assess whether or not the test was a success or a failure.
If the travel client launched an offline marketing campaign during the testing period, or their competitor went bust which resulted in a spike in travel related traffic, the results would be inconclusive as external factors played an influencing role in the increase in traffic and bookings.
What we are trying to ascertain here is if there is a statistically significant difference in the performance, i.e. one that is unlikely to occur by chance.
By testing with ACE, the travel client would be able to stipulate that increased bids are displayed 50% of the time against the original bid irrespective of outside market conditions, which means that an equal amount of exposure is given to both the control and the experimental bid.
This helps determine if there is a statistically significant difference in response rates between two different tested elements, which in this case is bid price. From an ROI perspective, advertisers can experiment with bid prices to see when they reach a point of diminishing returns. By testing regularly and in a controlled manner advertisers can find their ‘sweet spot’ and squeeze as much ROI out of their campaigns as possible.
Creating an Experiment
Setting up a campaign experiment is extremely straightforward and the ‘Experiment Beta’ can be found on the settings tab under ‘Advanced Settings’.
The Google system randomly decides to make either your control or experimental bid active for the auction which is based on the control / split percentage you choose during the set up process. In the example below I created a test for top performing keywords to be increased by 30% for 50% of the auction.
Once the test has been created, you then select the keywords you wish to be involved in the test and what percentage bid price increase / decrease you want to test.
Assessing the Impact
The exciting part is in keeping a close eye on the results, as checking the statistics can become somewhat of an obsession during the testing process. Assessing the impact that increasing or decreasing bids can have on your KPI’s can be fascinating and should lead to additional testing, as well as improved ROI.
As you can see from the table below, the impact on CTR, conversions and conversion rate have been extremely positive and represent a statistically significant increase in performance.
As soon as you have confidence in the results of your testing, you can then increase the exposure of the test, rolling it out to all elements of the campaign that performed positively during the testing period.
Making use of campaign experiments is certainly a low-risk approach to testing as it now allows advertisers to find the optimum bid price and position that provides the greatest return.
What Else Can You Test?
As well as a basic bid increase / decrease test I recommend being a bit more adventurous and also testing;
- The impact that a campaign restructure has on KPI performance.
- The quantity and quality of visitors acquired by different match types.
- New keywords from the opportunities tab.
- Ad group default bids against individual keyword bids.
- The use of Dynamic Keyword insertion.
- Text versus Image ads and additional placements on the Display Network.
What Can’t You Test?
Unfortunately there are limitations with campaign experiments which I hope are addressed in due course as there is so much more that I would now love to test! For example, you are unable to test different campaign settings, so CPC v CPA bidding cannot be tested at this moment in time. I would also like to test standard versus accelerated ad delivery to see which one converts at the most profitable rate!
Experiment reporting and comparing tests (especially for large campaigns) can be a bit awkward and as yet there is no clarification as to the impact testing has on quality score, but these are really minor issues compared to the value I believe this tool brings to the table.
I am also hopeful that experiments will be supported by AdWords Editor during the next wave of updates, so I can only see the functionality getting better!
Final Thoughts
Although this feature is still in its infancy, for advertisers with an inquisitive mind and a desire to constantly improve their KPI’s, ACE is simply a must have. Ongoing testing should be an integral part of campaign management and will result in a much more profitable AdWords account, which of course makes for a happy client!!
“To Test or Not to Test should no longer be the question”