You built out your new ad copy, tested out a bidding strategy, measured web and store sales to measure the online to offline effect; however, in the end you got the worst outcome possible – inconclusive results.

A negative result would have been better; at least you would have known that your hypothesis was wrong or that your strategy was not effective. But an inconclusive result tells you nothing, which can be incredibly frustrating as a marketer.

There are many reasons why a well-designed test might fail. For example, seasonal effects might be ignored, the dataset might be too small or the marketplace might change during the test.

However, a very common error in test design is not accounting for volatility – fluctuations in performance due to unpredictable events in the marketplace.

In this post, I shall delve into the issue of volatility, how it might lead to a test with inconclusive results and finally, how you can mitigate its effect on your test.

To understand the issue better, let us assume that you want to test the hypothesis that online SEM spending leads to offline store sales. To test this hypothesis, you ramp up your online budgets in increments every week.

Your plan is to run the test for 5 weeks, collect the data, do a regression analysis and answer the question, “What does one dollar spent online lead to in offline sales?”.  Now let us, put some real numbers in this thought experiment.

See more at searchengineland.com