Smart Experiments Improve Your Business by Will Phillips March 2010 Every day, managers fiddle with pricing and promotions based on their own common sense or perhaps recommendations from colleagues. This latter is like you saying you're a bit under the weather and some friends friends respond, "Oh, when I was sick, my doctor gave me this prescription, why don't you try it too?" This belief that all businesses are similar, not to mention that all managers and employees are similar, could not be further from the truth. Despite many businespeople's belief that they are hard-nosed and objective business thinkers, it turns out that far too many business decisions are soft-nosed instead. The absolute failures from giants like Enron to Lehman Brothers are perfect examples of this approach. For example, one bank that was concerned about low-costumer satisfaction ratings for long waiting lines, and so they tested showing television news in waiting areas to see if it shortened consumers' perceived waiting time. It turns out that perceived waiting time did drop in this particular branch, but it increased substantially in another branch. The bank of course, simply reported the success. Managers, if nothing else, are often extremely conservative, avoiding foolish decisions and mistakes if at all possible. For this reason, there's a strong commitment to only try a new approach if someone else has already tried it and reported success, thus failures like that bank's waiting area experiment. In a recent article in the February 2009 Harvard Business Review, Thomas H. Davenport reports on smart business experiments, and encourages businesses to yes, try new ideas, but to do so in a planned data oriented manner. Davenport cites examples such as: Will lobster tanks increase lobster sales at Food Lion Supermarkets? Do users bid higher in auctions when they can pay by credit card? Do Subway promotions on low-fat sandwiches affect all sandwich sales? The simplest process for building an experimental attitude in your business is to utilize this six step process: Write down the problem or the opportunity. Create a hypothesis or an idea of what you think might work in this situation to make things better. Design a test. Do the test. Measure the results. Assess how well your solution worked, fine tune it and start again. The key here is to figure out how to implement your proposed solution in such a way that you can measure the impact. One of the common ways of doing this is by using a comparison or control group. For instance, with a new promotion, you might make a promotional offer to every other customer and then at the end of the month, review the results for each group separately. Direct mail offers another useful way of gathering information on promotion success, using what's called a "split test," so that every Zip Code is split in half. One half of the mailings receive the one week of free service offer, and the other half a two week offer. After a month or so, it's fairly easy to measure which offer was the most effective. One of the areas where a great deal of experimentation can be done quite easily is with your website, but these principles of data gathering are still critical, perhaps even more so because of the instantaneous nature of the Internet. It's been reported that Jeff Bozos, president of Amazon, reportedly fired a group of web designers for changing the website without testing those changes first. Gary Loveman of Harrah's Entertainment agrees, stating that not using a control group is sufficient reason for termination. Thomas H. Davenport and Jeanne G. Harris recently published "Competing on Analytics:The New Science of Winning." Although this book may be a little heavy duty for your needs, it will provide the tools you need to test your business ideas in a cost-effective way. The Amazon review explains more about what this book offers: You have more information at hand about your business environment than ever before. But are you using it to "out-think" your rivals? If not, you may be missing out on a potent competitive tool. In Competing on Analytics: The New Science of Winning, Thomas H. Davenport and Jeanne G. Harris argue that the frontier for using data to make decisions has shifted dramatically. Certain high-performing enterprises are now building their competitive strategies around data-driven insights that in turn generate impressive business results. Their secret weapon: Analytics: sophisticated quantitative and statistical analysis and predictive modeling.