Win/Loss Surveys - What to Ask

OK...so you've decide to perform some win/loss analysis...you've either lined up some customers for interviews or you've decide to run an online survey (see my earlier post about this). Now you must decide what questions you are going to ask. This will determine whether your win/loss analysis provides a unique insight into the buyer's selection criteria and how your product's features and positioning match up to these criteria...or whether the responses you get from the customers will be the usual 'politically correct' platitudes.

If you ask a customer (or former prospect) why they selected product X (or even why they didn't select product X), you are immediately making them justify and defend their purchasing decision. This puts the person being interviewed on the defensive and things will go downhill from there. In an earlier post (The Challenges of Win/Loss Analysis) I explained how people will justify and defend their purchasing decisions and how this doesn't give you the information that you need for a meaningful insight into the buyer's purchasing criteria. You need to avoid putting the customer/prospect into this position.

Rather than asking the customer/prospect about their decision, ask them about their purchasing criteria. Give them a list of criteria and ask them to rank them in terms of importance to their particular purchasing decision. The criteria in the list should be based on your product positioning together with the positioning of competitor's products - after all, you want to know if there is something in a competitor's sales pitch that resonates with the buyer. So, your purchasing critieria should be something like:

1. Price
2. Ease of Use
3. Distributed Architecture
4. Performance/Scalability
5. Vendor Reputation
6. Development Environment
7. etc.

Some of these should be from your go-to-market product positioning and some from your competitor's. If you get the customer/prospect to rank their importance in the purchasing decision you will get a view as to whether your positioning aligns with the buyer's purchasing criteria. If there is a misalignment, you need to look at this and see if any adjustments are necessary.

As an example of why this works, I was involved with some win/loss analysis using this technique and the company that I worked for at the time thought that our product's distributed architecture was a key selling point. It turned out that for customers who needed this feature, it was very important in the purchasing decision, but for other customers it was not very important at all. This is perhaps what you might expect - but only a few of our customers needed this feature, so our primary differentiation and selling point missed the mark for most buyers. We also found out that the product demo and proof of concept (PoC) were critical to the purchasing decision - whether we won or lost the deal. In other words, customers wanted to see the product working in their environment before they would make a purchasing decision. There was a very strong correlation between good PoCs and wins and also between poor PoCs and losses. At that time our PoCs were dependent upon the knowledge and skill of the Sales Engineer assigned to the account, and this knowledge and skill varied greatly. As a result of the win/loss analysis bringing the importance of the PoC to light, we instituted better training and better support for the Sales Engineers performing the PoC.

In addition to asking the customer/prospect to rank purchasing criteria, you should also ask them about the sales process. List the various stages of your typical sales process and ask them to rate these - not in terms of importance, but in terms of how well they were performed. Also allow the customer/prospect some space to add comments. We found (from comments) that one opportunity was lost because we "didn't seem to understand their business and their problems". That pointed to a potential sales training issue. Also ask the customer/prospect to rate the sales process for other vendors who tendered for the deal - you might get a useful insight into your competitor's sales processes. Are they consistently performing well with a certain step (e.g. understanding the customer's problems)? Or do they struggle on certain steps (e.g. giving a demo)? All of this information helps your sales team to refine their approach to similar opportunities.

If you use online surveys to collect the quantifiable information, such as criteria ranking, sales processes rating, etc. you should look for interesting comments that might be an indicator that you should call the customer/prospect. You can then drill into the background to the comment to better understand the buyer's perspective. This can illicit further information that might be useful to sales and marketing.

Ultimately, the purpose of win/loss analysis is to improve your sales and marketing effectiveness- making sure that marketing positions the products in the most optimal way and that sales have the information and tools to address the buyer's purchasing criteria.