Developing successful products is hard
In today’s increasingly competitive and turbulent environment, a company’s ability to develop and commercialise new product innovation is a key determinant of its success and survival (Sandvik and Sandvik, 2003). In some industries such as the pharmaceutical and electronics industries successful new product innovation is virtually synonymous with success. Successful innovations can enable companies to consolidate their position in existing markets, enter new markets, and explore new market opportunities, and even enable them to gain competitive advantage (Tellis et al., 2009).
However, according to a leading market research agency, as many as 95% of new products introduced each year fail. Other researchers suggest that over 80% of new product ideas never even get past the development stages and half of the new products launched miss their profit objectives. An article in the April 2011 issue of the Harvard Business Review reveals that “less than 3% of new consumer packaged goods exceed first-year sales of $50 million—considered the benchmark of a highly successful launch”. Studies examining the general success rates of new products have shown that, only I in 4 new products that enter development ever becomes a commercial success. Although some industries do far better than others, in general, failure rates can be considered high across the board.
Considering the investment that goes into developing and launching a new product, these figures are astounding. The ability to identify key indicators of failure early on in the product development process can be vital to a company’s survival. Assessing and planning for the mitigation of risk before the product is marketed, can save a company 100s of £millions, and help them steer clear of the incalculable costs of revealing their failure in the market.
What constitutes failure?
Failure is a relative term. In this white paper, failure is defined as ‘a product that has underperformed relative to the original strategic and commercial expectations set for it’.Depending upon the specific context, a product can be considered a failure when one or more of the following is true:
- It is not launched It is withdrawn from sale
- It fails to achieve the forecasted market share
- It is unable to achieve the life-cycle anticipated
- It never achieves the anticipated profits
Why is there failure
In our experience, the major differentiating factor between comparable successful and unsuccessful products is the anticipation and deep understanding of what the targeted customer might actually want and be willing to pay for. Calantone and Cooper (1981) suggest that 30% of failure is attributable to this factor, and typically describes a technology based product. Notwithstanding, further research on new product development revealed that only 16% of managerial time is spent on marketing and customer related activities (mostly during the launch) whereas nearly 80% is spent on production and technical activities.
A top 4 global consultancy conducted a study looking at why some companies are consistently more successful than others at developing and launching innovation. The results were far from revolutionary: “Top performers were twice as likely as bottom performers to research what customers actually wanted.” Undoubtedly, the answer lies in uncovering an accurate view of customer perceptions and drivers of choice and behaviour. In spite of this, the same consultancy found in the same study, that “more than 80 percent of the top performers said they periodically tested and validated customer preferences during the new product development process, compared with just 43 percent of bottom performers.” While the statistic serves to validate some of the key points of this paper, what is conspicuous about this, is the fact that almost half of the bottom performers are also conducting research, raising the question: if conducting research to understand customer needs and preferences, is relied on by so many companies, why is the failure rate as high as it is? Could the clue be in how that research is being conducted?
Discovering true unmet customer needs
Uncovering what customers really want, value and will pay for, is a big challenge. Customers are in the main, unable to correctly predict what they might actually buy. Many examples exist where companies have asked customers what they want, given it to them then regretted it when the customers failed to deliver on their predictions.
There appears to be poor alignment between what they think they might do and what they actually end up doing.
The McLean burger, low fat KFC skinless chicken and New Coke are all examples of new products from some of the world’s biggest companies that failed commercially, in spite of testing well using traditional market research including surveys and focus groups. These are all examples that Kasocio Senior Partner, Philip Graves, uses to highlight the flaws in interpreting traditional market research where consumers are asked questions about their preferences and from this forecasts are made on future buying behaviour.
In an article published in the Financial Times on 5th June 2015, Philip suggests that the market research industry should have used “better” techniques to get past the rational conscious mind, and understand “the effect of the [Coke] logo on your emotions, the memories of summer evenings when you were a teenager, the shape of the bottle, the music from the ads.” These deeper unconsciously processed thoughts (or reactions) are largely ignored by the traditional market research methods that focus on blind taste tests.
Just before the recent UK General Election, a robustly selected sample of people, representing the population as a whole, when polled said they were going to behave and vote in a particular way. However, when voting actually occurred it was clear that the population as a whole behaved in a way that was different to that predicted. In this particular case, perhaps not enough rigour had gone into sampling. Or perhaps, the voters sampled were dishonest about their true intentions. Or, most interestingly, they might have been unaware of what their true intentions were, because, invariably, people’s true intentions exist outside of their conscious rational minds.
In his book Consumer.ology: The Truth about Consumers and the Psychology of Shopping, Philip writes: “The fundamental tenet of market research is that you can ask people questions and that what they tell you in response will be true. And yet . . . this is a largely baseless belief.” Philip suggests there is a simple, clear and obvious reason for this. “The unconscious mind is the real driver of consumer behaviour.” Put another way, there is no point in asking people what they want, or what they are going to do, because they themselves don’t know.
The above suggests that traditional market research is too inadequate and blunt a tool for predicting future buying behaviour and using this to create new product innovations. A Booz, Allen and Hamilton study on the subject of new products found that “Bad estimates of market potential (or other marketing research mistakes)” were a major cause of new product failure. Clearly, what people say they want (and do) should not be the only deciding factor in creating new product innovations. Given that companies are (presumably) only taking forward initiatives that they believe have merit based on their broader experience, the overall failure rate suggests that traditional research is no better a delineator of potential success than a coin toss. When one considers the complete disregard traditional market research techniques have for the unconscious mind, and the extent to which psychologists believe it is an important determinant of behaviour, this should not come as a surprise.
Kasocio’s opinion about traditional methods of market research is that they are often incorrectly interpreted and can frequently drive wrong decisions. We believe that asking customers what they think they will do in the future, or why they behaved as they did in the past, is a dangerously flawed way of actually predicting their behaviour, which means focus groups and surveys can be perilously misleading. With no evidence to justify basing decisions on such traditional market research methods, we believe that market research practice needs to adapt and take into account the evidence from psychology about how people think. One day in the not too distant future, we may all look in the rear view mirror at the way it used to be done and shake our heads in disbelief at how naive we once were.
But if you can’t build innovations based on what customers tell us, what can you do? You look at where advances in behavioural science, psychology and neuroscience research have taken us; Behavioural Economics.
What is Behavioural Economics?
Behavioural economics challenges the long-held traditional economic view that people are rational, price-minimising, value-maximising, socially-remote automatons with relatively constant preferences. While some of the traditional market research practices claim to go beyond superficial responses, most still rely on the idea that respondents understand their own thoughts and decision-making processes; this is something for which there is very little supporting evidence, whilst the contrary view is supported by countless experiments where behaviour is clearly influenced but the source of the influence isn’t recognised by the people taking part in the study. Traditional market research also often assumes that respondents are able to recall accurately, report fully and elucidate their behaviour, when often they cannot. Reported behaviour, attitudes and declared intentions often bear little relation to behaviour in the real world. This discrepancy can often be explained by behavioural and cognitive biases . If our preferences and choices are actually less the result of linear, deliberative and inhibited processes than the standard model would have us believe, then as innovators we need to understand the “hidden” influences on human behaviour and the normal inbuilt biases residing in all of us.
“Behavioural Economics can help market research to better understand, identify, predict and influence the drivers of choice and behaviour.”
Understanding customer ‘need’ during new product development
We would do well to consider all of the mechanisms through which human beings have attempted to see into the future. Most of us are sufficiently scientific to discount tea leaves and horoscopes (and their ilk) because we can see no meaningful connection between the mechanisms they use and their claimed visionary properties. However, we’re incredibly fond of the notion that we know what we think and that we know what we’ll do in the future and so it seems reasonable to trust a research approach that relies on this mechanism. Behavioural techniques provide two tools to improve new product development:
- The enhanced understanding we have of how people think and the role of the unconscious mind can enable us to better understand existing behaviour. The more effectively you understand ‘now’ the better your chances of developing something new that will resonate with your audience.
- The way in which potential new ideas are evaluated must use experimental techniques that minimise a reliance on respondents’ conscious evaluations and prioritise realistic contexts and actual behaviour.
Psychoanalytical interviewing techniques can explore existing behaviour and identify clues of the underlying psychological influences and drives that are shaping behaviour. New ideas can be explored in context and, via implicit techniques, at an unconscious level. Experiments can be designed whereby one variable is changed (covertly) and, through testing on different matched samples, changes in response can be attributed to the variable in question.
Key benefits of behavioural economics:
- Uses existing behavioural data to identify the most important opportunities: i.e. what are the economics that are driving the business?
- Provides a picture of existing behaviour that draws on several decades of research to deliver a much more developed understanding of how people actually think and make decisions.
- Highlights the differences between what people do and what they say they do, and as a result finds needs that would not and could not be directly expressed.
- Reveals the drivers and associations underpinning perceptions, choice and behaviour
- Gives valuable clues as to how to influence them.
- Provides a framework for what to do and what not to do when evaluating potential opportunities.
“New Product Development research is vital for any organisation seeking to satisfy the changing needs of its customers.”
Behavioural economics can play a major role in the discovery, design and evaluation of new products and services, not just to make sure that the needs of target customers are met. Meeting a customer need does not just mean creating a new product or service that fits a purpose but also having an understanding of how existing behaviour can be influenced and changed. Behavioural economics, properly applied can enhance both parts of the new product development process. In terms of the overall time and cost of developing a new product or service, the use of behavioural economics need not be time consuming or expensive. By leveraging existing (behavioural) data to identify the most important commercial targets, a small number of psychoanalytical interviews can unpick that behaviour and provide the insights into why people are behaving as they are. The underlying psychology provides the platform from which potential solutions can be developed. These ideas can then be evaluated both through internal measures, such as ease and cost, and via quantitative research tools that recognise how people think.
Although no methodology is 100% predictive, taking into account important non conscious factors can really help new product design. The key issue to consider when developing a new product or service is whether the real future behaviours of target customers have been properly researched and sufficiently evaluated. Gaining insight into this can form a more solid basis for designing new products and increase the chances of success.
There are a number of ways to improve the likelihood of market research being accurate:
Consider psychological validity: most people are familiar with the concept of statistical confidence, but very rarely to people consider whether respondents are capable of explaining their own decision-making.
Analyse existing behaviour: what people currently do is a fact, what they say they will do in the future is conjecture and should be treated with caution (however strongly they assert it).
Beware of priming: there is strong evidence to show that how we respond is determined by where our mental journey is influenced to begin. Often in research the recruitment, introduction or answers to early questions sets up a train of unconscious influence that leads people to a particular answer. Without this influence, they will often answer very differently.
Context is key: if you disagree with a referee at a football match you are likely to react very differently than if you disagree with, say, a librarian’s recommendation of a book. We factor in the context automatically and, typically, massively underestimate how important it is when we are reacting to something. Market research often creates its own context and unconsciously influences people to have a mindset that is quite different from the one they have when they are making a decision in the real world.
Download the PDF here: What do customers really really want