While I am general a strong proponent of research-based approaches to innovation, I often see research being misused in a way that prevents research. Either the wrong kind of research is conducted, or its results are used in the wrong way, and the result is that an innovation effort becomes an efficiency improvement: no significant changes are made, but the results are marginally improved.
This is because research, by its very nature, is focused on present practices and present results. The researcher seeks to discover what people are doing right now (or more accurately, he seeks to discover what people have done in the recent past) and those who consume the research use it to propose methods by which existing customers can keep doing what they are already doing and new customers can simply imitate the behaviors of existing ones. It’s more of the same, perhaps slightly different.
To be innovative, one must disregard what people are presently doing and define a new way to achieve the ultimate result toward which their behavior is intended. That is, rather than considering the things that people are presently doing, an innovator must seek to discover why they are doing those things – what goal they are attempting to achieve by their actions.
It could well be (and often is) that people are doing the wrong things to achieve their goals, or at least things that are not as efficient or effective as they could be. Even if people claim to be happy with what they are doing and the results they are getting, this is not to be taken at face value. There is the psychological tendency to justify past decisions and defend current practices, as well as the tendency to satisfice and accept a margin of frustration or disappointment.
And if the outcome of an “innovation” effort is to keep doing the same thing with minor adjustments, then the customer is being supported in the existing behavior without a substantial improvement. It will be easier to “sell” the new idea because it isn’t really new at all, and the customer will experience a slightly lesser degree of frustration and disappointment than he presently does, but with less need to change.
In this sense, most of the existing secondary research is not suited to innovation: it tells us what people are doing, but not why they are doing it. And that information is difficult to collect and even more difficult to quantify. It requires probing deeper than a multiple-choice questionnaire, and applying greater effort to the analysis of the data collected – which may be one reason that so few sources are conducting the proper kind of research for innovation, and cling to the methods that gain information that justifies a continuation of business as usual rather than disruptive change.