Managers are Trained to Optimize
Business schools train managers to optimize rather than innovate. The difference is that optimization begins with an existing set of activities and seeks to make them more efficient, whereas innovation requires considering if an altogether different set of activities might produce a better outcome.
The practice of "scientific" management that is presently the dominant theory is entirely about optimization: Frederick Taylor watched as men shoveled coal, theorized what physical movements might be less tiring, experimented with different sizes and shapes of shovel, and the like to make the workers more efficient at doing exactly what they were already doing. He did not pause to consider whether using a gasoline engine or geothermal energy might be more efficient than coal furnaces - there was no question of whether coal should need to be shoveled, just a question of how it could be shoveled more quickly.
The vast majority of business courses focus on optimization and efficiency, with maybe one or two chapters in the textbook that hint at innovation, but often shy away from it as if it is supernatural in nature and cannot be practiced or relied upon. As a result, the managers with formal education in management practice what they were taught and are often rather uninspired when it comes to exploring entirely new possibilities.
As such, it is no accident at all that many of the CEOs who are legendary for pioneering new businesses and ignoring the well-beaten path of standard industry practices are not formally educated in the field of business. Neither is it implausible that those who have formal training in business are very limited in their ability to innovate.
The Past is Safe
I avoided making this my opening point, because discussions of this nature usually dismiss decision-makers as being ruled by ignorant fear - and then the discussion degenerates into a litany of complaints and stereotyping. But in truth, those who have achieved positions of power by playing by the old rules are ruled by very well-considered fear that they may not prosper if the tules are changed.
Fear is not wholly a bad thing, because it keeps us safe from danger. The man who strides into the high grass and refusing to "fear" the snakes is not brave man, but very stupid. And before society made a mission of saving such people from themselves, many of them were removed from the gene pool by their own foolhardiness.
Fear is a survival instinct that gives us the inclination to avoid risk of harm. But beyond a certain healthy measure it also gives us the inclination to avoid risk of any kind, preferring to do what is safe and familiar, and foregoing opportunities that are unusual and carry with them the possibility of failure.
Fear of this kind can be ignorant, as the paralyzed individual may not be aware of the facts (like, there are not any snakes in that grass) or may not know the way to proceed with caution to mitigate risks (walking slowly, probing ahead, listening attentively, etc.)
Dragging this meditation back to business - to continue doing the very same thing you did in the past, which has been proven successful by years of practice, is always less risky than trying to do something different. You have no experience, no historical proof, that a proposed new way will be successful and are lead by careful consideration of this to cling to the present way ... and to reject innovation.
The Past is Proven
Along the same lines, the events of the past have resulted in evidence that can be perceived through the senses: we saw what was done, but can only imagine what might be done. As such, when there arises a decision of whether to continue to do what has been done in the past versus trying something different, there's a lot more proof for the past and very little - none, in fact - for the future.
A proposed future action is based on imagination, describing a plausible scenario, and speculating about the outcomes. You simply cannot prove the results of something that has never been done before with the same level of evidence that you can prove the past.
And because we are geared to trust more in proven facts of the past than unfounded speculation about the future, any choice between doing what is known and practiced versus what is unknown and unproven is weighted heavily against innovation.
Change is Difficult
To state the obvious, a business is composed of many people doing many things in coordinated patterns. It is comparatively easier for a single person to change his ways, break his patterns, learn new skills, and do the other tasks necessary to do things differently in future (though anyone who's tried to kick a habit will testify it's not as simple as it seems).
When a business wants to change its ways, many people must be trained in new skills, new facilities and equipment must be purchased, new suppliers engaged, new customers won, and new departments created and staffed. It's a long, painstaking, and expensive undertaking to do something different, and much cheaper and easier to keep doing the same thing as before.
For that reason most businesses are steadfastly focused on their present operations, and very few have much sense of a future beyond the current fiscal quarter or year. They do not question (and sometimes do not even remember) the reason that they do things as they presently do - and simply perpetuate their rituals without much thought unless some crisis makes them aware that they are becoming obsolete.
Many "Innovators" are Kooks
Perhaps the greatest barrier to innovation is the reputation and credibility of those who call themselves "innovators." Most of them are narcissistic and undisciplined types who have a constant supply of quirky ideas and boldly propose that others should bear the labor and risk of making their wildest dreams into a reality.
Granted, to stereotype an innovator as a foolhardy imbecile is just as wrong as it is to stereotype an opponent of innovation as a fearful coward. But there are no stereotypes without prototypes, and there are plenty of them on both sides of the argument over innovation.
The way to tell the difference is that a bona fide innovator is a very thoughtful person. He doesn't fall in love with an idea because it's his, because it's novel, or because it's cool - but because it makes sense. He can describe the reason for making a change in terms of sound theory, and the course of action he prescribes is based on solid and plausible logic.
Unfortunately, such individuals are rare. So instead, a different kind of self-proclaimed innovator tends to be dominant: the type who constantly comes up with wacky ideas he cannot justify, cannot explain in detail, and for which he cannot present a plausible argument. These "idea men" don't bother with the details, but leave them for others to sort out - which is sure evidence that they have not invested much thought into them.
Experience with a few such characters often poisons an organization against innovation, and gives the fearful all the more reason to reject any innovative idea without investing the time to consider it.
A this point, my memory is spent. I recall there being a few additional reasons that innovation has lost its appeal, and that organizations retreat to the comfort and safety of optimizing existing operations rather than exploring the possibilities ... but they've faded. Perhaps they will return to me later.