The hiding hand principle is a theory that offers a framework to examine how ignorance (particularly concerning future obstacles when person first decides to take on a project) intersects with rational choice to undertake a project; the intersection is seen to provoke creative success over the obstacles through the deduction that it is too late to abandon the project. The term was coined by economist Albert O. Hirschman.
Writing in The New Yorker, Malcolm Gladwell brought the concept to life, retelling the story of the construction of a railway tunnel through Hoosac Mountain in northwestern Massachusetts. Construction proved much harder than anticipated, but eventually was completed, with positive results. Gladwell was reviewing the book, "Worldly Philosopher: The Odyssey of Albert O. Hirschman," by Jeremy Adelman (Princeton University Press, 2013).
Bent Flyvbjerg and Cass Sunstein take issue with Hirschman's principle and argue that there are really two Hiding Hands, a Benevolent Hiding Hand, which is the one Hirschman talks about, and a Malevolent Hiding Hand, which obstructs projects instead of creatively saving them.[1] In an empirical test of 2,062 projects, Flyvbjerg and Sunstein found that the Malevolent Hiding Hand applied in 78% of cases, whereas Hirschman's Benevolent Hiding Hand applied in only 22% of cases, contrary to Hirschman's belief that the Benevolent Hiding Hand "typically" applies. Flyvbjerg and Sunstein also argue that the Malevolent Hiding Hand is the planning fallacy writ large.
Hirschman described the concept of the Hiding Hand principle in the second section of his essay "The Principle of the Hiding Hand"[2] where he states:
We may be dealing here with a general principle of action. Creativity always comes as a surprise to us; therefore we can never count on it and we dare not believe in it until it has happened. In other words, we would not consciously engage upon tasks whose success clearly requires that creativity be forthcoming. Hence, the only way in which we can bring our creative resources fully into play is by misjudging the nature of the task, by presenting it to ourselves as more routine, simple, undemanding of genuine creativity than it will turn out to be. Or, put differently: since we necessarily underestimate our creativity it is desirable that we underestimate to a roughly similar extent the difficulties of the tasks we face, so as to be tricked by these two offsetting underestimates into undertaking tasks which we can, but otherwise would not dare, tackle. The principle is important enough to deserve a name: since we are apparently on the trail here of some sort of Invisible or Hidden Hand that beneficially hides difficulties from us, I propose "The Hiding Hand."
Harvard University professor Cass Sunstein and Oxford professor Bent Flyvbjerg ran an empirical test of over 2,062 projects to determine the prevalence of the Hiding Hand. In the end, they found that the "malevolent" Hiding Hand (an instance of the planning fallacy) appeared 3½ times more often than the "benevolent" variant Hirschman described. In their words:
The theoretical implications of our findings are clear. The idea of a Benevolent Hiding Hand is a special case and as an effort to capture reality, it is misleading or even a distraction. The Malevolent Hiding Hand is pervasive, and it is a case of the planning fallacy writ large—i.e., it applies not only to schedule, but also to costs and benefits in the widest sense—aggravated by the effects of ignorance, power, and motivated reasoning. The policy implications are equally clear. It is bad policy to justify plans and projects based on faith in the Benevolent Hiding Hand. In most cases initial costs and difficulties will not be overcome by later creativity and benefits; it is a dead-end at best, a scam at worst.