On Leadership – Why more information is not always better

Malcolm Gladwell On Leadership - Why more information is not always better
Overconfidence has been proclaimed to be the most prevalent and potentially catastrophic problem in judgment and decision making and has been offered as an explanation for wars, business failures and stock market bubbles. Overconfidence is the psychological phenomenon that when people gain access to more information there is an increase in their level of confidence in their judgments, but not in the actual accuracy of their judgments.

A few months ago I had the pleasure of attending a presentation by Malcolm Gladwell, author of the best-selling books The Tipping Point, Blink and Outliers. He talked about expert failure, the paradox that people make mistakes and misjudgments, not because of incompetence or lack of information, but precisely because they have information. The reason is that with all this information comes overconfidence. As we get more information our level of confidence grows, but our level of accuracy does not always follow suit thus increasing the gap between what we know and what we think we know.

A classical study by the psychologist Stuart Oskamp investigated the obvious assumption that when psychologists accumulate case study material about another person, they naturally come to believe they know that person pretty well. The study examined whether that increase in confidence is justified by a corresponding increase in accuracy of conclusions. Oskamp showed that as psychologists and psychology students got more information their confidence level increased from 33% to 53%, but the accuracy of their judgments (<30%) did not improve.

Back to Malcolm Gladwell who talked about the American Civil War, in particular about the battle at Chancellorsville in 1863, where General Robert E. Lee, leader of the army of the South, and General Joseph Hooker, commander of the Army of the Potomac (the largest Union army), fought one of the biggest battles of the war. The American Civil War was presumably a turning point in the history of warfare due to its widespread use of and methodical approach to intelligence. General Hooker established a bureau of military information and is said to be the first general to do organized military intelligence. This led Hooker to have a lot of information about the enemy: positions, numbers, troop movements… you name it. He knew maybe even more about Lee’s army than Lee knew.

And the more information Hooker got, the more confident he felt. In fact he became certain he would win with the overwhelming amount of information about the battlefield and his opponent. Only thing: As he got more information, it did not necessarily make him a better decision-maker, because that overwhelming information did not improve his accuracy or leadership. Hooker got repeated intelligence on how General Lee’s troops were moving, and he believed that they were on retreat. But in reality it turned out they were preparing a flank attack. Three days of fighting ended in Union retreat.

Overconfidence comes in fact from two sources: Miscalibration and better-than-average-effect.

Miscalibration is for example seen in empirical studies when participants are asked to answer questions with two choices, and then asked to state the probability that their answer is correct. An example could be:

Who was born first, Charles Darwin or Charles Dickens?
How sure are you in terms of probability that your answer is correct?

The usual finding is that the proportion of correct answers is lower than the assigned probability. Bigger biases are found in studies where questions are to be answered with interval estimates (lower and upper bound), such as the length of the river Nile or future stock prices.

Another facet of overconfidence is the better-than-average-effect. What question to better exemplify this than:

Consider your driving skills. Do you think that you have above-average skills compared to the other people in this room?

I am sure both of us have excellent driving skills. If we try to exhibit a reasonable amount of self-criticism we may admit we may not be the best, but FOR SURE we are better than average. The problem is that everybody tends to say so. In a study, 82% of a group of students ranked themselves among the 30% of drivers with the highest driving safety. We tend to have an unrealistically positive view of ourselves. Numerous studies have shown that we often judge ourselves to be better than others with regard to skills or positive personality attributes, such as generosity.

Thus General Hooker’s overconfidence may have come not only from miscalibration but from believing himself and his army to be superior. In fact he said to his troops: “My plans are perfect. May God have mercy on General Lee for I will have none”.

Overconfidence is a problem to take seriously as it can severely impact the lives of millions of people. A US study from June 2001 to March 2011 concluded that financial executives are severely miscalibrated. During this study, more than 13,300 S&P 500 forecasts were collected. US CFOs were – on a quarterly basis – asked to predict one- and 10-year market-wide stock returns within an 80% confidence interval. As it turned out, the realized one-year S&P 500 returns only fell within their 80% confidence intervals 36.3% of the time. Now the occurrence of financial crises is not such a surprise after all. Other studies have shown that overconfident investors underestimate the variance of the value of a risky asset or overestimate their precision. Also they trade more aggressively. The higher the degree of overconfidence is for investors, the higher their trading volume is. Thus, overconfidence can explain bubbles in financial markets.

Overconfidence can lead to suboptimal decisions not only by investors, executives and generals. There is a lot of evidence that miscalibration tends to occur among experts in most fields. As Malcolm Gladwell put it, expert failure is more dangerous than incompetence, because when people lack skills and knowledge it is less likely that they are to be found in powerful positions. Experts on the other hand are the once in charge of banks, armies and countries.

So how do we handle the problem of overconfidence?

Of course it should not lead us to discard information, experts or education. Financial education and financial knowledge has in fact proved to be helpful to reduce biases in trading.

What we need to keep in mind is that making good decisions is not just a matter of having as much information as possible. It is just as much about our ability to turn information into knowledge and knowledge into wisdom. To overcome overconfidence we need to evaluate new information and our own abilities in better and more critical ways. And we need to look beyond the data – at the underlying mechanisms. This is particularly true when we consider diagnosing patients, making investments and going to war.

Information does not float around in free space – it is processed and interpreted by human beings. More information may thus prove invaluable without an increasing awareness and understanding of the human factor.

Itzhak Ben-David et. Al: Managerial Miscalibration, NBER Working Paper No. 16215, 2010 (revised 2013)
Robert W. Kolb: Behavioral Finance: Investors, Corporations, and Markets, Wiley 2010.
Stuart Oskamp: Overconfidence in case-study judgments, Journal of Consulting Psychology, 1965, Vol. 29, No. 3, 261-265

Share small business news, blogs and social media tips with Project Eve’s community of small business owners and entrepreneurs today. Our contributors come from a wide range of backgrounds; so whether you are a small business owner, social media strategist, financial adviser, serial entrepreneur, or write an amateur blog we urge you to contribute a blog to our 350,000+ community today. For more information, please refer to our Content Submissions Guidelines.

Add a Blog


  1. I’m seriously impressed with your writing capabilities and also with the structure of your web site. Did you build the blog on your own or is it a paid design template? Apart from the style and design, your content is similarly premium quality. It’s actually great to find a refreshing site once again.

  2. Hello from Reisterstown! I was bored to death at work, afterward thought to have a look at a thing or two on the net and then unexpectedly found this great post. I truly loved the way you presented the information and can’t wait to really digest the entire content when I get back home. Furthermore, I was really impressed by the rate at which your web site loaded on my cell phone even while I was on a 3G connection and not a WiFi. At any rate, you have got a really amazing weblog!

  3. 774078 646109Aw, this really is an incredibly nice post. In thought I would like to put in place writing like this moreover – spending time and actual effort to create a good article but exactly what do I say I procrastinate alot through no indicates appear to get something accomplished. 179100

  4. 162476 556529Amazing! This weblog looks just like my old 1! It is on a completely different topic but it has pretty considerably the same layout and design. Excellent choice of colors! 170017

  5. There are some interesting points in time in this article but I don?t know if I see all of them center to heart. There is some validity but I will take hold opinion until I look into it further. Good article , thanks and we want more! Added to FeedBurner as well

  6. Wow that was unusual. I just wrote an incredibly long comment but after I clicked submit my
    comment didn’t show up. Grrrr… well I’m not writing all that over again. Anyway, just wanted
    to say superb blog!

  7. Pretty portion of content. I simply stumbled upon your weblog and
    in accession capital to say that I get actually enjoyed account
    your weblog posts. Anyway I will be subscribing to your augment or even I success you get admission to consistently rapidly.

  8. The other day, while I was at work, my cousin stole my iphone and tested to see if it can survive a thirty foot drop, just so she can be a
    youtube sensation. My iPad is now destroyed
    and she has 83 views. I know this is totally off topic but I had to share it
    with someone!

  9. 746439 747977Making use of writers exercises such as chunking. They use a lot of websites that contain several creative writing exercises. Writers read an exercise, and do it. 642088

  10. 361675 128977thank you dearly author , I discovered oneself this web site quite useful and its full of exceptional healthy selective information ! , I as effectively thank you for the amazing food program post. 919281

  11. 171424 965563Thank you pertaining to giving this excellent content material on your web-site. I discovered it on google. I might check back again should you publish extra aricles. 540407

  12. 35980 620368Over and over once more I like to think about this problems. As a matter of fact it wasnt even a month ago that I thought about this very thing. To be honest, what could be the answer though? 184972

  13. 970417 946661Hello. Cool post. Theres an issue with the site in internet explorer, and you may want to test this The browser could be the marketplace chief and a large element of other folks will miss your wonderful writing due to this dilemma. 895580


Please enter your comment!
Please enter your name here