Several decades of laboratory experiments on human subjects (often students) have revealed that individuals suffer from psychological biases which affect decision making. One particular bias that proved to be persistent across subject groups is miscalibration. Miscalibration is defined as excessive confidence about having accurate information (Alpert and Raiffa 1982, Lichtenstein et al. 1982, Ronis and Yates 1987, Kyle and Wang 1997, Moore and Healy 2008). Miscalibrated people overestimate the precision of their own forecasts or underestimate the variance of risky processes; in other words, their subjective probability distributions are too narrow. There always exists a tension between psychological findings from the lab and their impact in the real world, especially in
voxeu considers the following as important:
This could be interesting, too:
Cheung, Chinn, Garcia Pascual, Zhang writes Exchange rate prediction redux
Eggertsson, Mehrotra, Robbins writes To accommodate or not: The Federal Reserve’s new normal
Kollmann, Pataracchia, Raciborski, Ratto, Roeger, Vogel writes Drivers of the post-crisis slump in the Eurozone and the US
Caliendo, Feenstra, Romalis, Taylor writes Theory and evidence for the last two decades of tariff reductions
Several decades of laboratory experiments on human subjects (often students) have revealed that individuals suffer from psychological biases which affect decision making. One particular bias that proved to be persistent across subject groups is miscalibration. Miscalibration is defined as excessive confidence about having accurate information (Alpert and Raiffa 1982, Lichtenstein et al. 1982, Ronis and Yates 1987, Kyle and Wang 1997, Moore and Healy 2008). Miscalibrated people overestimate the precision of their own forecasts or underestimate the variance of risky processes; in other words, their subjective probability distributions are too narrow.
There always exists a tension between psychological findings from the lab and their impact in the real world, especially in corporations and markets. The reason is that corporations and markets systematically penalise bad decisions through price, compensation, and governance mechanisms. Therefore, it is an open question whether psychological biases, such as miscalibration, transcend to top-echelon managers, or whether their biases in beliefs and decision making are mitigated by disciplining mechanisms.
Managers form beliefs and make predictions all the time. They are required to assess future outcomes all the time, whether these relate to sales, returns on investment, or the responses of consumers to a new product. Predictions ideally consist of a point estimate and a confidence interval, reflecting the range of potential predicted outcomes. For example, a manager may predict that the return on an investment that the firm makes will be 15%, with an 80% confidence interval of ±5%. This means that the manager believes that there is an 80% chance that the return on investment will be in the range of 10% to 20%.
How good are managers’ predictions? In our paper, we assess the ability of chief financial officers (CFOs) to make financial predictions (Ben-David et al. 2013). Each quarter for more than a decade, we surveyed hundreds of CFOs about their stock market predictions. Specifically, we asked for their point estimates for the S&P 500 returns over the following year (e.g. 8%) as well as 80% confidence bounds (e.g. lower bound: -3%, and upper bound 20%). Overall, we collected over 13,000 predictions. While we do not necessarily expect CFOs to accurately predict the returns of the stock market, we expect them to a have a good sense of the riskiness of the market and be able to define reasonable confidence intervals, reflecting the general uncertainty about the stock market.
Our test is very simple. We examine whether approximately 80% of the time, realised stock returns fall within the 80% confidence interval that CFOs provide. Think of it this way: we ask the CFO for a range of expected returns – we evaluate them, not by their point forecast, but by whether the range is reasonable compared to actual outcomes, and we do this each quarter for over ten years.
CFOs are severely miscalibrated, which means that the actual market return rarely falls within the CFO’s 80% confidence bounds – only 36% of future stock market realisations fall within their 80% confidence intervals. Figure 1 shows the fraction of CFOs for whom future stock market realisations fall within their confidence intervals. The average confidence interval provided by CFOs was only 9.4%, much narrower than the actual 40% range based on historical stock market volatility.
Figure 1. Percent of CFOs for whom future stock market realizations fall within their 80% confidence intervals
In addition, we document an interesting time-series pattern. Financial theory suggests that at times of high uncertainty (measured by the implied volatility of the S&P 500, the VIX index), executives should have wider confidence bounds (i.e. the lower bound should be lower and the upper bound should be higher). In contrast, we document asymmetric reaction of the upper and lower bounds. While the upper bound is not sensitive to uncertainty in the stock market, the lower bound declines as uncertainty increases. We interpret this result as suggesting that executives view uncertainty as having primarily downside implications on stock market returns.
Importantly, we also explore whether this miscalibration spills over to firm level decisions. We ask CFOs to provide predictions and confidence bounds for the internal rate of return (IRR) of their own firm projects. These own-firm IRR imputed volatilities seem very narrow at 5.3%, much lower than the volatility of the ROIC (return on invested capital) of Compustat firms in 2001–2007, which was 36.5%.
The same executives that have overly narrow confidence intervals for the S&P 500 forecasts also overly narrow confidence bands about their own-firms’ prospects. We therefore argue that the confidence interval on stock market predictions is a good proxy for an executive’s miscalibration with respect to his or her own company-specific decisions.
Our evidence shows that firms with miscalibrated executives (i.e. those with narrow confidence bounds) appear to be more aggressive in their corporate policies; that is, the overconfidence of the executive shows up in the company’s policies. These firms invest more on average and carry more debt.
To further demonstrate the impact of the specific CFOs on corporate policies, we examine investment and debt policies of firms before and after a given CFO took office. We show that after a miscalibrated CFO joins a firm, the company’s investment and leverage increase, on average.
These results join a literature in corporate finance that finds that managers’ characteristics have a real impact on firms’ operations. For example, Bertrand and Schoar (2003) show that there is a pronounced manager ‘fixed effect’ in corporate decisions, but they do not study the behavioral traits and other characteristics of individual executives. Cronqvist et al. (2012) find a relation between CEOs’ personal leverage and their firms’ leverage.
We draw two primary conclusions. First, we establish that senior level executives are severely miscalibrated. Their overconfidence is reflected in confidence intervals that are unrealistically too narrow for both the stock market as a whole and their firms’ own projects. Second, our research makes the case that it is important to understand the psychological profile of senior executives if we want to understand corporate decision-making.
Alpert, M. and H. Raiffa (1982), “A Progress Report on the Training of Probability Assessors,” in D. Kahneman, P. Slovic, and A. Tversky (eds.), Judgement under Uncertainty: Heuristics and Biases, Cambridge: Cambridge University Press, 1–49.
Ben-David, I., J. R. Graham, and C. R. Harvey (2013), “Managerial Miscalibration”, Quarterly Journal of Economics, 128:4, 1547-1584.
Bertrand, M., and A. Schoar (2003), “Managing with Style: The Effect of Managers on Firm Policies,” Quarterly Journal of Economics, 118, 1169–1208.
Cronqvist, H., A. K. Makhija, and S. Yonker (2012), “Behavioral Consistency in Corporate Finance: CEO Personal and Corporate Leverage,” Journal of Financial Economics, 103, 20–40.
Kyle, A. S., and A. Wang (1997), “Speculation Duopoly with Agreement to Disagree: Can Overconfidence Survive the Market Test?,” Journal of Finance, 52, 2073–2090.
Lichtenstein, S., B. Fischoff, and L. Phillips (1982), “Calibration of Probabilities: The State of the Art to 1980,” in D. Kahneman, P. Slovic, and A. Tversky, (eds.), Judgement under Uncertainty: Heuristics and Biases, (Cambridge: Cambridge University Press, 306–334.
Moore, D. A. and P. J. Healy (2008), “The Trouble with Overconfidence,” Psychological Review, 115, 502–517.
Ronis, D. L., and J. F. Yates (1987), “Components of Probability Judgment Accuracy: Individual Consistency and Effects of Subject Matter and Assessment Method,” Organizational Behavior and Human Decision Processes, 40(2), 193–218.