The Genesis and Failings of Market Predictions
In the year 1982, a leading global publication published a special dossier about what the world would be like in the year 2000. The publication invited a range of professional ‘futurists’ make their predictions. From a robotic revolution to the extinction of inflation to family experimentation practices, these predictions were published for all to read and gauge the future themselves. It’s not surprising that the most of the predictions remain unfulfilled till today. However, these futurists have not been completely discredited, nor have they become extinct. They live among us even today, with different roles and identities.
The practice of making predictions based on factual observations can be traced back to a time when markets didn’t even exist. From an instinctual perspective, making predictions is as much a part of human nature as is any other cognitive practice like rationalizing and reasoning. Over centuries, the art of making a prediction has undergone many-a transformation. This has been facilitated through philosophical, mathematical and even theological sciences. Even then, there still remains a strong element of uncertainty about the validity of making predictions. This is so because making predictions is essentially choosing one of two (or numerous) possible outcomes. There are some who will make the right choices and others who won’t. Those who make the right choice won’t be right all the time, and neither will the ones who made the wrong choice. However, the frequency or consistency in such a game of chance is anybody’s guess a.k.a. prediction.
For anyone who is looking to follow or disregard the plethora of predictions that are made every day, it is advisable to understand what governs such predictions and why they are so popular.
Five Famous Predictions That Never Happened
- In the 80s, Nobel Prize-winning economist Paul Krugman predicted that the internet was just another fad and would have no discernible impact on investing.
- Nobel Prize-winning economist Paul Samuelson was certain that the Soviet economy would surpass the American Economy by 1990.
- Just before the 2008 economic meltdown, AIG had endorsed and promoted the Credit Default Swaps (CDS), which was a prominent cause of the market crash and led to the $85 billion bailout for them.
- Economist Irving Fisher predicted the stock market was on the verge of a boom, 9 days before the great market crash of 1929 and lost most of his wealth in the process.
- In 2007, former Fed Chair Alan Greenspan claimed that interest rates would be in double digits to control inflation, but a year later Fed fund rates were at historical lows.
Making a Prediction: An erroneous human tendency
The start of this practice is probably anyone’s guess. In recorded history though, the end goal guiding the making of predictions has generally been to attain a strategic advantage over the competition. Whether it is in financial markets, war or sports, predictions are a guesstimate that have a tendency of giving confidence to those who are hopeful of a positive outcome. On the contrary, people also have a general tendency to ignore the possible outcomes that are not so positive, even though it might be most probable.
When Nostradamus, the French polymath, wrote his predictions in the form of prose, he did not write any specific details. Yet, today, people attribute many historical events (including 9/11 attacks, the atomic bomb and the rise of Adolf Hitler) to the predictions made by him. It is also important to note that there are many predictions of his that have not manifested in any way or form. That doesn’t make his supporters and followers discredit his work though. They continue to look for patterns where none might exist. Similarly, there are many who came before and after Nostradamus, making claims about future events and the people involved who are revered in different walks of life, including business, politics and economics.
The truth is human beings are geared to widely accept certain visions of the future that suit their view of things, even if those visions improbable or impossible. It confirms how one analyzes the world one is familiar with and wants to believe that their view is correct. When that hypothesis fails, it is put down to chance or an anomalistic inconsistency in one’s analysis. In light of this, in today’s information age, it is not unusual to find a range of predictions on a daily basis about almost anything and everything happening in the world. Even though these predictions may border on being totally ridiculous, it can find its own stream of subscribers who believe that a particular outcome would suit their worldview.
The questionable foundation of general predictions
“Predictions fail because the world is too complicated to be predicted with accuracy.”
Dan Gardner, Author of Future Babble
Experts have been trying to forecast the price of oil since the inception of the industry in the 19th century and they continue their hot streak of being wrong till today. Modern futurists use retrospective quantitative methods to predict socio-political scenarios decades into the future (known as cliodynamics) yet all they have achieved is the creation of a broad, over-arching theme that captures only part of any future. For decades, there have been innumerable predictions of the Dow Jones Industrial Average index either soaring through the proverbial ceiling or crashing with devastating effects. Just like in the case of cliodynamics, only a miniscule part of the prediction has actually occurred, enough to be branded as an inaccurate prediction.
This further begs the question, how does one qualify to be an expert? What is the criteria that separates the ones who make consistently successful predictions from those who are much less accurate in doing so? As it turns out, there is no such criteria and those who are considered to be ‘investing gurus’ are barely ever absolutely right with their predictions. According to a study by CXO Advisory Group which collecting predictions from 68 different forecasters, after study 6,500 different predictions, it was found that, on average, a forecaster will be correct 46.9% of the time. In other words, if the example of a coin toss were considered where the chances are 50-50, an average person would likely beat the average ‘market guru.’
Dr. Tetlock, The Hedgehog and The Fox
From 1984 till 2003, Dr. Phillip E. Tetlock, a political science writer and revered author, conducted one of the most comprehensive studies of analyzing expert predictions. He assembled a group of 284 experts from a myriad of fields, including government officials, professors, and journalists. They had a diverse range of opinions, from Marxists to free-marketers. In this 20 year period, Dr. Tetlock collected over 28,000 forecasts about the future, including questions about inflation, economic growth, the price of oil, the stock market, elections and the global order of power.
As he validated and analyzed the data sets that he had collected over 2 decades, the final result that he arrived at was that ‘the average expert was as accurate as the famous symbol of random guessing – a dart throwing chimpanzee.’ Of course, some did better than others but they were all far from perfect when it came to making predictions. However, there was a critical pattern he identified while studying these predictions. The way these experts were making predictions had a strong correlation with their individual style of thinking.
In order to simplify this observation, Dr. Tetlock drew inspiration from an essay written by a political theorist and writer, Sir Isaiah Berlin. The essay was titled ‘The Hedgehog and The Fox’ which divided writers and thinkers into hedgehogs (who view the world through the lens of a single defining idea) and foxes (who draw on a wide variety of experiences and for whom the world cannot be boiled down to a single idea).
In Dr. Tetlock’s analysis, the hedgehogs were those who use one big idea (usually an analytical tool) to make forecasts. They preferred simplicity and clarity, so they preferred to keep information pared down to what they thought was the bare essentials. They like using words like ‘certain’ and ‘impossible’ and are usually the ones on television. The foxes, on the other hand, gathered information from many sources, looked at issues with more than one analytical lens, readily admitted mistakes, and were comfortable with complexity and uncertainty. They often said ‘maybe’ and they were humble about their ability to see into the future.
The study also showed an inverse relationship between fame and accuracy. The more famous the person, the less accurate their prediction would be. One would assume that this finding would keep the inaccurate hedgehogs out of the limelight and the focus should remain on the humble foxes. However, hedgehogs are brimming with confidence and make absolute declarations, something that dispels uncertainty. The fox talks about probabilities and possibilities. For the general viewing audience, it’s always more attractive to have their concerns and uncertainties absolutely dispelled, even if it is through a hedgehog analysis which is less accurate than a dart throwing chimpanzee.
The role of bias in analyzing predictions
“Everything makes sense in hindsight. We cannot suppress the powerful intuition that what makes sense in hindsight today was predictable yesterday.”
Daniel Kahneman, Behavioral Scientist, Nobel Prize winner
In the path breaking study called Prospect Theory by Daniel Kahneman and Amos Twersky, the study of human decision making under conditions of risk was thoroughly analyzed. The study’s findings run contrary to the normative implications inherent within classical subjective expected utility theories. This theory gave validation to the idea that risk taking and risk averse behavior occurs not in isolation, but in conjunction with individual behavioral dispositions. These dispositions are driven by the biases individuals may have.
When it comes to making market predictions, such biases play an important role in influencing the final decision of an individual. Some of the prominent cognitive biases that affect predictions are as follows:
- Overconfidence bias: Overconfidence is one of the cognitive biases that underlie the illusion of validity. Overconfidence bias can be defined as an unwarranted and often times illogical faith that an investor has in their ability to predict the market. This bias is remarkably prevalent in the investment community. James Montier, former co-head of global strategy at Société Générale, once conducted a study with over 300 professional fund managers to measure the scale of behavioral biases. He asked if they believe themselves to be above average in their ability. 74% of fund managers responded in the affirmative, believing they were above average at investing. And of the remaining 26%, most thought they were average. In short, virtually no one thought they were below average.
- Confirmation bias: The illusion of validity persists because people fall prey to the confirmation bias; they focus on information that is consistent with their beliefs while neglecting inconsistent information. As physicist Robert Park once said, “People are awfully good at fooling themselves. They’re so sure they know the answer that they don’t want to confuse people with ugly-looking data.” This bias is more prevalent in market predictors with higher perceived knowledge about the market and higher strength of belief (i.e., sentiment). They will also have a negative interaction effect between the perceived knowledge and the strength of prior belief on confirmation bias.
- Representative bias: Representative bias occurs when the similarity of objects or events confuses someone while thinking about the probability of an outcome. According to Kahneman and Tversky, representative bias badly affects people’s decisions during their opinions building and reasoning. This may mislead investors and forecasters to believe they have already processed the information correctly right before they make a decision. While making a judgment, they are likely to ignore other related factors which may affect the value of the investment. Because of the representative bias, forecasters may ignore the law of small numbers where they incorrectly consider a small sample size as being representative of the population. Even though these numbers may reflect the current trend, they cannot possibly describe the properties of a whole population.
- Hindsight bias: Hindsight bias leads people to exaggerate the quality of their foresight. It is a behavioral trait where one becomes convinced they accurately predicted an event before it occurred. Based on this, they make predictions about the future. Once an event is part of history, there is a tendency to see the sequence that led to it as inevitable, as if uncertainty and chance were banished. Armed with such a bias, one can become overconfident of one’s foresight and make exaggerated predictions. Those who have hindsight bias, it is common to hear the phrase, “I knew it all along.”
The role of bias in analyzing predictions
It is easy to suffer the pitfalls of following the predictions of the numerous ‘hedgehogs’ that populate mass media nowadays. The markets, just like the future, will forever remain unpredictable. All the predictions that are made, whether scientific or instinctual, are essentially rooted in some form of cognitive bias and are probably driven by the possibility of making some personal gain. To base one’s investment decisions on ‘expert’ predictions is akin to sailing in a storm without a life jacket.
The advisable way to achieve one’s investment return goals is to have a clear objective and an innate ability to shut out the noise made by the so-called market gurus. Instead of listening and acting on the words of the ‘hedgehogs,’ it is more prudent to follow the analysis of the foxes who gear themselves for an uncertain future. These foxes will be advisors and managers who prefer diversification of risk and do not make claims about knowing what comes next. Instead of relying on instincts and emotions, an investor must trust those who take a calculated approach to mitigating possible losses and acting in time as the market moves.
While it might be considered a lot more amenable to do away with market gurus altogether, it is not possible in the current structure of information dissemination and market speculation. In certain markes, like Prediction markets where one trades the outcome of events and market prices indicate what all the investors think the probability of the event is, the role of the guru/expert is almost non-existent. There are many proponents for such markets but they remain a laggard to the popular markets where ‘crystal ball’ predictions still attract a majority of investors. These markets also have the largest market capitalizations and will always remain difficult to transform.
In order to facilitate diversification, there exists a broad range of assets and asset classes in the world now. It would be the rarest of rare occurrence for all of them to boom or go bust at the same time. By planning one’s investments wisely across these classes, based on a sound investment plan, would most likely achieve the long-term gains that any investor dreams of. While ignoring market predictions may not be easy, avoiding making knee-jerk investment decisions based on them can be easily avoided through discipline and consistency.
In Warren Buffet’s now famous article “The Superinvestors of Graham and Doddsville,” he presents indisputable data about how a specific number of asset managers who shared a common teacher have consistently managed to outperform the index over decades. None of them claim to have any supernatural prediction powers and neither do they boast of knowing something that others do not. They have followed their tested methods over a long time frame to achieve a goal they set for themselves. They dedicated their lives to achieving those goals and stayed true to the practices taught to them by Benjamin Graham. In the concluding paragraph of the article, Warren Buffett detailed an undeniable reality of the world which encapsulated the distracting and irrelevant nature of market predictions:
“There seems to be some perverse human characteristic that likes to make easy things difficult… It’s likely to continue that way. Ships will sail around the world but the Flat Earth Society will also continue to flourish.”
HCM-062122-64
