Type 1 & Type 2 Errors in Investing: Improving our Odds and Reducing Mistakes
- Rupam Deb
- Sep 7, 2023
- 7 min read
Updated: Nov 13, 2024
Our lives would be a lot simpler (but less interesting) if all our decisions had a guaranteed favourable outcome.
Unfortunately, that is not how the world works. Some decisions come with unpredictable favourable or unfavourable outcomes, with varying odds. Investing is one of them.
To improve our overall investing results, we need to increase the favourable outcomes and reduce the unfavourable ones. When achieving both simultaneously is difficult, there is a simple framework that can help us decide where to focus our efforts.
My sincere thanks to Gerd Gigerenzer, Nassim Taleb, Daniel Kahneman, Warren Buffett and most recently Pulak Prasad (Nalanda Capital), whose writings helped me to appreciate the elegance of this framework.
Type 1 & Type 2 Errors In Investing
There are two types of errors that we commit in our day to day lives – Errors of Commission (Type I Errors) and Errors of Omission (Type II Errors).
Understanding the impact of these two types of errors not only can significantly improve our investment performance, it can also help us to make better decisions in other domains.
These error types can be understood with the simple example of a maths exam that awards +5 for correct answers and penalises with a -5 for incorrect answers.
In this context:
‘Errors of Commission’ (Type I Errors) would be solving a problem incorrectly and scoring -5
‘Errors of Omission’ (Type II Errors) would be missing answering a problem (whose solution was known) altogether, and foregoing the benefit of scoring marks
For exams that are very easy, where one is reasonably confident of solving most of the problems (high probability of favourable outcomes), it would make sense to maximise the number of problems one attempts. Here chances of Type I errors are less and Type II errors or omission could prove costly
However, for a very difficult exam, where most of the problems are unknown (high probability of unfavourable outcomes). Here errors of commission could affect overall scores very adversely, so it would make sense to only attempt the very few that one is confident of solving…and in the process avoid accumulating the negative marks from the rest. In such an exam, even scoring a zero is not going to be the worst outcome.
The world of investing is similar to the ‘difficult maths exam’ (there are some differences which we will discuss in the subsequent parts of this article).
Imagine a total universe of 1,000 listed companies. Out of these 1000, let’s imagine only 10% or 100 companies are good investments (profitable) and the remaining 900 are bad (unprofitable). In the real world, the actual % of really good investments would probably be even lower.
Let’s say you are a very savvy investor who is right 80% of the time in all your investment related decisions. In other words, 80% of the time you can sniff out a bad investment and reject it…and similarly 80% of the time you are able to spot a good investment when you see one, and invest in it…So your rate of Type I as well as Type II errors are both 20% respectively.
In this scenario, when you actually make an investment into one of the listed companies, what is the probability of it being a good investment? Most people will say 80%, but that will be very far away from the right answer.
Read on to understand why…
Your 20% Type II error will result in you rejecting 20 out of the 100 good investments and you end up considering only 80 of them for investing. You forgo the benefit from the other 20 due to the ‘error of omission’
On the other hand, your 20% Type I error will result in you identifying 80% of the bad companies correctly (720 out of 900) and staying away from them … but failing to identify the remaining 20% (180 out of 900) bad ones. So you would mistakenly consider 180 of these bad companies for investing.
So your investment shortlist comprises 80 from the ‘good list’ and 180 from the ‘bad list’, or (80+180) = 260 companies. When you make an investment from this shortlist, it would result in a probability of success of only 80 (good ones) 260 (total) = ~31%…which is dramatically different from what one would have guessed.
Just like the difficult maths exam, this divergence of actual performance from ‘skill’ simply happens due to the fact that an overwhelmingly large number of investments are bad ones.
While Improving ‘skills’ definitely helps, understanding this probability framework and knowing where to focus your efforts can help you even more.
Impact Of Reducing Type 1 Vs Type 2 Errors
In the paragraphs above, we examined how the actual probability of investing success dramatically diverges from skill levels, depending upon the investable universe. In the earlier scenario discussed, we assumed a total universe of 1,000 listed companies. Out of these 1000, only 10% or 100 companies were good investments (profitable) and the remaining 900 were bad (unprofitable).
In the next part of this article, we examine how changing the universe of investments can affect our overall investment performance, given our Type I and Type II error rates. We then discuss the framework for evaluating the change in investment performance, by reducing our respective error rates.
The dramatic difference in results between reduction in Type I vs reduction in Type II errors is quite an eye opener.
Let’s say we find a different ‘pond to fish’ with a slightly better quality of fish. We find a universe of 1,000 listed companies, where 15% (150 out of 1,000) of the companies are good investments and 85% (850 out of 1,000) of the companies are bad investments. Your investment skills remain exactly the same (20% type I and type II errors)
In this universe, your 20% Type II error will result in you rejecting 30 out of the 150 good investments and you end up considering only 120 of them for investing. You forgo the benefit from the other 30.
On the other hand, your 20% Type I error will result in you identifying 80% of the bad companies correctly (680 out of 850) and staying away from them … but failing to identify the remaining 20% (170 out of 850). So you would mistakenly shortlist 170 of these bad companies for investing.
So your investment shortlist comprises 120 from the ‘good list’ and 170 from the ‘bad list’, or (120+170) = 290 companies. When you make an investment from this shortlist, it would result in a probability of success of only 120 (good ones) 290 (total) = ~41%
So you can see, by finding a slightly better quality universe, you can dramatically improve your chances of making a profitable investment, with the same set of skills.
Before we discuss how to find the ‘better universe’, let’s see how improving our respective skills of selection and rejection can affect our results.
Skill Improvement Scenario 1: You manage to improve your hit rate of selecting good investments from 80% to 90%, thus reducing your error of omission (Type II error) to only 10%). Your error of commission (Type I) however remains 20%
Skill Improvement Scenario 2: You manage to improve your ability of rejecting bad investments from 80% to 90%, thus reducing your error of commission (Type I error) to only 10%). Your error of omission however remains 20%
Let’s stick to the original universe of 10% good investments and 90% bad investments and compare the two improvement scenarios.
In scenario 1, because of the improvement in Type II error you now miss only 10% of the good investments and shortlist 90 out of the 100 good investments. However as your Type I error is still 20%, you erroneously select 180 out of the 900 bad investments … the total shortlist being 90+180=270. So your probability of winning = 90 (good) ÷270(total) = ~33%
In scenario 2, because of the improvement in Type I error to 10% you now erroneously shortlist only 10% of the bad investments i.e. 90 out of the 900 bad investments. However as your Type II error is still 20%, you miss 20 out of the 100 good investments. So in your shortlist you have 80 good and 90 bad investments … the total shortlist being 80+90=170. So your probability of winning = 80÷170 = ~47%
Do you notice the dramatic difference between the two scenarios? While an improvement in the Type II error rate gives you a measly benefit of just 2% (from 31% as we saw in Part 1 to 33%), an improvement in your Type I error rate improves the probability of your performance by a dramatic 16% (from 31% to 47%).
Here is an even more interesting and unexpected result, If you manage to improve both the error rates to 10%, then the probability of making a winning investment becomes 90÷180 = 50%…so just a 3% benefit from the Improvement scenario 2.
So in a universe where only a small % of the investments are good investments, the real benefit comes from reducing our Errors of Commission (reducing bad investments).
Quoting Pulak Prasad from his book ‘What I Learned About Investing from Darwin’ (one of the best I read recently and would highly recommend)
“ … whereas most investment books and college curricula focus on teaching how to make good investments, everyone would be better off by learning how not to make bad investments. An investment career is probably among the very few that rewards the sceptic more than the optimist.
Buffett is the best investor in the world because he is the best rejector in the world.” (highlighted by me for emphasis)
In the Part 3 of this article, we will examine
How we could changing our investing universe to enhance the probability of winning
How we could improve our odds despite our Type I & Type II errors
Comentários