What Do Psychic Animals and Mutual Funds Have in Common?

[Code for analysis available here, data from yfinance Python library]

Whenever there’s some big event happening like the World Cup, Olympics, or a presidential election, I often see headlines about psychic animals:

Sometimes the psychic animal seems to have a pretty impressive record. For example, Paul the Octopus, who predicted soccer games from 2008-2010, had a record of 12 correct predictions out of 14 (87.5% correct). This included a perfect record of 2010 World Cup predictions (image source):

Unfortunately, as you might have guessed, these animals aren’t really psychic. What’s probably happening is that for all of these events, many animals make some type of prediction. Then after the event is over and the results are known, the animals that happened to get it right by chance make the news. If you were to select some animals to watch before the results were known, or simply keep following the animals from these articles into the future, their performance would be consistent with regular probability from random guessing (unless there’s some kind of fraud or Clever Hans thing going on).

Eh, whatever. These stories are still cute and fun to read about, and probably most people aren’t taking them too seriously anyway.

---

Ok that’s enough about these cute psychic animal stories. Now on to a serious topic: financial investing. When you’re researching how to invest your savings, you might come across some articles like these, recommending mutual funds:

A mutual fund is a type of investment that contains a portfolio of stocks, bonds, and other securities. They’re usually¹ actively managed by financial professionals, who make decisions about the composition of the fund, and charge a fee of ~0.5% per year of the total investment. 

The way these articles from sites like Forbes, US News, and Nasdaq.com usually choose which mutual funds to recommend is by looking at past performance and picking the funds that have beat the market over the past year, 5 years, 10 years, or whatever time period. 

But wait… according to the Efficient Market Hypothesis it should be impossible for any investor to consistently outperform the market, other than due to luck. So what’s going on? Are these mutual funds really able to beat the market because they’re more skillfully managed and have better portfolios?

My guess is no. I’m willing to bet² that if you keep tracking the mutual funds recommended by these articles into the future, they will not outperform the market on average over the long run.

Can we test this prediction? Well for the articles recommending funds to invest in for 2023, we’ll just have to wait and see. But what we can do is go back and take a look at similar articles from the past. For example, Nasdaq.com’s list of the top performing mutual funds for 2016 is still up on their website. Here’s the top of the list:

It goes on to list 60 mutual funds total, chosen based on their 2016 return percentage. The numbers here look insanely good: 70.51% return, 55.65% return, etc. 

But how did these funds do in the year after the article was published?

I used the Python yfinance library to calculate the percent returns of these mutual funds in the year before the article was published, and the year after. I then compared these returns to the returns from SPY (a stand-in for overall market performance) over the same time period. The plots below show the distributions of fund-to-SPY return ratios. I put a vertical dashed line at 1, since a ratio of 1 means that a mutual fund had the same return on investment as SPY. Return ratios above 1 mean the fund beat the market, and return ratios below 1 mean the fund underperformed the market.

The top plot shows the return ratios in the year before the article was published. During this time period, the mutual fund returns substantially outperformed the market. The bottom plot shows the return ratios for the same mutual funds in the year after the article was published. For this time period, the vast majority underperformed the market.

A couple technical notes:

• Code for analysis available here

• The funds JHSVX, WTSVX, and ACRTX are excluded because I couldn’t find the data for them with yfinance.

• I used Adjusted Close here to calculate returns rather than Raw Close. I think this is the appropriate measure because Adjusted Close accounts for things like dividends and splits, but if anyone has a case for why I should have used Raw Close please let me know (I’m really not an expert with this stuff). 

That was the only article I found that had a long list of top-performing funds, enough to plot a distribution. But I also found a couple articles with short lists of top-performing funds, so I did a similar analysis on those.

Article

Publication date: February 6, 2014

Fund selection criteria: 1-year performance

Article

Publication date: October 15, 2015

Fund selection criteria: 10-year performance

Again, the numbers here are the fund/SPY return ratios, so a number above 1 means the fund beat the market, and a number below 1 means the fund underperformed the market.

So, what do psychic animals and mutual funds have in common? Both look pretty impressive if you select the ones with the best performance after the results are already known. But when you do this, you’re really just selecting the ones that got lucky, and if you continue following them into the future, the impressive results evaporate into the underlying probability distribution.

In this post I decided to focus on mutual funds, but this type of probabilistic fallacy, where results from random chance are misinterpreted due to a selection effect, is really a general problem found in many different fields besides just finance. For example, a closely related problem in academic contexts is called HARKing, an acronym coined by Norbert Kerr that stands for “Hypothesizing After Results are Known.” This is thought to be one of the causes of the replication crisis, and is why preregistration of experimental studies is so important.

Further Reading

• NYT Article by Jeff Sommer: “Mutual Funds That Consistently Beat the Market? Not One of 2,132.”

• A Random Walk Down Wall Street by Burton Malkiel – This book is basically the opposite of the “get-rich-quick” investing books. It advocates for a passive investing strategy (index funds and index-ETFs) rather than trying to beat the market through stocks or actively-managed funds, which he argues is impossible in the long run. Here’s a lecture the author gave if you don’t feel like reading the book.

• Academic paper on HARKing

1. Technically index-funds (which are passively managed, have a lower fee, and track an index like the S&P 500) are also a type of mutual fund, but for the purposes of this article I’m really talking about actively-managed mutual funds.

2. Seriously, if anyone wants to make an actual bet on this let me know!