Discussions related to success rates in active management are fairly common. For instance, at year end a frequent discussion is the percentage of funds that beat their respective benchmarks like the S&P 500, Wilshire 5000, and/or region/style averages/benchmarks. But there have not been many thorough discussions of predicting success rates and odds in active investing.
The earliest discussion that I'm aware of was in The Evolution of Passive versus Active Equity Management by Larry L. Martin in The Spring 1993 issue of The Journal of Investing. The relevant probability calculations were cited and summarized in Fact and Fantasy in Index Investing by Eric Kirzner (January 2000). See pages 27 & 28 for "Probability of Active Management Outperforming an Index." The probabilities for a single manager outperforming an index were 41% in 1 year, 29% in 5 years, 22% in 10 years, and 14% in 20 years.
In How A Second Grader Beats Wall Street, Allan Roth also discussed his analysis of this topic. After being asked to create a Monte Carlo simulation for John Bogle, Roth ran a simulation of mutual funds versus index funds. Roth did not list all his assumptions (varying past performance, cost, volatility, and correlations can significantly affect the results), but Roth does state that the index fund had a 0.23 percent total expense and the average mutual fund or separately managed account had a 2.00 percent expense ratio. Richard Ferri summarized the results in A Winning Fund Doesn't Equal A Winning Portfolio (4/22/2010) in Forbes.
Roth's estimate is that one active fund has a 42% chance of beating an index fund over a 1 year period, 30% over a 5 year period, 23% over a 10 year period, and 12% over a 25 year period. So we find that the Martin and Roth probabilities are virtually identical. Roth compares active funds to an index fund rather than an index, and Martin's is a statistical estimate, while Roth's is a Monte Carlo analysis. Yet despite those minor differences and at least partially different time periods, the similarity in the results suggests the results are very strong estimates and they tend to be confirmed by actual results.
Martin and Roth both came up with even worse odds if you have multiple funds/managers.
In his forthcoming book
The Power of Passive Investing: More Wealth with Less Work, Richard Ferri takes an in depth look at the odds of multiple fund portfolios outperforming passive funds from a theoretical perspective, as well as the empirical evidence. Roth also analyzed a higher cost differential to account for tax implications and emotions. Numerous studies have shown that investors get even worse dollar weighted returns than time weighted returns because they tend to buy and sell at the wrong times. An early study on that topic was Buy High, Sell Low: Timing Errors in Mutual Fund Allocations by Stephen L . Nesbitt in the Fall 1995 issue of
The Journal of Portfolio Management. Nesbitt found that timing costs investors 1% per year. More recent studies have come to similar or even larger gaps (For instance, see Terrance Odean and Brad Barber's The Common Stock Investment Performance of Individual Investors (or here)). See also The Zeroes.
There are many sources of actual results, one of the most useful being Standard & Poor's Indices Versus Active reports (2009 and 2008). The tables include Percentage of U.S. Equity Funds Outperformed by Benchmarks, Percentage of International Equity Funds Outperformed by Benchmarks (note the increasing %s over time), Percentage of Fixed Income Funds Outperformed by Benchmarks (note many categories are high 90s-100% at 5 years, which is consistent with low returns versus standard deviation and relatively high costs), and Survivorship and Style Consistency of Fixed Income Funds. Another source is Morningstar's Box Score Reports (BSR). For instance, here is their 2Q09 BSR.
I began researching the probabilities and odds of active management in the late nineties and the specific piece of information that kick started me was a section in the second edition of Jeremy Siegel's best-selling book Stocks for the Long Run. Siegel included a table with holding periods ranging from 1 to 30 years and expected excess return ranging from 1-5%. Siegel used the table to discuss finding skilled managers. In other words, given an expected excess return over the market and a specific number of years, the numbers project how statistically certain you could be that the performance was due to skill and not luck. Siegel's assumptions (based on data from 1971 to 1996) were a 14 Percent Expected Return, 16.6 Percent Standard Deviation, and 0.88 correlation coefficient. The return assumption is high, but given the length of the period, the assumptions should be reasonable for projecting the future.
While Siegel's discussion is interesting regarding the Luck vs. Skill debate, my twist on the topic is that it is a good tool for evaluating probabilities of success in active management if you substitute the investment costs (negative) for the expected excess return (positive). In other words, Siegel's expected alpha relative to the market is analogous to the inverted disadvantage of active investors due to costs. I see the luck/skill analysis and probabilities of success in active management as two sides of the same coin.
For example, with a 2% expected excess return, Siegel's numbers suggest that you can be 70.1% certain after five years (and 90% after 30 years) that outperformance was due to skill and there is a 30% chance it was due to luck. Using Siegel's data, I would say an investor (or fund, or manager) with a 2% disadvantage has a 30% chance of beating the benchmark after five years and 10% after 30 years.
At a 2% expected excess return, after one year Siegel's estimate is 59.3%, 5 years 70.1%, 10 years 77.2%, 20 years 85.4%. Assuming an inverse -2% for costs, we get 40.7%, 29.9%, 22.8%, and 14.6% projected outperformance projections. Comparing those rates, to the Martin and Roth estimates, we find that again they are virtually identical. The Fourth Edition of Stocks For The Long Run includes an updated version of the discussion (with more years of data), but does not include the assumptions. In the fourth edition, the probabilities are generally closer to 50% (+2% for 5 years changed from 70.1% to 66.6% and 30 years changed from 90% to 85.3%). Assuming an inverse -2% for costs, the 40.7%, 29.9%, 22.8%, and 14.6% outperformance projections (derived from the second edition data) changed to 42.4%, 33.4%, 27.2%, 19.6% (using the fourth edition data).
The assumptions are important, but ultimately this is a game of guesstimation. Yet we have a lot of empirical data to verify whether the numbers pass the smell test and in general they do quite well. We can't expect these to be perfect estimates for several reasons. For one, manager behavior changes over time depending on how well they do. Managers that do well tend to attract more assets and tend to become more index like due to either bloated assets, reduced risk tolerance from an incentive to stabilize their income, and the desire to not risk underperforming when they already have a lead on their benchmark (if they are ahead of the benchmark, they only need to match it going forward to have a long term track record of outperformance). Therefore closet indexing is common. But managers that fall behind tend to increase their risk and position sizes in an attempt to get positive, since they are likely to lose assets or get fired if they maintain their underperformance. Plus you have staff departures, and the possibility of changing conditions over time.
The next step is to evaluate using these tools to make estimates for other cases of active management. If we can build models for general equity mutual funds we should also be able to build models for predicting success of active bond investors, hedge funds, and sub categories of equities (styles, market caps, foreign). We should also be able to estimate how changes in the variables affect the results. For instance, what are the odds that a mutual fund with a front end load will beat the market (as opposed to a comparable no-load fund)? And how much do small changes in annual expenses affect the probability of beating the market? To be continued...
