Gary Karz, CFA (email)
Host of InvestorHome
Principal, Proficient Investment Management, LLC
Survivorship bias is an important issue that needs to be addressed when analyzing past performance. It has been the subject of a much important research recently and it continues to be debated extensively. Survivorship bias results from the tendency for poor performers to drop out while strong performers continue to exist. Thus when analyzing past performance of mutual funds for example, the sample of current funds will include those that have been successful in the past, while many funds that previously existed but underperformed and were closed or merged are not included. Survivorship bias results in an overestimation of past returns and leads investors to be overly optimistic in predictions of future returns.
Researchers have demonstrated in recent years that survivorship bias can play a significant role in biasing past returns of individual securities, mutual funds, and even equities of specific countries. For example, Professors
William Goetzmann and Philippe Jorion point out that the high historical returns from American stocks may represent an exception rather than the rule when evaluating equity premiums in a worldwide context. Their paper Global Stock Markets in the Twentieth Century
(earlier version) in the Journal of Finance (June 1999) was a Smith Breeden Prize winner (awarded annually to the top papers in the JF).
Some LBO Fund sponsors also had a tendency to leave out all the details when reporting their performance. In "The facts, but not all the facts" (Forbes 3/9/98) a telling quote from the article was "There is no legal requirement that the dealmaker must present a full picture of his or her career.
A technique that some companies use in launching new products is to "incubate" funds. For example, a company wishing to launch a new series of mutual funds might provide ten managers with a small amount of seed money to start aggressive funds. Each manager is given two years to test their stock picking ability. At the end of the period several of the funds are likely to have outperformed. Those successful funds are then made available to the public and marketed aggressively while the losers are silently discontinued. This is what is known as "creation bias."
Once you have an appreciation for the effects of survivorship and creation bias its easy to see how companies can effectively guarantee long term records of outperformance. By starting with a large number of funds and discontinuing or merging the poor performers, a company is left with a stable of cherry-picked winners. Even if a surviving fund only matches the returns of the market going forward, its long term record will continue to be better than the market as a result of the initial outperformance.
Another issue that investors should keep in mind is the tendency for strong performing funds to grow rapidly which has many implications. Many funds that outperform and experience large inflows, subsequently are unable to repeat the performance with substantially higher assets under management. Its also common for funds that outperform over short periods of time to turn around in the following period and underperform. The underperformance frequently occurs with more assets under management. The result is a fund with an average track record, but overall investors in the fund underperformed. See Nothing Fails Like Success by John Bogle.
The following is a simplistic and fictional example, but similar scenarios occur regularly in the investment business. A little-known stock fund with $10 million in assets experiences a period of significant outperformance rising 50% over the course of a year. As a result, the fund (now $15 million in assets) receives a significant amount of publicity and investors flock to the fund. The fund quickly grows to $150 million. The fund then proceeds to lose 10% over the following year.
What is the funds return over the two years? A positive 35% (a little over 16% annually). Yet investors in aggregate have lost more money in the fund than they've made. Gains in the first year were $5 million while losses in the second year where $15 million. The fund has a 2 year track record of 16% annualized returns but investors in the fund are down $10 million. The example demonstrates the difference between time-weighted and dollar-weighted returns (See also
What are Stock Investors' Actual Historical Returns? Evidence from Dollar-Weighted Returns from
Ilia Dichev.)
These issues demonstrate the importance of being skeptical of performance claims, particularly when the claims are coming from the company itself. Also keep in mind that there are numerous ranking systems which increases the likelihood that each fund will be highly ranked by someone over some period of time. Past decisions by the NASD and SEC regarding track records also complicate some of these issues. In many cases track records will be movable which could create more confusion.
Firms that comply with the
Global Investment Performance Standards (GIPS®) from the CFA Institute are generally not subject to the cherry-picking problem. They requires all accounts under management to be included in at least one composite that is included in their performance reports. Therefore firms that comply can't hide poor performance from their record.
Hedge Funds provide another unique example of the importance of evaluating survivorship bias. A very recent and well written summary of the major bias issues with Hedge Fund performance reporting can be found in Hidden Survivorship in Hedge Fund Returns from Rajesh K. Aggarwal and Philippe Jorion in the March/April 2010 issue of the Financial Analysts Journal. Hedge fund managers typically report their funds’ performance to databases only voluntarily. Therefore the common indices of hedge fund returns usually have two types of biases. First, "backfill bias, which occurs when a fund’s performance is not made public during an incubation period but is added to a database later, presumably following good performance." Second, "survivorship bias, which occurs when funds that no longer report are dropped from the database of 'live' funds." Malkiel and Saha (2005 - see below) found each of those averaged over 4%. Aggarwal and Jorion discovered a second major problem that could have inflated a prior database by another 5%.
Burton G. Malkiel and
Atanu Saha's paper was titled Hedge Funds: Risk and Return and appeared in the Nov/Dec 2005 edition of the Financial Analysts Journal. See this version, especially page 43, exhibit 15 which lists mutual fund and hedge fund attrition rates from 1994 to 2003). They reported that "hedge funds are riskier and provide lower returns than is commonly supposed." Further, "Investors in hedge funds take on a substantial risk of selecting a dismally performing fund or, worse, a failing one." See also Hedge Funds: Risk and Return - A Financial Analysts Journal Media Seminar with Atanu Saha.
Hedge Fund Attrition, Survivorship Bias, and Performance: Perspectives from the Global Financial Crisis from Xiaoqing Eleanor Xu, Jiong Liu, and Anthony Loviscek (2/12/10) discusses hedge fund performance through March 2009. In
Why Do Hedge Funds Stop Reporting Performance? in The Journal of Portfolio Management, Fall 2007, (earlier version), Alex Grecu, Burton G. Malkiel and
Atanu Saha summarize that "most funds stop reporting not because they are “too successful,” but rather because they fail." See also Survival, Look-Ahead Bias, and Persistence in Hedge Fund Performance from G. Baquero, Jenke Ter Horst, Marno Verbeek.
See also
Please send suggestions and comments to Investor Home