Investment Skill Measurement

Investment Skill measurement

This is probably the most important topic in investment management, yet the most neglected and misunderstood. It is critical to understand this whether you are a portfolio manager or overseeing allocation to external fund managers. While you need investment forecasting skill to generate alpha, alpha does not imply investment skill.

Before defining a measure of skill, let’s look at a critical and important ratio used in evaluating managers: the Information Ratio.

Information Ratio (IR)

The IR is the ratio of the value added divided by the volatility (or standard deviation) of this value added – also known as tracking error or active risk.

The following chart illustrates the IR of active US equity managers. The dotted line represents an IR of 0.5 at various risk level.

An IR of 0.5 seems to be a difficult result to obtain by investment managers over time. Some rule of thumb:

Question: Is good performance due to skill or luck?

From basic statistics, we know that:

Where N is the number of years

The rule of thumb to get statistical significance with 95% confidence is t = 2. How many years do we need to be able to assert if a manager’s positive performance is due to skill, not chance (with 95% confidence)?

     For managers with an IR of 0.50, we need 16 years!

     For managers with an IR of 0.33, we need 36 years!

     For managers with an IR of 0.25, we need 64 years!

And even after all those years, there is still a 5% chance that the positive performance was due to luck…

Most people do not fully understand the impact of the above on their portfolio construction and due diligence while allocating funds to external managers. When they are presented with the above analysis, they are shocked.

Active risk generally ranges from 2% to 12% with a bulk in the 4% to 8% range. The higher active risk levels are generally seen with style-biased managers measured against improper benchmarks.

Value added rarely surpasses 2% to 3%.

Simulation

Let’s assume two investment managers (manager 1 and manager 2) with the following return/risk profiles:

I did 10,000 simulations of such return distribution assuming the alpha of each manager is uncorrelated. The no-skill manager outperforms the skilled manager more than 40% of the time. And when he outperforms, it is by an average of more than 6%. Impressive for a no-skill manager!

But over time, skill will show. Or does it?

Many investors require a minimum of 3 to 5 years investment performance to consider an investment manager. This is well below the period required for statistical significance.

Looking at 2,000 simulations of 5 year performance, we get the following results:

As expected, the average return of the skilled managers is 2% (2.02%). We can interpret such simulations as if we would simulate the 5 year performance of 2000 equally skilled managers with expected return to risk ratio of 2% / 4%. 14% of these equally skilled managers had negative performance even after 5 years. Removing them from the average statistics have the effect of increasing the average return of the remaining luckier skilled manager to 2.45%. Note that all skilled managers in this simulation, the 14% we would drop, and the 86% we would keep, have the same future expected return of 2%.

As expected, the average return of the no-skill managers is 0% (-0.06%). We can interpret such simulations as if we would simulate the 5 year performance of 2000 equally NO-skill managers with expected return to risk ratio of 0% / 8%. As expected, 50% of these NO-skill managers had negative performance after 5 years. Removing them from the average statistics have the effect of increasing the average return of the remaining luckier NO-skill managers to 2.80%. Note that all NO-skill managers in this simulation, the 50% we would drop, and the 50% we would keep, have the same future expected return of 0%.

Therefore, if we eliminate from our candidates all managers with negative returns after 5 years, we eliminate only half of the no skill managers, and 14% of the skilled managers. Looking at managers who passed that first screen, we note that the no-skill-high risk managers have outperformed the skilled managers (2.80% vs 2.45%).

In real life, we do not know who is skilled and who is not. Therefore, based on historical performance alone, the no skill-high risk manager would still look very interesting.

We can get a bit tougher on our screen and focus only on managers with an observed IR of at least 0.50. Here are the results of the 2000 simulations:

It is good to see that the skilled managers have significantly higher probabilities of showing up in our screened universe but don’t forget that in real life, we cannot distinguish between the two groups. There are still one in seven no skilled managers with a very interesting IR, and because of the higher risk, these managers outperform by significantly more (5.95% vs 3.37%).

In conclusion, based on historical performance alone, it is very difficult to distinguish the skilled and no-skill manager.

Question: How many managers should be hired?

Most managers will tell you that you should hire just one: i.e. them. I don’t agree with that. Let’s go back to our simulation and let’s assume that we begin the simulation with 100 skilled managers, and 100 no-skilled managers. After 5 years, we retain only 50 skilled and 14 no-skilled for a total of 64 managers with an interesting IR. If we pick only one manager randomly, there is a 78% chance we pick a skilled manager (50/64), and therefore 22% chance of picking the no-skilled manager. If we pick the manager with the highest return, we will, with almost certainty, pick a no-skill-high-risk manager…

The solution:

Hopefully proper due diligence will increase the odd of finding the skilled managers (you should measure your investment skill at doing this) but we can’t rely only on that. We should hire more managers in order to increase the likelihood of being exposed to the skilled managers.

But if you still think that hiring only one good manager makes sense, I will still try to convince you otherwise, even after you hire me.

Investment Skill Measurement

Many investment managers sound like great stock pickers. The same is true about non-professional investors. All investors like to brag about their good investment moves and rarely talk about their worst investment decisions. When one hears such investors, they seem to have great forecasting ability:

It seems that when they forecast positive excess return (x axis), the subsequent return is positive (y axis). They seem to have a great crystal ball.

The reality is very different. For first quartile managers, a graphic of subsequent returns vs predicted returns looks more like the following:

While this graphic looks scary to most investors, there is some positive correlation between this manager’s forecasts and subsequent returns. It represents the average IC of a typical first quartile manager. The IC does actually fluctuate significantly from one period to the next but tends to be like the above for skilled managers. It is not strong, yet strong enough to be a first quartile manager if that forecasting ability is properly managed.

In order to outperform, a manager need investment skills. Investment skill requires two things:

1. Good forecasting ability
2. Good portfolio construction process

Information Coefficient (IC) as a measure of forecasting ability

The most widely known measure of forecasting ability is the Information coefficient (IC) which is the statistical measure of correlation between a manager’s forecasts and subsequent returns. In the previous section on Information Ratio, manager 1 had a positive IC while manager 2 had an IC of zero.

Typical IC for first quartile managers are:

• 0.05 - 0.06 for high breadth markets like the US
• 0.10 - 0.12 for low breadth markets

While these may seem very low, the return potential in the market is so large that when exploited properly, such low forecasting ability translates into significant performance over time. At the stock level, there is a direct link between alpha and IC:

Where σ_r stands for the cross sectional dispersion of stocks’ returns, and

             S stands for our standardized forecast

At the strategy level, we have:

Where:

             σ_r stands for the cross sectional dispersion of stocks’ returns,

             σ_t stands for the target tracking error or risk budget, and

             N is the number of independent bets possible (breadth)

Higher breadth markets (N) offer greater potential – but they tend to have lower ICs. We do not have much control on σr which fluctuates over time – although we could pick markets or segments of the market with higher return dispersion. Active skilled managers outperform better when the cross sectional dispersion of stock’s returns (σ_r  ) is higher. If we want to increase alpha, we need to invest in improving our forecasting ability. It is surprising that many managers don’t even measure their IC.

Investment managers would gain to have a person or group dedicated to Information Management which focuses on making better use of information and signals. Information management is much more than just IT. It focuses on improving a manager’s expected alpha.

IC used in determining a Portfolio Manager’s Bonus

In the section on IR, I simulated the performance of a skilled manager, and a no-skill-high risk manager. A similar simulation for negative-skill manager would show that some can deliver value added by pure chance. We saw that the performance and the IR of a manager can hide the absence of skill of that manager for a long period of time. A manager’s IC is a much better measure of a manager’s skill. I have long argued that a manager’s annual bonus should be linked to his IC which cannot hide a manager’s skill for very long.

Portfolio construction

Many active managers argue that if you have conviction in your forecast, you should have a concentrated portfolio of your best ideas. When you hear such an argument, you should immediately ask for an IC chart over a period of at least a year. Properly (and optimally) balancing expected return and risk, taking into account a managers’ typical low IC (even for first quartile managers) imply better diversified portfolios.

Not all managers know how to properly exploit their forecasting ability. I have seen managers with good positive ICs who neutralized part of their forecasting ability with improper portfolio construction. They would gain by spending time understanding the relationship between information, alpha, risk and how it transforms into portfolios exploiting their forecasting skill.

Everything we do in investment management is a risk budgeting exercise or an efficient frontier type of risk-return analysis. We have forecast returns or alphas, we have estimates of variances and covariances, and we want as much returns or alphas for a certain risk budget. The rest is mechanics… This apply to equity investments, fixed income investments, real estate investments, private equity investments, hedge fund investments, fund of funds investments, asset allocation investments and allocation to external managers..

Misconception

How many times did I hear “If you hire many managers, you get index returns”. This goes against Mathematics for Investment Management 101 ! The statement would be true, before fee, only if one has NO skill at picking skilled managers (i.e. the IC of the person hiring managers is zero…). I will elaborate on this in another text.

For more information on investment skill and its measure, please contact us.

Dominic Clermont, ASA, MBA, CFA