
Benefits and Pensions Monitor
Performance And Skill: Discerning The Signal From The Noise
Not Succumbing To Noise
The conclusion above may seem rather unsatisfactory. People tend not to like situations where experts in a field admit to limitations to the scope of their knowledge. After all, experts get paid for their knowledge, not for their modesty. However, appreciating that we cannot always distinguish between noise and skill within outcomes is a crucial understanding. Put more succinctly, our job – whether it is a fiduciary, an investment consultant, or a money manager – is very often to avoid succumbing to the impact of noise.
In conducting manager research, there are a number of approaches we take that recognize the difficulties discussed above:
When trying to predict prospective outperformance of active managers, we focus on process just as much as, if not more than, the outcome. We believe it is important to use qualitative research methods when carrying out investment manager research, in addition to quantitative techniques that focus on past track records.
We also believe it is important to consider factors that affect managers’ ability to shield themselves from the ‘noise’ in the marketplace and focus instead on the investment ‘signals.’ This only partly depends on the investment professionals directly involved. The business structure of the company, the culture that the investment firm’s leaders maintain, and the discipline brought to the investment process can all influence whether or not truly skilful investors are able to thrive and flourish in the long run.
When establishing long-term return assumptions for different asset classes, we rely on large and long-term datasets. This assumes greater stability in capital market conditions than within active manager return datasets.
When searching for active strategies that are likely to outperform, we may well favour those approaches that are less dependent on particular individuals, if the strategies have capacity. For example, we are relatively favourable toward the potential for non-capital, weighted indices to outperform capital-weighted benchmarks in a sustainable way given their potential capacity and independence from personnel change.
When reviewing an active manager, as well as ourselves, we recommend that clients use an array of different metrics, not just a single measure of past performance. We term this a ‘balanced scorecard’ approach which is captured in Chart 2.

Separating skill from noise is a challenge in all endeavours. One might think that with so many opportunities to quantify and dissect performance, investing would be an area where it would be easier than most in which to make this distinction. We would only caution that while reams of data are available, one must be very careful about the inferences drawn from past performance alone.
This challenge has been present ever since the first financial market data was published. We now have more sophisticated tools to apply to results that are distributed more widely and on a more timely basis than ever before. However, this has amplified the background noise, making signals harder to discern and, perhaps, skilful managers more difficult to identify. ■
Janet Rabovsky is practice leader, Central Canada, investment consulting, and Cameron Kerr is senior investment analyst at Watson Wyatt Worldwide.
|