Finance Questions, Answered: Can Academic Papers Help Investors Beat the Market?
Juhani Linnainmaa
Professor of Business Administration
Despite what Ken French argues in his research related to market volatility, academics have long been uncovering many trading rules that seem to promise a higher return than one would reap from investing in index or mutual funds. For example, a famous 1996 paper by Richard Sloan suggests that investors don’t properly distinguish between a firm’s earnings and its accruals (expected revenues or expenses), and thus the market overprices firms with high earnings. The practical rule to emerge from that paper says investors should sell companies with high accruals and buy companies with low accruals.
There are dozens of these so-called anomalies in the academic literature. The problem is, many of them don’t seem to work after they were discovered. “There are two possible reasons for this decay in profitability,” says Juhani Linnainmaa. “Maybe the effect was never real in the first place and was just a lucky finding in the data. Or maybe investors consulted the academic papers, started using the trading rules in them, and thereby wiped out the extra profits.”
Linnainmaa studies the first option closely in an award-winning paper published in The Review of Financial Studies. When an academic says an “effect was never real in the first place,” they are alluding to the issue of data-mining, or p-hacking. For decades, finance academics have used the same Compustat market data that goes back to 1963, testing hypotheses repeatedly, millions of times. If they find an effect that has a five-percent chance of being true even with false data, they claim it has a p-value of 5% and is statistically significant—in other words, that it would be unlikely to see this pattern just by random luck. But when academics do many tests, the p-values aggregate and can produce effects that look solid but are actually false. “When I teach Investments, I set up a spreadsheet where I create 10,000 nonsense trading rules, do the statistical testing, and show that I always find about 500 rules that seem to beat the market. This is the essence of p-hacking.”
The only way to verify if the trading rules in academic papers are real is to find new data. But Linnainmaa needed to find data that pre-dated the Compustat data that everyone was using, to avoid the influence of investors who might have traded using the rules from studies published after 1963. So he collected historical financial data going back to 1918, and tested the modern trading rules with it. “What we found is strongly consistent with p-hacking being a big concern,” Linnainmaa says. “We estimate that about 50 percent of the discovered and published trading rules are false, or that the profitability of the average trading rule that has been discovered was inflated by 100 percent in the original sample periods.”
The practical implication? Investors shouldn’t put much stock in the promises of trading rules in academic papers.
This article was originally published in print in Tuck Today magazine.