Not logged in. Login | Signup

Should you try to beat the market?

2019-10-26, Michael Thompson

Share on facebook Share on twitter

Become an insider
Get the latest updates.

Should you try to beat the market?

This article could save you a lot of time. We want to show you that attempting to beat the returns of a market (e.g. the S&P 500) is mostly futile. Most of you know someone who claims to beat the market significantly, year in and year out. However, a deeper look including all his assets, taxes and fees over an extended time will normally reveal the opposite. With a few exceptions provided down below, most of us would be best served with a simple collection of index funds.

Quick Analogy

Thanks to Mark Perry for this analogy. Let's say you need to take an exam like the MCAT. You're facing 10-15 hours of studying per week for the next 6 months. However, someone informs you of an alternate option. You can instead spend no time and receive a 95 percentile score. That's kind of what an index fund does for you (95% of experts don't beat it).


In whatever industry you choose to invest, there are thousands of experts studying it. It’s their full-time job and many work 60+ hours a week looking for assets that may be slightly undervalued. They have a lot more training than most of us, they have teams to bounce ideas off of and catch errors, and they have access to other information and resources we don’t. As soon as they identify an undervalued asset, large sums of money are invested causing the price to increase (and no longer be undervalued by their analysis). When you buy an asset, do you really think your analysis is superior to this? As Ray Dalio (founder of Bridgewater Associates) says: to beat the market you'll have to bet against this expert consensus and be right!

Successful active investing and luck

If you've beaten the market over the last year or two, you may conclude you will continue this feat indefinitely (or at least often enough to profit). About half the people that try are successful the first year, about a quarter are successful both of the first two years, but only a small fraction succeed over longer timescales. Depending on the study (and there are a lot out there, e.g. this one) less than 5 to 10% of professional investors beat the market over a 10 year period. It's even worse for nonprofessional active investors – studies indicate they beat inflation by less than 1%! Of the small fraction that do beat the market, most is attributed to luck. How can we say this? Because statistically there's little relationship between success one year and success another year.

If skill were a dominant factor in investing, then skillful investors would beat the market consistently (and unskilled investors would fail consistently). E.g. a world class tennis player will beat an average player well over 50% of the time, more like 100% of the time. However, numerous studies like this indicate that beating the market one year has little or no relationship with beating the market in another year (i.e. a poor performer last year is just as likely as anyone else to be a high performer this year). This doesn’t necessarily mean no one has investing skill, but that the number of people who do and the outperformance they achieve is very small. Warren Buffett even won a bet about this, described here. Even Warren himself - widely considered THE greatest investor in the world – now fails to beat the market with any consistency, see this article. Furthermore, this article points out that, even if skillful investors are out there (or if you yourself are one), you probably wouldn't even know. With some reasonable assumptions, they show that it would take about 150 years to be 95% confident that you're more than just lucky at beating annual market returns.

Many read this type of information but think they are different. They are smarter, they have a winning strategy, and they are part of a tiny group that wins over the long term. They can even point out the reasons why. Maybe it’s due to an advanced education or maybe they outperformed on high school math tests. This is likely due to well-known psychological weaknesses and brings us to the next section.


Human psychology is notoriously bad for playing the stock market. Our emotions are more likely to drive us to buy high and sell low than the opposite. They can give us a false sense of confidence and lead us to excessive risk. There’s too much to cover in this article, but we implore you to read more about this.

Taxes and fees

Even if you’ve picked assets that outperformed the market, if you’ve traded excessively and/or executed short term sales then the costs of taxes and fees may knock your net performance below what it would have been by simply holding an index fund. For more information about these "frictional costs" read this article from Forbes or this article by Joshua Kennon.

See this table showing how taxes alone sink the returns of an active trader.

What about my friends and neighbors who beat the market?

Most of you know someone who claims to beat the market significantly, year in and year out. These claims are often based on a short time frame or a subset of their assets. It's another psychology thing – our mind will automatically highlight our better results and disregard the inferior ones (read more here). I've seen people brag about beating the market, only to find out in later discussions they were referring to just one of their many investments (about half of your investments should beat the market!). If questioned about this, they'll often have a reason why the others failed and why it won't happen again (e.g. "I wasn't using this strategy when I bought those" or "I was just following Tim's advice when I did that"). In most cases they are not intentionally lying, but their psychology has done its job, convincing them that they are superior (remember, 90% of us believe we are above average).

If you ask enough people, you will find some that have truly beaten the market 10 or more years in a row (just as you'd find people who profited from the lottery). Most of us will accept that lottery winners are just lucky, but we may not recognize a lucky investor.

Payoff from your maket beating strategy

Let's say you spent 10 hours a week and found a formula to beat the market by 2% after fees and taxes (that’s a lot). If you’re only managing your own money (not investing for others as a living) then how much have you really gained? Let's say you were confident enough to put $100k behind your formula. Your hard work will pay you $2k for the year or about $3.84 per hour! You’d probably be better off working fast food or developing job skills. (It’s unlikely the formula will work as the market evolves in future years.)


Every hour you invest in active trading is an hour lost to other activities. Using that time to learn new job skills will likely be much more profitable in the long term. Spending that time with family or friends may rejuvenate you and make you happier, healthier and better performing in your career.

Exceptions: Cases for active investment

Not all investment should be passive. Three key exceptions are provided here.

Exception 1: Professional active investors

If everyone held cap-weighted index funds companies would keep their relative market caps indefinitely, regardless of performance. If Sears was worth 3x more than Amazon when this occurred, it would still be worth 3x more today! Clearly that wouldn’t make sense. Active investment keeps relative valuations within reason.

Exception 2: Promoting your ideals

Even if your political vote doesn't result in desired legislation, you're not completely powerless. Buying stock or bonds in socially or environmentally responsible companies is an alternative way to promote your ideals (as is buying the products of such companies). Indices have even been created around this, e.g. MSCI ESG. Vanguard provides an actively-managed fund VEIGX to facilitate this.

Be careful though: many have lost significant capital supporting beloved companies.

Exception 3: Entertainment or alternative to more risky behavior

Some people have fun researching companies, trying to pick winners or developing algorithms to do this for them. This is probably better than putting the same research and money towards racehorses or slot machines, but it's still risky. If you do this, keep an eye on risk and reserve a fixed amount of your capital for this.


In conclusion, the market is complex and stochastic. Stock picking is healthy to ensure money flows to the "right" businesses. However, unless you devote your life to it, you will probably only beat the market with luck. If you enjoy trying to beat the market, want to devote your life to it, or just want to use it as a voting machine then go for it! Otherwise, it's healthiest for most of us to spend just enough time to develop a simple strategy that closely follows the market. This article explains just how to do that.

Login to leave a comment.

Related Articles

Psychology and investing

Investing Basics: A most concise guide

Click here for a list of other recent articles.