Backtesting and shortfalls - survivorship bias

When backtesting trading ideas I usually focus on members of the DAX and MDAX indices, as those titles usually provide a sound liquidity and sufficient free float.

One of my systems traded snh.de (STEINHOFF) as this is (was part of the MDAX). Not surprisingly shortly after snh.de disclosed their financial problems my strategy was under water.

In March 2018 snh.de will leave the MDAX, as I don't keep a history of all Index members it turns out that this is a simplification in my backtesting approach, which is much misleading.

In March I would backtest the same strategy probably with a significant better result simply because snh.de is not longer part of the MDAX and I would not select this asset.

This is survivorship bias - so you might want to consider this in your backtests.

2 comments :

  1. I just read yesterday in a backtest just a brief disclaimer, that the backtest might contain survivorship bias. While the content of the blog is a great source for inspiration I believe that this is vast understatement. Given the current Steinhoff price I can clearly say that this strategy got ruined by this fraud, so if you think you don't need to take care or can safely ignore events like ENRON or STEINHOFF - remember this lines.

    ReplyDelete
  2. Now, fast forward 1,5 years - I have the very same problem with Wirecard - as this shares are (were) members of the DAX30. I know it is hard and maybe impossible to predict events like this and accomodate in your code but it is for me so critical that I will put a lot of research into this topic now. Diversification is only one solution and it is not my favorite one...

    ReplyDelete

Further Readings