Backtesting VIX ETP Strategies with Limited Data is Foolhardy

This has been covered many times in many other spaces, but the issue comes up all the time in my discussions with investors, so I think it bears repeating: designing strategies for trading VIX ETPs (like VXX and XIV) using only the historical data available since their launch, is foolhardy.

To illustrate, below is XIV (inverse 1-month VIX) since its launch in late 2010.

20140409.01

And here is XIV taken back to early 2004, using a combination of the indices and futures data on which XIV is based (read more).

20140409.02

A strategy designed to trade the first chart is likely going to be too strongly biased towards XIV (or short VXX), and too confident that that one significant drawdown is reflective of future drawdowns.

But a strategy designed to trade the expanded more complete dataset is going to be much more mindful of the huge risks that the inverse volatility trade actually poses, and much better at quickly shifting gears when the trade inevitably turns sour.

Designing strategies that actually work out-of-sample in the real-world is hard…like really hard. There are all sorts of cognitive biases that worm their way into our analysis no matter how hard we try to manage them. And even if they didn’t, markets are constantly evolving, turning even good strategies quickly bad.

Why then would you want to exacerbate that problem by both limiting the amount of data you have at your disposal, and limiting the types of markets that your data covers to just one: a bull market for inverse volatility.

The expanded data is not hard to acquire. Not using it is foolhardy.

* * *

Wonk notes:

  • The simulated (pre ETP launch) data of which we speak is only appropriate for what most refer to as “swing trading” strategies, or strategies that hold trades for multiple days (like ours). Any discrepancy between each day’s simulated value and where that ETP would have actually traded is small enough that it’s not going to have a significant impact on results.
  • The simulated data is not appropriate for a hyper-active strategy that relies on fine differences in the open, close, etc. I don’t see many of these in the VIX ETP arena, but it’s worth noting.
  • Had VIX ETPs actually been trading in early-2004, there’s a strong case to be made that VIX futures would have been significantly affected, making the simulated data less accurate and less valuable. I agree to an extent, but clearly the simulated data is not worthless. At the very least, it paints a broad picture of how VIX ETPs would have behaved, and clearly, having it, especially given the fact that it covers bull, bear, and sideways markets, is far better than not.

Good Trading,
Volatility Made Simple

Posted in Other Goodies.