Forums  > Trading  > Statistical significance of indicators with long lookbacks  
     
Page 1 of 1
Display using:  

trapezoid


Total Posts: 1
Joined: Oct 2012
 
Posted: 2012-10-16 10:34
Let's say we have a trading system that trades daily, holding for one day, but uses an indicator that looks back over the last 5 years. A simple example could be the percent change in price of an asset over that 5-year period.

There will be a high degree of serial correlation in the indicator values from one day to the next, due to the overlap of the look-back. Are there standard ways of taking this into account when assessing the performance of a trading system?

Intuitively, it seems that for a fixed length of backtest data (say 10 years) we should have more certainty of our performance metrics, such as Sharpe, when we have short look-backs, since then the samples of the indicator distribution are more independent.

Can anyone suggest a way to quantify this in the performance results? Or is it more of a rule-of-thumb situation?


Pike


Total Posts: 12
Joined: Jun 2012
 
Posted: 2012-10-16 11:05
If you play this strat every day, holding position for a day, your real holding period will be something like a year...

Indeed there will be a high degree of autocorrelation in your signal and because of that the strategy will stay long and short during long period: the turnover will be low.

If T is the characteristic time for your indicator to change sign, the holding period of your strategy is related to T: the performance metrics will be comparable to performance metrics of strats in the same frequency spectrum.

An example: if you trade every second a mid-freq strat with holding period of a week -indicators change every week on average- you do not expect the Sharpe ratio to be comparable with a high freq strat that has a holding period of a one sec.

My 2 cents...

xfd


Total Posts: 11
Joined: Mar 2008
 
Posted: 2012-12-07 20:56
Pike's point "If T is the characteristic time for your indicator to change sign, the holding period of your strategy is related to T: the performance metrics will be comparable to performance metrics of strats in the same frequency spectrum." is the right idea, I think.

For simplicity let's say your indicator each day is either +1 or -1, which maps to trades in the obvious way. An easy parametric way to think about testing for significance is to ask "is there statistical evidence to reject the hypothesis that the distribution of returns for which I=+1 is equal to that for which I=-1". If your indicator is worthwhile, those distributions should look different (for instance, have different means).

A t-test for example asks that question but about the means of the two groups and under various (probably wrong) assumptions. A permutation test is another approach -- a nice thing here is that you can do a permutation test for any statistic/performance metric you wish.
Previous Thread :: Next Thread 
Page 1 of 1