Forums  > Basics  > Testing trading algorithm  
Page 1 of 1
Display using:  


Total Posts: 13
Joined: Feb 2019
Posted: 2019-04-29 21:10

I have built a classic trend following strategy and backtested it on hourly data using quantstrat. It uses three components and none of them have been optimized in any way. It produced the following results:
- No of trades: 396
- Percent positive: 68,28%
- Profit factor(dollars made for each dollar lost): 1,59
- Avg. win trade: 814,74
- Med. win trade: 414
- Avg. losing trade: -1099,69
- Med. losing trade: -381,15
- Annualized sharpe: 1,42

I believe these statistics look good. My questions are:

Do you agree that this could be a viable strategy? Would you look at other statistics to determine whether it is?

How is it possible to in some way "stress-test" the strategy to in some sense see how it performs under other circumstances?


Total Posts: 1118
Joined: Jun 2007
Posted: 2019-04-30 07:19
That depends on what you mean by "tested".

I assume you tested it via back test on an out of sample data set? How large was that set? I mean 369 trades on a system that requires hourly data seem not to be that much of a sample.

Guessing from the median vs mean difference on wins and losses I assume your PNL is skewed in the right direction. That's a good thing.

Regarding stress tests you could look at "white's reality test", use Monte Carlo and or bootstrapping. You should definetly consider these things.

Just my humble opinion though.

Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...


Total Posts: 712
Joined: Jun 2005
Posted: 2019-04-30 10:44
> How is it possible to in some way "stress-test" the strategy to in some sense see how it performs under other circumstances?

try randomization of returns.

in stylized way:

returns = diff(prices)

for(k=0, k<n_trials, k + + )
returns_perm = random_permute(returns)
prices_perm = prices[0] + cumsum(returns_perm)
pnl[k,:] = apply_strategy(prices_perm)

You can also build some toy-MC, calibrate it with returns and find what you need by stressing model parameters. This method is longer to implement because you have to control ins and outs (for example, qq-plot of returns vs MC-returns). Basically, permuted returns is quasi-MC, where you conserve distribution of returns, perhaps also cross-correlations (if it is about portfolio), but destroy auto-correlation structure (which is behind any trend strategy) and marginal distributions.


Total Posts: 454
Joined: May 2006
Posted: 2019-04-30 14:10
> I believe these statistics look good.

Right now, they don't look like anything.

Forget stress testing for now. You are nowhere near that stage yet.

First of all, put in some realistic cost of trading and slippage.

Second, incorporate either closing positions out overnight, or incorporate overnight volatiliy.

Third, what is your universe, what period did you backtest on etc. Randomize that in some way. This looks like you tested ten symbols over one day, or something.

Profit per trade numbers are a bit big. Is that plus minus one thousand dollars per trade in intraday trading? What are you trading - million dollar lots?

The bad news is that this isn't a strategy.

The good news is that you have reached the stage where learning begins. Enjoy!

"There is a SIX am?" -- Arthur


Total Posts: 13
Joined: Feb 2019
Posted: 2019-04-30 17:09
> How large was that set?

Yes I did do a back test on an out of sample data set. The set contains 11300 observations. What is an appropriate amount of trades to draw conclusions for a system requiring hourly data? Just looking to get some intuition about these kinds of things :)

Thanks for the advice!


Total Posts: 13
Joined: Feb 2019
Posted: 2019-04-30 17:10
Thank you for the advice. I will keep that in mind!


Total Posts: 13
Joined: Feb 2019
Posted: 2019-04-30 17:32
> Cost of trading
Cost of trading was incorporated as a fee per trade appropriate for the exchange I am looking at(Binance), however, slippage has not been incorporated. I'll look into that.

> Closing out positions overnight
The market is Bitcoin(I know it's a shitty currency but I'm just trying to get some practice), and since it trades 24hr I would intuitively say that overnight vol is not relevant here? The positions should be opened on the open and closed on close for each hour as I set it up in any case though.

> Universe etc
The only asset is Bitcoin, trying to keep it simple since this is my first stab. Period backtested is a bit over one year.

> Trade size
As a matter of fact, I might be trading million dollar lots. Not sure if you are familiar with quantstrat but I set the trade size to 100 000 and the orderquantity to 10, so that might be the reason. In any case, I guess the percentages are what matter in this case, I am also aware of the fact that trading million dollar lots in Bitcoin isn't (I guess) feasible in real life.

>Randomization of returns
Also @nikol here. I just thought about that as you say @nikol, wouldn't that destroy the very basis of the strategy (since it is a trend follower)? Wouldn't I expect the strategy not to perform from the get-go with such a testing procedure?


Total Posts: 712
Joined: Jun 2005
Posted: 2019-04-30 19:24
You wanted to see performance of the strategy under stress-condition, so, you have it - trend disappears but strategy continues to trigger noisy and costly trades .

I also do inject completely random returns:

returns_random ~ Norm(0,std(returns_real)

Play MC multiple times and get distribution of PnL to get idea where it can end up in worse case etc. etc. Be aware, this exercise is only about losses, do not optimize on it.

Kaggle/OpenData has Terabytes of Bitcoin data.

PS. Some correction:
replace diff(prices) with log-difference: diff(log(prices)) and also
prices_perm = prices[0]*exp(cumsum(returns_perm))

PPS. Impact of correlation stress on PNL should be tested over 1-5 trades 'short' horizons or whatever is horizon when you think you are able to identify the trouble. If you run this test over a year, you will get huge and meaningless number.
'Short' range risk will tell you the level, when you have to stop in real life trading and rethink the strategy.


Total Posts: 13
Joined: Feb 2019
Posted: 2019-05-01 21:34
Thank you very much! I will try to implement your advice the best I can :)
Previous Thread :: Next Thread 
Page 1 of 1