Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
Slippage and Commission recommendations
I am noticing that while OOS performance in GSB impressively matches IS performance, when a good strategy is forward tested on a demo account,
performance is more often than not flat or losing. This is the case with both TS and MC, on NQ, CL, and ES.
I suspect the reason has to do with configuring slippage and commission in such a way that strategy calculations in forward testing resemble those in
backtesting. So far I have tried:
- Commissions set to both the actual total commission cost, TradePerSide, and set to zero. I suspect Commission set to zero is best, because it
wouldn't affect the profitability calculation in a live environment; the commissions are deducted after the fact.
- Fitness and Report Slippage set to both 1 tick @ TradeRoundtrip and 1/2 tick @ TradePerSide
It has been my experience, at least with MC and a colocated server, that there is rarely any true slippage during prime trading hours, but you lose a
tick because of the bid/ask spread. I have tried using MC "Trade" historical data (which is what TS supplies) as well as Bid price historical data,
but neither produces more reliable results.
Am I missing something? Do any of you have recommendations on how to make forward testing better resemble the profitability of OOS in GSB?
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Good questions. There are lots of issues to discuss.
A few observations. ES range is extremly low, and markets at new highs. This is highly likely to give poor trading results.
Years 2005 on ES are similar. There is a good chance this is the reason for poor oos results, more so than the results are OOS.
Of the 3 GSB ES systems supplied with GSB, I think 2 are at new highs. (I need to double check this)
I would expect the same issues for NQ. I would check how well CL is going for range too.
Im looking at abs of closed(0) - closed(1) and range of high - low.
Another obersvation in the current year is fitness of high PF is much better in current enviroment that fitness of
high corelation to a straight line.
Slippage is a part of trading, but if profit per trade is high its less significant.
I use some slippage in fitness to increase average trades.
Peter
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Ive been thinking about this more.
Did the systems pass a WF test? Do you know what too look for in a WF test?
Did the WF system reach a point where parameters are stable?
Parameter stability to me implies that the current parameters are robust and not likely to change in the future.
In a few weeks I may do another video on this. The olderGSB documentation on GSBES1,2,3 was done when GSB was less powerful than what it is now.
System building is now faster and simpler. I made about 8 systems in the last week and went live on a number of them.
Only reason I didnt go live on more of them is, not all systems were included when I put them in Portfolio Analyst & I ran out of time to implement
the execution of them before my current holiday.
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
No, I have not done a WF test. This is an area of GSB I have not yet explored. Will try that next, thanks.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
There are different views on calculating commission. Whether to include commission in strategy development or to include it afterwards. On balance, I
prefer including it. However, GSB doesn't always calculate it. It is not clear why.
As for the lack of volatility in equity markets, I expect it to continue. The markets have been drugged by central banks and behave accordingly.
Shocks fail to shock participants, because they have been trained to expect the central banks to step in and prevent a correction. BTFD has been a
winning strategy for the past 8 years. It has beaten the vast majority of hedge funds.
The fragility of the system, - economic, financial, social, political - has increased. Real risk is substantial but perceived risk is low. The central
banks cannot afford to let the massive global ponzi scheme, that they have created, to deflate.
The negative consequences of these policies have been enormous; and carefully covered up. But we are seeing increasing instability in the political
and social sphere, in many countries. Nobody knows when the black swan will arrive.
My own stance on risk is to avoid holding overnight positions in current circumstances, despite the manufactured calm.
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Cyrus68.
You open a number of cans of worms. I like to diversity greatly in markets and time frames.
Back to the Walk forward subject.
Its critical to do WF
For example you could have two similar looking equity curves. One system passes a wf, and the other fails.
Doing monte carlo testing etc, will not show this.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
I entirely agree about doing walk-forward analysis. This is one of the best features of GSB. You also make the excellent point of having a portfolio
that is diversified by markets and time frames.
However, you must take care about how you diversify. Some of the conventional "wisdom" on diversification is rubbish. A long stock portfolio by sector
and region will do OK in the current phony calm environment. But if genuine turbulence arrives, the correlation will go to 1, in a downward direction,
immediately. It helps to be diversified by broad market sector - stocks, commodities ...etc.. - with multiple long/short trading strategies.
Statistical arbitrage is a market neutral strategy. But in turbulent conditions, such as in 2007, your portfolio can take a big hit. For a slew of
hedge funds, their long positions went south and their short positions went north - a double whammy.
I wonder if anybody has experience with Portfolio Maestro in TS. I have it, but haven't used it. Looking at a few videos, I don't thing it can do what
I'm looking for, namely portfolio optimisation. I want to find the optimal portfolio allocation among a set of strategies in risk/return space.
Currently, I do this in Excel.
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
I havnt used PM in TS for a decade, but I didnt like it much.
Have you tried portfolio analyst?
http://trademaid.info/pa.htm
It has automatic portfolio selection to maximize netprofit / drawdown.
Your other comments are very valid but will be hard to implement fully. Its hard enough making money trading, let alone doing it with little risk. My
own theory is systems going poorly OOS is a bigger risk than most people think. Especially when you think your methodology is good, when its not. 2007
however was a very lucrative period for futures trading.
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
WF questions
I've started to experiment with WF and have some questions. The only documentation of substance I could find was pp. 11-12 of the 2017-08-31 Quick
Manual.
Perhaps I don't have the WF parameters set right, but more often than not, the last result of a WF run has NP far less than the original strategy. The
attached image shows once such example, where the original NP was $16,835, but the "Current" (final) WF NP was only $1,345.
Occasionally I will see a more appealing WF result, as in the attached ascending NP example, where the original NP was $21,195, and the "Current"
(final) one was $24,600. However, the graph of the results of this run shows the OOS curve well below the original curve, which is something the
documentation implies to be a failed WF run.
Please clarify what constitutes a successful WF run. Must the Current (final) NP exceed the original NP, as well as the OOS curve lie above the
original curve? Do I need to increase the number of WF runs to increase the likelihood that a winner will be found? I'm afraid I'm confused.
  
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Gregorian,
Your example is an excellent one of a system that looks great on the surface, but is a very bad failure in WF.
Looking closer there are a number of things that are very wrong.
You have used rolling WF, not anchored. This is a very bad idea unless you have very high volume of trades.
Also 30 WF runs, not the default 10. Best use defaults inn WF except bigger WF population /generations. (see below)
The second screen shot looks much better. Note also its on anchored with 20 runs.
Im not sure what the curves look like on this WF run.
The other issue is I see only data from 2017. My personal opinion, I wouldn't consider even building a system on such short data.
I also like 300WF population and 300 WF generations. 100,200 is ok for quick test but 300 300 will give more stable results.
Your questions are excellent and highlight why I need to do a video on this section.
I think the GSBsys1,2,3(ES) docs all show sucessful WF curves.
Its a bit too simple, but hope for the IS results, but expect the OOS results.
I like the OOS and IS curves to be parallel in the last 1/3 or so of the graph.
Even if parmaters have changed a bit, this implies that OOS results are similar to IS results.
The offset in the curves is due to changes before best parameter set has been found.
On the second picture, the results look much better. You have acheived relativly stable paramaters.
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
You can see in this picture, the OOS results look great. Its hard to see on the graph as its truncated.
The high Netprofit came at the expense of average trade and profit factor. (1012 trades compared to 433 & 422 trades)
I consider this a pass.
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Note here how the curves look ok relative to each other. The OOS results took a hit in 2015, but the curve is parallel to the IS curve since.
Note the profit factor results are higher in OOS and IS curves. Thats good.
Note the parameters are stable.
I just build this system and traded it ASAP.
In summary WF results can be a very clean pass, a clean fail,, or a more complex somewhere in between.
As systems are so quick to build in GSB, if your not happy with the resuls then make another system.
Its good to think about what symbols, setups seem to consistently work.
 
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
Quote: Originally posted by admin  |
Its a bit too simple, but hope for the IS results, but expect the OOS results.
I like the OOS and IS curves to be parallel in the last 1/3 or so of the graph.
|
Thanks for the detailed explanation.
So far I have yet to see a WF run produce an OOS line that is above the original line. Often it is just slightly below, representing NP I can
certainly live with.
Having the OOS and IS lines parallel with the last 1/3 of the original line is not uncommon.
The next step is to try some of these successful WF strategies in a forward test on a demo account. Will report back with results.
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by Gregorian | | Quote: Originally posted by admin |
Its a bit too simple, but hope for the IS results, but expect the OOS results.
I like the OOS and IS curves to be parallel in the last 1/3 or so of the graph.
|
So far I have yet to see a WF run produce an OOS line that is above the original line. Often it is just slightly below, representing NP I can
certainly live with.
|
The goal is not to have OOS aabove IS and the original curve, but its just an example chosen at random of a good system.
Why the OOS is the above line is due to trading in OOS being much more active pre-2015. You can see after this date the curves look parallel. We also
want to consider other metrics like how profit factor compares on the three curves.
The ideal is pf*netprofit in OOS is as good as IS and the original curve.
|
|
|
parrdo101
Junior Member
Posts: 71
Registered: 18-11-2017
Member Is Offline
Mood: No Mood
|
|
Super Admin said,
"
Ive been thinking about this more.
Did the systems pass a WF test? Do you know what too look for in a WF test?
Did the WF system reach a point where parameters are stable?
Parameter stability to me implies that the current parameters are robust and not likely to change in the future.
In a few weeks I may do another video on this.
"
Super needed!: A step thru (YouTube?) on taking the identified stable parameters - what do you do with them, where do you put them, how do you
associate them to anything? They've no names.
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by parrdo101 | Super Admin said,
"
Ive been thinking about this more.
Did the systems pass a WF test? Do you know what too look for in a WF test?
Did the WF system reach a point where parameters are stable?
Parameter stability to me implies that the current parameters are robust and not likely to change in the future.
In a few weeks I may do another video on this.
"
Super needed!: A step thru (YouTube?) on taking the identified stable parameters - what do you do with them, where do you put them, how do you
associate them to anything? They've no names.
|
Good you ask as this is important.
The current parameters are the ones that appear in the WF test and the end, and in the WF code..
See in blue. This is done in ewfo, not GSB, but the concept is the same.
This example is unusual, hence why I show it. The last parameter changed a little. Either would be ok, but in 99% of cases you would use the last
(current) parameters.
That argument in this case for using the second the last is its the most stable parameter set. The argument for the last is its the best over the
entire data set.
(the last or the more stable second the last) but to be honest Im only highly confident when I see three runs with the same parameters. (the last
three)
The names of them are in the order of whats in the ts code. Normally optimize data streams is off on EWFO, so they will be omitted.
|
|
|
parrdo101
Junior Member
Posts: 71
Registered: 18-11-2017
Member Is Offline
Mood: No Mood
|
|
Suggestion: Make these kinds of things default in out-of-the-box:
"Also 30 WF runs, not the default 10." (In one of Super Admin's posts, above.) Default to 30.
Another: In a YouTube where you save App Settings/Training/Pearsons to .98 and min trades to 100.
Tons of stuff you run into like these as your evaluation runs out and you've run into this too late to have done it better, yikes. Have them "run into
it" out of the box!
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by parrdo101 | Suggestion: Make these kinds of things default in out-of-the-box:
"Also 30 WF runs, not the default 10." (In one of Super Admin's posts, above.) Default to 30.
Another: In a YouTube where you save App Settings/Training/Pearsons to .98 and min trades to 100.
Tons of stuff you run into like these as your evaluation runs out and you've run into this too late to have done it better, yikes. Have them "run into
it" out of the box! |
The 30 runs in the post above is not my settings. I NEVER` use anything but 10 and unless your a guru, think you shouldnt change this. If your new to
GSB and or new to trading, the more you move away from default settings, the more of a hole you can fall in.
As for what in training. This has changed between pearsons to PF.
Both are valid but Im moving more to PF. GSB is a very powerful product, and its a whole galaxy of markets and settings to explore. So it will change
over time. + the fact that most of my time is spend on it + 2 programmers on GSB/PA/ewfo are constantly working on new features and bug fixes.
14 days should be enough to see the power of GSB, + you can get a feel from the forums and the size of the user base
I do extend trials in some cases if asked.
|
|
|
![[*]](images/davis/default_icon.gif){kind=icon}
