| Pages:
1
2
3
..
98 |
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
General support questions.
I'm getting lots of questions on GSB. It would be good to post them here so others can benefit from the questions and replies.
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
Hi Peter,
Suggestion for a future version of GSB: to be able to import any filename instead of only ES.30.txt.
The name ES.30.txt only shows the ticker and the bar size, not the session used nor the time period of the dataset.
Thanks,
Carl
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Carl,
I think thats a good idea.
Think like crude oil or gold could vary a lot in session time, and there's lots of room for confusion.
A modification on it could be ES.30.AnthingyouLike.txt or ES.30.830.1500.centralusa.txt
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
This is implemented in 19.5 build. It will be uploaded < 24 hours.
|
|
|
crazyhedgehog
Junior Member
Posts: 14
Registered: 9-5-2017
Member Is Offline
Mood: No Mood
|
|
Hi Peter, I noticed that WF optimization does not use the Commissions value (i.e. it always optimizes as if the Commissions were set to 0, rather than
the actual value I put in the field). This leads to the WF curves on the Graph being much higher than the curve from the original optimization. Also
this means that WF optimizes for smaller Average Trades, and the WF results would not be usable in actual trading systems (since the Average Trade
would be too small to cover for trading costs). Could you please look into this? Thanks!
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Crazyhedgehog,
I looked into this and found the s&c is not applied to the IS curve, but is applied to OOS curve & the trading/test curve.
It would be safer to use 0$ s&C for now. We are looking for entry techniques, not systems at this early stage of GSB development. So low profit per
trade should not be an issue.
GSB1es system (im the gsb\tradestation folder) has profits increase from $20 to about $200 per trade with one new truism filter, and re-optimization
of parameters.
The graph of the WF curves are not higher due just to S&C. They are typically higher because wf has made improvements in parameters.
You can see this at post 17 on http://www.trademaid.info/forum/viewthread.php?tid=1
Regardless the most import curve, the gray WF/OOS curve has had S&C applied.
If you change S&C on the report, and click on the left part of the line. (see red arrow) the curves is instantly recalculated on the graph.
This is not the case for WF, as WF will have to be calculated again. You can verify what I say by using $0 S&C
Regardless I will look into getting this issue fixed.
|
|
|
crazyhedgehog
Junior Member
Posts: 14
Registered: 9-5-2017
Member Is Offline
Mood: No Mood
|
|
Hi Peter, I understand the main goal is to look for entry techniques. However at least the way I think about it the entry technique should issue a
signal for the bigger moves, not for every small move, since the profits are in those moves. The best way I found to force the optimizers to look for
such techniques is to have some level of commissions set (say $5 for ES). Just my opinion though.
In terms of OOS curve, when I compare it to the curve I get in TS, it looks to me like the commissions are not applied to OOS curve either. They are
indeed applied to Trading/test curves. Not sure why we get different results?
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
19.96 build has the commission issue fixed. I will upload shortly. In the big picture soon I expect to have very significant increase in average trade
and profit factor with upcoming build.
Secondary filter is also going to help a bit.
Totally agree that adding commission shifts the results to fatter trades, but I doubt at this stage of gsb it will make a lot of difference. Later on
it will though.
fitness by default is netprofit * profit factor. You could also try netprofit * average trade^2
Bottom line is soon gsb is going to make high average trade much easier.
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
Hi Peter,
Have been testing version 19.3. Looking good!
1. Maybe an idea to be able to select a range of strategies to start the WFA on?
2. Maybe an idea to calculate the Pearson for the WFA/OOS curve and add this number after the WFA to the table with results?
3. Is it possible to quantify the difference between the Best/IS and the WF/OOS curve? Correlation? Pearson? If the difference is big, the number is
X. And if the difference is small, the number is Y. Is it possible to add this number to the table with the results?
4. In version 18.4 it was possible to copy the table with the results to Excel for further analysis (tab Unique systems). In version 19.3 this is no
longer possible. Can this be switched on again?
5. What is the relation between the parameter values in tab Parameters and the original system parameter values and the list of WF parameter values?
Thanks,
Carl
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Carl.
You point 1). Good idea. On the todo list.
2) We need all the metrics (or at least the main ones) for OOS and IS curve. Agreed
3) This is complex. We could do the ratio of fitness of orginal to OOS and IS. Why its complex is PF and Average trade might go up lots, and netprofit
down. So there is too many things to compare.
I'm open to ideas on this.
4) Im not clear on this. Can you do some screen shot(s)
5) Currently the parameter tab = the original system tab, though some people will find this hard to see. I think the parameter tab isnt needed.
Whats much too complex is the walkforward tab summary. I'm not going to document this as it will be explained in GSB in the next month or less.
In short this is the parameters and indicators in machine (GSB) readable format), not human format.
If you don' understand it, dont worry. Even I dont, though I have a good idea. ie 13 in the list I think represents fastK.
This needs to be explain in GSB as time permits.
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
In regards to crazyhedgehog post about lifting the average trade, please see this post.
http://www.trademaid.info/forum/viewthread.php?tid=12
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
May 10 2017
Suggestion:
"3. Is it possible to quantify the difference between the Best/IS and the WF/OOS curve? Correlation? Pearson? If the difference is big, the number is
X. And if the difference is small, the number is Y. Is it possible to add this number to the table with the results?"
A quality measure could maybe be the average absolute distance between the Best/IS and WF/OOS curves?
Carl
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
The perfect would be the two curves on on top of each other. I had that yesterday for the first time.
Next best thing would be the curve's are parallel for the later part of the graph. (more the better)
This is easy to detect. What I actually value a lot is parameter stability. This means the current parameters have locked in as being the best
parameters.
A very healthy sign. In WF, I tend to choose the fitness that gives the best parameter stability. This can be measured easily, and in time an auto
detect the best fitness
can also be done. It will require numerous WF runs though. I plan to included this in WF. Parameter stability should happen within a month, but auto
fitness detect some time away.
Whats still acceptable is that OS curve is not to far below IS curve.
Problem can be that IS curve has much better metrics. ie PF, Average trade, DD, but net profit is down.
This is still highly acceptable.
Whats not acceptable is OOS curve tanks. If that happens be glad that WF picked up a failed system early in the process.
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Carl, your suggestion to send multiple systems at once to WF is done in 20.3. Thanks for the suggestion.
|
|
|
gmoney
Junior Member
Posts: 14
Registered: 8-5-2017
Member Is Offline
Mood: No Mood
|
|
What is the difference between the Testing and Validation periods. The documentation says GSB only looks at the Training period. If that is the case,
then why are there separate Testing and Validation periods?
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
@ gmoney, good question.
Also I noticed that some of the price data files are not complete.
Didn't check all the price files, but for example NG.30.txt only covers the years 2002, 2003, 2006, 2007 and 2017.
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Gmoney,
At the moment I am using 40% training, so GSB is building systems on this part of the data. The rest GSB has not seen. However as we humans tend to
sort the results by full period,
we have seen the rest of the data. That's why I like people to view all systems both good and bad after this. What you find with GSB however is the
ratio of samples that work OOS is very high.
you can see this at the video here at about 4 minutes in. https://www.youtube.com/watch?v=bm6HQxN6ppk
The 60% remaining OOS gives us an idea of how the system has gone out of sample. Regardless a walk forward enables us to see a very large part of the
data out of sample. There is normally however a 40%
part of data in sample required for a WF test.
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
The NG data is bad. I dont have that on my TS login, so will take me some time to get the data, unless we have a volunteer to provide it.
We cant provide data with GSB soon as I dont think the exchange allows it. Instead we can provide the ts workspace to export the data.
|
|
|
gmoney
Junior Member
Posts: 14
Registered: 8-5-2017
Member Is Offline
Mood: No Mood
|
|
Thank you. I understand the difference between the training and OOS periods (testing and validation periods). My question is what the difference is
between the Testing and Validation periods.
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
You can choose to sort on test training results, and then see whats in validation.
Its sort of like a 3 step walk forward process. Its optional to use it.
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
Minor detail.
Order in left window is:
test
training
validation
But shouldn't it be?
training
test
validation
(And maybe add the text "IS" and "OOS"? Don't know if I am the only one, but I always mix training/test up...)
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Your correct that the ext is is the wrong order.
Not sure what you mean by
(And maybe add the text "IS" and "OOS"? Don't know if I am the only one, but I always mix training/test up...)
as I see them of the graph after a WF. Can you do a screen shot with arrows?
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
Hi Peter,
I meant the text in the left window:
Training (IS)
Test (OOS)
Validation
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
Sometimes a strategy makes a large part of the total profit in the 2008 volatile period. The rest of the equity curve is rather flat.
I would like to use the period 2009 until 2014 as the training period and the rest (1997 - 2008 and 2014 until now) as a test/validation period.
At the moment GSB only allows the sequence training/test/validation.
It might be a good idea to make this a bit more flexible?
|
|
|
admin
Super Administrator
Posts: 5062
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Try vbaseplusminusMedium fitness. Fitness is relative to how much of the range it gets, not the profit.
This was designed for this problem. I have not used it on GSB myself. Its a EWFO feature ported to GSB
If that doesnt work, try a fixed profit target of $1500 to $2000, but remove it when you go live.
Thanks received (3):
+1 TwntySQ at 2023-12-21 02:58:29 +1 Siem at 2022-05-03 04:00:34 +1 bizgozcd at 2020-06-24 05:41:18
|
|
|
| Pages:
1
2
3
..
98 |
![[*]](images/davis/default_icon.gif){kind=icon}
