| Pages:
1
..
4
5
6
7
8
..
98 |
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Quote: Originally posted by Gregorian  | In both 39.11 and 40.04, there is a significant slowdown after a few hours of generation. For example, on a PC where GSB is the only task running,
consuming 17% of CPU and 10% of RAM, at the beginning of the run it generates something like 4,700 strategies per minute, but a few hours later, with
no change in resource consumption, it degrades to 144/min.
Maybe I'm doing something wrong?
Also in 40.04 the stats window does not update with RAM use figures. Have to get those from Task Manager. |
Is that in a stand-alone, worker or manager? Can you repeat the identical test again?For a long time when I run 20 or 30 workers, I get one that
stalls. We have been try to address this for a while. Its improved but not fixed.
Im on 40.05 standalone right now, and havnt noticed the ram figures go, but I will look out for it.
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
It's a standalone. Just ran 40.4 on a different instrument, and after six hours went from 3,400/min to 239/min, so that's 3 for 3 degradations today.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by Gregorian | | It's a standalone. Just ran 40.4 on a different instrument, and after six hours went from 3,400/min to 239/min, so that's 3 for 3 degradation today.
|
Can you send me your entire folder ziped up, so I can test.
(ie send via dropbox url. sending a exe wont get to me)
Anything in exception folder?
Most likely its unique to your setup.
|
|
|
philcollins
Junior Member
Posts: 6
Registered: 10-11-2017
Member Is Offline
Mood: No Mood
|
|
Not getting any trades with SPY EOD data
I'm trying to generate some algos for EOD stock data (SPY as a first test), but GSB doesn't seem to produce any results.
The data (~15 years) seems to be imported alright. Data lines look like this:
| Code: | 20161201,09:00,219.437,219.437,217.866,218.244,64083890,0
20161202,09:00,218.383,218.960,217.975,218.383,54539260,0
20161205,09:00,219.357,220.103,219.129,219.646,49510242,0 |
I turned all backtest and train filters off and still nothing. The contract definition is as follows:
| Code: | "Name": "SPY",
"SecType": 0,
"Exchange": "GLOBEX",
"Currency": "USD",
"TicksPerPoint": 100,
"PointValue": 1.0,
"Digits": 2,
"SessionClose": "23:59:00",
"Session1CloseFrom": "23:59:00",
"Session1CloseTo": "23:59:59",
"Session2CloseFrom": null,
"Session2CloseTo": null |
Can anyone point me in the right direction? I've attached the optimization settings used.
Edit: Generating algos for the sample ES data works OK.
Attachment: Login to view the details
|
|
|
curt999
Junior Member
Posts: 51
Registered: 24-7-2017
Member Is Offline
Mood: No Mood
|
|
did you also turn off exit on close
|
|
|
philcollins
Junior Member
Posts: 6
Registered: 10-11-2017
Member Is Offline
Mood: No Mood
|
|
Yeah I did.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by philcollins | I'm trying to generate some algos for EOD stock data (SPY as a first test), but GSB doesn't seem to produce any results.
Edit: Generating algos for the sample ES data works OK.
|
Try turning fitness and reports commissions to zero initially.
All performance filters set to pass everything, except test to have pf of 1.1
You training period is also too high.
Under app settings,gui you can also have top update enabled, and latest update enabled to see each system gsb is running.
See these results in the tabs next to unque-systems.
Dont leave these settings to true unless you have to. Its much slower.
You can then see what metrics GSB is producing.
|
|
|
philcollins
Junior Member
Posts: 6
Registered: 10-11-2017
Member Is Offline
Mood: No Mood
|
|
Cheers I got it working now after some fiddling. I had to enable a couple of settings in the GUI update settings (update unique etc.) and disable the
secondary filter-mode.
I'm now running into another problem: all generated strategies seem to start trading at 01/01/2008 even though I have data from 2002 to 2016. Any
ideas? I tried lowering 'Max bars back' in app settings (lowest possible seems to be 300), but this didn't help. I'm using a 50/50 split.
I've attached my settings. I tried both 'spy.1440.Minute.foo.txt' and 'spy.405.Minute.foo.txt' as filenames but this didn't seem to make a difference.
Attachment: Login to view the details
Attachment: Login to view the details
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by philcollins | Cheers I got it working now after some fiddling. I had to enable a couple of settings in the GUI update settings (update unique etc.) and disable the
secondary filter-mode.
I'm now running into another problem: all generated strategies seem to start trading at 01/01/2008 even though I have data from 2002 to 2016. Any
ideas? I tried lowering 'Max bars back' in app settings (lowest possible seems to be 300), but this didn't help. I'm using a 50/50 split.
I've attached my settings. I tried both 'spy.1440.Minute.foo.txt' and 'spy.405.Minute.foo.txt' as filenames but this didn't seem to make a difference.
|
It might be that the systems require high volatility to meet your performance metrics. Lower them and I think you will get systems trade trade
earlier.
If this doesnt fix it, read below.
disabling SF will tend to give a massive drop in profit per trade.
You should try GA or CloseD and see what works best.
But on daily bars it might have merit as the amount of trades will have dropped due to daily bars.
Can you upload your data, and a saved system.
|
|
|
philcollins
Junior Member
Posts: 6
Registered: 10-11-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by admin |
It might be that the systems require high volatility to meet your performance metrics. Lower them and I think you will get systems trade trade
earlier.
If this doesnt fix it, read below.
disabling SF will tend to give a massive drop in profit per trade.
You should try GA or CloseD and see what works best.
But on daily bars it might have merit as the amount of trades will have dropped due to daily bars.
Can you upload your data, and a saved system. |
I chopped off 1 year at the beginning so that it starts at 2003 instead of 2002 and now the first trades come in at 01/01/2009 instead of 01/01/2008.
So I guess some offset ends up wrong somehow, but consistently wrong.
(SF enabled works btw, at first glance it just yields less systems.)
I've sent you the SPY data in a private message, perhaps you can tell what's going on.
Edit: lowering the filter restrictions didn't work.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by philcollins |
I suspect GSB thinks you have a hole in data, but im not sure.
try some other data like $spx.x. @ES.D I know its not tradable, but its just a test.
check just before 18 dec 2007
You have old settings for test, I would also loosing back test termination too.
 |
|
|
|
parrdo101
Junior Member
Posts: 71
Registered: 18-11-2017
Member Is Offline
Mood: No Mood
|
|
Peter had me make a custom session for Trade Station for using ES, $IDX, $SPX.X; it was 8:30am to 3pm exchange time. Those hours happen also to be
EXACTLY the regular session for $IDX AND $SPX.X. Careful inspection of reg session and my custom session shows they're exactly for the same
times and days - shaded green.
I am running my OOS exactly only in the Trade Station chart with the GSB optimizer script. $6333+ net profit for the OOS period. Now I switch - to
doublecheck things as I do - both $IDX and $SPX.X - I switch their regular session to my same-hours custom session. -$518 (minus) net profit.... What
about this?
 
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by parrdo101 | Peter had me make a custom session for Trade Station for using ES, $IDX, $SPX.X; it was 8:30am to 3pm exchange time. Those hours happen also to be
EXACTLY the regular session for $IDX AND $SPX.X. Careful inspection of reg session and my custom session shows they're exactly for the same
times and days - shaded green.
I am running my OOS exactly only in the Trade Station chart with the GSB optimizer script. $6333+ net profit for the OOS period. Now I switch - to
double check things as I do - both $IDX and $SPX.X - I switch their regular session to my same-hours custom session. -$518 (minus) net profit.... What
about this?
the important thing is that ts results = gsb. If this is the case, pursue this no further. If you did want to look more into this, My guess is your
computer time zone is not central usa time. (ok but the issue might relate to this) I would try closing ts, set computer to central time and test
again.
|
|
|
|
parrdo101
Junior Member
Posts: 71
Registered: 18-11-2017
Member Is Offline
Mood: No Mood
|
|
Checking in...If you had to pick only one you'd be stuck with forever:
1. Always optimize datastreams routinely on WF optimize. Set to True.
2. Never optimize datastreams routinely on WF optimize. Set to False.
Which would you?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by parrdo101 | Checking in...If you had to pick only one you'd be stuck with forever:
1. Always optimize datastreams routinely on WF optimize. Set to True.
2. Never optimize datastreams routinely on WF optimize. Set to False.
Which would you? |
This is a good question.
I would not optimize data streams as default
Typically I get the same final results or I get better results with much worse linearity in the equity curve. Ive only used opt data streams 10 or so
times.
It would also only be usefull when you have very similar data streams. ie es,$spx,$idx etc while if you had say advance/decline ratio custom indicator
its not likely to switch from that to $spx etc
|
|
|
parrdo101
Junior Member
Posts: 71
Registered: 18-11-2017
Member Is Offline
Mood: No Mood
|
|
One can't even select whether to optimize the datastreams per everything in the snippet, right?
But, I've been assuming it WILL always optimize the secondary datastreams in the GSB "first," "primal" optimize (not wf optimize)...(?) Do I have that
right?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by parrdo101 | One can't even select whether to optimize the datastreams per everything in the snippet, right?
But, I've been assuming it WILL always optimize the secondary datastreams in the GSB "first," "primal" optimize (not wf optimize)...(?) Do I have that
right?
|
Many features are hidden unless you have advanced mode on
(View from top menu, the advanced mode)
the GSB quick start guide might be worth a read again.
By the way, you were very good at attention to detail spotting that the test criteria had changed from pearsons to profit factor.
Note also that the profit factor figure used is lower on allmarkets settings, vs ES settings.
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
WF re-run with different dates
Is it possible to run a WF on an old strategy generated with GSB, but with different dates? For example, I'd like to re-run WF on a strategy generated
and saved by GSB six weeks ago, but this time including the last six weeks of data.
From what I can tell, the WF date range on a saved strategy seems to be hard-coded and is unchangeable.
Try to edit the gsb system file (save it by right clicking) and try changing the dates. 6 weeks unlikely to make any difference though
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
More through reply.
I havnt done this, but you should be able to right click and save the system. Then edit the saved system in notepad, change the dates. Then load the
system again. To be save load it in a new copy of GSB incase over writing the system didnt work.
6 weeks should be very little time compared to many years of data, so I wouldnt think its significant.
I like to see parameter stability for 3 or more wf runs, so again its not likely to change things.
|
|
|
parrdo101
Junior Member
Posts: 71
Registered: 18-11-2017
Member Is Offline
Mood: No Mood
|
|
Say you've no WF test in the GSB; you're going to pick something for the future directly out of the the GSB systems finds:
What are your top 3 look for's? Sorts in the columns? Prefer stated as a sorting of columns, not something like "look for equity curve closest to
straight line"; would prefer that stated more specifically e.g., like, "Step 1: Grab the next highest Pearson from the top which has the next lower
number of trades
than the top Pearson, in the Full Period column" (Please reference directly any discretions that may use such as whether Full Period or Test Period
columns, whatever, etc.)
It looks to be there could exist a very smart sort key given this sit. Could look good even without a WF? I'm beginning to think this GSB has it in
there actually somewhere.
|
|
|
parrdo101
Junior Member
Posts: 71
Registered: 18-11-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by admin | I'm intending to change this in the next week.
I want commission to be subtracted off system curve, IS curve and OOS curve.
One value for all
This value is NOT taken off fitness
The a commission value to be taken off FITNESS in system curve, IS curve and OOS curve.
One value for all
This value is not taken off equity curves.
EWFO works like this.
It means you can shift GSB to look for fatter trades, but get a totally level playing field for the equity curves,
but can choose to use commission on both as is the traditional usage of commission field.
I tested with $1 on each commission field and it worked fine. Probably your figure is to high to pass your filters in app settings.
If you use fitness netprofit * average trade, the commission value to fitness will in effect be irrelevant.
Fitness netprofit by itself, commission in fitness will be very significant.
Commission with Fitness in NP & PF I think we be irrelevant too.
|
So essentially, to incorporate a realistic commission within when the GSB is finding systems (say 29.76 round trip for ES), you have to use
Net Profit for fitness function? And, if so, this is going to show up in the graph curves as a slower rising equity curve? I'm not clear.
(Please also note an unanswered, 2nd question in my preceding post. I've noticed several instances when I'm piling on with the questions, you just
answer the last one. Several out there this way that got missed I saw.)
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
If fitness is np*at, GSB is going to look for high average profit trades. Adding commission in fitness will not likely change anything at all.
If fitness is net-profit only, commission is critical and I would double the actual value.
If you don't double this figure, you might get a system that does lots of trades, that makes all the money on 2007 to 2008, and makes nothing any
other time.
However I very rarely use anything other than np*AT fitness, and unless you have a very good reason- I wouldn't do anything else.
The commission in the reports section will show up in the graph only. (slower rising curve)
The commission in the fitness section will be internally used by GSB
Will reply to previous question next. sorry I missed it.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by parrdo101 | Say you've no WF test in the GSB; you're going to pick something for the future directly out of the the GSB systems finds:
What are your top 3 look for's? Sorts in the columns? Prefer stated as a sorting of columns, not something like "look for equity curve closest to
straight line"; would prefer that stated more specifically e.g., like, "Step 1: Grab the next highest Pearson from the top which has the next lower
number of trades
than the top Pearson, in the Full Period column" (Please reference directly any discretions that may use such as whether Full Period or Test Period
columns, whatever, etc.)
It looks to be there could exist a very smart sort key given this sit. Could look good even without a WF? I'm beginning to think this GSB has it in
there actually somewhere. |
I have filters on to only get the top % of the best systems. (Full period.)
The amount isnt critical but I dont want to look at 10,000 systems at it would take me months.
so for ES, i had tight metrics stored under app settings.
Something like $80000 np, 1.8pf? pearsons 0.985
Pearsons is what I mean by closest to a straight line. (R-F) Coulomb.
(Other markets specs might be much lower)
So I sort the entire period by np/dd and pick systems with decent pf, sort on pf and pick some with curves i like, sort on pearsons, sometimes sort on
np, or fitness. I feel for ES right now with very low range, high PF is the safest - not high pearsons.
On ES, as we are on extreme historic low range, I also want to see the last 12 months or so be profitable. Validation period of 8% is good for this
too. This can be added in the filters.
These are good questions to ask, and some of it was covered here in the walk forward video. This is a must see for GSB users.
https://www.youtube.com/watch?v=CQy-yP_kBMM
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
Till what date is this test. Till 2017 januar or december 2017?
| Quote: Originally posted by admin | | Quote: Originally posted by parrdo101 | Say you've no WF test in the GSB; you're going to pick something for the future directly out of the the GSB systems finds:
What are your top 3 look for's? Sorts in the columns? Prefer stated as a sorting of columns, not something like "look for equity curve closest to
straight line"; would prefer that stated more specifically e.g., like, "Step 1: Grab the next highest Pearson from the top which has the next lower
number of trades
than the top Pearson, in the Full Period column" (Please reference directly any discretions that may use such as whether Full Period or Test Period
columns, whatever, etc.)
It looks to be there could exist a very smart sort key given this sit. Could look good even without a WF? I'm beginning to think this GSB has it in
there actually somewhere. |
I have filters on to only get the top % of the best systems. (Full period.)
The amount isnt critical but I dont want to look at 10,000 systems at it would take me months.
so for ES, i had tight metrics stored under app settings.
Something like $80000 np, 1.8pf? pearsons 0.985
Pearsons is what I mean by closest to a straight line. (R-F) Coulomb.
(Other markets specs might be much lower)
So I sort the entire period by np/dd and pick systems with decent pf, sort on pf and pick some with curves i like, sort on pearsons, sometimes sort on
np, or fitness. I feel for ES right now with very low range, high PF is the safest - not high pearsons.
On ES, as we are on extreme historic low range, I also want to see the last 12 months or so be profitable. Validation period of 8% is good for this
too. This can be added in the filters.
These are good questions to ask, and some of it was covered here in the walk forward video. This is a must see for GSB users.
https://www.youtube.com/watch?v=CQy-yP_kBMM
|
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Not clear what you are referring too.
|
|
|
| Pages:
1
..
4
5
6
7
8
..
98 |
![[*]](images/davis/default_icon.gif){kind=icon}
