| Pages:
1
..
5
6
7
8
9
..
98 |
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Quote: Originally posted by parrdo101  | Say you've no WF test in the GSB; you're going to pick something for the future directly out of the the GSB systems finds:
It looks to be there could exist a very smart sort key given this sit. Could look good even without a WF? I'm beginning to think this GSB has it in
there actually somewhere. |
There is no way you should ever trade without doing a WF.
ES with secondary filter closed is high chance it will work, but I would not dream of live trading without WF. If no WF software existed, then you
could wait 6 months for out of sample results, but the WF software exits, is good and fast.
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
I already saw it was till november 2017 in the movie in tradestation
| Quote: Originally posted by admin | | Quote: Originally posted by parrdo101 | Say you've no WF test in the GSB; you're going to pick something for the future directly out of the the GSB systems finds:
It looks to be there could exist a very smart sort key given this sit. Could look good even without a WF? I'm beginning to think this GSB has it in
there actually somewhere. |
There is no way you should ever trade without doing a WF.
ES with secondary filter closed is high chance it will work, but I would not dream of live trading without WF. If no WF software existed, then you
could wait 6 months for out of sample results, but the WF software exits, is good and fast. |
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
Just wonder, how are the WF parameters chosen, based on the fitness criteria only?
Is it taken into account that while optimizing with WF there could be local peak? I mean sometimes systems work fantastic with x=10 but when x is 9 or
11 performance is much less.
That could also result in performance issues if markets change a bit OS.
For example there could also be a good performance at for example x=5 while at x=3 or 4 and x = 6 and 7 performance is more stable.
Do you consider the shape of the profit curve based on parameter(s)
| Quote: Originally posted by admin | | Quote: Originally posted by parrdo101 | Say you've no WF test in the GSB; you're going to pick something for the future directly out of the the GSB systems finds:
It looks to be there could exist a very smart sort key given this sit. Could look good even without a WF? I'm beginning to think this GSB has it in
there actually somewhere. |
There is no way you should ever trade without doing a WF.
ES with secondary filter closed is high chance it will work, but I would not dream of live trading without WF. If no WF software existed, then you
could wait 6 months for out of sample results, but the WF software exits, is good and fast. |
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by rws | Just wonder, how are the WF parameters chosen, based on the fitness criteria only?
Is it taken into account that while optimizing with WF there could be local peak? I mean sometimes systems work fantastic with x=10 but when x is 9 or
11 performance is much less.
That could also result in performance issues if markets change a bit OS.
For example there could also be a good performance at for example x=5 while at x=3 or 4 and x = 6 and 7 performance is more stable.
Do you consider the shape of the profit curve based on parameter(s)
|
This is a good question.
wf will choose whatever gives peak fitness in each run.
ideally that should give the last 3 or more wf runs using the same parameter.
This means the parameter is consistently the best value.
A "local peak" is possible but not likely with the GSB methodology.
if it happens you are going to see a parameter not be stable.
If I was manually optimizing a value, I tend to look for the best area, not the peak value. However GSB can be multi-dimensional, to this approach is
too simplistic. The exception would be say the stop value. I tend too add that after the system is built and look for the best area, not the best
value.
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
3 runs of the same WF parameters could also mean that the market has not changed just enough to cause troubles.
These optimizers are used in Amibroker when running WF:
CMA-ES, SPSO and Tribes engines
These engines avoid local peaks according to the software writer.
CMA-ES works very fast, even for bigger portfolios of 5 min tickers and
many years testing.
| Quote: Originally posted by admin | | Quote: Originally posted by rws | Just wonder, how are the WF parameters chosen, based on the fitness criteria only?
Is it taken into account that while optimizing with WF there could be local peak? I mean sometimes systems work fantastic with x=10 but when x is 9 or
11 performance is much less.
That could also result in performance issues if markets change a bit OS.
For example there could also be a good performance at for example x=5 while at x=3 or 4 and x = 6 and 7 performance is more stable.
Do you consider the shape of the profit curve based on parameter(s)
|
This is a good question.
wf will choose whatever gives peak fitness in each run.
ideally that should give the last 3 or more wf runs using the same parameter.
This means the parameter is consistently the best value.
A "local peak" is possible but not likely with the GSB methodology.
if it happens you are going to see a parameter not be stable.
If I was manually optimizing a value, I tend to look for the best area, not the peak value. However GSB can be multi-dimensional, to this approach is
too simplistic. The exception would be say the stop value. I tend too add that after the system is built and look for the best area, not the best
value. |
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by rws | 3 runs of the same WF parameters could also mean that the market has not changed just enough to cause troubles.
|
Yes, but if you think like that, you should put your money from brokerage account into fixed interested, or under your mattress.
Trading systems can and do fail- even when you know what your doing. If you dont know what your doing they are highly likely to fail.
Ideally you trade many systems, markets and time frames to diversify your risk. 3 runs with the same parameters, good performance metrics etc, with a
good architecture is a risk im willing to take. Ive been trading 17 years, and never had a period where I stopped trading. This is totally a personal
decision.
nothing wrong with CMA-ES, SPSO and Tribes engines but im not sure there is any great advantage.
For small runs of WF in the days when I did all WF in TS, brute force gave similar results to genetic, if the genetic sample was say 10% of the brute.
Go less than 10% your very often ok.
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
There is nothing wrong with optimizing and I think even less
with optimizing optimization and avoid local peaks if possible.
Local peaks is a issue that can be dealt with and I hope you
could improve that in the future.
Here a description of other optimizing software (not amibroker):
Optimize function works differently. It does not seek performance peaks; instead it looks for stable performance ranges and places the parameters into
their centers. This does not necessarily result in the maximum backtest performance, but in the highest likeliness to reproduce the hypothetical
performance in real trading.
| Quote: Originally posted by admin | | Quote: Originally posted by rws | 3 runs of the same WF parameters could also mean that the market has not changed just enough to cause troubles.
|
Yes, but if you think like that, you should put your money from brokerage account into fixed interested, or under your mattress.
Trading systems can and do fail- even when you know what your doing. If you dont know what your doing they are highly likely to fail.
Ideally you trade many systems, markets and time frames to diversify your risk. 3 runs with the same parameters, good performance metrics etc, with a
good architecture is a risk im willing to take. Ive been trading 17 years, and never had a period where I stopped trading. This is totally a personal
decision.
nothing wrong with CMA-ES, SPSO and Tribes engines but im not sure there is any great advantage.
For small runs of WF in the days when I did all WF in TS, brute force gave similar results to genetic, if the genetic sample was say 10% of the brute.
Go less than 10% your very often ok. |
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by rws |
Here a description of other optimizing software (not amibroker):
Optimize function works differently. It does not seek performance peaks; instead it looks for stable performance ranges and places the parameters into
their centers. This does not necessarily result in the maximum backtest performance, but in the highest likeliness to reproduce the hypothetical
performance in real trading.
|
I see the merits to what you say.
Its not the same, but the TS WF did have the option to stress test.
ie add +-5% and +-10% on the parameters, and look for the best area. My tests on this showed degraded performance in the OOS curve. To me this means
the concept looked good, but wasn't helpful in practice.
|
|
|
curt999
Junior Member
Posts: 51
Registered: 24-7-2017
Member Is Offline
Mood: No Mood
|
|
I also want to see the last 12 months or so be profitable.
I look at this first usually..maybe it might be beneficial to add some sort of column to the results that lists performance over the past 12
months..so you could sort by it..
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by curt999 | I also want to see the last 12 months or so be profitable.
I look at this first usually..maybe it might be beneficial to add some sort of column to the results that lists performance over the past 12
months..so you could sort by it.. |
I use validation for this, but if thats not going to work for you, it wouldnt be hard to add another filter in the app settings.
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
Once you have peaks in a parameter optimization, changing (other) parameters or changed market conditions could also cause OS degradation because of
that peak. There has been a lot of writing and testing about this issue lately (also on Quantopian).
I don't mind too much if WF parameters change a bit over the different iterations as long as the profit curve is linear. I would rather see a better
profit curve because of WF optimization than the exact the same parameters. Sure parameters should not change too much.
Parameters can also change a lot in WF because of peak optimization. I think if you look for an average good area of a parameter instead of a peak
when optimizing you will see less change in WF parameters too.
When market conditions change I think it is not more than logical that there could be a better setting for a parameter if that would not be a peak.
| Quote: Originally posted by admin | | Quote: Originally posted by rws |
Here a description of other optimizing software (not amibroker):
Optimize function works differently. It does not seek performance peaks; instead it looks for stable performance ranges and places the parameters into
their centers. This does not necessarily result in the maximum backtest performance, but in the highest likeliness to reproduce the hypothetical
performance in real trading.
|
I see the merits to what you say.
Its not the same, but the TS WF did have the option to stress test.
ie add +-5% and +-10% on the parameters, and look for the best area. My tests on this showed degraded performance in the OOS curve. To me this means
the concept looked good, but wasn't helpful in practice. |
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by rws | Once you have peaks in a parameter optimization, changing (other) parameters or changed market conditions could also cause OS degradation because of
that peak. There has been a lot of writing and testing about this issue lately (also on Quantopian).
I don't mind too much if WF parameters change a bit over the different iterations as long as the profit curve is linear. I would rather see a better
profit curve because of WF optimization than the exact the same parameters. Sure parameters should not change too much.
Parameters can also change a lot in WF because of peak optimization. I think if you look for an average good area of a parameter instead of a peak
when optimizing you will see less change in WF parameters too.
When market conditions change I think it is not more than logical that there could be a better setting for a parameter if that would not be a peak.
|
if this is your belief, you could just do rolling WF. Proof would be what the out of sample curve is like. The problem with WF is you have reduced
your sample size greatly of each run. Its particularly bad unless you have very big sample size.
I'm open to adding other options for genetic optimization, but its extremely low priority.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by admin | | Quote: Originally posted by curt999 | I also want to see the last 12 months or so be profitable.
I look at this first usually..maybe it might be beneficial to add some sort of column to the results that lists performance over the past 12
months..so you could sort by it.. |
I use validation for this, but if thats not going to work for you, it wouldnt be hard to add another filter in the app settings.
|
This is todays beta.
|
|
|
parrdo101
Junior Member
Posts: 71
Registered: 18-11-2017
Member Is Offline
Mood: No Mood
|
|
How many secondary data files can be loaded for a GSB run? Some limit? Completely forgot to ask.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Unlimted as long as you have the ram. It is a bit slower from memory if you increase this a lot.
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
Syncing and Stop Orders not activating
I have an NQ strategy with a Stop Loss of $1000. As you can see from the Trade Manager pic:
1. The open position for NQ shows over $4000 in losses, but the stop is not triggered and Trade Manager shows it as "unsent", even though the
Properties is set to send stops to the TS Stop Server.
2. The Strategy Performance Report shows the strategy as flat, so I then noticed that there are three one contract short positions open, whereas the
strategy only trades one contract. Apparently the strategy has gotten out of sync with the real world.
The Wait for UROut option is not checked. Might this be the problem? I was under the impression that if you had, for example, a long position and an
order to open a short position came through, the long position would be closed automatically.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by Gregorian | I have an NQ strategy with a Stop Loss of $1000. As you can see from the Trade Manager pic:
1. The open position for NQ shows over $4000 in losses, but the stop is not triggered and Trade Manager shows it as "unsent", even though the
Properties is set to send stops to the TS Stop Server.
2. The Strategy Performance Report shows the strategy as flat, so I then noticed that there are three one contract short positions open, whereas the
strategy only trades one contract. Apparently the strategy has gotten out of sync with the real world.
The Wait for UROut option is not checked. Might this be the problem? I was under the impression that if you had, for example, a long position and an
order to open a short position came through, the long position would be closed automatically. |
I am no epert on this. Wait for urout is normally ticked. If not I think you may get double fill under freak conditions. ie a stop order is filled as
the market hits it, moments later ts cancels the stop.
TS forum would be better to ask this. Often sims are not as reliable as actual execution. I do have trobule fro time to time with execution at TS. MOC
failed & last week I had a moc system that left a profit target after moc.
No execution is perfect.
I sold TS code that reconciles TS chart to live account, but it was only designed for one trading system per account. Think it was $200. It could be
improved beyond this but I dont have the time to invest into it as its not a two minute improvement
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
Regarding the contracts table in GSB, I'm not sure if it is set up wrong, or it was filled in a hurry or i'm reading it wrong. The column called
'Ticks' appears to refer to tick value NOT tick size.
the info for GC is correct. A tick size of 0.1 translates to a tick value of 10.
but for AD which has a tick size of 0.0001 and a tick value of 10, why is there a 10000 entry?
ES has a tick size of 0.25 and a tick value of 12.50, so what's 4 doing there?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
There are 4 ticks per point for ES. So 1/4 = 0.25
price scale is 1/10000. Hope this looks ok to you. Ive never traded AD so dont know it so well.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
Thanks for the clarification. So we need to enter ticks per point for each instrument.
In which case, 10000 for AD is correct.
Some software packages use tick size.
In the case of GSB it uses a different input, which is fine.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
When you use a validation period in GSB, does the test period remain OOS?
In other words, does the test period data remain unseen by GSB or is it seen?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cyrus68 | When you use a validation period in GSB, does the test period remain OOS?
In other words, does the test period data remain unseen by GSB or is it seen? |
very good question.
GSB fitness doesnt see this period, but in once sence because we the human have seen it, its not out of sample.
This is still ok as long as you do a WF.
Buy the way, its not easy or common to get a bad WF result on NG.
I got one today. Fantastic final equity curve, really bad OOS curve and parameter stability of zero
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
the choice of session times for CL was intelligent.
I see that you have the same ones for NG.
looks like it is working out well.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cyrus68 | the choice of session times for CL was intelligent.
I see that you have the same ones for NG.
looks like it is working out well. |
NG is just amazing.
note the stability is very healthy, curve consistent profitable in most runs too.
|
|
|
cotila1
Junior Member
Posts: 78
Registered: 8-5-2017
Member Is Offline
Mood: No Mood
|
|
Hi Peter, not sure if on yr plan already, but I think would be useful if WF shows the PASS/FAIL result flag at the end of the process?
|
|
|
| Pages:
1
..
5
6
7
8
9
..
98 |
![[*]](images/davis/default_icon.gif){kind=icon}
