| Pages:
1
..
8
9
10
11
12
..
47 |
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi All,
I have done a fair bit of testing along the paths here, https://trademaid.info/gsbhelp/Methodology.html
and have got some solid numbers to show tomorrow all going well.
I show results of 1000 wf systems with and without walk forward, and using the best anchored stability on 30 min, 29,30,31 min, and 25,30,35 minute
bars. All on ES.
Im very happy with the results, and keen to see others do similar tests on other markets too.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Here is the results.
The conclusion is also written in the picture. This research is very significant.
I have lots more combinations I am working on. ie just 30 min bars, 29,30,31 min bars, all 38 oscillators etc
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Methodology now has significant tweaks.
Would highly recommend all users read this.
https://trademaid.info/gsbhelp/Methodology.html
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Here is the next post.
Conclusion, On S&P500, much better results on 29,30,31 min bars than 23 30 35
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Today I have made a mistake in my testing that's not critical, but I need to do a number of tests over again.
"+" used instead of "*" operand
I have also found one more simple step that improves results.
So if anyone had workers they can donate, please send me the share keys via email.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
The macros used for methodology & WF testing are here.
Attachment: Login to view the details
Unzip them and put them in (or whatever location you have GSB manager in)
C:\gsb\Data\Settings\Macros\
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
I got lots of comments asking for more details.
Here is the docs. It will improve a bit more as time permits
https://trademaid.info/gsbhelp/Provingthemethodology.html
|
|
|
coccigelus
Junior Member
Posts: 73
Registered: 11-7-2018
Member Is Offline
Mood: No Mood
|
|
Peter, always appreciate all your efforts and generosity assisting users with your knowledge.
|
|
|
Bruce
Member
Posts: 115
Registered: 22-7-2018
Location: Auckland - New Zealand
Member Is Offline
Mood: No Mood
|
|
An update to an 'SI' (Silver) system built earlier in the year which I also posted here and have had running since which captured a nice trade
overnight. This system came about as the result of the earlier work Pater was sharing around multi-time frame (29-30-31) optimization and OOS stats
testing. The current report is attached and the system was put into TS end of Feb.
Given where GSB is today there's little doubt this could be improved, I've been working SI Data1 with GC as Data2 which looks promising.
Attachment: Login to view the details
Thanks received (1):
+1 RandyT at 2019-12-17 10:22:37
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Thanks for the comments coccigelus.
This below is a very important revelation.
By accident all my tests on the new methodology were using "+" operand, not "*"
hence weights were shown as in the picture.
I feel the methodology works much better with + and the weights like shown.
After reflecting on this, I have some very strong - but not obvious suspicions why.
More CPU consuming testing required....
For now I recommend using operand + with these weights. DO NOT use weights like -10 to 10 step 2, but keep as shown.
More on this later. Its possible we go back to operand * later, but there may be a new type of operand * in later build.
Note after the wf, gsb will tweak the weights. What Im saying is (for now) no tweak weights pre WF
|
|
|
cotila1
Junior Member
Posts: 78
Registered: 8-5-2017
Member Is Offline
Mood: No Mood
|
|
Quote: Originally posted by admin  | Thanks for the comments coccigelus.
This below is a very important revelation.
By accident all my tests on the new methodology were using "+" operand, not "*"
hence weights were shown as in the picture.
I feel the methodology works much better with + and the weights like shown.
After reflecting on this, I have some very strong - but not obvious suspicions why.
More CPU consuming testing required....
For now I recommend using operand + with these weights. DO NOT use weights like -10 to 10 step 2, but keep as shown.
More on this later. Its possible we go back to operand * later, but there may be a new type of operand * in later build.
Note after the wf, gsb will tweak the weights. What Im saying is (for now) no tweak weights pre WF
|
Very good to know! the operand and weights you suggest are for specific markets such as es, or do you think this is true in general?
|
|
|
avatartrader
Junior Member
Posts: 60
Registered: 1-10-2018
Member Is Offline
Mood: No Mood
|
|
So, just to confirm, since I have been implementing the methodology here as documented:
Even though the new section in the help on "proving the methodology" shows "Add" as disabled in the screenshot, the optimal setting as it stands
right now is "Add" enabled as the only operator?
And this is for both build as well as market validation/indicator evaluation and selection?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Cotila1
I think this is going to be universal across all markets.
In summary, weights on * operand dont work (and it was never claimed they did)
However before long weights on * will work.
ie result=indicator1*weight1*indiactor2*weight2 ...
What weight1 & 2 = doesnt matter, except for being zero.
with +
result=indicator1*weight1+indiactor2*weight2 ...
If weight1=100 and weight2=1, then indicator2 is noise. This is faulty architecture.
If all weights are fixed at 1(any non zero equal number) , its almost impossible to build a working system with an indicator that is bad or noisy
logic.
So if we WF the weights AFTER the systems are built, we have no systems with bad or noisy logic.
Hope this helps.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by avatartrader | So, just to confirm, since I have been implementing the methodology here as documented:
Even though the new section in the help on "proving the methodology" shows "Add" as disabled in the screenshot, the optimal setting as it stands
right now is "Add" enabled as the only operator?
And this is for both build as well as market validation/indicator evaluation and selection? |
Good to ask. This is something you want to make sure is correct.
Im updating the docs now, but not finished.
correct on all points.
|
|
|
edgetrader
Junior Member
Posts: 24
Registered: 16-5-2018
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by admin | with +
result=indicator1*weight1+indiactor2*weight2 ...
If weight1=100 and weight2=1, then indicator2 is noise. This is faulty architecture.
If all weights are fixed at 1(any non zero equal number) , its almost impossible to build a working system with an indicator that is bad or noisy
logic. |
Also depends on the indicators and what scaling/normalization is applied at which point in time.
1. For indicators that oscillate around zero, they could be pre-scaled with dividing by standard deviation. Even before that, oscillators with another
center value (e.g. RSI 0 to 100) can be made zero-mean by subtracting their mean (e.g. 50). Then weights could be on a common range like -1 to +1 and
always be valid.
2. For indicators that wander, like a raw close, moving average or highest high, their scale is tied to which data stream they originate from. A
signal like fastMA*1 + slowMA*-1 also makes sense and doesn't need scaling/weighting (other than flipping a sign with *-1) if both MAs are from the
same stream.
One solution would be to put all indicators of 1. in one group, and indicators of 2. in separate groups for each data stream. Group results could be
compared against their thresholds and then combined logically (like trade if all true).
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Edgetrader.
Interesting timing to bring this up. Right now we are working on some alternatives to normalization. There is nothing wrong with what we are doing,
but we are going to try some alternatives to see if we get better results or more diversity. What we have now is normalization that scales from -100
to 100,
so there is no need to subtract 50 for example.
Your point 2 is already done.
I will let you know if there are improvements in the other options. Dont want to release new features unless they are proven to work.
One thing we are looking at is building systems with fixed weights, but doing WF with variable weights, with soft coded ranges.
I will discuss your comments with the programmer.
Peter
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
In < 24 hours there are going to be some tweaks to this page
https://trademaid.info/gsbhelp/Methodology.html
They are not uploaded yet, but I will post on the forum when this is done.
See
3) Chosen the correct settings.
There are a number of changes in May 2019.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
https://trademaid.info/gsbhelp/Methodology.html
Point 3 had been enhanced.
Settings tweaked to improve results
|
|
|
appengineer
Junior Member
Posts: 61
Registered: 8-4-2019
Member Is Offline
Mood: No Mood
|
|
Proving the methodology
I followed the steps in Proving the methodology using the updated methodology settings and attached are my results for ES system.
I am not very good at interpreting the numbers but my result is not very good and quite different.
The Avg trade is very low on OOS and NP/DD is extremely low too.
Only 4 indicators are coming out as the best for ES and the performance degrades for the OOS period 06/30/2015 - 02/28/2018.
Peter - any ideas on the big difference?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by appengineer | I followed the steps in Proving the methodology using the updated methodology settings and attached are my results for ES system.
I am not very good at interpreting the numbers but my result is not very good and quite different.
The Avg trade is very low on OOS and NP/DD is extremely low too.
Only 4 indicators are coming out as the best for ES and the performance degrades for the OOS period 06/30/2015 - 02/28/2018.
Peter - any ideas on the big difference? |
Great you ask Dave,
send me teamviewr details. Wont take long to resolve
|
|
|
appengineer
Junior Member
Posts: 61
Registered: 8-4-2019
Member Is Offline
Mood: No Mood
|
|
Proving the methodology ES Result
| Quote: Originally posted by admin | | Quote: Originally posted by appengineer | I followed the steps in Proving the methodology using the updated methodology settings and attached are my results for ES system.
I am not very good at interpreting the numbers but my result is not very good and quite different.
The Avg trade is very low on OOS and NP/DD is extremely low too.
Only 4 indicators are coming out as the best for ES and the performance degrades for the OOS period 06/30/2015 - 02/28/2018.
Peter - any ideas on the big difference? |
Great you ask Dave,
send me teamviewr details. Wont take long to resolve |
Slightly better result after correcting the time to 00:00:00 - 23:59:59
And Max Stop Loss of $1000
Hi Peter,
Any suggestions on another market to test on and the settings?
 
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi Appengineer
Nat gas, CL,
see 2) in https://trademaid.info/gsbhelp/Methodology.html
There were also a few good posts on @EC on the forum
https://trademaid.info/forum/viewthread.php?tid=227#pid3769
Im still try to improve the existing methodology, as big improvements can be due to many small improvements cumulative.
|
|
|
avatartrader
Junior Member
Posts: 60
Registered: 1-10-2018
Member Is Offline
Mood: No Mood
|
|
Hi Peter,
I have a question on market validation and the updated methodology as it pertains to the most recent build(s):
When building a set of systems as a benchmark and then looking to incrementally test changes one by one to determine whether they enhance or degrade
the performance, do you have a reference as to exactly which settings can be changed and then tested against the existing collection of systems,
versus having to build a new set of systems (i.e., you can update the settings and then re-backtest and recalculate stats without having to generate
new systems)? I know there are some you can and some you can't, but I'd like to know for sure so I can continue to refine my process to be as
efficient as I can.
Thanks!
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by avatartrader | Hi Peter,
I have a question on market validation and the updated methodology as it pertains to the most recent build(s):
When building a set of systems as a benchmark and then looking to incrementally test changes one by one to determine whether they enhance or degrade
the performance, do you have a reference as to exactly which settings can be changed and then tested against the existing collection of systems,
versus having to build a new set of systems (i.e., you can update the settings and then re-backtest and recalculate stats without having to generate
new systems)? I know there are some you can and some you can't, but I'd like to know for sure so I can continue to refine my process to be as
efficient as I can.
Thanks! |
A good question Dave,
what you can do without a entire rebuild of your systems, is to select certain systems, put them in a favorites, and run a macro.
ie you can try selecting the top np/dd and pearsons - then run macro,
or you can select top 1000 systems, then run a macro.
However if you change the stop size, number of indicators, secondary filter(s), primary indicator(s) you need to do a build and (wf if used in your
testing) again.
The good news is all of this can be driven by macros.
ie, I build 50k systems, select the top 1000 of them by fitness, WF the top 1000, then do stats.
All that is done with one macro.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Hi avatartrader
I just found out this cant be done in one macro until 52.40 (to be confirmed)
The reason is the macro submits WF jobs, but doesnt wait for them to complete.
|
|
|
| Pages:
1
..
8
9
10
11
12
..
47 |
![[*]](images/davis/default_icon.gif){kind=icon}
