Update to GSB methodology. A must read, the backpacker and the Art of war by Sun Tzu

Quote: Originally posted by rws

That's a good idea. A bit similar to automatic features selection in machinelearning. But there is a drawdown. Sometimes input A and input B separately analyzed could show they don't work but while having A and B together they could work very good. I still think it's a very good idea.

Here is the summary on es.30
Closedbpv was excluded from all tests as its in this case the same as closed.

I will publish the best indicators in the private forum in a day or two.

For ES, using 2 indicators and operand +, this is a cut down list of what indicators work and dont.
I will publish the entire list in the private forum next week.

GSB is getting a modification to allow systems that are very similar, but not 100%. This is needed for me to get a larger sample size
using only two operators and multiply.
I will also publish the excel spread sheet too. But I'm waiting on GSB to have custom coulombs to be exported.

What Im interested in also is does the list vary when we use * instead of +
and will the list vary on other markets.


What I hope the cut down indicators will do is,

1) Build systems faster, possibly use less ram??
2) Make more robust systems
3) Give better our of sample results.

Here's what I got for NQ30, does this look reasonable?

Not sure why I got the #DIV/0! on row 16. halves the number of effective indicators!

Will do a new build based on these findings and see what comes of it.

Quote: Originally posted by TradingRails

Here's what I got for NQ30, does this look reasonable? Not sure why I got the #DIV/0! on row 16. halves the number of effective indicators! Will do a new build based on these findings and see what comes of it.

Quote: Originally posted by admin

Sorry I miss-read what you wrote tradingrails. Check ct2 is enabled

Quote: Originally posted by Bruce

Quote: Originally posted by admin   Sorry I miss-read what you wrote tradingrails. Check ct2 is enabled As I used only the * operator for this analysis, is it ok to use both * and / operators for the system build?

Quote: Originally posted by admin

Here is the template. Feedback welcome

I've performed a lot of testing with this new process focusing on NQ and SI, so I thought I would share the following observations.

There's no doubt that this analytical process significantly improves the quality of a system builds by eliminating what appear to be ineffective indicators in OOS testing. The quality of verification testing and VSS scores are also noticeably improved, all this leads to better WF results.

NQ which I have found to be a more challenging contract to build good systems for, upon performing Peters indicator test (I've published one of the test results here) the resulting system builds were noticeably better, more systems passing VF and delivering better quality WF output.

SI has been a lot easier, really good systems, with smoother equity curves. What I'm also noticing is the number of indicators that appear consistently to over or underperform across different assets as a result of this new testing process.

Quote: Originally posted by traderusa

That's interesting to see. How many systems were built to get this list? Peter found that results varied a fair bit on 10,000 sample, but 40,000 was consistent. Do you expect the same indicators to be good / bad on all markets, or market groups, or unique to specific markets?

Quote: Originally posted by traderusa

Thanks for the reply Bruce. Could you publish a silver GSB performance report. If any of it is sensitive, hide the trade list and or settings, but state what slippage and commission is used. There is no other reference in the forum to anyone using GSB on silver, so this is interesting.

Attached is the silver report form TS without any Slippage and Comm accounted for. Apart from the fac that the average trade is substantial so C&S is easily accommodated, I change the entry and exit orders to Limit Orders in most systems and use the TS function to change the limit order to a market order after 15 seconds which in reality rarely gets triggered.
For indexes I use buy 'minlist( C, AvgPrice) Limit;' or for shorts, 'SellShort MaxList( C, AvgPrice ) Limit;' have been doing this for years and found it to be really effective adding to the nett. There are a few other tricks as well!


Attachment: Login to view the details

Great result Bruce. I had never tried GSB on silver.
Here is an example of what secondary filter works best. Its is fairly simple but critical to get right.
For stock market indices, it always is CLosedMinusClose (not GA)

Quote: Originally posted by traderusa

Can someone publish how much results improve if we use all indicators, the top 1/3 only, the top half and the top 2/3 I would like an objective comparison to see if this is worth doing.

Hi TraderUsa. Here are the results.

Quote: Originally posted by admin

Hi TraderUsa. Here are the results.

Here is the results.
The conclusion is also written in the picture. This research is very significant.
I have lots more combinations I am working on. ie just 30 min bars, 29,30,31 min bars, all 38 oscillators etc

Here is the next post.
Conclusion, On S&P500, much better results on 29,30,31 min bars than 23 30 35

An update to an 'SI' (Silver) system built earlier in the year which I also posted here and have had running since which captured a nice trade overnight. This system came about as the result of the earlier work Pater was sharing around multi-time frame (29-30-31) optimization and OOS stats testing. The current report is attached and the system was put into TS end of Feb.

Given where GSB is today there's little doubt this could be improved, I've been working SI Data1 with GC as Data2 which looks promising.



Attachment: Login to view the details

Thanks for the comments coccigelus.
This below is a very important revelation.
By accident all my tests on the new methodology were using "+" operand, not "*"
hence weights were shown as in the picture.
I feel the methodology works much better with + and the weights like shown.

After reflecting on this, I have some very strong - but not obvious suspicions why.
More CPU consuming testing required....
For now I recommend using operand + with these weights. DO NOT use weights like -10 to 10 step 2, but keep as shown.
More on this later. Its possible we go back to operand * later, but there may be a new type of operand * in later build.
Note after the wf, gsb will tweak the weights. What Im saying is (for now) no tweak weights pre WF

Quote: Originally posted by admin

Thanks for the comments coccigelus. This below is a very important revelation. By accident all my tests on the new methodology were using "+" operand, not "*" hence weights were shown as in the picture. I feel the methodology works much better with + and the weights like shown. After reflecting on this, I have some very strong - but not obvious suspicions why. More CPU consuming testing required.... For now I recommend using operand + with these weights. DO NOT use weights like -10 to 10 step 2, but keep as shown. More on this later. Its possible we go back to operand * later, but there may be a new type of operand * in later build. Note after the wf, gsb will tweak the weights. What Im saying is (for now) no tweak weights pre WF

Quote: Originally posted by avatartrader

So, just to confirm, since I have been implementing the methodology here as documented: Even though the new section in the help on "proving the methodology" shows "Add" as disabled in the screenshot, the optimal setting as it stands right now is "Add" enabled as the only operator? And this is for both build as well as market validation/indicator evaluation and selection?

Quote: Originally posted by admin

with + result=indicator1*weight1+indiactor2*weight2 ... If weight1=100 and weight2=1, then indicator2 is noise. This is faulty architecture. If all weights are fixed at 1(any non zero equal number) , its almost impossible to build a working system with an indicator that is bad or noisy logic.

Proving the methodology

I followed the steps in Proving the methodology using the updated methodology settings and attached are my results for ES system.

I am not very good at interpreting the numbers but my result is not very good and quite different.

The Avg trade is very low on OOS and NP/DD is extremely low too.

Only 4 indicators are coming out as the best for ES and the performance degrades for the OOS period 06/30/2015 - 02/28/2018.

Peter - any ideas on the big difference?

Quote: Originally posted by appengineer

I followed the steps in Proving the methodology using the updated methodology settings and attached are my results for ES system. I am not very good at interpreting the numbers but my result is not very good and quite different. The Avg trade is very low on OOS and NP/DD is extremely low too. Only 4 indicators are coming out as the best for ES and the performance degrades for the OOS period 06/30/2015 - 02/28/2018. Peter - any ideas on the big difference?

Proving the methodology ES Result

Quote: Originally posted by admin

Quote: Originally posted by appengineer   I followed the steps in Proving the methodology using the updated methodology settings and attached are my results for ES system. I am not very good at interpreting the numbers but my result is not very good and quite different. The Avg trade is very low on OOS and NP/DD is extremely low too. Only 4 indicators are coming out as the best for ES and the performance degrades for the OOS period 06/30/2015 - 02/28/2018. Peter - any ideas on the big difference? Great you ask Dave, send me teamviewr details. Wont take long to resolve

Slightly better result after correcting the time to 00:00:00 - 23:59:59
And Max Stop Loss of $1000

Hi Peter,
Any suggestions on another market to test on and the settings?

Quote: Originally posted by avatartrader

Hi Peter, I have a question on market validation and the updated methodology as it pertains to the most recent build(s): When building a set of systems as a benchmark and then looking to incrementally test changes one by one to determine whether they enhance or degrade the performance, do you have a reference as to exactly which settings can be changed and then tested against the existing collection of systems, versus having to build a new set of systems (i.e., you can update the settings and then re-backtest and recalculate stats without having to generate new systems)? I know there are some you can and some you can't, but I'd like to know for sure so I can continue to refine my process to be as efficient as I can. Thanks!

Here are some results,
also shown is the results from https://trademaid.info/gsbhelp/Provingthemethodology.html
Improvements due to signpower function, and tweaks in methadolgy

User Bruce has also verified with his own testing.

Quote: Originally posted by admin

Here are some results, also shown is the results from https://trademaid.info/gsbhelp/Provingthemethodology.html Improvements due to signpower function, and tweaks in methadolgy User Bruce has also verified with his own testing.

Yes, this is certainly delivering some encouraging results and consistently so. I'm now running these newer 'tweaks' across other indexes, bonds, EC, etc to see what those builds deliver.

Peter, These latest updates are working great!

I just wanted to share a really simple build that is delivering solid results. Here I've just used the ES30 with the NQ30 and YM30 (instead of ES29,ES31), followed the same process you've outlined in the help using your Macros and the results can be seen. I know you are likely doing better, however, I'm just a hack and let's not forget you're the pro!

Here is the first system I chose using some of the new methodology. It also has a slightly different entry mode that I feel is slightly better.

There is one more entry mode that I have still to try
results out of sample since 28/3/2019

Includes $4.8 and $25 slippage and commission round turn

Hi Daniel,
More from me later
1) This will give long from one code and short from another.
Likely the tightest stop will apply to both codes

2) The next GSB build can do a wf thats long or short only
EWFO next build can do this, and combine reports
I am not certain of how well this concept works, esp on 2018 on ES which is a volatile but hard year


6) I used pearsons 0.91 on training, not the usual 0.95

I have been building on ES 29 30 31 to 6/2015 using the new methodology settings.
Running the BestIndicator Macro and then building 30k systems with top indicators, but I have not been able to find a system anywhere near the perf metrics of what Peter posted above. The Max Net Profit I could get was about $77k for example - Peters system >$132.

My stats for the first run had low deg -2.26 but after running again with Opt Indicators -22.0.

I'm wondering if anyone else has had same challenges? I've re-run a couple of time now. Posting my settings if anyone can see something incorrect?

Quote: Originally posted by admin

Hi Daniel, there are lots of issue here, and much more control when we get GSB 2.0 Check your fitness is on np*at Was the files in ewfo from gsb (ewfo wf) or ts? GSB doesnt always give the best parameter range. ie SF closeD way too big. I would be happy on es to make it 13 to 16. In gsb 2.0 the plan is to have much more control over these things.

At long last, I publish my default settings I am using for testing.
All files including data files are included, but not all the gsb exe,dll, xml files.
Included is gsb.config.xml to get GSB into admin mode.

There may be some settings that need a tweak, so consider this the first revision.
There are 4 ES files with 5 ticks of noise, 4 ES files with 10 ticks of noise

I think you can put this into the folder of your choice. (All paths relative, not fixed)
200mb in size zipped up.

My top 11 indicators are selected, and these are better than any other method Ive found. However sorry to say, im not sure how I found this group.


All macros used are included.

In summary you build 50,000 systems. Dates are 2000 to 2015.6.30 nth 1 / 80
The manager then populates but with nth set to all (workers built them with nth no trade)
Then macro 1 selects the top 250 systems and walk forwards them.

Manually put the top 250 into favouritesD.
Then run macro3 to get the stats.
record these stats.

Then run the macro5 astab-c macro that deselects all favouritesD, and puts the top 90 astab-c
Then run macro3 to get the stats.
The record these stats.

go to favourites A
Then run macro2 which changes dates back to 2015.6.30
after this is complete, run macro6 which selects the top 91 vss systems of the top 250
Then run macro3 to get the stats
Shown below is a summary of my stats.

zip file here of all my files in private forum
https://trademaid.info/forum/viewthread.php?tid=241#pid4133

Quote: Originally posted by appengineer

Hi Peter - Macros 4 is missing, can you share it, Thanks.

Quote: Originally posted by Daniel UK1

Hi Peter, May i ask, when you are referring to the above mentioned macros used and the process, is that how you go about the build process when you are doing a run for picking live trading system,, or is it your process to evaluate minor changes in the settings leading up to a build run in the future where you then pick real systems from? I notice that you are using 28 as a number for generations, not 1000 as i believe would be standard setting, i have used 40 for some time since it seems to make it easier to build system with, but can you explain how you come up with a such specific number as 28 and how one should think when it comes to changing this number... Cheers

Generations isnt critical, but if you reduce the indicator count to 11 as I have done, From (38/40) this gives less combinations.
So I see the systems graphs taper off in its gradient. It gets faster after the new random seeds are made when the 28 generations are hit.

Im still doing market validation, sorting out what sign modes, mutipliers etc I need

Quote: Originally posted by admin

Quote: Originally posted by appengineer   Hi Peter - Macros 4 is missing, can you share it, Thanks. macro 1) selected top 250 and wf them 2) change dates (back) to 2015-6-30 3) do stats from fav D 5) selected top 90 Astab-C into fav D 6) Select top 91 vss into fav D There was no macro 4. We can have 3 groups of macros, so we now have max of 18

Quote: Originally posted by Daniel UK1

Thank you Peter, Btw have anyone managed to get down degradation in general in SI Silver futures to good numbers? i cant seem to get lower than around 50% .. SI seems to produce great numbers like NP and DD, but the high degradation is kind of scary... anyone that manages to get lower degradation of SI and in this case any suggestions of settings?

Quote: Originally posted by admin

Quote: Originally posted by Daniel UK1   Thank you Peter, Btw have anyone managed to get down degradation in general in SI Silver futures to good numbers? i cant seem to get lower than around 50% .. SI seems to produce great numbers like NP and DD, but the high degradation is kind of scary... anyone that manages to get lower degradation of SI and in this case any suggestions of settings? Did you sue the top 50% or so of the best indicators?

Quote: Originally posted by admin

At long last, I publish my default settings I am using for testing. All files including data files are included, but not all the gsb exe,dll, xml files. Included is gsb.config.xml to get GSB into admin mode. There may be some settings that need a tweak, so consider this the first revision. There are 4 ES files with 5 ticks of noise, 4 ES files with 10 ticks of noise I think you can put this into the folder of your choice. (All paths relative, not fixed) 200mb in size zipped up. My top 11 indicators are selected, and these are better than any other method Ive found. However sorry to say, im not sure how I found this group. All macros used are included. In summary you build 50,000 systems. Dates are 2000 to 2015.6.30 nth 1 / 80 The manager then populates but with nth set to all (workers built them with nth no trade) Then macro 1 selects the top 250 systems and walk forwards them. Manually put the top 250 into favouritesD. Then run macro3 to get the stats. record these stats. Then run the macro5 astab-c macro that deselects all favouritesD, and puts the top 90 astab-c Then run macro3 to get the stats. The record these stats. go to favourites A Then run macro2 which changes dates back to 2015.6.30 after this is complete, run macro6 which selects the top 91 vss systems of the top 250 Then run macro3 to get the stats Shown below is a summary of my stats. zip file here of all my files in private forum https://trademaid.info/forum/viewthread.php?tid=241#pid4133

Quote: Originally posted by appengineer

Quote: Originally posted by admin   At long last, I publish my default settings I am using for testing. All files including data files are included, but not all the gsb exe,dll, xml files. Included is gsb.config.xml to get GSB into admin mode. There may be some settings that need a tweak, so consider this the first revision. There are 4 ES files with 5 ticks of noise, 4 ES files with 10 ticks of noise I think you can put this into the folder of your choice. (All paths relative, not fixed) 200mb in size zipped up. My top 11 indicators are selected, and these are better than any other method Ive found. However sorry to say, im not sure how I found this group. All macros used are included. In summary you build 50,000 systems. Dates are 2000 to 2015.6.30 nth 1 / 80 The manager then populates but with nth set to all (workers built them with nth no trade) Then macro 1 selects the top 250 systems and walk forwards them. Manually put the top 250 into favouritesD. Then run macro3 to get the stats. record these stats. Then run the macro5 astab-c macro that deselects all favouritesD, and puts the top 90 astab-c Then run macro3 to get the stats. The record these stats. go to favourites A Then run macro2 which changes dates back to 2015.6.30 after this is complete, run macro6 which selects the top 91 vss systems of the top 250 Then run macro3 to get the stats Shown below is a summary of my stats. zip file here of all my files in private forum https://trademaid.info/forum/viewthread.php?tid=241#pid4133 Hi Peter, I followed the steps described above 1. When I ran macro 5, 165 systems were selected, is it suppose to select the top 90 based on astab-c (what's astab-c, I don't see it on the columns?) 2. When I ran macro 6 no systems are selected, any ideas why? Thanks

Quote: Originally posted by admin

Quote: Originally posted by appengineer   Quote: Originally posted by admin   At long last, I publish my default settings I am using for testing. All files including data files are included, but not all the gsb exe,dll, xml files. Included is gsb.config.xml to get GSB into admin mode. There may be some settings that need a tweak, so consider this the first revision. There are 4 ES files with 5 ticks of noise, 4 ES files with 10 ticks of noise I think you can put this into the folder of your choice. (All paths relative, not fixed) 200mb in size zipped up. My top 11 indicators are selected, and these are better than any other method Ive found. However sorry to say, im not sure how I found this group. All macros used are included. In summary you build 50,000 systems. Dates are 2000 to 2015.6.30 nth 1 / 80 The manager then populates but with nth set to all (workers built them with nth no trade) Then macro 1 selects the top 250 systems and walk forwards them. Manually put the top 250 into favouritesD. Then run macro3 to get the stats. record these stats. Then run the macro5 astab-c macro that deselects all favouritesD, and puts the top 90 astab-c Then run macro3 to get the stats. The record these stats. go to favourites A Then run macro2 which changes dates back to 2015.6.30 after this is complete, run macro6 which selects the top 91 vss systems of the top 250 Then run macro3 to get the stats Shown below is a summary of my stats. zip file here of all my files in private forum https://trademaid.info/forum/viewthread.php?tid=241#pid4133 Hi Peter, I followed the steps described above 1. When I ran macro 5, 165 systems were selected, is it suppose to select the top 90 based on astab-c (what's astab-c, I don't see it on the columns?) 2. When I ran macro 6 no systems are selected, any ideas why? Thanks 1) you need to wf the 250 systems in fav a to get astab 2) you need to do verifcation of the systems in fav A. run macro 2 before you do this

I already did WF for the systems in Fav A.
Is the verification step manually done? When was it supposed to be done, is before running macro5?

Thanks

Quote: Originally posted by admin

ideally its best to run verification after the top 250/500 are in in fav a. if you dont, then run macro 2 which puts dates back to 2015.6.30 Yes its manually done. Select the 250 systems, right click and do verify.

here is astabc. Your correct that memory is not freed up as much as it should. This is on the todo list.

Quote: Originally posted by Daniel UK1

Peter, would it perhaps be good to make a short video about the Process for testing with the macros etc, your reasoning behind he steps that you shared recently?

Quote: Originally posted by admin

here is astabc. Your correct that memory is not freed up as much as it should. This is on the todo list.

Quote: Originally posted by appengineer

Quote: Originally posted by admin   here is astabc. Your correct that memory is not freed up as much as it should. This is on the todo list. Thanks Peter It's on the Walk Forward tab, was looking for it in Fav A Also macro 5 has to be run when in Walk Foward tab. Is the Walk Forward data saved? GSB suddenly closed..

Quote: Originally posted by admin

Quote: Originally posted by appengineer   Quote: Originally posted by admin   here is astabc. Your correct that memory is not freed up as much as it should. This is on the todo list. Thanks Peter It's on the Walk Forward tab, was looking for it in Fav A Also macro 5 has to be run when in Walk Foward tab. Is the Walk Forward data saved? GSB suddenly closed.. There are settings on the left side to save systems. default is off as it takes too much disk space. Check your windows event log for what happened. ie ran out of ram etc yes, macros can be run from wf or favA in most but not all cases

Quote: Originally posted by admin

appenginer, programer says "For this user a .net framework or win repair could fix it" There are no reports of this error from other uses. If that fails reset the ram modules in the mother board and or reload win 10

I built a new ES system this week. Here are results. Out of sample after 2019/2/28
results hypothetical but included $25 slippage and $4.80 round turn brokerage







Attachment: Login to view the details