Methodology of GSB

Training, Test, Validation

Hi Peter,

apologies for what might seem a simple question to you, but I'd be grateful for a bit of help. I understand walk forward optimisation, in sample and out of sample optimisations. I'm trying to get my head around that in context with the training and test sections of the equity curves. So let me see if I've got this right:

-GSB combines 3 to 5 different indicators out of a pre-loaded set of 36 or so, plus any extra customs ones a user loads up or you may include in the future. Lets say a set of three like: MACD, RSI and CCI for example.
-It then adjusts the values of these indicators (eg Fast or Slow period) within that set over their various permutations and combinations (or a sample of them) and for any systems that meet any filter criteria, these are displayed as unique systems in the table in the lower centre of the screen.
-After it has finished with that set, it replaces say the CCI with another indicator, like a stochastic, and then off it goes again.

Please correct me if I'm wrong.

What then is the difference between the training data, the test data the validation data and the walk forward optimisation that is able to be performed on those unique systems.

Thanks very much.

I'm enjoying the videos, working through the documentation and trying different aspects of the GSB to see what they do.

Best regards, Jason.

Quote: Originally posted by JasonT

Hi Peter, apologies for what might seem a simple question to you, but I'd be grateful for a bit of help. I understand walk forward optimisation, in sample and out of sample optimisations. I'm trying to get my head around that in context with the training and test sections of the equity curves. So let me see if I've got this right: -GSB combines 3 to 5 different indicators out of a pre-loaded set of 36 or so, plus any extra customs ones a user loads up or you may include in the future. Lets say a set of three like: MACD, RSI and CCI for example. -It then adjusts the values of these indicators (eg Fast or Slow period) within that set over their various permutations and combinations (or a sample of them) and for any systems that meet any filter criteria, these are displayed as unique systems in the table in the lower centre of the screen. -After it has finished with that set, it replaces say the CCI with another indicator, like a stochastic, and then off it goes again. Please correct me if I'm wrong. What then is the difference between the training data, the test data the validation data and the walk forward optimisation that is able to be performed on those unique systems. Thanks very much. I'm enjoying the videos, working through the documentation and trying different aspects of the GSB to see what they do. Best regards, Jason.

Some questions about IS/OOS 'testing' sections of GSB.

Hi Peter,

I understand that Training% is the amount of data that is used to optimise/maximise GSB fitness. That data is In Sample (IS). Is this correct?

Test% applies the optimal parameters found in the prior Training% IS data to unseen Out of Sample (OOS) data. There is no further optimisation/maximisation/changes in the parameters. So if the equity curve continues on a similar trajectory with similar performance attributes, then this shows the strategy has some resiliency and is expected to continue to perform similarly in the future (although it may not). Is this correct?

If the Validation% data is also OOS, what is its purpose? What additional value does the method expect it to add?

Note that I've got some additional questions about Walk Forward optimisation that I'll ask next, but I am assuming that building strategies using this Training/Test/Validation approach to IS and OOS data is a step prior to conducting walk forward optimisation on particular strategies that you like the look of (as shown in some of your videos).

Many thanks,

Jason.

P.S. I'm very glad I've purchased the software and looking forward to mastering this incredible capability you've developed. Thank you.

Quote: Originally posted by JasonT

Hi Peter, I understand that Training% is the amount of data that is used to optimise/maximise GSB fitness. That data is In Sample (IS). Is this correct? Test% applies the optimal parameters found in the prior Training% IS data to unseen Out of Sample (OOS) data. There is no further optimisation/maximisation/changes in the parameters. So if the equity curve continues on a similar trajectory with similar performance attributes, then this shows the strategy has some resiliency and is expected to continue to perform similarly in the future (although it may not). Is this correct? If the Validation% data is also OOS, what is its purpose? What additional value does the method expect it to add? Note that I've got some additional questions about Walk Forward optimisation that I'll ask next, but I am assuming that building strategies using this Training/Test/Validation approach to IS and OOS data is a step prior to conducting walk forward optimisation on particular strategies that you like the look of (as shown in some of your videos). Many thanks, Jason. P.S. I'm very glad I've purchased the software and looking forward to mastering this incredible capability you've developed. Thank you.

Hi happy you are happy and wish you the best with GSB. Your are on of two users I have meet in person.
Validation gives you the human, the change to look at another out of sample period. This is fraught with danger as we the human will abuse this and just go for the best results over the entire period. There would be merit in making gsb not sort or include validation in its default metric displayed. Bottom line is you dont need to use validation. Im leaning towards make good market validation tests, then build systems. Then wf the best ones, then chose one with good metrics, wf stability and validation on other time frames.
Soon we will have WF multiple time frames.