GSB Forums

Filter for Training-period-only and avoiding cherry-picking systems

kiran - 20-11-2017 at 08:59 PM

How are the Unique Systems picked? I assume it's based on systems from each generation that pass the filters.

To avoid cherry-picking successful systems (i.e. PF>2 and Pearson>0.98) throughout Total (Training & Test) period, I changed the Filter in Tools->AppSettings to have PF>2 and Pearson>0.98 (as shown in attached screenshot), to see how many of these survive Test period.

Ironically, I'm only seeing systems that are successful in Test+Training (with Total_period PF>2) in Unique Systems.
- Is there some other filter that's not visible applied to Unique Systems?
- How do i find out the success rate through the evolution - i.e. (# of systems also successful in Test)/(Total # of systems successful during Training)?



TrainingOnlyFilters and UniqueSystems.PNG - 150kB

admin - 20-11-2017 at 10:37 PM

I tried this. Was a good experiment. No other filters apart from back test termination which should not affect things.
All non duplicate systems should pass to the GUI
You've only been running for 2 minutes, so you will get more good and bad systems over time.
So in your example you have 19 systems.
If your definition of success is net profit of 40k, put that in the min net profit and see the number 19 drop.
Your definition of success and how good GSB is, are the two big factors here.

kiran - 21-11-2017 at 01:08 AM

Still surprising, as i removed all filters from Training, Test, Total and Backtest termination. However, all the Unique systems reported have Total PF>1.8 (and Test Period PF>1.6), which is really good.
- Statistically, the Builder should generate good systems (that survive the Test period) and bad systems (that don't survive Test period due to over-fitting), as I've seen with most genetic programming builders.

Is there some self-selection of indicators or some other filter or setting, that's ensuring that only robust systems (i.e. those that have performed well in Test and Training period) are filtered in Unique systems tab?

admin - 21-11-2017 at 02:46 AM

Quote: Originally posted by kiran  
Still surprising, as i removed all filters from Training, Test, Total and Backtest termination. However, all the Unique systems reported have Total PF>1.8 (and Test Period PF>1.6), which is really good.
- Statistically, the Builder should generate good systems (that survive the Test period) and bad systems (that don't survive Test period due to over-fitting), as I've seen with most genetic programming builders.

Is there some self-selection of indicators or some other filter or setting, that's ensuring that only robust systems (i.e. those that have performed well in Test and Training period) are filtered in Unique systems tab?

There should be no other filters running apart from back test termination. These should not affect any of this on the default settings.
ES 15 is a very easy market for GSB. Other markets can be harder.

REMO755 - 16-4-2023 at 11:26 AM

Hello,


What do I have to do if I search for system in Performance Filter Validation P.F > 2?
Where are the validation statistics?
Do I have to configure the Optimization module?

Is the explanation possible?

Thank you.

REMO755 - 16-4-2023 at 04:31 PM

I have performance training: Pearson MIn = 0.9
Gsb is giving me systems with R-F 0.068, I don't understand anything.

a little light how to use performance filters?

Captura.JPG - 272kB

admin - 16-4-2023 at 06:09 PM

@Remo
my guess is you built systems with nth 80 meaning 80 days in sample, 80 days out of sample
Then when nth is changed to all, results drop massively because you are now also looking at the 80 days out of sample which has poor results on the system you selected.
hence rf0.068
It is critical we dont filter out the oos results, as if these are good are bad, we need to know the truth

Big picture, a lot of work is being done on Cl by some of the most experienced GSB users, and I hope in time some of this will be shared.
The work was paused due to gsb to ts missmatch issues, much (but far from all) of which only showed when CL price was < 0
Most of these issues are fixed in 65.55 build which is released today.
It would be good for you to read more here, https://trademaid.info/gsbhelp/Methodology.html
so you have more understanding