kiran
Junior Member
Posts: 19
Registered: 10-11-2017
Member Is Offline
Mood: No Mood
|
|
Filter for Training-period-only and avoiding cherry-picking systems
How are the Unique Systems picked? I assume it's based on systems from each generation that pass the filters.
To avoid cherry-picking successful systems (i.e. PF>2 and Pearson>0.98) throughout Total (Training & Test) period, I changed the Filter in
Tools->AppSettings to have PF>2 and Pearson>0.98 (as shown in attached screenshot), to see how many of these survive Test period.
Ironically, I'm only seeing systems that are successful in Test+Training (with Total_period PF>2) in Unique Systems.
- Is there some other filter that's not visible applied to Unique Systems?
- How do i find out the success rate through the evolution - i.e. (# of systems also successful in Test)/(Total # of systems successful during
Training)?
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
I tried this. Was a good experiment. No other filters apart from back test termination which should not affect things.
All non duplicate systems should pass to the GUI
You've only been running for 2 minutes, so you will get more good and bad systems over time.
So in your example you have 19 systems.
If your definition of success is net profit of 40k, put that in the min net profit and see the number 19 drop.
Your definition of success and how good GSB is, are the two big factors here.
|
|
|
kiran
Junior Member
Posts: 19
Registered: 10-11-2017
Member Is Offline
Mood: No Mood
|
|
Still surprising, as i removed all filters from Training, Test, Total and Backtest termination. However, all the Unique systems reported have Total
PF>1.8 (and Test Period PF>1.6), which is really good.
- Statistically, the Builder should generate good systems (that survive the Test period) and bad systems (that don't survive Test period due to
over-fitting), as I've seen with most genetic programming builders.
Is there some self-selection of indicators or some other filter or setting, that's ensuring that only robust systems (i.e. those that have performed
well in Test and Training period) are filtered in Unique systems tab?
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Quote: Originally posted by kiran  | Still surprising, as i removed all filters from Training, Test, Total and Backtest termination. However, all the Unique systems reported have Total
PF>1.8 (and Test Period PF>1.6), which is really good.
- Statistically, the Builder should generate good systems (that survive the Test period) and bad systems (that don't survive Test period due to
over-fitting), as I've seen with most genetic programming builders.
Is there some self-selection of indicators or some other filter or setting, that's ensuring that only robust systems (i.e. those that have performed
well in Test and Training period) are filtered in Unique systems tab?
|
There should be no other filters running apart from back test termination. These should not affect any of this on the default settings.
ES 15 is a very easy market for GSB. Other markets can be harder.
|
|
|
REMO755
Member
Posts: 181
Registered: 11-4-2021
Member Is Offline
|
|
Hello,
What do I have to do if I search for system in Performance Filter Validation P.F > 2?
Where are the validation statistics?
Do I have to configure the Optimization module?
Is the explanation possible?
Thank you.
|
|
|
REMO755
Member
Posts: 181
Registered: 11-4-2021
Member Is Offline
|
|
I have performance training: Pearson MIn = 0.9
Gsb is giving me systems with R-F 0.068, I don't understand anything.
a little light how to use performance filters?
|
|
|
admin
Super Administrator
Posts: 5069
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
@Remo
my guess is you built systems with nth 80 meaning 80 days in sample, 80 days out of sample
Then when nth is changed to all, results drop massively because you are now also looking at the 80 days out of sample which has poor results on the
system you selected.
hence rf0.068
It is critical we dont filter out the oos results, as if these are good are bad, we need to know the truth
Big picture, a lot of work is being done on Cl by some of the most experienced GSB users, and I hope in time some of this will be shared.
The work was paused due to gsb to ts missmatch issues, much (but far from all) of which only showed when CL price was < 0
Most of these issues are fixed in 65.55 build which is released today.
It would be good for you to read more here, https://trademaid.info/gsbhelp/Methodology.html
so you have more understanding
|
|
|
![[*]](images/davis/default_icon.gif){kind=icon}
