| Pages:
1
2
3
4
5
..
47 |
cotila1
Junior Member
Posts: 78
Registered: 8-5-2017
Member Is Offline
Mood: No Mood
|
|
Quote: Originally posted by admin  | Finally, new video on market testing, nth feature, date range * inverse date range, and most importantly- A method to validate if a market is good for
GSB.
The method is fast & simple.
Please try this on other markets too.
The video is not as polished as I would like, but I hope it will be useful.
There is a minute of duplicated content that needs to be removed.
File is now fixed
|
Very good Video! thank u.
Just one confirmation: when I set ''Nth day'' to a choosen value (say 1 for instance), then this is applied to training +test periods together and not
only to training period solely, am I correct?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cotila1 |
Very good Video! thank u.
Just one confirmation: when I set ''Nth day'' to a choosen value (say 1 for instance), then this is applied to training +test periods together and not
only to training period solely, am I correct? |
Yes, it applies to all periods.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
The videos were informative. I agree with the contention that one should focus on implementing a strategy that produces a high proportion of
attractive systems, rather than doing extensive runs. As Big Dog mentioned on another thread, such a data-mining approach may not produce successful,
robust systems.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cyrus68 | | The videos were informative. I agree with the contention that one should focus on implementing a strategy that produces a high proportion of
attractive systems, rather than doing extensive runs. As Big Dog mentioned on another thread, such a data-mining approach may not produce successful,
robust systems. |
Thanks for the comments. I have spend much of the day testing this methodology on ES500, and doing minor tweaks.
ie 15 min vs 30 min. No data2 etc.
Likely I will publish the results in the private forum. Still fine tuning some details before I share results.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
I was intrigued by some earlier references to the System Quality Number. Being unfamiliar with it, I searched the net and found the following
reference:
https://www.aussiestockforums.com/threads/strategy-performan...
In it, Howard Bandy, who has done lots of work on performance testing, offers the sound suggestion to use the t statistic on OOS results.
I agree with him.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
My tests for CL looked good. Long looked excellent, short failed, and long and short together were ok.
I haven't checked in detail, but my CL systems overall have been good out of sample.
No systems were terrible in results and some excellent.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
I have 7 CL systems
The worst cl system had pl -$50 less slippage and commission.
All systems our of sample from December to Jan 23 2018
This was the best system
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
Good to hear that you have found some attractive CL systems. May I suggest an additional metric in GSB for ranking system performance.
The t statistic, calculated on the P/L, by trade, of the Trd curve would give the best overall measure of OOS system performance.
I think this would constitute an important addition to GSB's capabilities.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
whats "The t statistic"
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
Check out the web link I gave in my earlier post.
In it, Howard Bandy describes what it is, and how to calculate it.
It is a better alternative to the System Quality Number.
I have a background in Econometrics. So this stuff comes naturally.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
I will talk to programmer about T stats.
Here is a vix system I built. Zero slippage used.
Out of sample 20171020
I havnt dont market tests on vix
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
Why short only? Did you try long/short?
Alternatively, you may prefer directional bets on the short side
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cyrus68 | Why short only? Did you try long/short?
Alternatively, you may prefer directional bets on the short side |
Long will not work well on vix. If you look at the chart you will see strong downward trend.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
You're right. Over that period the trend has been down.
You don't hold overnight positions. So you are less exposed to upward spikes, which are more likely in the future.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
44.64 is out with improved internal nth methodology and nth metrics now dont have to used via a mouse over.
http://trademaid.info/forum/viewthread.php?tid=39&page=6
I have found that ES works great (no surprise)
ES works better long than short, but both are ok, and long a little better than long and short.
CL works well (long & short), and even better if long only. Cl short was a fail.
I want to finish and finalize the video on this in < 1 week. I'm still doing more tests.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Here is the part2 draft video of updated validation method.
Its best to look at part1 as well, though the methodology has been refined somewhat.
Part 1 was on the forum a few weeks ago.
search for
Market_testing1f_r1.01.mp4
Attachment: Login to view the details
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
CORRECTION TO CONTENT OF VIDEO.
My CL session times were not correct, so Nth oos metrics have improved. Will publish them tomorrow
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Update to market methodology with the new build of GSB features.
Please note.
The last section on crude oil needs a complete re vamp. Its not explained clearly.
This video says that SESSTIONTIMECALCS.ELD is explained in this video, but it is not. Its covered in the eld comments and the previous video in this
thread. I am Unexpectedly over seas for roughly another week, so cant finish this off.
SESSTIONTIMECALCS.ELD is with all the TS code that comes with GSB.
Attachment: Login to view the details
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
The video proposes a useful test that I may adopt because it is quite fast. However, I would like to note that it is a test of the viability of a
particular strategy applied to a given symbol. For example, in the case of ES, if you selected 15-minute data for the primary symbol, different
secondary datasets, 5 indicators and 2 operators – this would constitute a different strategy. And let’s not forget different historical periods over
which the test may be carried out. The test may produce different results compared with your first strategy.
One thing I noticed is that, for ES, you set the max # of unique systems at 600. Yet at the end you had 1735. Either GSB didn’t stop or you let it run
further. I have the same problem with max # completed. Typically, I set it at 5 million. So, if GSB is processing 35,000 a minute, it should stop
after 2 hours and 20 minutes. That doesn’t happen. So, I have to stop it manually.
Currently I use a rather biased, but simple method, to gauge the overall viability of a strategy. I look at the proportion of WF results with PAS
scores above 50. If, out of 60 WF, there are 40 with scores above 50 - and most are bunched closer to 100 – that is a good sign. If there are only 10
or 15 with acceptable scores, that is not a good sign. The test is biased because WF is carried out on systems that are pre-selected on the basis of
attractive metrics.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
I would also like to propose a much simpler way of comparing the IS and OOS results. This could be done via the average t-value of the IS and OOS
systems. This is a system quality number that I proposed for inclusion in GSB, a while back.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cyrus68 | | I would also like to propose a much simpler way of comparing the IS and OOS results. This could be done via the average t-value of the IS and OOS
systems. This is a system quality number that I proposed for inclusion in GSB, a while back. |
Is this it? If not for all the viewers of this thread can you re-state it?
What do we do or risk? I know its a minor point, but I personally use very big stops that rarely get hit, so not sure if the risk needs tweaking.
I have a question however regarding the system quality number Van Tharp uses to determine if a strategy is good or not. Basically it is a creative way
of working out whether a strategy will perform well long term, based on x amounts of trades. The premise behind it is that it relates everything back
to how much you are risking on each trade, in terms of "R" multiples (where "R" is the amount risked). You determine your average R profit over all
trades, and also the std deviation of these trades. This is then used along with the number of trades to work out your SQN :
SQN = average of trades in terms of R multiples / stddev of r multiple trades * sqrt(number of trades).From Van Tharp, if your SQN > 1.7 you
have a system that "statistically" should generate profits.
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
I think I remember Tharp uses 100 trades in his calculation.
So if SQN = 1.7, then the average of trades in terms of R multiples / stddev of r multiple trades = 0.17
If the risk is the same for all trades (so a dollar stop), the SQN is equal to 1 divided by the number in the TS strategy report, tab "Trade
analysis", "coefficient of variation"
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by Carl |
I think I remember Tharp uses 100 trades in his calculation.
So if SQN = 1.7, then the average of trades in terms of R multiples / stddev of r multiple trades = 0.17
If the risk is the same for all trades (so a dollar stop), the SQN is equal to 1 divided by the number in the TS strategy report, tab "Trade
analysis", "coefficient of variation"
|
shouldnt we use in sample number of trades & the out of sample number of trades to give us IS sqn and OOS sqn?
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
I gave a web link, in an earlier post, where Howard Bandy explains why we should use the t statistic instead of Tharp's SQN. Here it is again:
https://www.aussiestockforums.com/threads/strategy-performan...
Bandy's command of these issues is better than Tharp's. He also explains how to calculate it.
It is also possible to do a Wilcoxon test of paired sample values of IS and OOS. My preference is for the t test, because we can rank the systems
according to their quality, as well as get the average for all systems.
Of course, you can implement both tests. But I suspect your programmer will have hiccups.
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
Hi Cyrus68,
The web link in your post seems to be truncated
The full link is:
https://www.aussiestockforums.com/threads/strategy-performan...
|
|
|
| Pages:
1
2
3
4
5
..
47 |
![[*]](images/davis/default_icon.gif){kind=icon}
