| Pages:
1
..
7
8
9
10
11
..
98 |
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Quote: Originally posted by Gregorian  | Just want to share with fellow GSB users a very important concept that was not obvious to me and therefore might not be evident to everyone else:
If you want your strategy to perform well going forward live, you must:
1. Go through the WF procedure as Peter has explained here and in his videos.
2. Use only strategies whose PAS is at least 40, preferably 50 or higher.
3. Use only strategies whose WF optimal and WF OOS curves are close or similar to one another.
I've wasted a lot of time trying out strategies that looked appealing but did not meet all of these criteria and ultimately lost money going forward.
While conducting this sort of due diligence takes time, the odds of your ending up with a reliable strategy going forward will be greatly increased.
It's also worth pointing out that TSL, SQ, and AT either do not offer WF at all or fail to emphasize its importance, so on this essential front GSB
stands out from its peers. |
Thanks for restating this. I am a very big believer in the WF process.
50 parameter anchored stability is ideal, but I do go lower in some circumstances. if you get a parameter that goes from 99 to 98, it will drop the
stability score. The reality is this 1% change in settings won't change things significantly.
This is touched on in this video.
https://www.youtube.com/watch?v=UNAEy1wYgho&t=22s
There are more ideas in the GSB pipeline in this area to come.
However my logic is, the higher the parameter stability score, the more likely it is to go well out of sample. This doesn't guarantee success but it
helps. Now GSB system portfolio can be very diversified.
We have nat gas, crude oil, soybeans, copper, vxx, ES, NQ etc
Sorry you lost some $, but systems also need time to work, and loosing money for a period that matches in sample performance is to be expected.
Market conditions are another. ES has been flat for a long time, but of recent systems are going to either make or loose more that is normal.
Another tip is use entire contract data length from where data is not patchy. Exception is ES where I often use 2001, but 1997 is also valid.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
Interesting observations by Gregorian. I have 2 examples from my recent experiments with NG. One system has excellent stability scores, but its recent
performance in the market is mediocre.
The second system has rather modest stability scores, but good performance in recent market conditions. Its Pearson score on the optimal curve is
0.993. Also, visually, you can see that its Pearson score post-2010 looks to be excellent. The parameters are stable from period 7 onward.
My own inclination would be to trade the second system but monitor its performance, and replace it with another when its performance deteriorates.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
I tried to post the images via Imgur but it did not work. Maybe the attachments will work.
   
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
cyrus68: Yes, even after all three WF steps have been performed, a strategy can still disappoint. However this is the best way to increase a
strategy's chances of success. Your concept of monitoring each strategy is sound.
As Peter suggested, a diversified set of strategies that have passed these steps is probably the best way to achieve ongoing success. Likely not more
than a few of them will fail, so the good ones should help you achieve overall profitability.
I often wonder if the big boys at Goldman Sachs et al go through this sort of analysis with the tools they have. They may have vast computing power at
their disposal, but the underlying principles and challenges they face are probably the same.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Im sympathetic to your argument. The first curve is not such a consistent performer. Not all runs are profitable.
Second system you could stick a ruler on the line and see its just nice and linear past the volatile 2008 period.
You can have 100% stability, but the curves are all poor. Doesn't happen a lot, but it does happen. Note in your second parameter values, you cant see
the right most side. No big deal for the forum post but make sure you check them all.
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
I think Goldman Sachs has many more ways to evaluate trading performance.
Next to walkforward which is more than 10 years in Amibroker, this dr Howard Bandy has for example 2 more additional measures.
http://blueowlpress.com/system-development/all-things-being-...
This measure is used in the Amibroker community and some users do have reported better and more robust OOS performance
| Quote: Originally posted by Gregorian | cyrus68: Yes, even after all three WF steps have been performed, a strategy can still disappoint. However this is the best way to increase a
strategy's chances of success. Your concept of monitoring each strategy is sound.
As Peter suggested, a diversified set of strategies that have passed these steps is probably the best way to achieve ongoing success. Likely not more
than a few of them will fail, so the good ones should help you achieve overall profitability.
I often wonder if the big boys at Goldman Sachs et al go through this sort of analysis with the tools they have. They may have vast computing power at
their disposal, but the underlying principles and challenges they face are probably the same. |
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
I think Goldman Sachs has many more ways to evaluate trading performance.
Next to walkforward which is more than 10 years in Amibroker, this dr Howard Bandy has for example 2 more additional measures.
http://blueowlpress.com/system-development/all-things-being-...
This measure is used in the Amibroker community and some users do have reported better and more robust OOS performance
| Quote: Originally posted by Gregorian | cyrus68: Yes, even after all three WF steps have been performed, a strategy can still disappoint. However this is the best way to increase a
strategy's chances of success. Your concept of monitoring each strategy is sound.
As Peter suggested, a diversified set of strategies that have passed these steps is probably the best way to achieve ongoing success. Likely not more
than a few of them will fail, so the good ones should help you achieve overall profitability.
I often wonder if the big boys at Goldman Sachs et al go through this sort of analysis with the tools they have. They may have vast computing power at
their disposal, but the underlying principles and challenges they face are probably the same. |
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
I didnt fully understand that article, but it part of it seems sensible money management and trying to figure out true risk.
I'm open to more education on it.
Now I have so many new and diversified markets, I'm trading a max of 2 contracts on any one system (normally 1), and spreading capital fairly evenly
over diversified markets. Portfolio Analyst Pro is excellent at doing this now as you can limit contracts per system, and total per symbol.
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
Gregorian: I am not about to trade either of the 2 systems. They were just examples. The main point is that linearity matters.
You are absolutely right about having a diversified portfolio and testing its performance. Peter's PA is useful, and you can also do quite a lot in
Excel. This can be complementary. I also use MSA for various things. I don't use TS's portfolio maestro.
Institutions such as Goldman often use Var or cVar for risk assessment. cVar is better. What I currently can't do - and they are capable of doing - is
to dynamically adjust portfolio weights according to changing market conditions.
rws: I got Bandy's book 2 years ago and unfortunately laid it down after a few chapters. I will look at it again. He was into Python programming and
I'm more into Matlab.
|
|
|
Carl
Member
Posts: 342
Registered: 10-5-2017
Member Is Offline
Mood: No Mood
|
|
So what we need to investigate this is a bunch of systems and their metrics.
See what systems are still profitable in the future and investigate their historical metrics.
The backtest and WF results in GSB are great.
The only thing I am worried about is the fact that a GSB system on for example CL isn't profitable on highly correlated RB. And vice versa. Some
traders use this multimarket test to see if a strategy is robust and not overfit.
Any thoughts?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by Carl | So what we need to investigate this is a bunch of systems and their metrics.
See what systems are still profitable in the future and investigate their historical metrics.
The backtest and WF results in GSB are great.
The only thing I am worried about is the fact that a GSB system on for example CL isn't profitable on highly correlated RB. And vice versa. Some
traders use this multimarket test to see if a strategy is robust and not overfit.
Any thoughts? |
Often the same can be said for emd vs Russell 2000, but it doesnt always apply. ES SOMETIMES works on Er and EMD as well.
What you say about CL & RB MIGHT work on some systems, but not others.
GSBsys1ES also was ported to NQ (Emini nasdaq), but the data2 was changed and the parameter set also changed. 11 months out of sample results were
excellent on NQ.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
You could test how well the nq went on es without changing parameters, and optionally change the data2.
Regardless of the results, I dont see that as a conclusive test. From memory the ER wasnt so good on gsbsys1ES, which is closer than the NQ overall.
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
Because GSB is a peak optimizer it is always hard to have consistent performance. It is always the best to search for a stable range of parameters
which may but should be not be the parameter with the highest performance if this is a peak. I understand doing WF is a good way of looking for stable
parameters but no guarantee. After having the stable range of parameters the performance should be stable with small variations of these parameters
just like a market could change.
Instead of optimizing for 1 index, I optimize (non peak optimizer) on all the tickers of an index and sofar my impression is that this gives more
robust OOS performance. Depending on how many stocks in the index have a buy signal and the weight of them I am currently testing this idea.
| Quote: Originally posted by admin | You could test how well the nq went on es without changing parameters, and optionally change the data2.
Regardless of the results, I dont see that as a conclusive test. From memory the ER wasnt so good on gsbsys1ES, which is closer than the NQ overall.
|
|
|
|
cyrus68
Member
Posts: 171
Registered: 5-6-2017
Member Is Offline
Mood: No Mood
|
|
A useful feature would be Monte Carlo simulation that allowed parameter variation, within limits. Currently, the only kind of Monte Carlo simulation
that is possible is price randomisation, done outside GSB.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cyrus68 | | A useful feature would be Monte Carlo simulation that allowed parameter variation, within limits. Currently, the only kind of Monte Carlo simulation
that is possible is price randomisation, done outside GSB. |
The newest version of EWFO has updates to the nth feature.
You can see a graph of the 1 to the 50th (for example) best parameter set as 50 equity lines. Lot of work to do compared to a WF in GSB.
What is Nth. It shows the best to the nth best paramater set and plots it. TO do this you would have to re-optimize your GSB (or other system) in TS.
That would take a lot of time as TS is slow. At leaset you have an idea of the paramter range expected from GSB own WF. I'm not convinced its worth
the effort.
There is always more improvements to be made in methodology, but I feel the current GSB setup is very fast from a human perspective and robust.
Nothing however is bullet proof and there is always risk.
Some markets are easier than others and the harder markets can be very CPU intensive requiring GSB to spend a lot of processing power.
Copper is deadly if you don't do a WF. However I've found the diversification in copper excellent compared to other markets. Portfolio analyst chooses
a lot of copper systems compared to other markets. This is likely my next video.
|
|
|
rws
Member
Posts: 114
Registered: 12-6-2017
Member Is Offline
Mood: No Mood
|
|
Peter,
I understand you looked at Adaptrade, why not look at the these additional confirmation tool like price randomisation and Monte Carlo that is in
Adaptrade. It could be an additional confirmation which would make GSB better.
| Quote: Originally posted by cyrus68 | | A useful feature would be Monte Carlo simulation that allowed parameter variation, within limits. Currently, the only kind of Monte Carlo simulation
that is possible is price randomisation, done outside GSB. |
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by rws | Peter,
I understand you looked at Adaptrade, why not look at the these additional confirmation tool like price randomisation and Monte Carlo that is in
Adaptrade. It could be an additional confirmation which would make GSB better.
| Quote: Originally posted by cyrus68 | | A useful feature would be Monte Carlo simulation that allowed parameter variation, within limits. Currently, the only kind of Monte Carlo simulation
that is possible is price randomisation, done outside GSB. |
|
Portfolio analyst pro has Monte Carlo. While I don't mind features such as price randomization, im not sure to what degree its helpful.
There are also much more important issues in the job que.
The most important and significant is more secondary filters and pattern filters etc. Plus other features like enhanced order exits.
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
Exit at Daily Profit Goal
On more than a few days, my portfolio of GSB strategies will, for example, achieve a peak profit of around $1,600, then close the day at only around
$800 of profit. Some other auto-trading systems have a notion of a "Daily Goal", where trading is stopped and positions are closed when a daily
target profit level (or loss) is achieved.
I've done extensive testing with [simple, not ATR] trailing stops - using my own code, as the built-in EL call backtests too optimistically - but have
never gotten them to increase the profit of my strategies, on a strategy-by-strategy basis. Experience suggests the Daily Goal concept would be
better, however, especially when applied across all strategies being run.
Has anybody had any success with this concept? It would be easy to write into a GSB strategy, but I don't know how to write a routine that would
control all strategies. This concept works best when profit across all strategies/charts is the criterion.
Have you found a third-party strategy that accomplishes this across all charts? All I've found so far is "DailyProfitTarget_ForAllStrategies" in the
TS App Store, but it only works on the strategy(s) on a single chart, not across all open charts.
Peter has already stated that this is not on his to-do list, so for the time being any solution has to come from elsewhere.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by Gregorian | On more than a few days, my portfolio of GSB strategies will, for example, achieve a peak profit of around $1,600, then close the day at only around
$800 of profit. Some other auto-trading systems have a notion of a "Daily Goal", where trading is stopped and positions are closed when a daily
target profit level (or loss) is achieved.
I've done extensive testing with [simple, not ATR] trailing stops - using my own code, as the built-in EL call backtests too optimistically - but have
never gotten them to increase the profit of my strategies, on a strategy-by-strategy basis. Experience suggests the Daily Goal concept would be
better, however, especially when applied across all strategies being run.
Has anybody had any success with this concept? It would be easy to write into a GSB strategy, but I don't know how to write a routine that would
control all strategies. This concept works best when profit across all strategies/charts is the criterion.
Have you found a third-party strategy that accomplishes this across all charts? All I've found so far is "DailyProfitTarget_ForAllStrategies" in the
TS App Store, but it only works on the strategy(s) on a single chart, not across all open charts.
Peter has already stated that this is not on his to-do list, so for the time being any solution has to come from elsewhere. |
Long them, I think the idea is a bad one sorry to say. Look at 2018. There have been speculator big moves and it means you would only get part of
them. You will get all the choppy moves.
It might work better with daily loss .
Its not the same but related. Yesterday I put 2 copper systems on the same chart. Much better metrics, no increase in dd, but lost 30% of the over all
profit compared to running them in separate charts. Lost about 6 k extra profit in the last year. Basically when there is a big profitable move, most
systems trigger
|
|
|
cotila1
Junior Member
Posts: 78
Registered: 8-5-2017
Member Is Offline
Mood: No Mood
|
|
I've got PAS = 0 for the WF-ed system, however there is an high parameters stability (see the other screenshot)
Is it not a bit strange in this case that the PAS is equal to 0 despite the high parameters stability ??
 
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cotila1 | I've got PAS = 0 for the WF-ed system, however there is an high parameters stability (see the other screenshot)
Is it not a bit strange in this case that the PAS is equal to 0 despite the high parameters stability ?? |
Anchored compares the last line to the others, and every case is a miss-match. Rolling compares the lines next to each other.
The last line as a very big change in parameters. You could compare the last settings and second the last parameters in ts and compare the equity
curves. Unless results don't change significantly, I would not use that system
|
|
|
cotila1
Junior Member
Posts: 78
Registered: 8-5-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by admin | | Quote: Originally posted by cotila1 | I've got PAS = 0 for the WF-ed system, however there is an high parameters stability (see the other screenshot)
Is it not a bit strange in this case that the PAS is equal to 0 despite the high parameters stability ?? |
Anchored compares the last line to the others, and every case is a miss-match. Rolling compares the lines next to each other.
The last line as a very big change in parameters. You could compare the last settings and second the last parameters in ts and compare the equity
curves. Unless results don't change significantly, I would not use that system |
Thanks, so in general what are the minimum PRS and PAS values you would reccomend?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by cotila1 | | Quote: Originally posted by admin | | Quote: Originally posted by cotila1 | I've got PAS = 0 for the WF-ed system, however there is an high parameters stability (see the other screenshot)
Is it not a bit strange in this case that the PAS is equal to 0 despite the high parameters stability ?? |
Anchored compares the last line to the others, and every case is a miss-match. Rolling compares the lines next to each other.
The last line as a very big change in parameters. You could compare the last settings and second the last parameters in ts and compare the equity
curves. Unless results don't change significantly, I would not use that system |
The higher the better, but i prefer 30 or more, the last parameters not changing significantly. But a change like for 95 to 96 will detract from the
stability score, but in real terms such a small change is not significant. I'm less concerned with rolling stability.
I think I spoke on this in the last youtube video I did.
Thanks, so in general what are the minimum PRS and PAS values you would reccomend? |
|
|
|
Gregorian
Junior Member
Posts: 97
Registered: 23-5-2017
Member Is Offline
Mood: No Mood
|
|
44.09 Workers don't start
Since upgrading to 44.09, my workers on the same machine as the manager do not start. The manager remains in Waiting mode. None of the Workplace
settings were changed on the workers or manager. The Standalone version works fine. Suggestions?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by Gregorian | | Since upgrading to 44.09, my workers on the same machine as the manager do not start. The manager remains in Waiting mode. None of the Workplace
settings were changed on the workers or manager. The Standalone version works fine. Suggestions? |
I have the same issue regardless of what version.
Its taking about 10 to 15 minutes for the workers to start.
Im aiming to fix asap, but its a job for one of my programmers.
I rebooted the sql server, and it didnt help
|
|
|
| Pages:
1
..
7
8
9
10
11
..
98 |
![[*]](images/davis/default_icon.gif){kind=icon}
