GSB Forums

Enhanced Walk Foward Optimizer updates. EWFO EXE

 Pages:  1    3

admin - 8-4-2018 at 06:57 PM

20210621.1 released via auto update when you open EWFO. This will be the last update for some time.
This build has a number of improvments, and very important bug fixes. All users should upgrade
If users did not install under c:\ewfo and they are not running as windows administrator, then the update won't occur successfully.
You most open again runing EWFO as windows adminstrator.

New fitness types are also included.


EWFO (With no installer file) 20210611.10 version

Attachment: Login to view the details


Full installer file. This should auto-update to EWFO-Pro 2020 version when run
Attachment: Login to view the details
This build will come via auto update next week.
It installs into c:\ewfo to save issues of auto update not working when you are not windows administrator.
The last build has bug fixes, and error traps and new feature of last run when bars mode used. To be documented later.


EWFO UPDATES. (exe file only) over write existing ewfo files
may 29 2021
All the new fitness types added into GSB
Important bug fixes
Attachment: Login to view the details



System quality number
This was put into EWFO at the request of a client. I have not used this myself yet.
My question to this client was, did SQN work well compared to net profit / drawdown or net profit * average trade.
"Absolutely YES !!!
At least with my own trading systems, I am experimenting that the table of SQN value is true and when there are setting having, at the same time minSQN > 2.5 and Rank > 15 the stability is really good and OOS data are very close with IS. On the other side using P/DD only (with the same trading system) it is more frequent to get better IS but less robust and then worst OOS.
The results are extremely accurate when the trading system has time filter and or day of week or other seasonality filter, in this case the core edge of market flow is exactly to generate very consistent trade statistics and then using SQN you should easily achieve values > 3.5 and very high stability."

admin - 8-4-2018 at 07:02 PM


boosted - 15-6-2018 at 02:25 PM

Doing some minimal testing with the added list of SQN Fitness Criteria in EWFO is giving me some interesting results.

Will you be adding the SQN Fitness Criteria to EWFO so those can be used in GSB WF?

I think it would be a nice addition.

boosted - 15-6-2018 at 02:26 PM

sorry, made a mistake. SQN is already in EWFO, will it be added to GSB?

admin - 15-6-2018 at 03:41 PM

Do you mean in fitness, or in performance metrics?
Anyone else tried it? EWFO is not used a lot as GSB WF is much faster.

boosted - 15-6-2018 at 04:00 PM

yes, both. Will SQN and metrics be added to GSB?

I prefer using GSB WF version than EWFO.

Also, while on the subject, does GSB allow me to create code in TS then somehow dump it into GSB to do WF work vs having
to use EWFO since GSB WF is faster as you say?

admin - 15-6-2018 at 09:17 PM

It's possible to add sqn but it's likely some time off as there are more pressing features.
You can't take ts code and wf in GSB. That's the whole purpose of ewfo.

boosted - 15-6-2018 at 11:37 PM

The SQN calculation that you included in EWFO, is that calculation Van Tharp, Bandy t-statistic or some other algorithmic measure?

admin - 15-6-2018 at 11:39 PM

Quote: Originally posted by boosted  
The SQN calculation that you included in EWFO, is that calculation Van Tharp, Bandy t-statistic or some other algorithmic measure?

Its Van Tharp. Was done by special request of a customer

Petzy - 16-6-2018 at 06:30 AM

Quote: Originally posted by admin  
Do you mean in fitness, or in performance metrics?
Anyone else tried it? EWFO is not used a lot as GSB WF is much faster.


I use sqn sometimes. I like to use EWFO to compare different metrics on different strategies.
Sometimes np/dd is best for one strategy and something else for another strategy. I dont believe that you can have one metric that is best for everything. That is what I use ewfo for anyway

boosted - 16-6-2018 at 07:33 PM

Hi Peter,

I was using EWFO tonight and happen to look at Stability Scores tab and noticed there are some of the Fitness Criteria missing in the list.

Is this something you can fix.

Would very much like to see Rank and MisSQN in the list of missing Stability Scores.

Thanks.

admin - 17-6-2018 at 04:24 PM

Quote: Originally posted by boosted  
Hi Peter,

I was using EWFO tonight and happen to look at Stability Scores tab and noticed there are some of the Fitness Criteria missing in the list.

Is this something you can fix.

Would very much like to see Rank and MisSQN in the list of missing Stability Scores.

Thanks.

This is all programmable by the user, but it was a bit fiddly to do it. Its been a long time since I looked at it - but I may add to the next release of EWFO

admin - 23-8-2018 at 10:43 PM

Quote: Originally posted by boosted  
Hi Peter,

I was using EWFO tonight and happen to look at Stability Scores tab and noticed there are some of the Fitness Criteria missing in the list.

Is this something you can fix.

Would very much like to see Rank and MisSQN in the list of missing Stability Scores.

Thanks.

This is now added in the 20180817 build

admin - 23-8-2018 at 10:46 PM

20180818 build has minor update to documentation, Sqn and other new fitness scores etc added into stability scores.
Note stability scores will take a long time as it does a new WF for every fitness used. GUI shows stuck on 87% for some time.



new-fitness.png - 14kB

admin - 2-1-2019 at 10:32 PM

all ewfo code has expired.
You need 20190103 version
This is a 21 day trial
File deleted, newest file at top of this thread.


boosted - 31-1-2019 at 02:09 PM

My EWFO is at least 1+ yrs old. I recently tried to use it but said my password had expired.

I am a paid client so does this EWFO update resolve the expiration issue?

admin - 31-1-2019 at 02:39 PM

Quote: Originally posted by boosted  
My EWFO is at least 1+ yrs old. I recently tried to use it but said my password had expired.

I am a paid client so does this EWFO update resolve the expiration issue?

It should fix it. If not get back to me and I will get a fix.

admin - 21-3-2019 at 07:49 PM

A free update to EWFO is in the pipeline. It will have cluster analysis like the TS wfo.exe

I am hoping to have a modification to TS wf efficiency metric. I think its useless as is.
It measure the gradient of the in sample curve compared to the out of sample.
If the systems starts in mid 2007 the later results will be really poor efficiency metric.
The same system if started in 2005 would give outstanding figure.
Market conditions need to be factored in.
Im also add a few tweaks over the TS version too.

admin - 25-3-2019 at 11:59 PM

Any one want to be a beta tester? Its nearly ready
Also has anchored and rolling parameter stability. in the past it was rolling only.

Shown here is results of GSBsys1es, the free GSB system





ewfo-cluster.png - 42kB

Bruce - 26-3-2019 at 04:08 PM

Quote: Originally posted by admin  
Any one want to be a beta tester? Its nearly ready
Also has anchored and rolling parameter stability. in the past it was rolling only.

Shown here is results of GSBsys1es, the free GSB system





You can put me down for a beta tester, this will this sit on my current license?

admin - 26-3-2019 at 04:12 PM

Quote: Originally posted by Bruce  
Quote: Originally posted by admin  
Any one want to be a beta tester? Its nearly ready
Also has anchored and rolling parameter stability. in the past it was rolling only.

Shown here is results of GSBsys1es, the free GSB system





You can put me down for a beta tester, this will this sit on my current license?

Correct, just over write all files in current ewfo folder. WIll send in a few hours if it passes testing

petrfin - 26-3-2019 at 04:53 PM

Perfect Peter!
please add me also as a beta tester.

cotila1 - 27-3-2019 at 02:29 AM

Thanks Peter, yes pls add me too

admin - 28-3-2019 at 03:37 AM

Here are some screen shots of todays version.
It has added equity curve match, and oos / is ratio match
rolling parameter stability, anchored parameter stability, average of anchored ad rolling parameter stability



ewfo-gui.png - 95kBtab-cluster.png - 22kB

admin - 15-5-2019 at 11:23 PM

New EWFO is out. 20190516.6 build
Has cluster anaylsis, abilty to do a WF backwards and more.
EXE at the top of this post.
updated documentation
https://trademaid.info/gsbhelp/EnhancedWalkForwardOptimizerf...

admin - 29-5-2019 at 06:29 PM

The next release of ewfo will have optional filters on .

GSB now can make the ewfo/ts files so there is no need to go through the slow and time consuming process of putting code into TS if you want to do your own walk forward.

Not many users do this. I think there is value in it, but its not essential.

EWFO has a lot more tools than GSB when it comes to walk forward


filter_outSOme.png - 130kBafter.png - 73kB

Bruce - 30-5-2019 at 09:56 PM

Quote: Originally posted by admin  
The next release of ewfo will have optional filters on .

GSB now can make the ewfo/ts files so there is no need to go through the slow and time consuming process of putting code into TS if you want to do your own walk forward.

Not many users do this. I think there is value in it, but its not essential.

EWFO has a lot more tools than GSB when it comes to walk forward


Can't wait to see this Peter!

admin - 28-6-2019 at 02:23 AM

New ewfo with batch mode.
see https://trademaid.info/gsbhelp/Batchmode.html


New build at top of thread. File removed.


admin - 30-7-2019 at 07:19 PM

New ewfo, You can build and optimize in TS or GSB a long short version.
Then you can optimize long only, short only and long and short combined.

This is NOT the same as just seeing the performance for long and short.

The reason is different parameters are in effect used for long and used for short

See this url for the documentation on it
https://trademaid.info/gsbhelp/LongShortWalkforward.html

Download the file at the top of this thead.

saycem - 31-7-2019 at 02:25 AM

Hi Peter
I'm selecting systems that I have run a standard gen alg multi thread wf in GSB that had good stab scores and then running them again as ewfo.
However when I load them into ewfo their oos equity curves are worse. Is ewfo harder to pass?


GSB wf.JPG - 104kBewf graph.JPG - 78kB

admin - 31-7-2019 at 02:35 AM

Quote: Originally posted by saycem  
Hi Peter
I'm selecting systems that I have run a standard gen alg multi thread wf in GSB that had good stab scores and then running them again as ewfo.
However when I load them into ewfo their oos equity curves are worse. Is ewfo harder to pass?


First thing is check your fitness.

gsb default = np*at. ewf default = np

saycem - 31-7-2019 at 03:20 AM

made a big difference - thx

saycem - 31-7-2019 at 04:45 PM

I'm running a number of different efwo multi thread in GSB and looking at the results in EWFO.

How do I get the script for a system that I'm happy with from GSB. There is no option for Tradesation WF scripts in the drop down and I'm not sure that the settings are matching between the two (highlight yellow)

ewf result.JPG - 330kBGSB result.jpg - 325kB

admin - 31-7-2019 at 06:26 PM

Hi Saycem,
there is no wf code in drop down as WF has not been done. So you are comparing pre wf parameters of GSB with post WF param from EWFO.

Even if this was not the case, there can be all sorts of valid reasons why the results between the two might not match.

Input ranges on WF may have been different, dates different to name just a few

saycem - 31-7-2019 at 09:00 PM

Are you not performing a WF when you select EWFO multi thread in GSB?
I open the file in EWFO and the results are similar to a normal GA WF from GSB ( but you can't compare them exactly bc you can't see the EWFO curves in GSB - it shows an error message)
I have a system that where the IS and OOS results are close in EWFO and it has also passed Cluster Analysis which I assume is a good result??
So I want to use this system and the param settings in TS.

select ewf.JPG - 66kB

admin - 31-7-2019 at 09:02 PM

Quote: Originally posted by saycem  
Are you not performing a WF when you select EWFO multi thread in GSB?
I open the file in EWFO and the results are similar to a normal GA WF from GSB ( but you can't compare them exactly bc you can't see the EWFO curves in GSB - it shows an error message)
I have a system that where the IS and OOS results are close in EWFO and it has also passed Cluster Analysis which I assume is a good result??
So I want to use this system and the param settings in TS.


My guess is a GSB_ewfo doesnt create the script like normal GSB script.
If so you can always do both.
Whats the error message?

saycem - 31-7-2019 at 09:19 PM

The error message attached pops up when you select the system in GSB and no WF curves come up.
But you can open fine in EWFO and see the IS OOS equity curves and do Cluster. But what do you do then when you want to trade the system? Not sure how you get a script to copy into TS?
I assume the 10 inputs that you can see in EWFO (yellow highlight) are the settings you need but in the GSB script there are 14 different variables - so how do I marry them up? Or is this incorrect?

error ewfo.JPG - 22kBewf result.JPG - 330kBGSB result.jpg - 325kB

admin - 31-7-2019 at 10:00 PM

Hi Saycem,
the names match up, and likely the order to. The data streams for each indicator are absent in EWFO as they were not optimized.

Not sure on the error message, but likely ok to ignore it.

saycem - 31-7-2019 at 11:33 PM

Ok so what do you think is better for robustness:

If system passes cluster in EWFO then use these settings and copy them into TS manually
OR
Just run GSB GA Multi Thread WF even you will get different astab scores each time you run it and pick the best one you can get?

admin - 1-8-2019 at 05:55 PM

Quote: Originally posted by saycem  
Ok so what do you think is better for robustness:

If system passes cluster in EWFO then use these settings and copy them into TS manually
OR
Just run GSB GA Multi Thread WF even you will get different astab scores each time you run it and pick the best one you can get?


Im not decided on this, and stability should be examined in closer detail if it fails.
I still like to manually optimize each input (via AI wf), then use this range and put into ts to to a wf via ewfo. Its not essential do do this, but Im still looking closely to see what works best, and refined what Im doing.
As I said before, astab-c is likely a little better than astab. I also liked vss score.
using 30 min bars, with 5 and 10 tick random noise. 4 data streams of 5 ticks, and 4 data streams of 10 ticks. I often pick vss >-(somewhere between 14 and 20)
Note this is all on ES, which is what im doing all my methodology testing on.

Bruce - 10-8-2019 at 09:42 PM


Hi Peter,

Unable to get the new version to render correctly on the screen. See the attached I'm unable to get to the 'Run' button.

What am I missing?


Screen Shot 2019-08-11 at 3.38.07 PM.png - 1MB

edgetrader - 11-8-2019 at 05:54 AM

Quote: Originally posted by admin  
The next release of ewfo will have optional filters on .

GSB now can make the ewfo/ts files so there is no need to go through the slow and time consuming process of putting code into TS if you want to do your own walk forward.

Not many users do this. I think there is value in it, but its not essential.

EWFO has a lot more tools than GSB when it comes to walk forward

Are the filters working in a way such that they hide the fitness functions in the stability spreadsheet that don't meet the filtering criteria?

I'd like to see filters built into the fitness functions themselves that work during walk-forward optimization. E.g. with a filter of at least 50 in-sample trades, do everything as usual if met, but set fitness to zero if less than 50 trades.

Bruce - 11-8-2019 at 04:52 PM


Please notice the exception error, is this an issue along with the loss of data from the WF and EWFO file creation?



Screen Shot 2019-08-12 at 10.50.24 AM.png - 2.1MB

admin - 11-8-2019 at 05:06 PM

Quote: Originally posted by Bruce  

Please notice the exception error, is this an issue along with the loss of data from the WF and EWFO file creation?



Im not sure, but I think that data center had an upgrade at midnight usa time.
It is still a problem let me know. Most exceptions are not critical.
Any connectivity outage doesnt matter too much,as when things are restored, the information is resent

admin - 11-8-2019 at 05:13 PM

Quote: Originally posted by edgetrader  
Quote: Originally posted by admin  
The next release of ewfo will have optional filters on .

GSB now can make the ewfo/ts files so there is no need to go through the slow and time consuming process of putting code into TS if you want to do your own walk forward.

Not many users do this. I think there is value in it, but its not essential.

EWFO has a lot more tools than GSB when it comes to walk forward

Are the filters working in a way such that they hide the fitness functions in the stability spreadsheet that don't meet the filtering criteria?

I'd like to see filters built into the fitness functions themselves that work during walk-forward optimization. E.g. with a filter of at least 50 in-sample trades, do everything as usual if met, but set fitness to zero if less than 50 trades.


Hi Edgetrader,
sorry but I see this post was not replied too.
There are user definable settings, when you do fitness stabilty scores.
So on the defaults you will only see the fitness types that work best for your system. You have complete control over this.
This should do what you want - ie with the 50 trades.

Bruce - 11-8-2019 at 07:10 PM

Quote: Originally posted by Bruce  

Hi Peter,

Unable to get the new version to render correctly on the screen. See the attached I'm unable to get to the 'Run' button.

What am I missing?




Any response to this pls?

admin - 11-8-2019 at 07:15 PM

Quote: Originally posted by Bruce  
Quote: Originally posted by Bruce  

Hi Peter,

Unable to get the new version to render correctly on the screen. See the attached I'm unable to get to the 'Run' button.

What am I missing?




Any response to this pls?

You cant do run till you open files. Its blank up the top right

Bruce - 11-8-2019 at 07:32 PM

Quote: Originally posted by admin  
Quote: Originally posted by Bruce  
Quote: Originally posted by Bruce  

Hi Peter,

Unable to get the new version to render correctly on the screen. See the attached I'm unable to get to the 'Run' button.

What am I missing?




Any response to this pls?

You cant do run till you open files. Its blank up the top right


Once I've loaded a file and open the Run button doesn't appear.

Bruce - 11-8-2019 at 07:38 PM


I've seen this raised previously however there's a casim of difference between the GSB-WF and the imported EWFO file into EWFO, why is this. What are the expected variances, if any? does this mean that the GSB-WF is overstated?


Screen Shot 2019-08-12 at 1.31.12 PM.png - 2.6MB Screen Shot 2019-08-12 at 10.26.15 AM.png - 2.2MB

admin - 11-8-2019 at 10:42 PM

Quote: Originally posted by Bruce  

I've seen this raised previously however there's a casim of difference between the GSB-WF and the imported EWFO file into EWFO, why is this. What are the expected variances, if any? does this mean that the GSB-WF is overstated?



I suspect the fitness in gsb is np*at and the fitness in ewfo is np*sharpRatio^2

If so the comparison in this case is useless.
There are features in GSB 54.65 which means you can not optimize any input. This is going to be good for things like secondary filter closed. Im happy with that being locked, or ideally given a narrow range. Locked is now possible but narrow range wont happen until GSB 2.0

saycem - 13-8-2019 at 12:15 AM

Thanks Peter. I wanted to understand your comment better. Are you saying that EWFO could be optimising an input that GSB is not optimising and that should not be optimised. This is critical for me to understand as I am using the inputs from EWFO for live trading given I rely on Cluster Analysis in my process.

admin - 13-8-2019 at 12:29 AM

Quote: Originally posted by saycem  
Thanks Peter. I wanted to understand your comment better. Are you saying that EWFO could be optimising an input that GSB is not optimising and that should not be optimised. This is critical for me to understand as I am using the inputs from EWFO for live trading given I rely on Cluster Analysis in my process.


GSB WF I think should be indentical in GSB and EWFO IF EVERYTHING IS THE SAME.
ie fitness, dates used. So remember if we WF in GSB from 2000 to 2015-06-30
but in ewfo wf form 2000 to 20190812, this is not the same thing.

Also a wf in TS/MC to ewfo will be significantly more different as the parameters used will not be the same.

I hope this helps.

If your market validation steps are solid and healthy numbers, then the specs on typical systems will be good weather wf has been done or not.


Bruce - 13-8-2019 at 02:32 AM

Quote: Originally posted by admin  
Quote: Originally posted by saycem  
Thanks Peter. I wanted to understand your comment better. Are you saying that EWFO could be optimising an input that GSB is not optimising and that should not be optimised. This is critical for me to understand as I am using the inputs from EWFO for live trading given I rely on Cluster Analysis in my process.


GSB WF I think should be indentical in GSB and EWFO IF EVERYTHING IS THE SAME.
ie fitness, dates used. So remember if we WF in GSB from 2000 to 2015-06-30
but in ewfo wf form 2000 to 20190812, this is not the same thing.

Also a wf in TS/MC to ewfo will be significantly more different as the parameters used will not be the same.

I hope this helps.

If your market validation steps are solid and healthy numbers, then the specs on typical systems will be good weather wf has been done or not.



Is there any merit with unpacking the GSB_EWFO files with the GSB_WF settings loaded into EWFO so "everything is the same" and then perform the analysis (cluster test, etc) from that datum?

saycem - 13-8-2019 at 03:56 AM

I am not able to get GSBWF = EWFO, see attached. Same dates, same fitness, 20%oos 10 runs.
GSB WF different every run. AStab can range from 20-70
EWFO gives same result every time.
Not sure why?

OUEtw GSB WF.JPG - 344kB OUEtw GSB WF params.JPG - 392kB OUEtw EWFO.JPG - 319kB

admin - 13-8-2019 at 04:13 AM

Quote: Originally posted by saycem  
I am not able to get GSBWF = EWFO, see attached. Same dates, same fitness, 20%oos 10 runs.
GSB WF different every run. AStab can range from 20-70
EWFO gives same result every time.
Not sure why?


Looks like you have a good system there.
Can you send me teamviewer tomorrow?
The performance is close, but not exact.

zug7 - 28-8-2019 at 12:19 AM

In case someone is interested: I have used a system with 324 iterations and have run this through the Walk-Forward Analyzer of Tradestation. First I have run a Walk-Forward Analysis and afterwards a Cluster Analysis. After that I have done the same with EWFA with the following results:

WF Tradestation: 4 seconds, EWFO: virtually immediatly, but let’s call it 1 second

CA Tradestation: 54 seconds, EWFO: 10 seconds

admin - 8-9-2019 at 10:51 PM

at top of thread, todays version that has auto updates for all future versions. Zip file only. No installer.

BigDog - 29-9-2019 at 04:40 AM

Hi Peter,

I tried downloading and running EWFO in a machine running windows server 2012. It hangs on loading 46.67% of any TS WFO data file. Doesn't seem to make any difference if I set compatibility = Win 2007.

Any suggestions?

admin - 29-9-2019 at 08:40 PM

Quote: Originally posted by BigDog  
Hi Peter,

I tried downloading and running EWFO in a machine running windows server 2012. It hangs on loading 46.67% of any TS WFO data file. Doesn't seem to make any difference if I set compatibility = Win 2007.

Any suggestions?

Hi BigDog

As a test use ts wfo.exe and or do a quick optimization in ts. Likely the files are corupt is my guess. Make sure your on a recent build of ewfo too. New build likely out this week sometime

admin - 30-9-2019 at 06:46 PM

There is a new build, Has options to save profiles etc. Not documented yet but fairly straight forward.



save.png - 8kB

Sten - 31-10-2019 at 11:00 AM

I took a look at EWFO 20190724.1. And found two major issues:

1. "Run \ Save Parameters" does not save Cluster Analysis settings (only saves "ClusterAnalysisEnabled" flag). So when I run EWFO from command line it uses default settings for Cluster Analysis and not the ones I wanted to save into json file.

2. When I run EWFO from a command line like this:

WalkForwardOptimizer.exe -p tst_parameters.json -d C:\GSBInstaller\Data\ewfoFiles\20191026-222718-200359-T7a65 -o c:\tst

It does the job, calculates and saves output .txt files. However after that the app GUI window remains open, the application does not quit automatically when it finishes processing. This makes much harder to use EWFO in some automated workflows.

Can you, please, fix these issues?

admin - 31-10-2019 at 04:13 PM

Quote: Originally posted by Sten  
I took a look at EWFO 20190724.1. And found two major issues:

1. "Run \ Save Parameters" does not save Cluster Analysis settings (only saves "ClusterAnalysisEnabled" flag). So when I run EWFO from command line it uses default settings for Cluster Analysis and not the ones I wanted to save into json file.

2. When I run EWFO from a command line like this:

WalkForwardOptimizer.exe -p tst_parameters.json -d C:\GSBInstaller\Data\ewfoFiles\20191026-222718-200359-T7a65 -o c:\tst

It does the job, calculates and saves output .txt files. However after that the app GUI window remains open, the application does not quit automatically when it finishes processing. This makes much harder to use EWFO in some automated workflows.

Can you, please, fix these issues?


I have passed this into the programer. Thank you for pointing out the issues.
The fix will be applied via auto update when you run ewfo again.
Sometime next week.
Peter

admin - 4-11-2019 at 03:48 AM

Quote: Originally posted by Sten  
I took a look at EWFO 20190724.1. And found two major issues:

1. "Run \ Save Parameters" does not save Cluster Analysis settings (only saves "ClusterAnalysisEnabled" flag). So when I run EWFO from command line it uses default settings for Cluster Analysis and not the ones I wanted to save into json file.

2. When I run EWFO from a command line like this:

WalkForwardOptimizer.exe -p tst_parameters.json -d C:\GSBInstaller\Data\ewfoFiles\20191026-222718-200359-T7a65 -o c:\tst

a lot of this work has been done. Im on working holiday so no time to test it.
there will be new command line -e = -- auto exit when done + other changes
Should be out late this week.

It does the job, calculates and saves output .txt files. However after that the app GUI window remains open, the application does not quit automatically when it finishes processing. This makes much harder to use EWFO in some automated workflows.

Can you, please, fix these issues?

admin - 6-11-2019 at 04:26 PM

Quote: Originally posted by Sten  
I took a look at EWFO 20190724.1. And found two major issues:

1. "Run \ Save Parameters" does not save Cluster Analysis settings (only saves "ClusterAnalysisEnabled" flag). So when I run EWFO from command line it uses default settings for Cluster Analysis and not the ones I wanted to save into json file.

2. When I run EWFO from a command line like this:

WalkForwardOptimizer.exe -p tst_parameters.json -d C:\GSBInstaller\Data\ewfoFiles\20191026-222718-200359-T7a65 -o c:\tst

It does the job, calculates and saves output .txt files. However after that the app GUI window remains open, the application does not quit automatically when it finishes processing. This makes much harder to use EWFO in some automated workflows.

Can you, please, fix these issues?

new build for you to test is here


Attachment: Login to view the details


Sten - 7-11-2019 at 07:07 AM

I did a quick test, Cluster Analysis & Test Criteria settings are now saved correctly into .json file. This issue is solved.

- DEL -

UPD: I found -e command line switch which solves issue No 2. I am testing it right now.

Sten - 7-11-2019 at 07:33 AM

The -e or --autoexit switch solves issue No. 2 for the most part.

The only minor problem is that EWFO with -e switch leaves shell window with a directory with output files open (see screenshot).
Consider an automated workflow which runs EWFO 1000 times. EWFO would open 1000 shell windows, which may not be what end user wants. :)





EWFO_autoexit_leaves_shell_open.png - 10kB

admin - 7-11-2019 at 02:50 PM

Quote: Originally posted by Sten  
The -e or --autoexit switch solves issue No. 2 for the most part.

The only minor problem is that EWFO with -e switch leaves shell window with a directory with output files open (see screenshot).
Consider an automated workflow which runs EWFO 1000 times. EWFO would open 1000 shell windows, which may not be what end user wants. :)




I think whats done is deliberate (and fine) but we need a non shell option.
It will be in next build - likely by auto update.

admin - 8-11-2019 at 10:05 PM

new ewfo out via automatic update. Has bug fixes and features described above few posts

admin - 11-12-2019 at 06:51 PM

WalkForwardOptimizer20191209.2.exe is out. For users with more recent builds, it will auto update when you run it.
If you have an older build, download the zip files and put in your EWFO folder C:\Program Files (x86)\EWFO,
OR use the installer file.
Whats new is bug fixes, and improvements in batch processing. Likely this product will more into EWFO & EWFO PRO (BATCH)
The upgrade will be free for all current users of EWFO, but new users will have a choice the two products which some premium in the price.

admin - 14-1-2020 at 03:44 PM

EWO PRO is out soon.
It will look very similar to last EWO, just slightly refined and faster.
It is 3 times faster than the previous build.
What takes Ts wfo.exe 21:12 minutes takes EWFO pro 4 min 25 sec
You can also run multiple EWFO with little speed penalty

There now is time stamp logging.
There will be a mild price increase for future purchasers, likely next month. A lot of programing time has gone into this.
The biggest feature in the last year has been batch processing of the EWFO input files.
For existing users, there are no upgrade charges.

It will be released by automatic upgrade when you open EWFO.

admin - 15-1-2020 at 12:15 AM

The documentation has been updated, but could do with a few tweaks.
There are a few minor bugs to be fixed but its working well.
If any one is after the new build in a hurry, then email me.

edgetrader - 22-1-2020 at 07:22 PM

How can I roll back from 20200116.2 to previous version? What I had before worked fine on same computer and same data folder, but 20200116.2 doesn't work, see image.

20200116.2.png - 35kB

admin - 22-1-2020 at 07:26 PM

Quote: Originally posted by edgetrader  
How can I roll back from 20200116.2 to previous version? What I had before worked fine on same computer and same data folder, but 20200116.2 doesn't work, see image.


how much ram on the pc in total, how much is free?
How big are the data files from ts?

I can get you the old build, but the new build is miles better.
I also dont expect any change in memory usage between old and new build

edgetrader - 22-1-2020 at 07:30 PM

Data file is 30.6 GB and computer has 32 GB RAM + 16 GB SSD page file. The very same setup worked flawlessly in previous version.

admin - 22-1-2020 at 07:38 PM

Quote: Originally posted by edgetrader  
Data file is 30.6 GB and computer has 32 GB RAM + 16 GB SSD page file. The very same setup worked flawlessly in previous version.


Thats at the limit of what you can do due to your ram.
I dont think its the new build at all.

Ive done 60 gb files in the past, but have much more ram.

You can try two things
1) increase swap file size, then reboot the computer
2) download the old build, and click no when it asks to update
https://trademaid.info/sw/ewfo_20191106.2b.exe

Just a reboot itself may make enough difference to free up some ram

you could also run
C:\GSB\ResourceMonitor\ReduceMemory.exe


or buy more ram. Its cheap now.

edgetrader - 22-1-2020 at 08:20 PM

Thanks for your help.

I increased page file to 32 GB and rebooted. Tried again and had same issue.

Now on old version and it's looking good. The CPU doesn't support more RAM.

admin - 22-1-2020 at 08:23 PM

Quote: Originally posted by edgetrader  
Thanks for your help.

I increased page file to 32 GB and rebooted. Tried again and had same issue.

Now on old version and it's looking good. The CPU doesn't support more RAM.


thanks for update.
Likely you can run both versions depending on file size.
you can rename the new version to wfo20190123.exe etc
new version is 3 times faster, and i didnt expect ram size to alter between them

admin - 22-1-2020 at 08:32 PM

Hi Edgetrader
can you note the ram usage on both the old and new version?


peak.png - 70kB

edgetrader - 22-1-2020 at 08:41 PM

Yes, I'll report in the next few days. Now looking to let the run complete on old version.

edgetrader - 24-1-2020 at 01:08 AM

version 20191106.2
32,896,300 K

version 20200116.2
32,472,620 K

Seems the new version used slightly less peak memory when it crashed. Older version had longer runtime and completed the run.

admin - 24-1-2020 at 02:03 AM

Quote: Originally posted by edgetrader  
version 20191106.2
32,896,300 K

version 20200116.2
32,472,620 K

Seems the new version used slightly less peak memory when it crashed. Older version had longer runtime and completed the run.


there is an update a few minutes ago.
For your files, old version used 4.2 more cpu than new version, but new version used more ram.
even on machine with 320 gb ram, last weeks version failed on your files.
todays is fine.
I may improve ram useage. We increased some accuracy of things, and that used more ram.

todays version also has dd mae of each trade, and option to use open or closed dd in fitness (advacned)

admin - 27-1-2020 at 11:52 PM

EWFO PRO 20200128 VERSION OUT SOON VIA AUTO UPDATE.
Option for the amount of CPU cores. Need to limit CPU cores in the case of having 30GB data files on machine with 32 GB of ram.
One of the reasons EWFO is faster, is more processing is muti cores.

Option of close or open trade for fitness draw-down.
Longer paths support in user interface.
Bug fix were nth mode is used. (need to export Json file again.) You will know about this bug if you see it, so most users can ignore this.
Smarter rego file system, for a few users who have unique rego file.


advacned1.png - 9kB

admin - 3-2-2020 at 01:24 AM

New EWFO features
There are a few more features, like dd%, medium trade, ave win loss in the output files
There will be much faster auto update of EWFO.
A few minor tweaks.

EWFO to EWO pro is a free upgrade, but prices will increase Feb 15 2020
EWFO pro will be $390, and addition licenses $130 each. Currently $300 / $100.

This is to pay for considerable investment in new features, and more speed.
EWFO pro is 300% faster than EWFO, and 400% faster than Tradestation WFO

admin - 3-2-2020 at 09:55 PM

Feb 4 2020
New full installer file for EWFO.
You dont need this if your EWFO is a 2020 version.
All future updates will be via auto update when you open EWFO.
See the top of this thread.

cipher - 10-2-2020 at 09:41 AM

im seeing virustotal.com pick up viruses on the 2 EWFO executables posted -- i know the current solution is to exclude it from windows defender / anti virus, but is there a chance we can get a new release that gets rid of these false positives down the road?

VirusTotal.com: WalkForwardOptimizer20191209.2.exe

VirusTotal.com: EWFOPro_20200203.2b.exe

admin - 10-2-2020 at 05:21 PM

Quote: Originally posted by cipher  
im seeing virustotal.com pick up viruses on the 2 EWFO executables posted -- i know the current solution is to exclude it from windows defender / anti virus, but is there a chance we can get a new release that gets rid of these false positives down the road?

VirusTotal.com: WalkForwardOptimizer20191209.2.exe

VirusTotal.com: EWFOPro_20200203.2b.exe

Its just too problematical to do.
Here are some issues.
Many av companies will not accept files as big as the the installer file.
Others dont respond or take action.
A file can be fine, and then the av company add wefo to its list of virus's etc.
On a postive note, microsoft av is really good. They respond to false postives in just a few hours. However to release the update to users will take longer.
Often there are much less false virus issues if the installer is not used. This in itself shows you how fickle the anti virus vendors are.
Im happy to release the EWFO with no installer, but there are still a few more tweaks out in the next week. So give me a few days.

admin - 10-2-2020 at 07:44 PM

At the top of this thread, is the newest build with no installer file.
It has copy to clip board of future dates in the eld script.



dates.png - 63kB

admin - 20-2-2020 at 04:13 AM

Alot more work has gone into EWFO.
We shortly should have astab coarse, and rstab coarse,
windows can be resized to cope better with the various font scaling on 4k + monitors
New release out in the next week or so.
Various other small refinements and bug fixes.

moresi522 - 26-2-2020 at 12:08 PM

Hi Peter,
Every time I click on "Show close prices" I got this error.

Code:
System.Collections.Generic.KeyNotFoundException: La chiave specificata non era presente nel dizionario. in System.ThrowHelper.ThrowKeyNotFoundException() in System.Collections.Generic.Dictionary`2.get_Item(TKey key) in DataIO.TradeHelper.<>c__DisplayClass6_0.<GetTradesClosingPrices>b__0(Trade t) in D:\dev\upwork\Peter\wfo\walkforwardoptimizer2019\build\DataIO\TradeHelper.cs:riga 62 in System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext() in System.Linq.Buffer`1..ctor(IEnumerable`1 source) in System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) in WalkForwardOptimizer.FormWalkForward.DrawEquityGraph() in System.Windows.Forms.CheckBox.OnCheckedChanged(EventArgs e) in System.Windows.Forms.CheckBox.set_CheckState(CheckState value) in System.Windows.Forms.CheckBox.OnClick(EventArgs e) in System.Windows.Forms.CheckBox.OnMouseUp(MouseEventArgs mevent) in System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) in System.Windows.Forms.Control.WndProc(Message& m) in System.Windows.Forms.ButtonBase.WndProc(Message& m) in System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


It's italian but as you can figure it out the main message says something like "The key is not present in the dictionary"

Let me know if you need more info

Cattura.PNG - 45kB

admin - 26-2-2020 at 03:28 PM

Quote: Originally posted by moresi522  
Hi Peter,
Every time I click on "Show close prices" I got this error.

Code:
System.Collections.Generic.KeyNotFoundException: La chiave specificata non era presente nel dizionario. in System.ThrowHelper.ThrowKeyNotFoundException() in System.Collections.Generic.Dictionary`2.get_Item(TKey key) in DataIO.TradeHelper.<>c__DisplayClass6_0.<GetTradesClosingPrices>b__0(Trade t) in D:\dev\upwork\Peter\wfo\walkforwardoptimizer2019\build\DataIO\TradeHelper.cs:riga 62 in System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext() in System.Linq.Buffer`1..ctor(IEnumerable`1 source) in System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) in WalkForwardOptimizer.FormWalkForward.DrawEquityGraph() in System.Windows.Forms.CheckBox.OnCheckedChanged(EventArgs e) in System.Windows.Forms.CheckBox.set_CheckState(CheckState value) in System.Windows.Forms.CheckBox.OnClick(EventArgs e) in System.Windows.Forms.CheckBox.OnMouseUp(MouseEventArgs mevent) in System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) in System.Windows.Forms.Control.WndProc(Message& m) in System.Windows.Forms.ButtonBase.WndProc(Message& m) in System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


It's italian but as you can figure it out the main message says something like "The key is not present in the dictionary"

Let me know if you need more info


Please email me the ts data files. They are smallest using .7zip, but zip will do.
use dropbox. google drive etc as the files are often to large to email

admin - 2-3-2020 at 01:08 AM

next ewfo has many font options for the users with 4k monitors & or font scaling > 100 %

see top of post for the exe.
It will be released by auto updates after more user testing



fonts.png - 96kB

admin - 3-3-2020 at 01:32 AM

EWFO 20200302.1 Released by auto update.
Open up EWFO and it should update

admin - 6-3-2020 at 11:38 PM

some users are getting auto update to fail. The fix for now is either run EWFO as windows administrator, or copy your files from
C:\Program Files (x86)\EWFO to c:\ewfo.

Then make a shortcut from c:\ewfo\WalkForwardOptimizer.exe and put on your desktop

moresi522 - 11-3-2020 at 10:16 AM

Hi,
is it possible that when I select Fitness selection and click on Run (It takes quite long time) the Fitness Selection tab is always empty? (see images attached)

and another question. The empty resulting table looks very similar to the image showed at Documentation -> Parameter Stability -> Stability Score.
Sincerly I dont' understand what I should see, the image of StabilityScore is too low resolution.

are But Stability Score and Fitness Selection the same thing right?

Could you please explain better how to use it?

(I can't find Fitness selection in documentation.. maybe it's my fault)


ewfo1.png - 45kBewfo2.PNG - 31kB

admin - 11-3-2020 at 03:41 PM

Quote: Originally posted by moresi522  
Hi,
is it possible that when I select Fitness selection and click on Run (It takes quite long time) the Fitness Selection tab is always empty? (see images attached)

and another question. The empty resulting table looks very similar to the image showed at Documentation -> Parameter Stability -> Stability Score.
Sincerly I dont' understand what I should see, the image of StabilityScore is too low resolution.

are But Stability Score and Fitness Selection the same thing right?

Could you please explain better how to use it?

(I can't find Fitness selection in documentation.. maybe it's my fault)

unclick the data function. This means no fitness gave results that were good.
the docs are not uptodate. fitness stabilty scores has been renamed to fitness selection.
I hope to do a video on the new feature soon, but thats taking a while due to a bug in gsb wfo output

data.png - 42kB

moresi522 - 12-3-2020 at 10:57 AM

Quote: Originally posted by admin  

unclick the data function. This means no fitness gave results that were good.
I hope to do a video on the new feature soon, but thats taking a while due to a bug in gsb wfo output


Ok, now it works.

I understand that you have a lot of work to do and, however, I admit that a video with a methodology would be highly appreciated. at this moment for a newbie like me it is not clear how is the better way to use this tool.

Thanks

admin - 12-3-2020 at 03:59 PM

Quote: Originally posted by moresi522  
Quote: Originally posted by admin  

unclick the data function. This means no fitness gave results that were good.
I hope to do a video on the new feature soon, but thats taking a while due to a bug in gsb wfo output


Ok, now it works.

I understand that you have a lot of work to do and, however, I admit that a video with a methodology would be highly appreciated. at this moment for a newbie like me it is not clear how is the better way to use this tool.

Thanks

EWFO has taken massive amount of work this year, but its now finsihsed apart from docs and video. GSB is a bottomless pit of things to do, so time is very presured and torn between many things that need to be done. There is years of work to go in GSB, let alone know the many ways to use it.
I also take 3 years off the dates when i use fitness selection.
otherwise you have too much danger of a fitness curve fit using all the data

dates.png - 5kB

admin - 13-3-2020 at 01:01 AM

The documentation has been updated, though a bit more to be done.
https://trademaid.info/gsbhelp/Newfeatures.html
A number of other sections are updated.

admin - 3-4-2020 at 07:24 PM

EWFO 20200402.2 IS OUT
Bug fixes, error trap for faulty TS data, empty data files.
It will be released via auto update in the next week.
Installer file now installs to c:\ewfo to stop problems of auto update not working when not run as windows administrator.

moresi522 - 21-4-2020 at 10:34 AM

Hi,
with this new version I'm not able to use fitness selection anymore. As you can see in the video attached I always get an error.
I've attached also the file I used (generated by GSB)

Attachment: Login to view the details

Attachment: Login to view the details

 Pages:  1    3