| Pages:
1
..
54
55
56
57
58
..
98 |
NickW
Junior Member
Posts: 29
Registered: 12-2-2020
Member Is Offline
|
|
Quote: Originally posted by Daniel UK1  | If what user pronos is stating is correct, then that would be a serious issue for MC users, have any other users seen this issue also with MC?
Cheers |
MC with IQFeed data will not provide you with up/down volume/tick data. (It will capture that information when you do realtime data collect, but
that's pretty useless)
MC with TS data DOES provide you with up/down volume/tick data historically.
See screenshot of the data. Top: IQFeed data, Bottom: TS data.
I don;t have any systems using volume/tick data, so i use MC with IQ, however if i put one of these systems in prod, i will be using TS data against
it.
Last year major issues with TS data. I had missing bars for days to months in the data and since switched to IQFeed. Make sure to build in checks to
make sure data has no huge gaps.
Thanks received (1):
+1 Daniel UK1 at 2021-06-23 17:22:28
|
|
|
Arndt
Junior Member
Posts: 8
Registered: 12-5-2021
Member Is Offline
|
|
| Quote: Originally posted by NickW | | Quote: Originally posted by Daniel UK1 | If what user pronos is stating is correct, then that would be a serious issue for MC users, have any other users seen this issue also with MC?
Cheers |
MC with IQFeed data will not provide you with up/down volume/tick data. (It will capture that information when you do realtime data collect, but
that's pretty useless)
MC with TS data DOES provide you with up/down volume/tick data historically.
See screenshot of the data. Top: IQFeed data, Bottom: TS data.
I don;t have any systems using volume/tick data, so i use MC with IQ, however if i put one of these systems in prod, i will be using TS data against
it.
Last year major issues with TS data. I had missing bars for days to months in the data and since switched to IQFeed. Make sure to build in checks to
make sure data has no huge gaps.
|
OK. So that means that when using GSB for Systems building we can use the TS data. Once going to live trading we are fine with IQFeed as the ticks are
delivered with the real-time data supply.
What I really donīt get are the discrepancies between TS and IQFeed (or in generell data providers...)in the data... When I look at the screenshot not
even the open is the same point....
Thank you very much for these insights!
|
|
|
NickW
Junior Member
Posts: 29
Registered: 12-2-2020
Member Is Offline
|
|
The problem with using MC with IQ is that historical data will NOT have the volume data. Thus your indicators will be incorrect for 500 bars until
real time data is collects.
Now whenever you have a futures contract rolling, you will have to reload data and you will lose all that collected volume/tick data.
BTW i reload data 3 times a week to prevent bad data from giving false signals (costed me thousands in losses!)
We are retail traders and the tools we have really suck. There's really nothing we can do about it. All we can try to do is mitigate the problems.
|
|
|
Daniel UK1
Member
Posts: 470
Registered: 4-6-2019
Member Is Offline
|
|
Hey Nick, i am curious... i am also using MC since years back... and i try to avoid using Custom future contracts since this has caused lots of issues
with reliability for me, freezing, taking lots or resources etc.. are you using custom or normal adjusted ? whats your experiences with custom
futures?
This tick data issue is causing some concerns for me now, i restart server every 5 days, and empty data cash every day ... so the 500 bar solution is
not really working ..
There must be better data providers for MC that can solve this ? QGC ? TS data into MC is not really optimal..
|
|
|
Daniel UK1
Member
Posts: 470
Registered: 4-6-2019
Member Is Offline
|
|
Does anyone know, if the issue in regards to up down volume, is in MC or with IQ dtn ? just want to check before i contract MC support to try to
figure out a solution.
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by avatartrader | This is not necessarily a support question, but based on the areas where I can post, this seems the most appropriate option:
I was curious how others are tracking per-strategy performance for the purposes of potential increase/decrease of position sizing, strategy health and
degradation, etc.? I am using Multicharts/IB and am becoming increasingly annoyed by the fact that they do not maintain any real-time performance of a
strategy in a given account, so the only way to really track it is to do it manually and often in case it needs to be turned off or restarted for any
reason.
I suppose another way (as a partial solution) is to also track the historical performance separately and use an equity curve or position sizing
indicator based on needs, but I wanted to see if anyone else was handling this in an efficient and scalable manner.
I like what Peter has done with IBLink and that it seems to maintain a DB of strategies and historical live performance, so there is a lot of
potential there. But since that is not commercially available, I would have to write my own middleware should I actually want to abandon MC and use TS
for signals and then execute through a gateway.
Thanks for the input-
|
you should be able to write the daily PL to a file in real time for each strategy.
This is not as good as live fills but its a big step in the right direction.
|
|
|
NickW
Junior Member
Posts: 29
Registered: 12-2-2020
Member Is Offline
|
|
| Quote: Originally posted by Daniel UK1 | Hey Nick, i am curious... i am also using MC since years back... and i try to avoid using Custom future contracts since this has caused lots of issues
with reliability for me, freezing, taking lots or resources etc.. are you using custom or normal adjusted ? whats your experiences with custom
futures?
This tick data issue is causing some concerns for me now, i restart server every 5 days, and empty data cash every day ... so the 500 bar solution is
not really working ..
There must be better data providers for MC that can solve this ? QGC ? TS data into MC is not really optimal.. |
Hi Daniel,
I have tried custom futures contract before, but there were a lot of issues with it, when to manually roll them, pain to set up each contract in MC,
etc. I just use the standard @ES or @ES#C contracts and have a workspace that alerts me when the @ES contract price is different from the current
contract i'm trading the ESZ1 or whichever is current. Then i know it's time to roll.
I have looked at many data providers, recently again after discovering IQ does not provide up/dn volume data. Some vendors don't have all the
instruments or indexes we need.
|
|
|
Daniel UK1
Member
Posts: 470
Registered: 4-6-2019
Member Is Offline
|
|
Hey Nick,
I see, yes i manually roll also similar like you, then you have the added "spice" when you need to roll NG and CL earlier than IQ dtn rolls, because
they roll after the 6 day cutoff from IB not allowing trades for front month issue due to negative prices issue.
I assume you manually keep track of this with alarms or something?, i have not found a better way like MC telling me i need to roll before 6 days
limit.
Btw, i checked with MC and it is really as it has been said here... using minute bars from IQ Dtn , fetching historic data, provides just Up volume
bars, no matter if one choses Tick volume or traded volume.
So in essence chosing tick volume in GSB, will/could provide another outcome compared to backtesting in MC with IQ data, when using the strategy on
live charts in MC using IQ, since on LIVE CHARTS you get Up Ticks and Down Ticks that the strategy then will use.
This ads a new complexity to the practicallity of execution..
Only data that provides Up tick and down tick volume while fetching historic data, is TS in MC as far as i understand it..
meaning that one needs to use TS data when building in GSB, backtesting in MC... but trading live... one is perfectly fine using IQ dtn 
Sounds clear? not really right... very messy situation
Thanks received (2):
+1 bizgozcd at 2021-06-25 07:10:03 +1 engtraderfx at 2021-06-25 06:19:57
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
@Daniel
Why ive never picke dup on this issue is Im using TS data in MC. Thats not a bad option if it works for you. It wont work for all users as TS wont
even supply to all countries etc
You are going to have to build systems using up volume only. NQ systems will be affected a lot by this, as they often use accumDist - which needs
volume. Other markets like gold don't use volume in inidcators chosen
IN TS I have custom futures that roll before the continuos on data1, and the continuous on data2, and alertmon to give me an alarm when out of sync.
MC can do this, but setting up custom futures is more complex
|
|
|
Daniel UK1
Member
Posts: 470
Registered: 4-6-2019
Member Is Offline
|
|
Hi Peter,
Not your fault in regards to Volume issue, its good that the issue came to light.
As of now its either get TS data to MC, or build on up volume only and use any other data source.
Until someone find another datasource to MC that supplies up down ticks historic minute data, as of now this does not exist to my knowledge.
The rollover issues was more in regards to Nicks comment on how he handles it in MC, rollover can be tricky sometimes and prone to human error.
Custom futures in MC has been more prone to causing issues compared to normal adjusted.
Alertmon i believe you was going to release to GSB clients ?
Thanks received (1):
+1 engtraderfx at 2021-06-25 06:16:45
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
@Daniel,
Alertmon has had massive amount of work done on it, but I'm only likely to supply it with my VM service. Both have been running well for a long
time... but I'm just too busy on more pressing topics.
You could make a alert that says if close of data1 <> close of data2 then ... alarm...
but if you let the contract expire where data stops, you will not get the alert. This is one of the many areas where alertmon shines
|
|
|
portfolioquanttrader2020
Member
Posts: 269
Registered: 30-4-2020
Member Is Offline
|
|
Hi peter
What you call minor modifications, so as not to deviate from the path that you are all following. Can you give me an example?
I apologize, for using the inappropriate forum for support questions.
|
|
|
portfolioquanttrader2020
Member
Posts: 269
Registered: 30-4-2020
Member Is Offline
|
|
For example, I have changed the fitness function, and I have changed the size of the stop for the search of the 10 green indicators.
Would that be a minor change?
Should you use the same fitness function and the same stop to look for green indicators, and to look for better systems in out of sample before the
wfo?
I understand that to discover improvements, we must go through the process by continually making "minor" changes.
Do you have documentation on all fitness functions?
|
|
|
portfolioquanttrader2020
Member
Posts: 269
Registered: 30-4-2020
Member Is Offline
|
|
For example, I have changed the fitness function, and I have changed the size of the stop for the search of the 10 green indicators.
Would that be a minor change?
Should you use the same fitness function and the same stop to look for green indicators, and to look for better systems in out of sample before the
wfo?
I understand that to discover improvements, we must go through the process by continually making "minor" changes.
Do you have documentation on all fitness functions?
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by portfolioquanttrader2020 | For example, I have changed the fitness function, and I have changed the size of the stop for the search of the 10 green indicators.
Would that be a minor change?
Should you use the same fitness function and the same stop to look for green indicators, and to look for better systems in out of sample before the
wfo?
I understand that to discover improvements, we must go through the process by continually making "minor" changes.
Do you have documentation on all fitness functions? |
Unless you have high skill in GSB, its best to change nothing as generally whats published works well. The exception is when newer things have been
found that improve on whats been done before hand. (IE new fitness)
Fitness functions are nearly all standard functions. The exeption being the mroe recent additions. These at high level were documented under beta
builds when they made there way into the current build.
So you can change fitness or stop (But not both at the same time), but if so change nothing else until you have done 4 identical tests to compare to
your previous benchmark.
I think I did publsih better fitness for nq in this section.
https://trademaid.info/forum/viewthread.php?tid=117
|
|
|
NickW
Junior Member
Posts: 29
Registered: 12-2-2020
Member Is Offline
|
|
| Quote: Originally posted by Daniel UK1 | Hey Nick,
I see, yes i manually roll also similar like you, then you have the added "spice" when you need to roll NG and CL earlier than IQ dtn rolls, because
they roll after the 6 day cutoff from IB not allowing trades for front month issue due to negative prices issue.
I assume you manually keep track of this with alarms or something?, i have not found a better way like MC telling me i need to roll before 6 days
limit.
Btw, i checked with MC and it is really as it has been said here... using minute bars from IQ Dtn , fetching historic data, provides just Up volume
bars, no matter if one choses Tick volume or traded volume.
So in essence chosing tick volume in GSB, will/could provide another outcome compared to backtesting in MC with IQ data, when using the strategy on
live charts in MC using IQ, since on LIVE CHARTS you get Up Ticks and Down Ticks that the strategy then will use.
This ads a new complexity to the practicallity of execution..
Only data that provides Up tick and down tick volume while fetching historic data, is TS in MC as far as i understand it..
meaning that one needs to use TS data when building in GSB, backtesting in MC... but trading live... one is perfectly fine using IQ dtn
Sounds clear? not really right... very messy situation
|
Hi Daniel,
My trade data uses the continuous contract, QCL#C. I map the front contract in IB symbol mapping for which contract to actually trade.
Attach is my setup for checking when a symbol has rolled. On the left is current contract and next contract. On the right is current contract and
continuous contract with an alert at the bottom that compare the current contract price with the continuous price every 15min and emails me when it's
different. When there's a roll alert, everything just rolls forward, these charts, the symbol mapping, reload data, restart systems.
Yes it is very messy to manage the tick volume from IQ in MC. If you need to trade a system live with it, i would just use TS data for that system.
That's what i would do.
Thanks received (1):
+1 admin at 2021-06-29 20:24:06
|
|
|
Daniel UK1
Member
Posts: 470
Registered: 4-6-2019
Member Is Offline
|
|
| Quote: Originally posted by NickW | | Quote: Originally posted by Daniel UK1 | Hey Nick,
I see, yes i manually roll also similar like you, then you have the added "spice" when you need to roll NG and CL earlier than IQ dtn rolls, because
they roll after the 6 day cutoff from IB not allowing trades for front month issue due to negative prices issue.
I assume you manually keep track of this with alarms or something?, i have not found a better way like MC telling me i need to roll before 6 days
limit.
Btw, i checked with MC and it is really as it has been said here... using minute bars from IQ Dtn , fetching historic data, provides just Up volume
bars, no matter if one choses Tick volume or traded volume.
So in essence chosing tick volume in GSB, will/could provide another outcome compared to backtesting in MC with IQ data, when using the strategy on
live charts in MC using IQ, since on LIVE CHARTS you get Up Ticks and Down Ticks that the strategy then will use.
This ads a new complexity to the practicallity of execution..
Only data that provides Up tick and down tick volume while fetching historic data, is TS in MC as far as i understand it..
meaning that one needs to use TS data when building in GSB, backtesting in MC... but trading live... one is perfectly fine using IQ dtn
Sounds clear? not really right... very messy situation
|
Hi Daniel,
My trade data uses the continuous contract, QCL#C. I map the front contract in IB symbol mapping for which contract to actually trade.
Attach is my setup for checking when a symbol has rolled. On the left is current contract and next contract. On the right is current contract and
continuous contract with an alert at the bottom that compare the current contract price with the continuous price every 15min and emails me when it's
different. When there's a roll alert, everything just rolls forward, these charts, the symbol mapping, reload data, restart systems.
Yes it is very messy to manage the tick volume from IQ in MC. If you need to trade a system live with it, i would just use TS data for that system.
That's what i would do.
|
Hi Nick, thanks for sharing.
Never relied on a signal to warn before, but after one to many human mistakes with rolls, i will setup similar as you.
One thing, though.. i believe you are using IQ, issue with CL i believe is that IQ rolls CL after the 6 day (days before end date for the contract)
limit IB has, IB dont allow you to trade close to 6 days for front month, meaning that you have a few days before it rolls that you are not able to
trade using the contract you on. My point is, if not mistaken about the 6 day rule, you need to roll a few days befoer iq rolls.
|
|
|
portfolioquanttrader2020
Member
Posts: 269
Registered: 30-4-2020
Member Is Offline
|
|
Hi peter
I don't have these flags on my server where I have Alertmon configured.
What are they for? I would like to use them.
|
|
|
portfolioquanttrader2020
Member
Posts: 269
Registered: 30-4-2020
Member Is Offline
|
|
Hi Peter
Alertmon can be used with Ninjatrader?
I have an interested person
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
Likely its not hard to make it work for NT, as long as we don't need the position match from nt to brokerage account.
I will look into it.
|
|
|
NickW
Junior Member
Posts: 29
Registered: 12-2-2020
Member Is Offline
|
|
| Quote: Originally posted by Daniel UK1 | | Quote: Originally posted by NickW | | Quote: Originally posted by Daniel UK1 | Hey Nick,
I see, yes i manually roll also similar like you, then you have the added "spice" when you need to roll NG and CL earlier than IQ dtn rolls, because
they roll after the 6 day cutoff from IB not allowing trades for front month issue due to negative prices issue.
I assume you manually keep track of this with alarms or something?, i have not found a better way like MC telling me i need to roll before 6 days
limit.
Btw, i checked with MC and it is really as it has been said here... using minute bars from IQ Dtn , fetching historic data, provides just Up volume
bars, no matter if one choses Tick volume or traded volume.
So in essence chosing tick volume in GSB, will/could provide another outcome compared to backtesting in MC with IQ data, when using the strategy on
live charts in MC using IQ, since on LIVE CHARTS you get Up Ticks and Down Ticks that the strategy then will use.
This ads a new complexity to the practicallity of execution..
Only data that provides Up tick and down tick volume while fetching historic data, is TS in MC as far as i understand it..
meaning that one needs to use TS data when building in GSB, backtesting in MC... but trading live... one is perfectly fine using IQ dtn
Sounds clear? not really right... very messy situation
|
Hi Daniel,
My trade data uses the continuous contract, QCL#C. I map the front contract in IB symbol mapping for which contract to actually trade.
Attach is my setup for checking when a symbol has rolled. On the left is current contract and next contract. On the right is current contract and
continuous contract with an alert at the bottom that compare the current contract price with the continuous price every 15min and emails me when it's
different. When there's a roll alert, everything just rolls forward, these charts, the symbol mapping, reload data, restart systems.
Yes it is very messy to manage the tick volume from IQ in MC. If you need to trade a system live with it, i would just use TS data for that system.
That's what i would do.
|
Hi Nick, thanks for sharing.
Never relied on a signal to warn before, but after one to many human mistakes with rolls, i will setup similar as you.
One thing, though.. i believe you are using IQ, issue with CL i believe is that IQ rolls CL after the 6 day (days before end date for the contract)
limit IB has, IB dont allow you to trade close to 6 days for front month, meaning that you have a few days before it rolls that you are not able to
trade using the contract you on. My point is, if not mistaken about the 6 day rule, you need to roll a few days befoer iq rolls.
|
Hi Daniel,
I have not had this issue at all before. Lets connect with the next roll of CL and see if there's any issue.
Nick
|
|
|
Daniel UK1
Member
Posts: 470
Registered: 4-6-2019
Member Is Offline
|
|
Hi Nick,
I believe this wording at IB creates the issue i am mentioning.
So as far as my understanding, you can close out positions, but you can not open a new trade, "within 5 days to expiry of closing contract"
And i Believe IQ rolls 3 days before contract ends for CL.
"Additionally, beginning five business days prior to expiration, only margin-reducing transactions in expiring month contracts will be accepted for
oil-based products eligible to trade at negative prices."
https://www.interactivebrokers.com/en/index.php?f=deliveryEx...
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by Daniel UK1 | Hi Nick,
I believe this wording at IB creates the issue i am mentioning.
So as far as my understanding, you can close out positions, but you can not open a new trade, "within 5 days to expiry of closing contract"
And i Believe IQ rolls 3 days before contract ends for CL.
"Additionally, beginning five business days prior to expiration, only margin-reducing transactions in expiring month contracts will be accepted for
oil-based products eligible to trade at negative prices."
https://www.interactivebrokers.com/en/index.php?f=deliveryEx... |
I think this is correct. in ts the correct symbol for cl (via ib) is
|
|
|
REMO755
Member
Posts: 181
Registered: 11-4-2021
Member Is Offline
|
|
Hello, what is the name of the macro that makes the top 300, group as a family and then do w.fs. ?
I can not find
|
|
|
admin
Super Administrator
Posts: 5060
Registered: 7-4-2017
Member Is Offline
Mood: No Mood
|
|
| Quote: Originally posted by REMO755 | Hello, what is the name of the macro that makes the top 300, group as a family and then do w.fs. ?
I can not find |
doing wf by macro is not recommended. Not part of the methadolgy.
First you examine all members OOS in a family, then wf them if you like them
macros see june 8 post here
https://trademaid.info/forum/viewthread.php?tid=262
|
|
|
| Pages:
1
..
54
55
56
57
58
..
98 |
![[*]](images/davis/default_icon.gif){kind=icon}
