[ts-gen] How symbols and contracts are defined during database creation
sam
sam_backtester at yahoo.fr
Fri Sep 12 18:17:33 EDT 2008
Thx for answers.
I know the database cannot be involved in my issue, it was just to tell that I've made a new/fresh install from the latest tarball, including an empty database.
I've just tested with the sample TWS java client, and it seems that it might be an account specific issue, or somethong else but not a Shim issue, because I have the same problem when trying to "Req Contract Data" from this client. Quite strange...
Thanks for futures data explanations, I'll repopulate my databases with my favorites futures contracts next week.
Sam.
--- En date de : Ven 12.9.08, Bill Pippin <pippin at owlriver.net> a écrit :
De: Bill Pippin <pippin at owlriver.net>
Objet: Re: [ts-gen] How symbols and contracts are defined during database creation
À: sam_backtester at yahoo.fr, "trading-shim general mailing list" <ts-general at trading-shim.org>
Date: Vendredi 12 Septembre 2008, 17h15
Sam,
About wildcard contract details for AA:
> ... I've found that the "wild" things looks broken to me,
> with the 080909 tarball, tested with new databases.
> A "select wild STK AA all;" for example returns only
> "No security definition has been found for the request",
> which is not supposed to do. Historical data for AA, in
> this example, still working ...
I'm not sure why your system fails for the STK:AA query.
Here's what mine, leaving aside account details, right hand
side message text, and some reformatting, provides when
running against a paper account:
4|100| 0|# |4|100|0|****************|
4|101| 0|# |4|101|0|0.83|080820|data|
4|100| 0|# |4|100|0|****************|
4|102| 0|# |4|102|0|23|1|40|20080912 10:00:04 EST|Connect with:
3| 9| 1|1|
3| 4| 2| |2104|Market data farm connection is OK:usfarm|
1| 9| 0|select wild STK AA all; |
1| 2| 0|wait 1;|
2| 9| 3|9|3|AA|STK||1.00||1||||1|
3|10| 4|AA|STK||0.0||SMART |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||ARCA |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||BATS |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||CBSX |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||CHX |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||CSFBALGO|USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||DRCTEDGE|USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||EDGEA |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||ISE |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||ISLAND |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||LAVA |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||NSX |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||NYSE |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||PHLX |USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||TRACKECN|USD|AA|AA|AA|4005|0.01|
3|10| 4|AA|STK||0.0||EBS |CHF|AA|AA|AA|10173157|0.01|
3|10| 4|AA|STK||0.0||MEXI |MXN|AA|AA|AA|38708037|0.01|
3|10| 4|AA|STK||0.0||SMART |EUR|ALU|USSTARS|USSTARS|13013280|0.0010|
3|10| 4|AA|STK||0.0||FWB |EUR|ALU|USSTARS|USSTARS|13013280|0.0010|
3|10| 4|AA|STK||0.0||IBIS |EUR|ALU|USSTARS|USSTARS|13013280|0.0010|
3|10| 4|AA|STK||0.0||SWB |EUR|ALU|USSTARS|USSTARS|13013280|0.0010|
1| 0| 0||
1| 1| 0|exit;|
Even for the demo account, and leaving aside the entry MEXI, I get
essentially all of the above info.
If you are willing provide more information to the list,
please run the shim with the following script, named
say, wild, as input:
#!./shim -f
select wild STK AA all;
wait 3;
exit;
using the command line:
./wild --data file save wild
The wildcard option disables database checks for the symbol AA, so
I don't see how your database could be involved with the problem.
The save option keeps binary traces for the request and message
traffic as shim2tws.bin and tws2shim.bin, respectively.
If you would filter those via:
bin/req.filter < shim2tws.bin > wild.aa.req
bin/msg.filter < tws2shim.bin > wild.aa.msg
and remove any sensitive data such as the account details lines
with your account code, that is lines including text such as:
3| 6| 2|AccountCode | DU15015| |DU15015| ...
3| 6| 2|AccountType |UNIVERSAL| |DU15015| ...
3| 6| 2|AccruedCash | 0.00|BASE|DU15015| ...
3| 6| 2|AccruedCash | 0.00|USD |DU15015| ...
and then post the two text files to the list, I can look them over
for something suspicious. You are able to collect contract details
via the tws gui, right? Otherwise it's not an api issue, but rather
a problem with your account.
> I hope I'll be able to investigate with the IB sample client next
week.
I understand it might be awhile until you can respond; that's no problem
Take your time, and when you have a chance, please realize that I much
appreciate the usage reports that users send us.
> ... adding some contracts in LocalFut.sql, I'll be
> happy with CAC40, FEF, Q, NQ, ER2 and ES, if possible ;).
All of these have index products in the database, so that
you should be able to collect price data:
30257 30250 946200 IND MONEP CAC40 EUR FR
30209 30202 41868091 IND MONEP FEF EUR EU
30221 30214 16483960 IND LIFFE Q EUR EU
29189 29180 11004958 IND GLOBEX NQ USD US
29076 29067 13486931 IND GLOBEX ER2 USD US
29078 29069 11004968 IND GLOBEX ES USD US
Some also have stock entries:
14509 14500 4811268 STK SMART FEF EUR DE
10915 10872 3882734 STK SMART Q USD US
21501 21492 14893400 STK TSE Q CAD CA
15640 15631 1254464 STK SBF ES EUR FR
16395 16386 6604196 STK BVME ES EUR IT
20001 19992 26975496 STK TSE ES CAD CA
Give my previous post, I suspect you realize that you would
expect there to be data for futures in the database, but
that it not yet be reflected in futures contracts accessible
to the shim, and for the most part that is indeed the case.
For the symbols you mention: CAC40, FEF, Q, NQ, ER2 and ES,
and looking at the load files, there are entries for all the
futures but NQ and ER2, so you would want to add load file
entries for them, repopulate the database via create.sql, and
then update the expiry attribute using data you've collected
via contract details; mod/LocalFut.sql is the standard place
to record such data in case you need to recreate the database.
Below are entries from the load files:
27291338, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20081219', 'FCEZ8' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
28454973, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20090619', 'FCEM9' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
31332313, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20091218', 'FCEZ9' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
33682842, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20100618', 'FCEM0' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
36163287, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20101217', 'FCEZ0' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
38532726, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20110617', 'FCEM1' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
40996520, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20111216', 'FCEZ1' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
43331097, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20120615', 'FCEM2' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
44729080, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20080620', 'FCEM8' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
46266078, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20080919', 'FCEU8' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
47697612, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20121221', 'FCEZ2' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
48448308, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20080418', 'FCEJ8' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
49016315, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20080516', 'FCEK8' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
49713431, 'FUT', 'CAC40' , 'EUR', 'FR'
, '20090320', 'FCEH9' , 'CAC 40'
, '' ,
'MONEP' , 'MONEP' ),
46454370, 'FUT', 'FEF' , 'EUR', 'EU'
, '20080620', 'FEFM8' , 'FTSEUROFIRST 80
NEW' , '' ,
'MONEP' , 'MONEP' ),
47697618, 'FUT', 'FEF' , 'EUR', 'EU'
, '20080919', 'FEFU8' , 'FTSEUROFIRST 80
NEW' , '' ,
'MONEP' , 'MONEP' ),
49713432, 'FUT', 'FEF' , 'EUR', 'EU'
, '20081219', 'FEFZ8' , 'FTSEUROFIRST 80
NEW' , '' ,
'MONEP' , 'MONEP' ),
46266105, 'FUT', 'Q' , 'EUR', 'EU'
, '20080620', 'QM8' , 'FTSE EUROTOP 100
INDEX' , '' ,
'LIFFE' , 'LIFFE' ),
47697658, 'FUT', 'Q' , 'EUR', 'EU'
, '20080919', 'QU8' , 'FTSE EUROTOP 100
INDEX' , '' ,
'LIFFE' , 'LIFFE' ),
49713474, 'FUT', 'Q' , 'EUR', 'EU'
, '20081219', 'QZ8' , 'FTSE EUROTOP 100
INDEX' , '' ,
'LIFFE' , 'LIFFE' ),
46255901, 'FUT', 'ES' , 'EUR', 'IT'
, '20080919', 'ESU8' , "Gruppo
Editoriale L'Espresso SpA" , ''
, 'IDEM' , 'IDEM' ),
47685897, 'FUT', 'ES' , 'EUR', 'IT'
, '20081219', 'ESZ8' , "Gruppo
Editoriale L'Espresso SpA" , ''
, 'IDEM' , 'IDEM' ),
48437689, 'FUT', 'ES' , 'EUR', 'IT'
, '20080418', 'ESJ8' , "Gruppo
Editoriale L'Espresso SpA" , ''
, 'IDEM' , 'IDEM' ),
48980563, 'FUT', 'ES' , 'EUR', 'IT'
, '20080516', 'ESK8' , "Gruppo
Editoriale L'Espresso SpA" , ''
, 'IDEM' , 'IDEM' ),
49668639, 'FUT', 'ES' , 'EUR', 'IT'
, '20090320', 'ESH9' , "Gruppo
Editoriale L'Espresso SpA" , ''
, 'IDEM' , 'IDEM' ),
44718750, 'FUT', 'ES' , 'EUR', 'IT'
, '20080620', 'ESM8' , "Gruppo
Editoriale L'Espresso SpA" , ''
, 'IDEM' , 'IDEM' ),
You've probably heard that the symbol ER2 is changing, I
believe to RU2. The IB tws does not yet reflect such recent
changes; once IB updates the entry, you would then need to change
the load file you had added for ER2 to, say, RU2.
Thanks,
Bill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.trading-shim.org/pipermail/ts-general/attachments/20080912/e5092764/attachment-0001.html
More information about the ts-general
mailing list