[ts-gen] cross session orders, client ids

[Nils Gebhardt]

Bill, 



On Thu, 2008-12-18 at 12:01 -0500, pippin at owlriver.net wrote:
> Nils,
> 
> Three updates from my message of yesterday:
> 
> 1.  If you are still seeing problems, and in particular if your session
> is ending with a 524 exit, please consider running the shim-081218.tgz
> tarball I just released; I've added some print statements at the point
> where failure to read in account data is detected, to display some of
> the program state at that point.  And, as I wrote yesterday, if you'd
> be so kind as to post:
> 
>     a.  the text the shim displays when it exits; and
>     b.  notes about any exceptional state
>         with respect to the log files, e.g., 
>         missing log directory, or missing log files; and/or
>     c.  the results of running:
>         bin/req.filter < log/shim2tws.bin
>         bin/msg.filter < log/tws2shim.bin
> 



here is a step by step description:

starting a tws and shim session from very scratch - everything is fine,
provided there are no open API orders. Other open orders don't harm.

Once there is an API order launched, all follow up connections  fail. 
This is the console output: 

  
n at gb07:$ ./shim --risk save

          The trading shim has connected to the database server
          and loaded 51649 products; connected to the IB tws as
          client id  8; 
Problem: 524 could not obtain IB account info at startup
Exiting


The output is allways the same; the rate of failing is almost always,
say 97%, but appearently not 100%.

This is the log output: 

n at gb07:$ cat log/tws2shim.bin | bin/msg.filter 
39|
20081218 20:34:17 CET|
 9| 1|67|5|10|66|ESTX50|FUT|
20081219|0.0|?|DTB|EUR|FESX DEC 08|SELL|2|LMT|2490.0|0.0|GTC||DU33186|C|
0||8|890934513|0|0|0||890934513.0/DU33186/100|||||||||0||0|||||||0|0|0|
0||3|0|0||0|2|3|6|66|Submitted|0|2|0.0|890934513|0|0.0|8|

(bad line wrap).
 There are 2 contracts for sell at limit 2490.00, which might be
correct. The 

Also: 

n at gb07:$ bin/req.filter < log/shim2tws.bin
23|8|6|
 2| 1||

well, I can't interpret it, looks short. 
 
Removing the order changes the picture drastically: 

correct start up: 
n at gb07:$ ./shim --risk save

          The trading shim has connected to the database server
          and loaded 51649 products; connected to the IB tws as
          client id  8; and queried for the account code, which
          seems to be DU33186.  Program initialization has been
          completed.


here is the log output: 

n at gb07:$ bin/req.filter < log/shim2tws.bin
23|8|6|
 2| 1||

which is the same as above, and 

n at gb07:$ bin/msg.filter < log/tws2shim.bin
39|
20081218 20:48:40 CET|
 9| 1|68|
 4| 2|-1|2104|Market data farm connection is OK:cashfarm|
 4| 2|-1|2104|Market data farm connection is OK:eurofarm|
 4| 2|-1|2107|HMDS data farm connection is inactive but should be
available upon demand.ushmds2a|
 6| 2|AccountCode|DU 
[...]
6| 2|WhatIfPMEnabled ....

a lengthy message with all the complete account details - the stuff what
was missing before. 
In the rare case the connection doesn't fail in spite of a pending order
the log is the union of account message and report of pending order as
above. 


A short look with tcpdump at traffic to and from the tws port doesn't
reveal so much to me at a first glimpse. I can see request and response
flooding around, seems to be okay without inspecting the packets more
closely, but there is traffic also if the connection fails, which is
somewhat obvious because it leaves an entry in the log, but no dropped
or rejected packets. At some occasion I could observe packets dropped by
the kernel, actually more than I thought that it would be the case, but
it doesn't seem to the source of the problem.  


The last try is with the very latest version: 


[20:50:03][~/T/TEST/shim-081217]
n at gb07:$ ./shim --risk

          The trading shim has connected to the database server

Problem: 414 mysql query evaluation failed
Error 1054 (42S22): Unknown column 'mark' in 'field list'
select uid, mark, var, oid, tag, ecn, client_id, order_tag from
CreateEvent where uid >= 1 order by uid;
Scan: 2 4 1 <1>
Term: ChangeOrder eid       (nil)
Grammar rule: Terminal (eid)
Grammar rule: Compound (ChangeOrder)
Syntax error:
Cursor state: probe; text is:
Terminal tag: name:eid  index:1
1|1|create|LMT|2|5.0000|0.0000|

Dbms: 0 ChangeOrder 1
Dbms: 1 ChangeOrder 1
Dbms: 2 ChangeOrder create
Dbms: 3 ChangeOrder LMT
Dbms: 4 ChangeOrder 2
Dbms: 5 ChangeOrder 5.0000
Dbms: 6 ChangeOrder 0.0000

Problem: 514 a db tuple could not be typechecked
Exiting

which seems to  be a problem prior to problem 524. 



OK, that's it for the moment. If something is missing please let me
know. 


Thanks,

Nils