[ts-gen] Do I really need to tail -f ShimText?

[Paul C]

On Mon, Sep 14, 2009 at 09:25:31PM +0900, Ken Feng wrote:
> 
. . .
> Question:  If I need to know the moment that a response to an order is
> coming from upstream, is the right approach to open a pipe to "tail -f
> ShimText" and select() on that pipe?

Hi Ken,

A few notes on what I do, in case it helps:

I'm not sure I'm using the 'right' approach, but I also look for the 3/3/6 'Filled'
response in the shim output.  However, I don't tail -f ShimText but
instead run the shim with the 'cout' option (shim --risk cout) so
the response is on the stdout, which in turn is connected to a
pipe/handle (popen in ruby?). (I'm not actually using ruby, hence the
lack of an example/code).

> If there is no better approach, that is exactly what I will do - check
> for the 3/3/6 from that pipe and try to match it up with a row in the
> OrderStatus table so I can get better processed details about what
> just happened.

I only send one order at a time though, so I've not (yet) bothered
trying to match up order id's with the mysql tables.  

 
> I can't help but feel that I'm doing something dirty/hacky by doing a
> "tail -f ShimText" into Ruby.  Someone please tell me that I am wrong
> and show me the light (ie a better way), before I fall into the
> dark-side...  Thanks in advance.

I'd agree that using the ShimText file doesn't seem right.  I wonder
if you just need that 'cout' option? - or maybe I'm missing
something. I run the shim from within a haskell program and get
'handles' to the shim's stdin, stdout and stderr. I read from the
stdout, write to the stdin, and ignore the stderr.

I also use the shim's logd option (shim --risk cout logd) so that I
can put a tail -f on the appropriate syslog file; but that's just so I
can see what's going on.  In the past I've also setup
/etc/rsyslog.conf so that the shim's 'logd' output goes to a named
pipe(FIFO), which I then opened as a file - rather than use the ShimText on the
hard disk. Maybe that is another way forward.
 
> I was hoping that Shim's stderr would show raw messages (like what's
> in ShimText) while Shim's stdout would show "cooked" messages (nice
> cooked stuff would also show up in MySQL as we see in the journal
> tables)... ...oh, well...

My approach may be too simplistic - I don't actually utilise any of
the MySql info, or deal with multiple orders.  I run two instances of
the shim from within the downstream program: one in --data mode for
reading prices (select tick...) and the other in --risk mode (as
above) for sending trades.  With both shim processes I parse the
output looking for the appropriate |src|tag|_| lines.

Regards

Paul C