[ts-gen] Downstream architecture (3rd)

[Ken Feng]

Hi Bill,

I appreciate your taking the time to comprehensively describe a
sensible approach to designing the downstream client.  It's a great
wrap-up, answering many of the questions I have had this week on the
mailing list.

The examples in exs are fantastic for getting our feet wet, but as you
mentioned,

> It would be nice to provide example code via a ruby sample client,
> and that's just one of many things we haven't completed yet.

I look forward to seeing an example one of these days, but I believe
you have provided enough of a roadmap for me to continue coding an
approach in Ruby.  Generally, I am indifferent to a particular
language or platform, insomuch as picking a tool doesn't "make life
hard for me" in terms of implementation or maintenance.
Unfortunately, my engine for generating signals/orders in C++ is
sufficiently large/complex that I do need to interface to it somehow.
As you have very well described in your posts, I share you abhorrence
of any thought of duplicating such code.  I was walking down the
approach of a "poor man's IPC" via MySQL and crontab'ed writes and
reads into the database, but I may be able to avoid that with your
Ruby hints.

My indifference towards languages has also led me to a hodge-podge of
programs and tools for getting the job done.  I am fine with this, but
it is certainly not pretty for other developers who are trying to
understand my code - but that's a different story.

Having said that, I actually like Ruby and prefer it over all others
for any type of scripting (though I hear Python is also quite good) -
Ruby's conciseness, intuitiveness, richness of API, and delivery of
its promise of "least surprises" are all quite impressive.  I will
definitely search popen3() and select() in Ruby - thank you for
pointing me in the right direction.

I will hard-code my triplet response code this time in Ruby, but I do
look forward to the day when I can refactor it to be driven by a shim
table instead.

BTW, if you haven't done so already, I hope you do copy-and-paste
these three posts into the new documentation.

I am certain that users who are at the last stage of downstream client
design/development would find these notes invaluable, as I have.

Regards,
Ken

On 8/29/09, Bill Pippin <pippin at owlriver.net> wrote:
> Ken,
>
> With respect to your questions about downstream architectural
> approaches to the IB tws api, this post focuses on the following
> question:
>
>     How might a ruby sample client, an example program to interface
>     with the shim, be designed, what parts of the code would be
>     provided, what parts would be supplied by client users, and in
>     the absence of such a sample client in the shim tarball for now,
>     what might users write in its place?

>
> To recap, such a program would apply user-specific criteria to perform
> the following types of tasks:
>
>     * ask for market data;
>     * interpret such data;
>     * generate orders; and
>     * manage positions.
>
> Once given an IB tws account, and the shim, then the shim command
> language would be used to generate api requests, and the shim cout
> option could be used to ensure that the resulting message traffic,
> and indeed the entire integrated event stream, were accessible to
> the client.  That much is obvious, and leaves us a long reach from
> the task goals above.  What else?
>
> A.  Main.
>
> Once given initialization, about which more later, there would be
> a main loop to process events; input for that loop would be read
> via the select() system call, provided in ruby; and the two input
> streams from the shim provided via the popen3() system call, the
> stdin and stderr, would be multiplexed via select() .  The input
> would be selectively converted to event objects via table-driven
> code, which brings us back to initialization.
>
> B.  Init
>
> Prior to entering the main loop, event descriptions from a sql
> database would be read in, and used to generate: event classes, the
> related data members, and constructor code which would take as
> input a split array from a text input line.  The result would be an
> admittedly large quantity of automatically generated boilerplate
> none of which would need or warrant human maintenance.
>
> C.  Processing
>
> The client user would still be responsible for writing: the
> definitions of non-empty process operations for each message event;
> as well as strategy code that would use input from such processing,
> presumably to generate signals and/or orders.
>
> D.  User input
>
> The client user would still be responsible for writing: gui screens,
> or code to accept yet another layer of downstream command input, if
> so desired.  As previously noted, the select() system call could be
> used to multiplex the three input streams of other client/human
> control, events from the shim cout, and exceptional messages from
> the shim stderr.
>
> E.  What for now?
>
> As for (c) and (d), above, that's the user's responsibility in any
> case, and so about which no more.
>
> As for (a), users should already be using popen3() or its equivalent
> in their favorite scripting language, and select() ditto, in any case;
> or else they may choose threads, in which case that's their business,
> and outside the scope of these list messages.
>
> It would be nice to provide example code via a ruby sample client,
> and that's just one of many things we haven't completed yet.
>
> As for (b), this is the serious gap, and all I can say is that study
> of rule.c and apis.c may help, but is a non-trivial effort.  I fully
> intend to define sql create table and insert statements to provide
> table definition of the api events, but such work is intimately coupled
> with versioned api-level event processing, our next major work effort.
> It's just not done yet.  Sorry.  Feel free to read rule.c, and apis.c,
> and for that matter EClientSocket.java and EReader.java.
>
> Thanks,
>
> Bill
>
> _______________________________________________
> ts-general mailing list
> ts-general at trading-shim.org
> http://www.trading-shim.org/mailman/listinfo/ts-general
>