[ts-gen] Triple 8 release

[Bill Pippin]

Order journalling for the shim is now complete.  You don't need to refer
to it --- the same information has been and still is available through
the output log channel printed by the shim --- but it's there if you
need it.  Note also that multiple shims can share the same database;
this has been possible for a while now, although the documentation
has not yet been updated to reflect the fact.

In short, CreateEvent and ChangeOrder records track order requests by the
shim towards the IB tws, while OrderStatus, ActiveOrder, OrderResult, and
OrderReport records reflect the order status, open order, portfolio value,
and execution report messages from the IB tws to the shim.

In more detail, the shim posts information as it becomes available from
the IB tws as follows:

The table:        is written to by the shim:			(notes:)
-----------       -----------------------------------------------------
AccountCode       At startup, if and when an account code is
                  seen for the first time.                      (a     )

CreateEvent       When an order is created; more precisely, each
                  time a place order request is made to the IB tws
                  as the result of a create order command.

ChangeOrder       When any order-related command occurs; more 
                  precisely, when a place order request is made
                  for any reason, whether create, modify, submit,
                  modsub, or urgent; and also, if and when 
                  a cancel order request is made.

OrderStatus       For each IB tws api order status      message (  b   )
ActiveOrder        "   "   "   "   "  open order          "     (  b   )
OrderResult        "   "   "   "   "  portfolio value     "     (     c)
OrderReport        "   "   "   "   "  execution report    "     (  b, c)

notes:
------------------------------------------------------------------------

  (a) The shim tries to guess whether the account is live or paper/demo
      from the account code prefix, but you can update the AccountCode
      record type attribute, or for that matter create the entire record
      yourself to start with, as you choose.

  (b) OrderStatus, ActiveOrder, and OrderReport records can be matched
      with each other via the permanent order id, with attribute name
      "perm".  They can be matched with CreateEvent records via the
      triple of attributes (acc, client_id, order_tag), that is the
      account code; per-session connection id; and per-order line
      item tick id, which together serve to designate a unique order.

      The IB tws maintains the tick id sequence numbers for each
      (account-code, client-id pair), at least as long as you continue
      to use the same settings directory; if that changes, it's up to you
      to choose a different client-id session value using the shim's
      ClientId configuration variable, currently set via the .shimrc file.
      
  (c) The shim attempts to lookup the IB contract id, attribute name ibc,
      using, from the message, the security type, currency, symbol name,
      and as applicable, the expiry, option right, and strike price; and
      for the session, the locality or region.  The region is also set
      via a configuration variable, Locality, included in the .shimrc
      file, or else defaulting to "US".

      If the region is incorrect for the given product, then one of 
      two things will occur.  If some contract id is found, it will
      be wrong, and the "ibc" entry in this record will be wrong; or
      else the lookup will fail, and the shim will refuse to write
      the portfolio value or execution report message to the 
      OrderResult or OrderReport table.  This problem goes away once
      we upgrade the shim to or past client version 35 of the IB tws
      protocol.

      Until then, you'll need to check the product locality via the
      sql/bin/get_id.sql script, which can be used to dump product
      information to the standard out.  Or for that matter, you can
      search the product symbol load files in sql/sym for the Source
      and FutSrc tables, where the region is referred to by attribute
      name "code".

      Without the region value, the information in the IB tws api
      is incomplete up through client version 34.  We feel that 
      the risk of failed writes (the shim prints a transient error
      message to the stderr and keeps on going) is preferrable
      to ambiguous product information.  
       
--------------------------------------------------------------------------

These features have been a long time coming.  Their implementation has
required extensive changes to the database design, command language, and
initialization process.  (The code for the database posts is itself
relatively simple, and has been dropped in over the last couple of
weeks.)

As a result of the aforementioned earlier changes to the database design, 
downstream clients have no longer needed to update the database during
a shim session, and so there has been little or no need to carry data
between versions of the database.  This changes now that there is
effective order journalling; if you make orders with the shim, and
care about this data, you'll want to transfer it from one database
version change to the next.

This should be relatively straight-forward.  The history bar and order
journal records all include duplicate information to simplify reloads.
The IB contract id has been added to history bar, as an alternative
to the (product, exchange) key pair, and the contract descriptions
from the open order, portfolio value, and execution report messages
are included in their respective tables.

This should not be an issue yet --- after all, order journalling is 
a new feature --- but if you have questions about database migration,
please ask on the list.  We'll probably include a database reload
script with the distribution at some point in the near future.

Please note that the database version has increased a number of
times over the past few weeks, and that there have been changes
to the sort ordering for table "Symbol", which has changed
the product and contract numbers used by the shim.  A near-term
future change should allow you to use IB contract numbers in
commands instead, but for now, you'll need to update downstream
scripts to reflect the jitter in trading-shim contract uids.

Thank you for your patience, and I hope this is useful.

Thanks,

Bill