[ts-gen] How symbols and contracts are defined during database creation

[Bill Pippin]

Sam,

Based on your report, I've fixed the problem with STK:35052259
in the most recent release.

That being said, such incremental fixes to the database are
probably not satisfactory to either of us, certainly not for
the long run, and I suspect that eventually there will be
scripts that each user will need to run to preen the database.

Perfecting the symbols data in the distribution beyond what
we can obtain externally is problematic, for at least two reasons:
we can not distribute information received via the api, and api
contract details vary from one account to the next.

By way of example, for queries against symbols STK:GSZ and STK:GAZ,
for a shim running against my current paper account, and with the
wild card command line option, e.g.:

    ./shim --data file save wild

then the commands:

    select wild STK GAZ  all;	wait 1;
    select wild STK GBZ  all;	wait 1;

return only three distinct contracts:

    3|10| 4|GBZ|STK||0.0||ASX     |AUD|GBZ|GBZ|GBZ|46933587|0.0010 ...

    3|10| 4|GAZ|STK||0.0||SMART   |CAD|GAZ|GAZ|GAZ|44531001|0.0050 ...
    3|10| 4|GAZ|STK||0.0||VENTURE |CAD|GAZ|GAZ|GAZ|44531001|0.0050 ...

    3|10| 4|GAZ|STK||0.0||SMART   |USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||ARCA    |USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||BATS    |USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||CSFBALGO|USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||DRCTEDGE|USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||EDGEA   |USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||ISE     |USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||ISLAND  |USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||LAVA    |USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||NSX     |USD|GAZ|GAZ|GAZ|46943368|0.01   ...
    3|10| 4|GAZ|STK||0.0||TRACKECN|USD|GAZ|GAZ|GAZ|46943368|0.01   ...

In particular, I don't see a ib contract 35052259.  Presumably you
and I have different accounts, and the api filters contract details
accordingly. 

For your other symbols, would you be willing to consider using
wild-card contract details queries to determine the correct entries
to the database load files, and submitting patches to the list?

For that matter, for proprietary trading, you can make these changes
locally and not even say anything.  The reason to submit changes
to the list is just for your convenience, so that new releases
capture your earlier updates.

To make such changes, you could edit the load files, and then
run the database create.sql script to drop, recreate, and repopulate
tables.

Please post to the list if you have questions about any of the above
points.

Thanks,

Bill