[ts-gen] Speculation about lost order status messages

[Bill Pippin]

Ken,

About your speculation as to the cause of the lost order status messages
that Nils was dealing with:

> Could this buffering/LiveLock problem potentially cause OrderStatus  
> messages from the upstream not to make it through the Shim and down to  
> the Ruby client?

No; the problem is observed as a hang, not intermittent lost messages.
And no one has yet noted a missing order status *message* to the shim's
log --- Nils' problem was with silent, lost posts, which issue seems to
have been pinned down to a problem, since fixed, with duplicate keys
interacting with the mysql "ignore" attribute in insert statements; see:

"Debugging intermittent lost posts:
http://www.trading-shim.org/pipermail/ts-general/2009-October/000664.html

Now, back to your speculations about order status messages reaching the
client.  All upstream IO uses low-level IO via reads and writes to a
socket.  As I noted in your quote from my original post below, the key
issue is lost IO events; a busybody stdlib IO layer that insists on
buffering multiple IO events is *discarding* the initial subsequence of
those events from the viewpoint of upper layers in the IO stack, and
non-trivial concurrent systems will not tolerate such interference.

>> The livelock would occur where commands to the shim were accumulating
>> in a stdlib buffer, the shim did not know of them and was loop waiting
>> using select for commands, and the client was loop waiting for events
>> that would not occur until the commands were processed.

Buffering is legitimate for a layered API such as TCP/IP, where pieces
for one layer, on one side, are buffered, or aggregated, by the *same*
layer on the other side of the stack.  E.g., network splitting and
coalescence of packets occurs all over the Internet.

Problems arise when some busybody library component throws away
event notifications for a client layer higher in the stack.

Since C stdlib IO and ruby do not do TCP/IP style encapsulation to
distinguish the multiple IO layers --- C file descriptor, C file
pointer, and ruby IO object --- it is up to the application developer
to expose low-level IO state, and the related events, by eliminating
buffering entirely for input, and limiting it to line-buffering for
output.

Thanks,

Bill