To: stik@ON-Luebeck.DE
Subject: Re: [8] STIK: STIK/DIALER                                                       

Hi Guy!

When I first read your latest posting (below) I got it all wrong !
Fortunately I didn't reply at that stage, but took my time to understand what
you really must have meant, filling in by deduction som things never stated.
I'll summarize my conclusions here so you can easily confirm or deny them.

You propose an alternative messaging system to be used by STiK/clients for/in
event loops.  The message codes should be chosen such that they may as well be
used through the AES message system, though that is not the intended normal use.

Instead a set of new functions are to be used for calling a new part of STiK,
which will handle the messages by the aid of a new buffer (oft-referred below).
The purpose would be to achieve a message system fast enough for STiK needs, and
yet similar enough to the AES message system to allow some parallel usage. 


So, am I right or wrong...?

If I am wrong in this, then all my replies below are a load of garbage.  :-)


On Fri, 23 Aug 96 22:17:42 G Guy Harrison <test@swampd wrote: 
>
>  Peter Rottengatter <perot@pallas.amp.uni-hannover.de> wrote:-
>>
>>There has been reached consent throughout many programmers that the Fsfirst and
>>Fsnext design should be dismissed because it's got a fundamental flaw :
----- snip -----
>
>I gave the wrong impression - no memory of this sort needs allocating. 
>The idea was so that the code could look for messages to which it 
>needs to respond. is_first_msg() would lock onto the first message for 
>apid in the buffer, is_next_msg() would simply look for subsequent 
>ones. It would only issue read_msg() if it found one it was interested 
>in. In retrospect appl_search(1,) would have been a better analogy!

Well, the fundamental thing here then is that 'is_first_msg' and 'is_next_msg'
both only test for the presence of messages, they do not affect storage at all.


>>What I wanna say is that if we want to invent the wheel again, then 
>>let's not invent a cornered one ;-), instead let's do it properly, and 
>>use the open/do_things/close concept for splitting is_msg().
>
>My thoughts on the buffer atm are that it is maintained by STiK and 
>the calls implemented via the cookie. This shouldn't be a problem for 
>Dan because the only connection it will have with STiK is that STiK 
>will have to allocate the buffer. Someone else can write it, it can be 
>tested as a seperate TSR if necessary, then the code could be passed 
>to Dan once debugged.

True, although the 'test of fire' would of course be for STiK itself to use it
for some purpose connected with actual network transfers.
I'm still not convinced that this method is suitable for data transfer though.


>If the buffer is a reasonble size, it could be used for data transfers 
>between clients if the need arises.

I do not agree, it would be far easier to stick to the standard AES message size,
and let clients use this block to tell each other where data buffers really are.
The main message buffer becomes a global STiK resource, and should not be so
easily abused by any client, perhaps preventing others from efficient use.


> It should at least be big enough not to run out of msg space in practice and
>to hold a reasonable number of filenames - 5K or so.

I don't see the connection to filenames, but that size should be ok.


>Specific size is not really important - it could be specified in the config file.

Sure, or even use a default size decided by STIKTSR.


>Here's a spec to start us off. Discussions from all please!
>
>void	init_mbuf(void *the_buffer, long buffer_size)
>
>Internally issued by STiK after it has obtained the buffer. Saves Dan 
>from having to write more than a couple of lines of code. :-) Gets the 
>buffer working.

You mean the separate code module (not by Dan) can initialize variables etc ?


>void	kill_mbuf(int apid)
>
>Flushes any messages waiting for apid. (below)

I agree that STiK must be able to do so, but in fact it may also be useful for
the application to get rid of messages when aborting communication with some
other client.  It can still do that a message at a time though.


>The rest are accessible through the cookie ...
>
>int	put_mbuf(int apid, void *app_buff, int msg_size)
>
>writes msg_size bytes from app_buff into the buffer. Returns the 
>number of bytes written or a negative error.

Are you sure we want to allow extended messages in this buffer...?
Ok, so this format is defined for the AES too, but when is it used ?


>int	is_msg(int apid, int flag, int &msg[0])
>
>Returns ibuf (below) - unique index to locate message with get_sbuf(), 
>get_mbuf() and clr_mbuf(). Returns zero if the are no messages waiting 
>for apid. If flag is passed as zero it gets the first message waiting 
>for apid, 1 gets the second, 2 the third etc - will eventually return 
>0. On a valid non zero result msg[0] will be the contents of the 1st 
>byte in the message. I envisage ibuf to be a direct index into the 
>buffer so get_mbuf() can consist of little more than a transfer 
>operation.

That means we define the first byte of the buffer as unusable (no msg at ibuf==0)
and for access efficiency this should be a 16-bit word.

A standard AES message does not consist of bytes but of 16-bit words, so at least
one word must be moved to &msg[0], which should be of 'uint16 msg[8];' type.
Otherwise testing the message type will require more operations.

In fact we gain a lot of speed by declaring the entire buffer as an array of
16-bit words, especially if we stick to the standard 8-word message format.

Extended messages may be padded to an even length, since the true length is
always given in the 8-word message heading the extension.


>int	get_sbuf(int ibuf)
>
>Return size of message ibuf or negative error.

Why not follow the normal AES usage instead, and let 'is_msg' get all 8 words
of the standard message, since they will normally be needed anyway.

This should cost less time than an extra call to find the size, especially as
we're no more likely to use extended messages here than in normal AES.


>int	get_mbuf(int ibuf, void *app_buff, int msg_limit)
>
>Transfers msg_limit or less bytes from buffer to app_buf, returning 
>the number of bytes still waiting to be transferred or negative error. 
>Stored in the buffer for each message will be a "number of bytes 
>transferred so far" value - is_msg() will reset this to zero.

Ok, but if 'is_msg' gets the first 8 words, this function will only be needed
for extended messages, which I don't really see any need for having.


>bool	clr_mbuf(int ibuf)
>
>Removes message ibuf from the buffer.

Ok, but why bool ?
I know 'void' would prvent us from using the statement in 'dummy' expressions,
but then why not use some type less likely to cause conversions. eg: 'uint16'.


>Problems:
>
>The cookie may not be initialised. For apps which refuse to run 
>without STiK this is of no concern but some apps do! Solution: 
>is_msg() will have to have a source binding to check for the cookie 
>etc before it calls the STiK is_msg(). If there's no cookie etc then 
>this source has to return 0.

I think this should be irreleveant by now, from Dan's description of the new TSR
it appears that the cookie is always defined, which does not mean that any
connections are open either physically or logically.

The messaging system could very well be used anyway, however, since this should
be initialized at boot by STIKTSR.  So why worry ?


>At some stage the messages for an app will need to be force cleared 
>with kill_mbuf() else the next app that gets that apid will get the 
>previous app's unread messages. I guess it would need to be done once 
>the app has closed its last socket but this does not cover crashes. 
>This one needs your input Dan!

Maybe, but I will have my say too.

This can easily be solved, if an array is maintained holding a structure for
each application having a message waiting in the buffer.  This structure would
contain the applications identifier, the number of messages waiting, and the time
when the application last polled for messages.

A variable also keeps track of the time we last made what I call a 'safety check'
of the structure array, so this is repeated at regular intervals.  That variable
should be checked against current time at start of each 'put_mbuf' and those
'is_msg' calls with flag==0, to clear out messages to 'dead' applications,
by checking the array for applications not polling recently.


>Advantages:
>
>Tos only programs can use it.

Not as you describe things, because they have no application id.
We'd have to use some other identifier for that.


>The is_msg(0) get_msg() clr_msg() combo will behave like normal AES 
>messages, arriving in the order they get written.

Sorry I can't make sense of that sentence.
As I see it they are all functions, with the first two roughly corresponding
to 'evnt_mesag' and 'appl_read', whereas the third has no equivalent.


>is_msg() can be made selective. Useful for getting out of code without 
>the overhead of evnt_multi().

I suppose you mean we can select which of several incoming messages to obey first.


>Use of ibuf means a message can be located fast.

Once found by is_msg, yes.


>Possible Addition:
>
>It might be worth considering implementing a timer message. After a 
>specific time the sender gets the message back.

That requires some highly regular interrupt for STiK itself, such as the 200Hz
system interrupt, but that must be highly speed-optimized and may even be needed
for entirely different purposes in a new improved STiK version.


>Well let's get discussing folks - then we can either trash the idea 
>or get designing the buffer format :-))

Ok, I think the idea is worth looking into as a complement to the interface.
(Provided that is, that my assumptions of what you really mean are correct.)

-------------------------------------------------------------------------
Regards:  Ronald Andersson                     mailto:dlanor@oden.se
                                               http://www.oden.se/~dlanor
-------------------------------------------------------------------------
