To: stik@ON-Luebeck.DE
Subject: Re: [9] STIK: STIK/DIALER                                                       


On Thu, 22 Aug 1996 10:00:26 Peter Rottengatter <perot wrote: 
>
>Please read the whole message before responding.

Ok, I have done so, and will now add some answers below.


>On Wed, 21 Aug 1996, Ronald Andersson wrote:
>
>> On Wed, 21 Aug 1996 09:41:21 Peter Rottengatter <perot wrote: 
>> >
>> >This sounds feasible. But there is still a point I do not understand.
>> >Why are you so keen to have a GEM event driven interface ?
>> 
>> Because this is the only way to have STiK applications that do not 'freeze'
>> during transfers like all the current ones do.
>
>Naah, cannot understand this. The calls to receive data do not block.
>If there is no data they return with a corresponding 'error'.

I am not a client programmer (yet), so I am not fully conversant with the transfer
protocol, but I have seen comments even from Dan about the 'blocking' effect.
If this is merely due to a misuse of the function then all client authors should
correct their code immediately.

NB: This requires that STiK actually starts processing of a reception packet for
    this client after releasing him for other things with a 'data_not_yet_ready'
    type of error code, so that client can call back later.

This does not match any traffic description I've seen before...
I say again, this may be due to current misuse, but we need Dan's opinion on this.


You do not mention the transmit routines above, so what about them...?


>There is absolutely no reason why the application should not be able to do a run 
>through it's event loop before calling the receive call again. This is 
>completely up to the application. Think about the purpose of STiKs 
>housekeep function. It's pretty much the same. Think about how the Dialer 
>achieves a continually processing dial function, with all the other 
>functions still available. It's pretty much the same.

Unless I've misunderstood something the dialer as yet does not use the transfer
functions so I don't see how this example proves anything either way.
As for _how_ it does what it does, I wouldn't know.  I don't have that source.


>It's not even that the burden for extra code is shifted to STiK,
>the applications structure is again the same. Place the call for checking
>for data next to the evnt_multi(), and you really have the same program flow
>as with messages DATA_IS_THERE delivered by evnt_multi().

Let me repeat again: I have _never_ said that transfer handshaking is to be done
by AES messages.  For this purpose I (like you) prefer other _faster_ means.
I just don't think the current means are sufficient, though this may be partly
due to misuse or non-use of existing features (but only partly).


>So the reason why STiKs clients (apart from CAB.OVL, thats a different story)
>freeze is not that it's not possible to avoid it.
>(CAB.OVL would freeze even with event driven STiK).

Only if Alexander utterly refused to accept any changes in the interface between
CAB.OVL and CAB.APP, and why should he start to refuse to do that all of a sudden.
How do you think the "mailto:" links were implemented, except by cooperation on
the part of Dan and Alexander on how to add it.

In fact Alexander himself has complained that the current interface limits smooth
functioning of CAB, so I think he would welcome an improvement of this nature.


>> >Why can't you content yourself with the interface we have right now ?
>> 
>> Because I am not satisfied with the way all STiK applications are handicapped
>> in their ability to update windows etc.  The delay time for loading an entire
>> web page would not matter so much, if I could at least use the ready segement
>> of the page meaningfully, with scrolling etc.,  or if I could start loading a
>> second page into another window while studying the first page, and so on.
>
>See above. For normal clients it can be achieved without an event interface.

No, I don't think so, even if all you say above is true, that still leaves several
unanswered questions, such as transmit operations and simultaneous clients.


>CAB however (sorry that I gotta rob your illusions ;-) )wouldn't benefit anyway,
>unless Alex gives up on separating the net code from the browser code.

Where did you get that idea.  All it takes is for him and Dan to agree on an
altered interface to CAB.OVL, so that Alex can have a special event_loop handling
things while waiting for transfer completion.


>> For IRC true multi-channel multi-window capability could be possible,
>> and in FTP one could continue searching directories during downloads.
>> (GETting more files might be possible too, but only with lowered cps.)
>
>I can draw immediately the program structure on an IRC client with 
>multi-channel multi-window capability with the _current_ STiK interface.

Good, then I suggest you do that (at least roughly), and send it as a suggestion
to Lonny and Timpy.


>If you do pure sequential programming you end up with applications that 
>freeze. You gotta dismiss the sequential programming, not the STiK 
>interface, in order to avoid freezing.

Do you never read my actual words ?

I have _never_ dismissed the current interface, only suggested added improvements,
but your comments on sequential programming are of course true.

As I've understood some of Dan's remarks one problem of the current interface is
that clients are considered purely sequential and alone, whereas we need them to
work as parts of a multitasked environment (even under single-TOS).


>So looking for non sequential paradigms you came up with a GEM event.
>You figured you had to do it via messages,
>because there is no proper way to introduce new events.

Do _what_ via messages...?

If you are still talking about data transfers then you are putting (false) words
in my mouth, because I have many times iterated that I do not believe that AES
messages are a good way to implement this.  This time please note it.

There are things for which messages are appropriate, and there are things that
need a low-level interface for reasons of speed etc. That includes data transfer.

The only event I've even mentioned in connection with packet transfers is the
timer event, which could be used after 'unfinished' STiK responses,
to let other tasks use the CPU during tardy transfers.


>You thought about this way because you're so familiar with GEM events and messages.

Did I...?   No!

It seems that you are recollecting a lot of stuff said by others, from the older
TFC debates, when some people wanted to replace the cookie interface completely
by an AES messaging system.  (Which I often have argued _against_ !!!)

So when I've insisted on talking about the continuing need for interface
improvements of an event-driven kind (meaning that APPs are so driven),
you identify me falsely as holding all of those opinions myself.


>But remember that the current STiK already provides events
>(at least you can look at it this way),
>it's just that these events are not delivered by event_multi(),

In principle I agree, but even Dan thinks the current implementation insufficient,
and I agree because we need better multi-client support, and a consistent way of
letting clients stay non-blocking in their event_loops (for both RX and TX).


>but by any receive_data call() from the current STiK interface !

And what about other calls of the interface...?  (none must block)
And what about simultaneous (well nearly) client calls...?
This is where the current interface fails to do the job.


>> These things are possible on other computers, so I see no reason why they
>> should not be implemented on our systems too.
>
>No problem ;-)

Agreed, but I think the interface needs to be updated for it.


>> Yes, and for that to work, the routines must await STiK results in event_loops,
>> since that is where the AES messages are delivered to applications.
>
>Ok, I'll sketch a rough example for the event_loop :
>
>do {
>     event = event_multi (MU_BUTTON | MU_MESAG | MU_TIMER,
----- snip -----
>     if (CNbyte_count (our_connection) > 0) { /* get data and process */ >        }
----- snip -----
>Note this is _very_ rough, and out of my head, don't worry about missing 
>parameters in evnt_multi(). Hopefully you get the point ?

I sure do, since this is one of the forms of "event_driven" uses of the interface
which I favor, but the current routines need alterations for several reasons,
since some _are_ blocking, and most lack multi-client support.


>> First, only those operations _requiring_ AES messaging should use it.
>> Wherever appropriate the traditional method of direct function calls should be
>> used instead, since that is one of the fastest interfaces at our disposal.
>
>So, which operations do *require* AES messaging
>(You see, I'n not convinced yet ;-) )

For basic communication, none at all !
I keep telling you, you're picking on the wrong guy about that message stuff.

But there are lots of not time-critical things for which messages are a good way
to communicate between different programs, so STiK should retain the ability to
use these (and all other) AES capabilities, by having a main ACC or ACC/APP.
This can be linked to the TSR as it is now, or through some other means (below).


>> Ah, here we reach agreement.  This is precisely the sort of 'Semaphore' I speak
>> of above (hadn't gotten this far when I wrote it), except that I think that both
>> the flag and the buffer should be provided by the application, because it is not
>> acceptable that one APP lock up STiK's internal resources any more.
>
>Ok, just that this 'Semaphore' (it's not quite a semaphore, but let it 
>call it this way) is already implemented in STiK !

No, that is a common flag for all clients, whereas I speak of a unique flag
for each client, and therefore supplied by the client.


>The only trouble is that you can get buffer overflow if the application spends
>to much time doing other things, that's why I proposed a set_buffer() call in
>STiK, so that the application can have a larger buffer.

Which I agree with, except that each client should have separate buffers.
STiK still needs a master buffer, but packets should be transferred to the buffer
of the client that ordered the data.  This way overflow only affects the buffer
of the misbehaving client, and not the master buffer or other clients.


>> In fact a new structure could be declared, encompassing this flag, the buffer,
>> and any other stuff we find necessary. Such might include latest poll time to
>> allow STiK to judge if a client has 'locked up' and then set a flag telling the
>> client (if he ever wakes up) that he must restart his operation.
>
>This sounds reasonable. But don't you think the semaphore is there already ?
>STiK tells you if data is present.

Present, but insufficient for multi-client use.


>> For multi-client use STiK can then have a:
>> 
>> 'struct client_blk *client_list[ MAX_CLIENTS ];'
>> 
>> or something similar to keep track of all currently active clients.
>
>Yes, this should be up to Dan to implement.

Naturally.


>> The necessity is surely obvious to anyone who has seen how poorly CAB, or most
>> other clients respond to GEM operations during transfers, though this is more
>> obvious under multitasking systems than under singletasking TOS.
>
>The sad thing is that just CAB probably doesn't gain from all this. CAB 
>requests a www page from the OVL and expects it in the cache once the OVL 
>returns. This prevents the use of such techniques, no matter how well 
>designed they are.I've got the OVL interface development kit for CAB, just 
>hadn't time to scutinize it thoroughly, so there might be a clever way of
>doing it.

AFAIK the CAB.OVL is still supported by Dan, and I'm sure he could do something
to alter the interface so that CAB can stay in the event-loop awaiting data.
And if your description is indeed correct and reception calls never block then
this should be possible immediately without waiting for STiK changes.

The altered interface between CAB and CAB.OVL will have to be agreed upon by
Dan and Alexander first of course, and at the same time they could fix the
remaining cache problems (ie: Latest OVL is incompatible to latest CAB already).

When STiK itself is updated (which I still think it needs),  this should not then
affect the interface between CAB and CAB.OVL again, so at that time Dan can make
the changes without even consulting Alex.


>> I fully sympathize with your worry concerning speed, but I happen to believe
>> that we are currently _losing_ speed by having the APPs locked up while waiting
>> for network data limited by modem speeds.  Such waiting is improductive.
>
>I haven't written any data traffic client for STiK yet, so my thoughts 
>are a little theoretical. But maybe my sample code from above inspired 
>one of the client programmers to give it a try this way.

Hopefully, and if it really works that way.

But why then have none used it yet, and why do they say that the calls block ?


>If it's proven to be possible this way we don't need to worry about this anymore,
>and a lot of Dans programming time will be saved.

That would be well, but even if current reception routines are not blocking,
there are other aspects to our current problems that still need solutions.
I won't reiterate all of them here since I have done so several times above.


>Can I have some feedback from the clients programmers ?
>If nobody wants, I would give it a try.
>My problem is that I got to write something like an ftp-client from scratch 
>just for this.

Yes, can someone who has worked with the transfer routines please give us the
straight dope on this...?

Are they in fact blocking or not...?

-------------------------------------------------------------------------
Regards:  Ronald Andersson                     mailto:dlanor@oden.se
                                               http://www.oden.se/~dlanor
-------------------------------------------------------------------------
