To: stik@ON-Luebeck.DE
Subject: Re: [11]  STIK: Runtime error Ag                                                 


In this letter I make some alternative suggestions in the final part that
replace some of my earlier ones, so to avoid wasting time I suggest that
all who want to reply to this finish reading before starting writing.


On Thu, 10 Oct 1996 23:13:10 Peter Rottengatter <perot wrote: 
>
>On Thu, 10 Oct 1996, Ronald Andersson wrote:
>>
>> A timer interrupt may use AES functions correctly only if the interrupted code
>> was itself not a GEM function, since there is no telling at what stage of such
>> a function the interrupt occurred.  If the interrupt occurs directly after some
>> data has been placed in any of the arrays, calling any AES function may erase
>> that data before it has been properly used.
>>
>> Possibly the input data of 'evnt_multi' is not needed again after entry and
>> this might make the usage safe, but then the interrupt routine must have some
>> way of testing that this 'stable' part of the code has been reached.
>
>This is not a problem, as easily a copy of the original AESPB can be 
>used. This copy could even be made by STiK itself. Then your scenario 
>cannot happen.

Sorry, I don't get this.

As I see it copying the array of pointers makes no difference.
It is the data in the various arrays they point to that needs protection.
Copying and later restoring that data would be too time-consuming.

I'd rather copy just the global pointer and use brand-new arrays for the
rest, although I still think that there could be other reentrancy problems.


>> >Just to get sure, I'll talk to Andreas Kromke and Craig Graham to this
>> >(They are two programmers of AES versions, both know these things very well.)
>> 
>> Good idea!  Note however, that we will still have to test that their ways of
>> implementing such things also corresponds with that of Atari's TOS.
>
>Sure. Geneva's Dan Wilga is on the net, and maybe someone can figure out the 
>address of the author of NAES. Those people we can ask directly.

Good!  The more the merrier :-)


>> Yes, I agree, but if that's what it takes to achieve fully fluent operation
>> of STiK, that will just have to be accepted.
>> That is why I have repeatedly said that there must always be one resident
>> GEM APP in a STiK system capable of using all system functions legally.
>> 
>> Whether that APP is STiK itself, a dummy APP, or even your Dialer
>> (then mandatorily resident) doesn't really matter.
>> What matters is that having more resident APPs should be an option of 
>> the user.
>
>We might still get away without any GEM app, and still being able to call 
>AES, if my first idea is not turned down by the AES authors I mentioned.

Yes, but that would mean that there is no internal AES capability of STiK
when there is no client registered to borrow id and arrays from.

I'd rather go for a scheme where the dialer takes over as resident APP/ACC,
to give STiK a continuous AES capability as well as the ability to request
callbacks from the dialer's event_loop.


>> If we go for the latter of the three alternatives, your Dialer 
>> effectively will become an integral part of STiK itself after each 
>> runtime installation, so you will need some method to coordinate the 
>> use of its AES capability with STIKTSR. This is easily solved by using 
>> a 'test and set' semaphore to indicate whether the arrays are already 
>> in use or free to be used.  Interrupt routines must still be careful 
>> though, since _no_ AES or GEMDOS calls are legal when the system
>> was already at work in such a function at the time of interrupt.
>
>The latter restriction is only valid on old single-tasking systems !

No, AFAIK only MagiC has such a rewrite of GEMDOS, but none of the other
multitaskers, and (also AFAIK) not even MagiC has interrupt-reentrant AES.
Besides, do you mean we should abandon single-tasking compatibility...?


>I'd not need any semaphone, as STiK could use a copy of the AESPB, like 
>explained above.

One way to do it is of course to copy the 'global' array ptr into a new AESPB,
but use separate arrays for the rest.  Simply copying all of AESPB won't work.


>> One way of testing this, similar to how MagiC and MTOS avoids tasker problems,
>> is to simply test whether the interrupted routine was in supervisor mode.
>
>MagiC only prevents task-switching in supervisor mode because MiNT does it
>and MagiC is required to be compatible. MagiC has no problems with calling
>AES functions in supervisor mode, or even from an interrupt. 

Of course it is legal to call the functions in supervisor mode.
AFAIK this has always worked in all TOS versions.  
Calling them from interrupts is something completely different.

The problem is not that the call is made from the interrupt, but that there is
no way of knowing what the system was doing when interrupted.  This means that
only such routines that use semaphores properly and/or disable interrupts while
initializing critical function data are reentrant.  The MagiC doc's state that
such changes have been made for bios, xbios, and for gemdos, but I have not
seen any such statement regarding VDI or AES.

But we are not writing for MagiC alone so that really doen't matter !
None of the other multitaskers have reentrant system functions, and we must
also stay compatible with single-tasking TOS so reentrancy is not available.


Do we really need to have this kind of timer interrupt....?

If we accept having one resident GEM APP (eg: the dialer) we could use its
'evnt_multi' for this purpose instead.  So instead of installing an interrupt
routine, each STiK module/client that needs one gives a callback address to
this main APP, which calls the registered modules at timer events and passes
its own AESPB to them so they can use AES without being GEM APPs.

I know there's a lot of overhead involved in 'evnt_multi', which is of course
more serious when making the call with short timeouts, but much of the overhead
can be removed by using an AESPB with reserved constant arrays for 'contrl',
'addrin' and 'intin'.  Since their data is never altered the only overhead that
remains is the testing/handling of output data, which is a lesser problem
since the resulting event type is then known.

There are of course other overhead costs in the AES routines, but that is
beyond our control.


I'm feeling a bit confused at the moment about how best to achieve the abilities
we've been discussing for STiK.  On the one hand I'd like interrupt-based timing
for the precision and extremely low overhead costs, but on the other hand I'm
unsure about the legality (actually safety) of doing some things we might need
from within an interrupt routine.  But 'evnt_multi' timing is imprecise, and
has a lot of overhead when used with short timeouts.

Perhaps we should have both...!

An interrupt-based linked list scheduler allowing reliable periodical calls to
STiK modules that request it by a login to the scheduler with a callback address.
These callback functions should not use system functions directly but could set
flags telling other routines to do so later, and should direcly handle those
things that need good regularity.

An evnt_multi-based similar scheduler allowing the callback functions to use all
system functions.  This would then use a longer timeout to make the overhead
insignificant, and these callback functions could handle those AES jobs requested
by the flags (and other data) set by the interrupt-based routines.
These calls should of course pass a suitable AESPB pointer to the modules.

This dual scheme would be 100% legal and hopefully flexible enough for our needs.

A module wanting to use this dual feature would then log in to both schedulers
whereas some modules might only need the interrupts.  Note that by 'module'
I here mean any resident STiK-related code, and most especially STIKTSR itself.

Additionally a pointer to the 'global' array of the main APP should be made
available to all modules that need AES access when called in other ways.
Such modules should define their own AESPB and the other arrays, and simply
fill AESPB[0] with the 'global' ptr of the calling APP, or the main APP.

Thus such modules may also serve non-GEM programs with full AES capability,
even though they need not be GEM programs themselves.

Even temporarily resident programs can be allowed to log in to the schedulers,
provided they link a routine into 'etv_term' to guarantee logout in case of
errors causing that program to be aborted.

This means that any client can have three code segments running in parallel,
normal program code, the interrupt callback, and the event callback,
and all this with full compatibility in single-tasking TOS as well.

To allow login of TSR modules these schedulers could be written as a separate
TSR placed before all others (incl STIKTSR) in the \AUTO\ folder.

Naturally no callbacks will occur until it is switched on by the main APP
(dialer?) calling a reserved 'sched_init' function with the initial 'evnt_multi'
arguments.  These will then be moved into constant arrays for later repeated
use (without overhead) and the AESPB pointer values will be copied into a local
copy that will be passed to modules called by the event-based scheduler.

Thus the main APP can optionally use separate arrays in its own routines without
consulting anyone, by changing ptrs in its normal AESPB, and it will never have
to prepare input data for evnt_multi calls, which are replaced by a simple call
to a second reserved function 'sched_multi'.

The function interface could be implemented as a new cookie, or even included
with the entire scheduler code into STIKTSR (if Dan prefers this).

I think I may be on to something here, what do you think ?

-------------------------------------------------------------------------
Regards:  Ronald Andersson                     mailto:dlanor@oden.se
                                               http://www.oden.se/~dlanor
-------------------------------------------------------------------------
