To: stik@ON-Luebeck.DE
Subject: Re: [2] STIK: Work on STiK                                                          


Hi again Martin!

Seeing that you have adressed some questions below to me, I thought
I might as well add some comments to your dialog with Peter too.

On Tue, 24 Dec 1996 00:22:32 Martin Mitchell <mem@mode wrote: 
>
>On Fri, 20 Dec 1996, Peter Rottengatter wrote:
>
>> The conclusion on this is that we are really tight on CPU time here, as 
>> the IP core is executed within an interupt, we cannot allow *any* 
>> unneccessary overhead here, and I fear separating the IP from the kernel 
>> will produce quite a lot of overhead. Note that we need to care for 
>> people with a 8 MHz 68000, one meg RAM and TOS 1.0.

NB: The most critical thing here is not RAM or TOS version, but speed.


>Indeed. I have such a machine, a 1040STFM. However I believe your fears
>are at best unfounded, and at worst completely inaccurate. For example,
>you advocate a scheme using Pexec() for loading modules. This is indeed a
>simple system, which is its main strength.
>
>However consider the memory wasted by this method. If you want each
>program to be in standard format then 256 bytes is wasted per module for a
>basepage. Let's look at the real cost of this:
>
>Likely modules necessary for normal Internet usage on a 1040STFM:
>
>Modules       |    Estimated minimum overhead per module (bytes)
>Needed        |      Pexec() method    |   Custom (NVDI) method
>------------------------------------------------------------------
>              |                        |
>TCP.STX       |          256           |           0
>UDP.STX       |          256           |           0
>RESOLVE.STX   |          256           |           0
>SERIAL.STX    |          256           |           0
>------------------------------------------------------------------
>Total:        |         1024           |           0
>------------------------------------------------------------------

Actually this is exaggerated, since it is quite legal to use the
later half of the basepage for application data.  Thus the real
losses do not need to be larger than half of those given above.
512 bytes extra per four modules is not all that much really.


>I estimate the size of the kernel code needed to implement a custom system
>is no greater than 64 bytes larger than using Pexec().

This amount too I find quite tolerable.  (So, no big deal.)


>Thus using Pexec() wastes a minimum of 1Kb, which on a 1Mb TOS 1.00 system
>is better used for more folder entries to alleviate the 40 folder bug.
>Indeed I believe my estimates above are quite conservative, and I think
>the actual savings might be even larger. I have not considered the
>space needed for a Mshrink() call for each module, for example.

True, but I still say that as long as the RAM loss is on this order of
magnitude it is negligible, even on small systems.  Also, since half of
the basepage is reusable this more than balances the extra cost of Mshrink
and Pterm/Ptermres (which you missed ;-).  Actually a separate Mshrink is
unnecessary if a module uses Ptermres directly (or almost) on entry.


>Also, the time to load each module will be less with a custom system, as
>the overhead of creating a basepage will not be performed for each module.
>A program such as STiK should not take 10 seconds to initialise upon
>bootup.

The actual time taken is mainly dependent on the file and folder structure
leading to the 'STIK_DIR', and the number of files in that folder.
Assuming that all modules have the 'fastload' bit set there should not be
a very large difference between reading 'data' modules or using Pexec.
I've tried his demo code with a lot more modules than 4, and the load time
was quite tolerable.


>Peter, you say in another message that:
>
>> I like simple, efficient methods
>
>( I think you were saying it in a general sense, so I don't think I'm
>quoting out of context. )
>
>I believe the custom method is simple and much more efficient, so please
>reconsider what I'm proposing.
>
>Ronald, as someone familiar with assembly, don't you think a custom module
>system is quick and easy to implement?

Yes I do, and have myself argued similarly with Peter.  I do not really
think that this issue matters however, and using some non-program format
would have several drawbacks.  eg: We would have to do a lot more work on
the dev-kits to ensure that all compilers can produce the special format.
Also, we would lose the use of 'FastRAM' and MiNT 'protection' flags in
the program headers.


>> able to say it's modular now. I needs benefits on the horizon for doing 
>> this. Put in yet another way : It's unneccessary overkill.
>
>I don't think it's overkill, but a more useful design. It is only one of
>several reasons why I suggest that IP should be made a module. I refer you
>to my previous message for the other reasons.

Here I am definitely inclined to agree with Peter, for timing reasons,
but more on this further below.


>> > However from the high level view, looking at the design as a whole, and
>> > indeed at the C code, I think it is far more desirable to make IP a
>> > module. It improves the modular nature of the structure, designed as it is
>> > in a more 'object-oriented' way than the current STiK.
>>
----- snip ----- re: OOP vs other programming methods
>
>I quote the entire paragraph so you can see what I said. I merely said
>that your design is more object oriented than the current STiK.
>
>And what do I read? A lengthy diatribe completely irrelevant to my
>comment.
>
>What I did _not_ say was: let's write STiK in C++, I know the overheads
>involved.

I agree that he overreacted to your single reference to OOP, but please note
that although C coded modules are allowed, no part of the kernel itself will
be entrusted to the care of any C compiler.  This program is being written
completely in human-optimized assembler for maximum speed optimization.


----- snip ----- re: some more OOP stuff

>Also let me correct you by reminding you that design and implementation
>are two separate issues, whether in an object oriented context or not.

Then I can agree with you that the _design_ of the IP code can be quite
separate from the rest of the kernel.  But I see no reason for doing
that since the speed we want requires an integrated implementation.


>> > It will also simplify the coding of the kernel, which will have enough
>> > disparate code hanging off it as it is. (misc functions, internal memory
>> > allocators, modules handling, general init, vector bending etc).
>> 
>> This is true, but no reason at all to make it separate.
>
>That's right, but it's equally valid for me to say: 'Why should IP be
>included along with all the miscellaneous functions?'

Because they include an interrupt driven threader which may interact with
some IP routines quite a lot.  If IP was written as a separate C module,
that would require a much larger function call overhead, than when it is
done in optimized assembler integrated with the calling (and called)
routines.


>If you think that having it there will produce the speed and efficiency
>needed to service interrupts quickly, well you're very mistaken. That is
>an implementation detail, not a design issue.

True, it is an implementation detail, and successful implementation of the
speed we need requires integrated assembler implementation.  Given this I
see little point in merely designing it separately, though I agree that it
might be of value if we later want to build a similar  kernel for another
type of network.

If support of other network types becomes necessary it should still be
possible to use the source of STiK to build a new kernel for that network.
As you yourself have said, STiK was always intended to be a general purpose
TCP/IP stack, so why attempt to make it so much more general than that.

Using another network protocol than IP will AFAIK mean that _none_ of the
STX modules intended for IP can be used, so a whole new set of modules
will then have to be developed for this network.  This means that the part
of kernel + modules which is not network specific is a very low percentage.
So no relatively large loss of RAM, or speed, will occur by running the new
network through its own kernel, separate from STiK.

But making them separate from each other, each network-specific protocol
integrated with its own interrupt routines and module handler, will make
both networks faster.  Especially in the normal situation, when only one
network type will be installed.

For some cases specialization is a good thing, just as generalization is
for other cases.  It is necessary to use a well balanced compromise between
these two principles to achieve the best results. Both I and Peter believe
that separating IP from the STiK kernel means severe speed penalties, though
we also agree that the other stuff gains usefulness by its modularity.

-------------------------------------------------------------------------
Regards:  Ronald Andersson                     mailto:dlanor@oden.se
                                               http://www.oden.se/~dlanor
-------------------------------------------------------------------------
