Hi Vassilis.

on Sun 03-10-1999 00:19 Vassilis Papathanassiou wrote:
>
>Seems the time has come to find out if some of my doubts about STinG
>are right or wrong :-) In either case I'd like to help in the STinG
>development anyway, I think we need more (or better) connectivity
>software than development of the stack itself.

Well, of course we need quality software on many levels, so as to
achieve a useful 'whole'.  The best stack in the world would still
be just so much garbage if no one made any clients and/or servers
for it.  And vice-versa of course.


>Since the above approach requires you to send me part of the STinG
>sources (which are wrongly not free IMO), I'd like to tell you what
>I think about it, and you'll tell me of course if you agree or not!

Ok.  As for the 'not free' part, that is Peters decision, and though
we may disagree on this, I do know more about his reasoning behind it.
Remember that this has always been freeware, and intended to remain so,
so at least there is no personal greed involved.  Instead it is mainly
that he has seen other projects being ruined by 'too many cooks', as
the saying goes...  He wants to avoid that happening to STinG.

That does not mean that STinG should be a one-man project of course.
But it does mean that it should be a 'one release point' project,
and that development and releases should be coordinated.


>Whatever I receive I'll try to optimize, debug or add new features
>that we have previously discuss. I'll never publish all or part of
>a module without your permission (in source or binary form).
>I'll always return the modified sources to you so that we'll be
>able to talk on real facts and not obscured results based on our
>systems differences. Thus, whatever changes will be kept by you
>(the administrator) and we should use these 'central' sources as
>a reference.

This all sounds ok to me, and in accordance with Peters intentions.


>Hmm, looks a bit official ;-)

Well, all agreements of this kind tend to be a bit formal, as that
is the best way to ensure that both parties are clear on what the
agreement really means.


>but since Peter and you want to keep STinG's development under control,

We do.  Not to gain anything for ourselves, but to gain something for the
users.  They will always know that the STinG release published is indeed
the 'valid' latest version as no other should exist.

Some other projects where collaborators had no common system of control
have been 'lost in limbo', because users never knew what was a valid
version and what was not.  Most large projects with 'open sources' have
had to institute some form of release control anyway, just to avoid the
chaotic situation which did develop before those controls existed.
(That goes for both Linux and MiNT, just to mention two.)

With STinG the control has a different form, and the sources are only
to be released to people who are to take part in the development.


>I think such an agreement is needed for any 'external' member.

Yes, as I too have mentioned before.  But you need not exaggerate that
feeling of being 'external'.  The extent to which you participate is
mostly up to yourself. I certainly will not stand in your way, though
I have to abide by the conditions Peter made in passing his sources.

The main conditions of his were:

I become responsible for the new development group.
Membership requires a serious attempt to contribute.
Only members of the group have access to STinG sources.
No member of the group will make separate STinG releases.
No matter what we change, STinG itself will remain freeware.

This may not be quite as 'open' as GNU stuff, but then again, it is a
lot more 'open' than the majority of software projects in existence.


>Please let me know if you have something else in mind.

No, what you said is sufficient in my opinion.


>One more thing I think you should know is that I've ported MintNet
>for MagiC (well, you know that, but now several other people know
>it too). This has advantages and disadvantages :-) On the plus side,
>we have something to compare and get ideas etc. On the minus, you
>can call me a traitor ;-) and exclude me from STinG development!

Let's not be silly  ;-)

But I certainly want to discuss the possibilities of porting some of
that code for use as an alternative STinG interface, possibly through
some new dedicated module.


>If you think there is such a problem (technology 'leaks' ;-) etc),
>feel free to say it.

I see no problem, as I don't think of you as the kind of person who
would make it a problem.  (Unfortunately there are such people too.)

If we share knowledge back and forth in ways that lead to improved
functioning of either one or both of these stacks, I see no harm in it.
Quite the contrary.  Whichever stack benefits from it, it will mean
more and/or better software available for our platform.


>I hope you understand now why it took me rather long to respond to
>your 'STinG sources here' (IIRC) mail, I thought it might be
>inappropriate to take part in it, being the developer of another
>stack! But then again why not ? :-)

Precisely.  I see no reason why you should not take part.


>Note also that maybe contrary to what you may believe, I also like
>the idea of a compatible stack down to singleTOS level, but not ignoring
>the new features the multitaskers might offer (or need!).

That is not contrary to what I believed at all  :-)

It is what I have thought all along, although I am also aware that you
have shown more interest in working on MagiC-specific things yourself.
But I do not take that as meaning that you dislike all else. It is just
a case of priorities, as I see it.  There is not time enough to do all
the things we might like, so we have to choose which ones we prefer.

It is the same way for me.  But whereas I too prefer to use MagiC rather
than singleTOS, I do have a special interest in compatibility issues, and
how to solve them.  This has caused me to seem concentrated mostly on the
use of STinG under singleTOS, merely because that is harder to do well,
so that it demands more of my attention.


>---- My ideas / 'real life' results on debugging ----
>
>Now, on the practical side. Debugging under MagiC is difficult and
>sometimes dangerous (eg never use writeback cache with these tests).
>But some times you can get better results under MagiC since you
>can leave the debugger in the background and go to desktop where
>you can work normaly until a breakpoint is hit.

True, but I do not like the way PD does some things, though I will
have to live with it until something better comes along.


>What I've done with the ethernet pocket driver for STinG is this:
>
>#include <mt_aes.h>	/* or <aes.h> :-) */
>
>void  main (int	argc, char  *argv[])
>{
>    bla bla (you know them already ;-)
>
>    /* just before Ptermres */
>
>    test_loop:
>  /* We can use either appl_yield or evnt_timer() or both */
>        appl_yield();
>    	evnt_timer (2000L);
>
>        goto test_loop;
>
>     /* Ptermres (_PgmSize, 0); */ /* temporarily out */
>   }
>}
>
>Simple eh? But now you can compile the module as a TTP, press ALT D,

I assume you mean the Alt-D command available in the standard PC
environment (PC.PRG).  I do not use that as I consider the editor
it has to be pure crap.  I use PC_SHELL 2.21 instead, as it allows
external editors to be used (for which I use Everest).


>fill the form with "STinG_Load", go to PD, set your breakpoints
>and click on R(un). Click to make the mouse visible, go to MagiC
>menu and click tidy up. You're working on your desktop now! When any
>of the breakpoints is hit you can investigate your code with all
>the labels, variables, stacks etc etc.

*IF* the multitasking survives such a breakpoint, which is dubious
in most kernel tests, and some module tests where problems develop
in interrupt-driven parts.


>One problem is that when any program terminates, PD also terminates.
>Thus some attention is needed, eg if you need a viewer, leave it running
>after use etc. When you think you've finished debuging, do a cold reset
>since both PC and PD will not understand SHUTDOWN messages and since
>many vectors are taken I haven't risk 'ignoring' them in shutdown.inf.
>You can try though :-)

Frequent rebooting is a normal part of system software development.
I don't see that as a problem.  It is the only way to make sure that
results of further tests are not affected by preceding ones.


>Well, this method should work with all the modules and debuging is
>as easy as reading your own sources and single-steping them at the
>same time!!

Not for me.  Many of my own sources are in DevPac form, and PD is
really only good for showing PASM/PC stuff properly. I have to settle
for seeing a raw disassembly of the code.  Not that this is any real
problem, as that is what I am used to with other debuggers too.


>Probably some connections will time out while you're
>viewing the code, take it as a good opportunity to check the
>time-out behaviour too! :-)

Yes, I know, but with careful breakpointing it is often possible
to avoid a timeout too when needed, simply by patching an argument
to TIMER_elapsed, or some data register used for comparisons.


>A similar (but of course more complicated) method could be used for
>the kernel also, or use the external program approach that I've
>described under 'BNET problems'. In the first case you should not
>single step the 200Hz vector installation (that's for sure ;-) but
>the rest of it might work (normally it _should_ work).

I often single-step the installation of XBRA vectors, not excluding
such for timer_C.  If it is done correctly there is no danger in it.

But the safest way to handle such things is to set a breakpoint some
place after the installing code, and let the program run untraced up
to that breakpoint. That is especially useful if several vectors are
to be linked, that interact with each other. Then the breakpoint used
should be placed in the code to executed after the last vector has
been patched, to minimize the risk of problems.


>For the second method the kernel can be run from PC's shell as usual,
>(with the timer C installation removed of course ;-) and the start
>of the handler (ie the pointer) moved to the STinG cookie (or another
>'test' cookie). Breakpoints set as usual.

The routine executed by timer C is highly specific to that particular
mode of invocation, and it will not perform in the same way when it
is invoked in any other way. Such tests will not yield valid results.
This limits the usefulness of this method for testing the kernel.


>After going to the desktop using the above method, you can run the
>external program, something like this:
>
>------------------------------- cut here ----------------------------
>#include <PORTAB.H>
>#include <tos_old.h>
>#include <VDI.H>
>#include <MT_AES.H>
>
>#include <string.h>
>#include <stdio.h>
>#include <stdlib.h>
>#include "sock_dv.h"
>
>SOCK_F	*sock_dv;
>ULONG	sock_val;
>short	app_id;
>
>void	sldm (void);
>
>static long	get_cookiejar (void)
>{
>	return (* ((LONG *) P_COOKIES) );
>}
>
>ULONG	*get_cookie (ULONG  cookie, ULONG  *value)
>{
>	register ULONG *cookiejar = (ULONG *) Supexec (get_cookiejar);
>
>	if (cookiejar)
>	{
>		while (*cookiejar)
>		{
>			if (*cookiejar==cookie)
>			{
>				if (value)
>					*value = *++cookiejar;
>				return (cookiejar);
>			}
>			cookiejar += 2;
>		}
>	}
>	return (NULL);
>}
>
>void	evnt_mess (SOCK_F *sock)
>{
>	EVNT	events;
>	short	end = 0;
>
>	do
>	{
>		EVNT_multi( MU_MESAG + MU_TIMER, 258, 3, 0, 0L, 0L, 20L, &events );
>
>		if (events.mwhich & MU_TIMER)
>		{
>
>			Supexec (*sock->tim_alarm);
>			sldm ();
>		}
>		if ( events.mwhich & MU_MESAG)
>		{
>			if (*events.msg == AP_TERM)
>			{
>				end = TRUE;
>			}
>		}
>	} while (!end);
>}
>
>short	serdv_open (char *dev)
>{
>	return ( Fopen (dev, FO_RW) );
>}
>
>short	serdv_close (short han)
>{
>	return ( Fclose (han) );
>}
>
>short	main (void)
>{
>
>	if ( (app_id = appl_init() ) == -1 )
>		return (app_id);
>	if (! get_cookie ('SCKM', &sock_val))
>	{
>		Cconws (" MAGX-NeT device driver NOT installed!\r\n");
>		Cconws ("Press any key...");
>		Cnecin ();
>		return (-1);
>	}
>	if (sock_val)
>		sock_dv = (SOCK_F *)sock_val;
>	sock_dv->ser_open = serdv_open;
>	sock_dv->ser_close= serdv_close;
>
>	Supexec (*sock_dv->inst_vect);
>	evnt_mess (sock_dv);
>	appl_exit ();
>	return (0);
>}
>------------------------------- cut here ----------------------------
>
>I'm using it to run MintNet's "timer alarms" (a rather good method for
>taking care timings AND the relevant buffers for both the stack itself
>and user programs).

This is closely related to a topic that was long discussed in the older
discussion list, namely what to do about the need we felt for extended
GEM event types.  So that a network APP could simply use an evnt_multi
call to await completion of some network event.  The conclusion then
was that the only way available was to use message events, with some
special message types.

What we never did solve satisfactorily was how we could make GEM send
such messages without having to do it from a parallel APP, so that we
would effectively have real network events merged with the system. I
have some hope that my current work may lead to such a method, although
it is still too early to be sure of that.  I know that I can do it in
some system versions of course, but I want a generic method for all
existing GEM versions.

One drawback of implementing any event-like work as part of an APP is
that the distribution of timeslots for short timer events is not regular.
That is one of the main differencies between getting timeslots this way
and getting them from timer C interrupts. The difference between regular
and irregular timeslot use can be crucial for efficiency, as Peter and I
found when developing the current STinG methods (it was a joint effort).


>The second call is for the infamous 'serial line
>daemon' sld (sldm for MagiC ;-) Even using this approach TCP transfers
>using the loopback driver run at 220 KB/sec with no optimisations!

I am not familiar with this stuff, but I assume you refer to some 'daemon'
used to transfer data blocks between the network layer and the low-level
drivers.  Doing this from an APP might seem risky, but as long as polling
is done often enough to keep the buffers flowing it does not have to cause
lowered transfer rates.  But it will cause response delays proportional
to the intervals between timeslots of that APP of course, and those delays
will be subject to irregularity dependent on current multitasking load.

Some such effect is unavoidable with any timesharing method, so that is
not particular to any one implementation. STinG too has a similar effect
such as you noticed a while ago in your RTT observations, but for STinG
the timeslots available are very regular, as they are not dependent on
the multitasking used for APPs.


>Note that this is the exact source as used by me, you need to modify it
>for STinG (ie remove the sldm() call and replace the "sock->tim_alarm"
>with STinG's handler) or I can do it if you like.

No, that is no problem.  The real problem is that there are some parts
of STinG that this method can't be applied to.  It should work well
for various modules, but hardly for the kernel itself.  There simply
is no substitute for the real timer C routine.  At least I can't see
any way of calling the same code from a normal APP function, and still
make it do all that it is supposed to do.

It may still be possible to make some tests of the kernel with this method,
but for those tests a special shell will have to be made allowing the timer
routines to be called in other ways, and then no time critical results can
be taken seriously.


>Using this method, PD will stop inside the STinG kernel and not at
>the above program of course ;-) thus kernel's labels, stacks, variables
>etc can be examined as usual.

Perhaps, although I am a bit doubtful about PD's ability to correctly
interpret stack usage when an API is called from alien APPs, as the
stack used is external to the program under test.  I'm no expert on
PD usage however, as my old-time favourite debugger was AMON.


>Well, I'll stop here and wait for your reply. As you can see the above
>methods can provide _very_ useful results, at least for the external
>modules that I've tested them already (including BNET.XFS). I'd never
>been able to debug the ethernet driver without using such methods.

I naturally do see the value in being able to test things at 'desktop'
level this way.


>And since STinG has been improved/changed here and there for a long
>time, I thing we can 'sweep' it without endless tests with clients and
>servers. After the test period, when you think everything is OK we can
>return to the current handler (and at least for SingleTOS I see no
>reason why it should change).

I'm not sure what you mean here, so let me make one thing clear.
Under no circumstance whatever will the main API in a released version
of STinG be implemented as part of an APP.  For test versions, ok, as
anything goes when it comes to finding bugs, but for release versions
it will always be a TSR, and the same one for all TOS.

Having several different versions of STinG is one of the things I know
that Peter wanted to avoid, so I do not even have the right to make a
decision to change that (IMHO), even if I wanted to.


>In case you have other things to do, I can take care the debuging
>of any module under 'normal' ATARIs or MagiC PC (very useful for
>testing 'strange behaviour').

It is not just that I have other things to do, but also that I have
suffered some equipment loss this summer.  I have no monitor at all
that a normal ST/STe can work with right now, and don't really have
time to repair any either.  In any case it is good that you can do
this testing instead, as well as MagiC PC tests, as I fear that the
active participation in the beta list has fallen too low for us to
rely on those reports.


>Note also that most PCs are using a
>16550 UART and thus HSMODEM for them is different. For the moment
>serial.stx is unable to use the modem2 device driver and many PC
>internal modems. I'd like to see this problem solved, as it is
>now, one has to select either dialing with MagiC PC or using the
>local network.

Peter's intention was to make SERIAL.STX able to handle any driver
that is implemented as a proper HSModem driver.  The question here
is whether the failure is due to some error/oversight in those
drivers, or in SERIAL.STX, which I have not yet done any work on.


>Hmm, you need the header file for the above program. Normally you
>only need this SOCK_F struct modified for STinG.
>
>------------------------------- SOCK_DV.H ----------------------------
>
>#define P_COOKIES	0x5a0
>
>#ifndef	TRUE
>	#define FALSE	0
>	#define	TRUE	1
>#endif
>
>typedef  struct sock_f
>{
>	long	(*inst_vect)(void);
>	long	(*tim_alarm)(void);
>	long	*kernel;
>	short	(*ser_open) (char *dev);
>	short	(*ser_close)(short han);
>	long	(*ser_send) (void *inf);
>	void	*allslbufs;
>} SOCK_F;
>
>/* extern	SOCK_F	*xdd_ptr; */
>
>typedef struct
>{
>	long	cookie_id;
>	long	cookie_value;
>} COOKIE;
>
>#define	vec_inst ()		(*sock_f->inst_vect)()
>#define alarms_r ()		(*sock_f->tim_alarm)()
>------------------------------- cut here ----------------------------
>
Hmmm...  I'm not sure if I missed something somewhere.

As we agree that I will not use your example program as-is, but only
the general method it illustrates, I don't see why I would need the
header file.  But it doesn't hurt any to have it of course...  :-)


Btw:
As you may have noted above I have been sparing in the use of 'snips',
quoting most of your statements intact.  I believe this is best for
'first-level' quotes on important subjects.  It helps the other party
remember the precise context of the reply, without which it may lose
part of its meaning.  For second level quotes (if present at all), I
will use much more brutal snipping of course.


Now for the subject of sources:
-------------------------------
You and I both use fairly similar 'flavour' in our C coding, using hard
tabs and strict vertical alignment of symbols significant to structure.

Peter on the other hand does not.  He uses 'soft spacing' and places most
leading structure symbols at the end of the preceding predicate (if any),
such as exemplified by these 4 lines:
    if (some_condition) {
        do_some_work_1();
        do_some_work_2();
      }
Note also that his placement of the closing structure symbol is not fully
consistent, so that it sometimes is aligned with the predicate, while it
may also be indented one or more spaces extra in some cases.

For this same code I believe that you would have used these 5 lines:
	if (some_condition)
	{
		do_some_work_1();
		do_some_work_2();
	}

And I myself would use the following 4 lines for the same code:
	if	(some_condition)
	{	do_some_work_1();
		do_some_work_2();
	}
The only exception to this is that I sometimes do use exactly the same
form that you use, as they are identical in alignment, and my form is
most suitable for code already debugged, so that the top line 'inside'
the structure is not subject to further change.


I am not going to argue about taste here, but a simple fact is that you
and I both use fairly similar methods in these things, making direct use
of Peter's sources rather inconvenient, and prone to cause mistakes.

Another thing I have always seen as a drawback of most C 'flavours' is
that no significant distinction is made to attract the eye to border
areas of function definitions and structure declarations etc.  For this
I myself systematically use full-width horisontal 'dash' lines, which
I think helps a lot in reading any source code (regardless of language).

I am gradually reediting Peter's code, to conform to my normal methods,
but that is slow work, as the sources are quite extensive.  Naturally
I can send you sources in the 'raw' form as written by Peter, but I do
not think you will find them easy to work with.  I know I didn't.

Sources for TCP.STX and UDP.STX are almost completely converted now,
and the kernel source conversion is begun, but still far from complete.
Conversion of the many other STinG components has not yet started...
(Mainly because I want to do some real programming too, not just edit.)


I know from earlier mail that you have been wanting a peek at STinG
insides for a long time, but that Peter never sent you any, and I see
no reason why I should prolong this wait of yours any further.

So for a start I will send you an archive of the latest TCP beta,
as I remember that this was one component whose sources you did want.

I will also include my lab beta binaries of the kernel and UDP module,
both changed so as to include the UDP_info function, together with a
new TRANSPRT.H file with some prototypes and some other stuff for this
function.  This will allow you to experiment with use of the new DEFER
mode both for UDP and for TCP without futher delay.

The archive should arrive in the same mail batch as this mail.

-- 
-------------------------------------------------------------------------
Regards:  Ronald Andersson                  mailto:dlanor@ettnet.se
http://dlanor.atari.org/    ICQ:38857203    http://www.ettnet.se/~dlanor/
-------------------------------------------------------------------------
