Development Systems and Created Code (reentrant, ROMable, ...)

[pjw]

I do not agree. An example is e.g. The Editor. It's a very good editor (the only one supporting column blocks afaik), but it has the disadvantage (e.g. compared to MasterSpy) that it can only edit one file, and it's not reentrant. If you want to edit more files, you must start a full copy for every file. As it's size is >100K this consumes a lot of memory. If it would be reentrant, you could HOT_CHP it and run many copies of it, with the code just loaded once.

Its a real pity that Turbo-compiled tasks are none-re-rentrant! I dont suppose anything will be done about that anymore.. Ive always liked The Editor and still use it occasionally, now mainly only for bulk operations on text files.

I have it on a HOT_LOAD key, and just now disproved the assumption made in my previous message: Each HOT_LOAD instance loads a new copy of the program. (Funny the need was felt to add, and document, the "I" (for impure) parameter for HOT_LOAD then, as it doesnt make any sense.)

[ql_freak]

I don't know or use the hotkey systems but pjw pointed out:

On a Hotkey which loads the code into memory, you will always have N + 1 copies of the code in memory,
where N is the number of copies of the code actually executing.

So doesn't this suggest even with Hotkey you are not saving memory?

This is wrong. If you HOT_CHP (HOT_RES) an executable program it uses the same code. From https://superbasic-manual.readthedocs.io/en/latest/H/index.html:

"The function HOT_CHP will load an executable file with the specified filename into the common heap and make it into an Executable Thing. Now, each time that you press <ALT> plus the specified key$, a new copy of the program will be started up in memory (although the same code is used, meaning that very little memory is used by each additional copy)."

I also don't use MasterSpy or SEDIT...are they written to open multiple internal buffers for different text files?

SEDIT not, but as it's reentrant code it can be HOT_CHPed WITHOUT the "I"(gitigit) option from HOT_CHP. Spy and the much improved version MasterSpy can load different files in multiple buffers, as can MicroEMACS, DEM (which also has a command language as MicroEMACS) AND the new version of QED (QED202). Thank You Jan, I must have a look at it. It was my standard editor for QLFED the MAUS-Tausch application for reading the MausNet (R) newsgroups, which had a German, an international QL and a international C68 group plus it imported (via a Gateway) the international QL group from Usenet. As far as I can remember, Jan hosted the gateway which linked the Maus groups to FidoNet and other mailbox systems.

[pjw]

BTW, where is "Thing" officially defined. I just looked through Dilwyn's SMSQ/E link and it refers to it but there doesn't seem to be a "a Thing is a blah, blah, blah" so one can get their bearings.

The Hotkey system depends largely on Things, but not vice versa. Youll find both the Thing and Hotkey documentation in the QDOS / SMS Reference Manual.

[janbredenbeek]

"The function HOT_CHP will load an executable file with the specified filename into the common heap and make it into an Executable Thing. Now, each time that you press <ALT> plus the specified key$, a new copy of the program will be started up in memory (although the same code is used, meaning that very little memory is used by each additional copy)."

I also don't use MasterSpy or SEDIT...are they written to open multiple internal buffers for different text files?

SEDIT not, but as it's reentrant code it can be HOT_CHPed WITHOUT the "I"(gitigit) option from HOT_CHP. Spy and the much improved version MasterSpy can load different files in multiple buffers, as can MicroEMACS, DEM (which also has a command language as MicroEMACS) AND the new version of QED (QED202). Thank You Jan, I must have a look at it. It was my standard editor for QLFED the MAUS-Tausch application for reading the MausNet (R) newsgroups, which had a German, an international QL and a international C68 group plus it imported (via a Gateway) the international QL group from Usenet. As far as I can remember, Jan hosted the gateway which linked the Maus groups to FidoNet and other mailbox systems.

Thanks, but QED has always been re-entrant from the beginning. So you could have HOT_RES it and invoke it multiple times with different files, saving just 8K of code space on each subsequent invocation. Only problem with v1.x was that it didn't show the name of the file being worked on .
AFAIK the only impure program I've written is Multimon, and that's being worked on (it's dirty written anyway, at least the exception code which takes over control from the current job, bypassing the scheduler!). And the QBOX configuration program which was written in C68, which is almost as big as QBOX itself...

(To be honest, with current emulators having plenty of memory and fast access to storage I don't see much advantage of HOT_RES over HOT_LOAD. In fact, I have replaced all HOT_RES items in my BOOT file by HOT_LOAD).

Jan

[ql_freak]

Thanks, but QED has always been re-entrant from the beginning.

I have not said in my original message, that QED is not reentrant. I know, that it has always being reentrant.

But there is one thing which could be improved: QED has as some other editors not the capability to mark character blocks, just line blocks (at least as far as I can remember). That's OK, and I know, it may be hard to implement.

But one thing should be easy to implement and would be very useful: A "character block" which must be inside of exactly one line. This would be very useful, if you e.g. want to insert a long string (e.g. "my_very_long_variable_name=") in a lot of lines.

Example:

Code: Select all

...
        my_very_long_variable_name="something";
        ^cbs                      ^cbe
...

And than you can insert/move this character block in any line with "cbi" (or cbm, ...) at the current cursor position.

Perhaps a message like "Character block start/end missing in current line", if cbs/cbe is entered in a line, where one of the 2 is missing (difficult to declare what I mean) maybe useful for the user.

[BSJR]

I have it on a HOT_LOAD key, and just now disproved the assumption made in my previous message: Each HOT_LOAD instance loads a new copy of the program. (Funny the need was felt to add, and document, the "I" (for impure) parameter for HOT_LOAD then, as it doesnt make any sense.)

Of course HOT_LOAD1 and HOT_RES1 will pick an already Loaded or Resident copy of the program.
It depends on the program type which HOT_ option is the best to start with. I often run many QDs but only need one Xchange.

BSJR

[pjw]

I have it on a HOT_LOAD key, and just now disproved the assumption made in my previous message: Each HOT_LOAD instance loads a new copy of the program. (Funny the need was felt to add, and document, the "I" (for impure) parameter for HOT_LOAD then, as it doesnt make any sense.)

Of course HOT_LOAD1 and HOT_RES1 will pick an already Loaded or Resident copy of the program.
It depends on the program type which HOT_ option is the best to start with. I often run many QDs but only need one Xchange.

I did study the manual, Bob, back in the day
My point is that HOT_LOAD loads and executes a new copy on each call. That means that HOT_LOAD does not care about whether a program is impure or not. Hence no need to specify the I option, AFAICS.

The following information may or may not be clear from the documentation. I have gathered
this through observation and testing, not by reviewing the code. Please correct anything
you know to be wrong:

Code: Select all

Command type          Self modifying   Copies in memory
HOT_RES/HOT_CHP  impure              N + 1
HOT_LOAD                pure/impure     N
HOT_RES/HOT_CHP  pure                 1

where N = number of instances running, 0..

The reason impure code needs an extra copy is because this is the reference copy. After that,
each instance needs its own copy as each instance's code may be slightly different (eg due to
relocation) or change due to local circumstances while running.

In the case of HOT_LOAD, if no intances are running then no code is loaded.

For pure code invoked by HOT_RES/HOT_CHP, one copy of the code is alway present, whether
any instances are running or not.

Code loaded with HOT_CHP can be removed again, and its Hotkey released, with HOT_REMV.

Code loaded with HOT_RES, even if it does not actually end up in the Resident Procedure Area
(due to jobs running, etc) cannot be removed with HOT_REMV; only the Hotkey is detached.
The Thing and code are still present. There is no command in Hot_rext or SMSQ/E to remove
such a Thing, an external toolkit is required.

The HOT_XXX1 option only means that a given hotkey will never invoke more than one instance
of the program it refers to. If an instance is already running, it is Picked and Woken, otherwise
a first (and only) instance is invoked. This behaviour is the same for all the different
invocation types.

[ql_freak]

The following information may or may not be clear from the documentation. I have gathered
this through observation and testing, not by reviewing the code. Please correct anything
you know to be wrong:

Code: Select all

Command type          Self modifying   Copies in memory
HOT_RES/HOT_CHP  impure              N + 1
HOT_LOAD                pure/impure     N
HOT_RES/HOT_CHP  pure                 1

where N = number of instances running, 0..

I think this is (at least mostly) correct.

But even code, which must be relocated when loading may be reentrant (but not ROMable of course). Albeit such programs are not supported by HOT_CHP, you must write a new code, which will create a new instance of this program (from second instance on it must not execute the relocator). The problem with reentrancy is, that the static data must be addressed via a base register. Than any new instance has another address for its static data in its base register. Of course if you don't change any of the static data, code should always be reentrant (unfortunately at least the C compilers have static data inside, which will be changed by the runtime system of the compiler libs, e.g. errno). The same is true for Turbo compiled programs. You have no chance to make a reentrant program with those development systems. In Assembler you can do it. E.g. use only DC and never (albeit possible, but bad style) change a value defined with DC, and of course do not use DS (which normally is used for uninitialised data, which makes no sense, if you must not change it, because you want to get a reentrant program). Note you may change DC and DS values, if they are always addressed via a base register, which is initialised when the program is started. You may then, as A6 in SuperBASIC, never use this base register in your code.

In short: A self modifying program (except relocation when first loading it) is not reentrant.
--
My dream: A 68xxx processor with 16 address registers ;-)

[tofro]

Peter,

it's not quite that simple. While it is true that most fully re-entrant programs use a base register for their data, there's some severe limits to that approach induced by limits of the (generic, the 68020+ models can do significantly more here) 68k adressing modes:

• Register-relative addressing can only reach +-32k from the base address
• PC-relative branches can also only reach +-32k from the jump origin

Without further action, that limits both your code and data size to 32kB (64kB, if you manage to offset the base register). There are ways around these limitations, but each comes at a cost. (Notwithstanding that the base registers can't be used for anything else in your program, which is sometimes a pain). For machines like the QL that can have nearly 1MB of memory, that is a severe limitation.

C68 uses absolute addressing for everything (both code and data), normally. During the build, it keeps track of these addresses and puts them in a relocation table which is run through at startup and all absolute addresses patched to (have the load address added) the proper runtime values. That relocation able is cleverly re-used for data space once it is no longer needed, so no memory is wasted.
That allows it to have unlimited code and data size, at the cost of non-pure programs, but it doesn't need to spend scarce registers or add a run-time overhead other than the initial relocation to programs. This is probably the most convenient approach for the programmer.

Prospero compilers use a different relocation mechanism: They use PC-relative addressing for intra-SEGMENT calls, and long absolute addressing for inter-SEGMENT calls (which limits the size of a Pascal code "Unit" to 32kBytes) - all of those long jumps are collected in a specific section (ATAB) that is relocation-patched at startup (this also means that Prospero Pascal doesn't produce "pure" programs.) During run-time, all jumps need to be re-directed through that jump table, which adds a bit of runtime overhead. Data is reached by always having A4 pointing to yet another relocation table, so the address of far-away data needs to be obtained by a double-indirect reference into the relocation table (yet another runtime overhead).

MCC (and, very probably its derivates EJC and PDQC) use reserved base address registers for both code and data, setting a limit of +-32k on both these segments, thus (in my opinion, severely) limiting code and data size for "pure" programs.

QLiberator has no limits in that respect - But it's not really a compiler ( ) - hit me for that).

As a side note: What you can see from the above, "back then", the Intel 8086 has always been blamed for its quirky segmented architecture. The highly-praised 68000 has similar quirks, just not that obvious (all of these limitations were lifted, however, in later CPUs)

Tobias

[Peter]

Peter said in the above thread: "PDQC can create ROMable code".

Well that is correct and (I'm afraid) also wrong. The C-Compiler of PDQC (Lattice as in EJC and Metacomco C Development Kit [MCDK]) can create reentrant and even ROMable code. Therefore you must use the compiler option "-b" in phase 1.

But the PDQC Development System does (I'm afraid) not create reentrant code, not even position independent code. The reason is, that the libraries (at least qlib_l) are NOT compiled with the -b option.

I did not use C libraries for the QL-SD driver code. The only use would have been formatted debug output, but for that I had some assembler routines aynway.

By this practical experience, which was based on QL-HD, I still think that the impact on speed and code size of partially using C for ROMs is not as bad as some assume.
It could fit into 16 KB ROM and the speed difference (using same kind of SDHC card accesses) was small. One has to look where the bottleneck really is.

Unless one is constantly training assembly language programming, C provides faster development, lower likelyhood of bugs and more portable code.
In this case, only the possibility of ROMable C code enabled the whole QL-SD hardware and QL-native FAT32 and SDHC card support!

Without PDQC, I would not have been able to test QL-SD and provide working hardware. It is also unlikely that the luxury of data interchange between emulators and QL/GC/Q68/Q40/Q60 by containers in FAT32, which we all now take for granted, would ever have appeared! I don't have the time to write non-trivial drivers purely in assembler, and poineering FAT32 and SDHC drivers without PDQC would just not have happend. All later driver developments re-used the (then already tested and debugged!) code, translated to assembler.