Extending the operating system/Basic in compilied programs

Anything QL Software or Programming Related.
Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Extending the operating system/Basic in compilied programs

Post by Martin_Head »

Does anyone know if it is safe to add device drivers and SuperBASIC proc/functions while within a compiled program?

Would the SuperBASIC procs/functions just be added to the compiled program, or be available to SuperBASIC itself?

Or is it likely to crash the system?


User avatar
tofro
Font of All Knowledge
Posts: 2685
Joined: Sun Feb 13, 2011 10:53 pm
Location: SW Germany

Re: Extending the operating system/Basic in compilied programs

Post by tofro »

Martin,

if you do it wrongly, it's likely to crash the system ;)

SMS.LIOD and SMS.LFSD (functions to link in drivers) are QDOS functions and should work without an SBASIC context (that is an assumption, I have no proof for that, but also no idea what in dd initialisation would need a working SBASIC job)

SB.INIPR is an SBASIC vector (see the prefix!) and needs a working (and accessible) SBASIC environment, namely access to the name table and other internal data structures unique to SBASIC jobs. I would strongly assume it can only be called from an SBASIC context, even if the vector documentation does not specify a6 as an input register. As you cannot specify an (or rather: which) SBASIC job you refer to in the call I'd assume it finds the name table using a6 which wouldn't point to a sensible place in a compiled job.

With regards to device driver linking, some caveats:
  1. Make sure the memory you load into is actually owned by job 0, the only job that cannot be removed in a QL. ALCHP from TK2, for example, will make the current job the owner of the memory, and as soon as this job ends, the memory will be released and your QL will crash. Other memory allocation commands like for example ALLOCATION from Turbo TK will allow you to specify job 0
  2. Such allocations as above will definitly create undesirable memory fragmentation - not much of a problem on QPC2 with 128MB of memory, but watch out on smaller systems
  3. Loading and linking in device drivers that do not contain SBASIC extensions shouldn't be a problem as long as you observe rule #1
  4. SBASIC jobs other than job 0 have a private name table. Anything you LRESPR in there will only be available to child jobs of that SBASIC job. The extensions will no longer be accessible as soon as the SBASIC daughter job is gone. I have tried in a short example to load an SBASIC extension in a TURBOed job started from job 0, expecting it to fail and it did - Extensions weren't loaded. It didn't crash, though, so I would simply assume it couldn't find a proper SBASIC environment and silently failed.
  5. The QLiberator manual "Using Basic extensions" explicitely states "it [initialising Basic extensions] doesn't work because QLiberator jobs have a non-expandable name table"
Last edited by tofro on Wed Sep 20, 2017 10:42 am, edited 1 time in total.


ʎɐqǝ ɯoɹɟ ǝq oʇ ƃuᴉoƃ ʇou sᴉ pɹɐoqʎǝʞ ʇxǝu ʎɯ 'ɹɐǝp ɥO
Martin_Head
Aurora
Posts: 847
Joined: Tue Dec 17, 2013 1:17 pm

Re: Extending the operating system/Basic in compilied programs

Post by Martin_Head »

Thanks for the response, I was asking because someone has reported to me that they are trying to start my IP Network driver in a compiled program, and it is crashing the system.

I don't know exactly what they are doing, or which compiler they are using . But my initial thought was that it was a risky thing to do for most of the reasons you give.


User avatar
Giorgio Garabello
Gold Card
Posts: 277
Joined: Tue Jun 30, 2015 8:39 am
Location: Turin, Italy
Contact:

Re: Extending the operating system/Basic in compilied programs

Post by Giorgio Garabello »

tofro wrote:Martin,


[*]The QLiberator manual "Using Basic extensions" explicitely states "it [initialising Basic extensions] doesn't work because QLiberator jobs have a non-expandable name table"[/list]
With QLiberator for an extension to be recognized it must have been loaded so that it can copy new keywords into its table-name (if I'm not mistaken) then in theory the tab name should already be expanded.


User avatar
tofro
Font of All Knowledge
Posts: 2685
Joined: Sun Feb 13, 2011 10:53 pm
Location: SW Germany

Re: Extending the operating system/Basic in compilied programs

Post by tofro »

Giorgio Garabello wrote:
tofro wrote:Martin,


[*]The QLiberator manual "Using Basic extensions" explicitely states "it [initialising Basic extensions] doesn't work because QLiberator jobs have a non-expandable name table"[/list]
With QLiberator for an extension to be recognized it must have been loaded so that it can copy new keywords into its table-name (if I'm not mistaken) then in theory the tab name should already be expanded.
Giorgio,

You are correct with that, but I don't think that was the gist of the question - Martin was asking on whether it's possible to initialize a basic extension from a compiled program so that it is globally known - not just to the compiled program itsself.

Tobias


ʎɐqǝ ɯoɹɟ ǝq oʇ ƃuᴉoƃ ʇou sᴉ pɹɐoqʎǝʞ ʇxǝu ʎɯ 'ɹɐǝp ɥO
User avatar
tofro
Font of All Knowledge
Posts: 2685
Joined: Sun Feb 13, 2011 10:53 pm
Location: SW Germany

Re: Extending the operating system/Basic in compilied programs

Post by tofro »

tofro wrote:
Giorgio Garabello wrote:
tofro wrote:Martin,


[*]The QLiberator manual "Using Basic extensions" explicitely states "it [initialising Basic extensions] doesn't work because QLiberator jobs have a non-expandable name table"[/list]
With QLiberator for an extension to be recognized it must have been loaded so that it can copy new keywords into its table-name (if I'm not mistaken) then in theory the tab name should already be expanded.
Giorgio,

You are correct with that, but I don't think that was the gist of the question - Martin was asking on whether it's possible to initialize a basic extension from a compiled program so that it is globally known - not just to the compiled program itsself. And that is what the manual says, is not possible.

Tobias


ʎɐqǝ ɯoɹɟ ǝq oʇ ƃuᴉoƃ ʇou sᴉ pɹɐoqʎǝʞ ʇxǝu ʎɯ 'ɹɐǝp ɥO
User avatar
Giorgio Garabello
Gold Card
Posts: 277
Joined: Tue Jun 30, 2015 8:39 am
Location: Turin, Italy
Contact:

Re: Extending the operating system/Basic in compilied programs

Post by Giorgio Garabello »

tofro wrote:
tofro wrote:
Giorgio Garabello wrote:
With QLiberator for an extension to be recognized it must have been loaded so that it can copy new keywords into its table-name (if I'm not mistaken) then in theory the tab name should already be expanded.
Giorgio,

You are correct with that, but I don't think that was the gist of the question - Martin was asking on whether it's possible to initialize a basic extension from a compiled program so that it is globally known - not just to the compiled program itsself. And that is what the manual says, is not possible.

Tobias
I'm also talking globally
What do you mean by the term "initialization"?
Do you refer to the upload using the LRESPR command or something else?

Gio


User avatar
tofro
Font of All Knowledge
Posts: 2685
Joined: Sun Feb 13, 2011 10:53 pm
Location: SW Germany

Re: Extending the operating system/Basic in compilied programs

Post by tofro »

Giorgio,

I refer to the "linking the command into BASIC" (the BP.INIT system call).

My understanding of Martin's question was "Can you have an LRESPR (or LBYTES + CALL) of a basic extension in a compiled program and expect that extensions be known to all other basics afterwards?" And the answer is "no, you cannot".

A detailed answer might add "no, you cannot at all, the extension wouldn't even be known to the compiled program itself, but it would rather crash"

If you include basic extensions into liberated programs using $$asmb, those commands will only be known to the program itself.

Tobias


ʎɐqǝ ɯoɹɟ ǝq oʇ ƃuᴉoƃ ʇou sᴉ pɹɐoqʎǝʞ ʇxǝu ʎɯ 'ɹɐǝp ɥO
User avatar
Giorgio Garabello
Gold Card
Posts: 277
Joined: Tue Jun 30, 2015 8:39 am
Location: Turin, Italy
Contact:

Re: Extending the operating system/Basic in compilied programs

Post by Giorgio Garabello »

tofro wrote:Giorgio,

I refer to the "linking the command into BASIC" (the BP.INIT system call).

My understanding of Martin's question was "Can you have an LRESPR (or LBYTES + CALL) of a basic extension in a compiled program and expect that extensions be known to all other basics afterwards?" And the answer is "no, you cannot".

A detailed answer might add "no, you cannot at all, the extension wouldn't even be known to the compiled program itself, but it would rather crash"

If you include basic extensions into liberated programs using $$asmb, those commands will only be known to the program itself.

Tobias
No, the question is different, I know because I created the problem and involved Martin.

The situation is as follows
Martin has written an extension that works in the following way:
- is loaded via the LRESPR command
- As soon as it is loaded it makes available a single command: NET_START
- When this command is executed it makes available new commands.

What I did:
- I created a job that runs the net_start command (with QLiberator)
- before uploading it, I uploaded the extension and activated the NET_START command to "translate" all keyborad
- I turn off the emulator
- reboot the emulator
- loaded the extension with the LRESPR command
- I did my job to run the NET_START command but the job crashes

We are trying to understand the reason.

Gio


User avatar
tofro
Font of All Knowledge
Posts: 2685
Joined: Sun Feb 13, 2011 10:53 pm
Location: SW Germany

Re: Extending the operating system/Basic in compilied programs

Post by tofro »

OK. Understood.

I guess the culprit is here:
Giorgio Garabello wrote: - When this command is executed it makes available new commands.
That clearly indicates the NET_START command calls BP.INIT to link in "more procedures".

This call extends the S*Basic nametable with the basic commands you hand in. QLiberator is a sort of "interpreting compiler" that has it's own private name table - If you call BP.INIT from compiled code's context, the system will inadvertently try to access the "private" name table in your liberated job and try to expand it. As this table is of static size in a QLiberated job, this extending will overwrite vital parts of your job and crash your program.

The QLiberator manual says this:
The BOOT program is separate from the main program so that all the new procedure names are recognised before the main program is loaded. Such boot programs CANNOT BE COMPILED by Q_Liberator for the following reasons:
a) The standard function RESPR gives an error if any jobs are running and so has been modified (see below).
b) Each file of extensions contains a small piece of code to link the new names into SuperBASIC's name table which is designed to grow as necessary. The Q_Liberator name table is of a fixed size, determined during compilation.
c) LRUN is an illegal procedure as far as Q_Liberator is concerned,
(see chapter 8).
Clearly, case (b) applies here, although it is not done when the loaded code is CALLed for the first time, but rather when NET_START is executed. The fact that your program was compiled with a copy of the (global) name table that already was expanded at compile time (because you had already executed NET_START before) doesn't really change much - BP.INIT does not check whether a command it tries to link in is already present - It will still try to expand the name table.

You should definitely only call such BASIC procedures from interpreted basic (TK2_EXT is another famous candidate for the same problem).

Tobias
Last edited by tofro on Fri Sep 22, 2017 2:05 pm, edited 1 time in total.


ʎɐqǝ ɯoɹɟ ǝq oʇ ƃuᴉoƃ ʇou sᴉ pɹɐoqʎǝʞ ʇxǝu ʎɯ 'ɹɐǝp ɥO
Post Reply