QL / PASCAL

[Chain-Q]

Wow, thanks for the quick uptake Obviously I'm completely foreign to the codebase and in that case I tend to change minimalistically, so if anything can be implemented better I'm all for it. I also very much noticed that my Pascal days have been a while

Sure, for a codebase I'm new to, I also tend to do the same approach. And as I wrote, the patch was very helpful anyway.

The assembler reader was the only major component which we reused and just patched up from the very early 1.0.x m68k Free Pascal, when we resurrected the m68k port for v3.0, and it sadly shows.

That explains it. I actually went "svn blame" to see when that got broken and which is the "newer" version, only to see that, within the SVN history, it has never worked in the first place.

Yes. The previous CVS history was not imported in 2005, when the project moved to SVN. This was intentional, due to some code licensing issue at the time, and all the code which was affected have been left behind.

Is there any mechanism of passing an option to the backend to select which style of output should be produced?

What do you mean by "style of output"? Assembler style? Not currently, on the backend, the m68k always produces GNU AS style assembler, even for vasm.

I didn't check the register order! Good point. I did see some funny "move.l d1,d1" in the resulting code when the register is already in the right place, though

Yes, the optimizer completely skips the assembler blocks, including the dead-move elimination. If you want to optimize those, I'd still leave the parameter-move code in place, commented out maybe, or a small comment in its place, that the parameter is already at the right register, or something. This way the next person who comes around will probably understand better what is going on.

I thought about the linker generating the header, too, but actually no QL linker does so either, so I'm fine with it being linked into the code as a normal object to stay generic.

OK, we'll keep it like it is currently then.

Having a smaller relocation section would be very nice, as it already takes some 2.5kb in the Hello World example.

Sure. The only drawback will be that the relocation code gets slightly more complicated and slower, as it will be a byte-stream with no longer guaranteed alignments. But it's not too bad.

However, if any new formats could be established I'd like to put one further possible optimization up for consideration: relocation is not a common concept in the QL world and as the QL does not have any "loader" in that sense, it cannot throw away the reloc info after relocation like other systems would. So they end up occupying memory without any further benefit. It would however be possible for .reloc and .bss to share the same address space in memory, with the loader clearing the info during relocation for .bss usage. Not sure how difficult that would be to implement on the linker side, but definitely worth thinking about for small systems I guess.

This was already like this, for the BASIC loader version, .bss overwrote the relocation table after it has been used. I guess to see if we can keep this in the exe version or not, depends on the if the dataspace is allocated right after the binary when loaded, forming a continuous address range, or not. If yes, this should be possible. When calculating the dataspace needed, we only have to sutract the relocations size from it, and reuse the reloc as part of the .bss area. If not, and the dataspace is allocated elsewhere, then it gets complicated...

[Chain-Q]

Marcel, I also integrated your qdos.inc and qdosfunc.inc patches now, with some minor changes. It's mostly about spaces/tabs and other whitespace changes, but I also added d3 save/restore to one of the functions where d3 was overwritten.

I also merged the tiny patch of yours to change the exe extension from bin to exe, where in the commit message I "successfully" mistyped your name as "Marcus", as I mentioned already, I'm working on the Amiga version of FPC with a fellow called Marcus Sackrow, and I wrote his name so many times, my brain just wrote Marcus anyway... I'm sorry!

[Chain-Q]

Few more things integrated:

the .map file generation during linking, this is again modified compared to the original patch, and instead of being always active, it's hooked up to some already existing compiler infrastructure. Activate with the -Xm command line switch. (This was actually also backported to other targets using vlink where it was also missing, so again, great.)

sysfile.inc and system.pp changes are integrated too.

Now the only big change remains is the migration away from the BASIC loader and move to executable. For a starter, is there more documentation about that QL header array you added to t_sinclairql.pas? I don't really like magic values. Also, see the questions I had earlier regarding the relation of .bss + dataspace vs. relocation overlap, I have a feeling that might not entirely be correct, but I can be wrong.

[mk79]

First of all, thanks for taking the time to look and improve my stuff, I know how it is.

Is there any mechanism of passing an option to the backend to select which style of output should be produced?

What do you mean by "style of output"? Assembler style? Not currently, on the backend, the m68k always produces GNU AS style assembler, even for vasm.

No, I mean the two methods of attaching the meta data, QemuLator header and XTcc trailer.

Yes, the optimizer completely skips the assembler blocks, including the dead-move elimination. If you want to optimize those, I'd still leave the parameter-move code in place, commented out maybe, or a small comment in its place, that the parameter is already at the right register, or something. This way the next person who comes around will probably understand better what is going on.

Yes, that's what I thought, too, but that's an optimization that can wait until the very end

However, if any new formats could be established I'd like to put one further possible optimization up for consideration: relocation is not a common concept in the QL world and as the QL does not have any "loader" in that sense, it cannot throw away the reloc info after relocation like other systems would. So they end up occupying memory without any further benefit. It would however be possible for .reloc and .bss to share the same address space in memory, with the loader clearing the info during relocation for .bss usage. Not sure how difficult that would be to implement on the linker side, but definitely worth thinking about for small systems I guess.

This was already like this, for the BASIC loader version, .bss overwrote the relocation table after it has been used.

Oooh, of course you're right. So that's fine already, but then the relocation code should clean up after itself to zero .bss, does it not (at least in C this is the rule)?

I guess to see if we can keep this in the exe version or not, depends on the if the dataspace is allocated right after the binary when loaded, forming a continuous address range, or not.

Well, for normal execution a single memory block is allocated for both binary and dataspace, so the area is continuous.
There is one special case, EXE files can also be loaded as a so-called "thing", which can be executed multiple times. In this case only the dataspace is allocated for each instance, the code is shared. But of course this only works when the code uses relative addressing to its data area, for relocated code this is not an option in any case, so it doesn't matter either way.

Marcel, I also integrated your qdos.inc and qdosfunc.inc patches now, with some minor changes. It's mostly about spaces/tabs and other whitespace changes

I started writing the code in a TMUXed SSH session on the Mac with VIM as the editor but later switched to VS Code on my PC in order to not go insane. Might have scrambled the tabs a bit. What is the official tabs policy?

where in the commit message I "successfully" mistyped your name as "Marcus"

No worries, in the English speaking world I'm called Marcus so often it almost doesn't register anymore

the .map file generation during linking, this is again modified compared to the original patch, and instead of being always active, it's hooked up to some already existing compiler infrastructure. Activate with the -Xm command line switch. (This was actually also backported to other targets using vlink where it was also missing, so again, great.)

That's great, thanks!

Now the only big change remains is the migration away from the BASIC loader and move to executable. For a starter, is there more documentation about that QL header array you added to t_sinclairql.pas? I don't really like magic values.

IIRC you have QemuLator installed? Have a look at the manual, appendix II. The QDOS hdr_ fields are listed in the SMS reference manual chapter 7 "DIrectory Device Drivers". The first field hdr_length is skipped in QemuLator headers as it's superfluous, it starts with offset 4 (hdr_access):

Code: Select all

OFFSET LENGTH(bytes) CONTENT
0      18            “]!QDOS File Header“
18     1             0 (reserved)
19     1             $f (total length_of_header, in 16 bit words)
20     1             0 hdr_access field (unused)
21     1             1 hdr_type field (1 = EXE file)
22     4             x hdr_data (data space)
26     4             0 hdr_extra (unused)
30     end of header

Cheers, Marcel

[Chain-Q]

No, I mean the two methods of attaching the meta data, QemuLator header and XTcc trailer.

There isn't one currently, but I'll figure something out. There are a few platform specific switches already, I'll probably add one for this one too.

This was already like this, for the BASIC loader version, .bss overwrote the relocation table after it has been used.

Oooh, of course you're right. So that's fine already, but then the relocation code should clean up after itself to zero .bss, does it not (at least in C this is the rule)?

That is correct. There was a FillChar() call in the original startup code by me for this, which zeroed out the .bss.

Well, for normal execution a single memory block is allocated for both binary and dataspace, so the area is continuous.

OK, that's great then.

There is one special case, EXE files can also be loaded as a so-called "thing", which can be executed multiple times. In this case only the dataspace is allocated for each instance, the code is shared. But of course this only works when the code uses relative addressing to its data area, for relocated code this is not an option in any case, so it doesn't matter either way.

There's a quick & dirty hack in the compiler for register-relative addressing of the .bss area, which I used in the PalmOS port, but it limited the .bss size to 32K, at least on a standard '000. You still needed relocations, but because no relocations pointed into the .bss are, you were good. It would be possible to use that, maybe even improve on it. Something for later maybe, I'd say.

What is the official tabs policy?

The official policy is no tabs, indent with spaces. (There's still tab indentation sneaking in here-and-there, I use to fix them from time to time.)

[Chain-Q]

No, I mean the two methods of attaching the meta data, QemuLator header and XTcc trailer.

There isn't one currently, but I'll figure something out. There are a few platform specific switches already, I'll probably add one for this one too.

I figured it out. There are two new platform specific arguments, as documented here. They're also shown in the compiler help text on the command line. Commit here. The integrated patch to write the headers (based on your work, but modified) is here. I wonder if it would make sense to add a "NONE" style as well, which would avoid writing any style. In case someone needs that.

So that's fine already, but then the relocation code should clean up after itself to zero .bss, does it not (at least in C this is the rule)?

That is correct. There was a FillChar() call in the original startup code by me for this, which zeroed out the .bss.

The new system init code is now integrated too, it's slightly modified compared to your patch, and includes zeroing out the .bss among others. I tested it in Q-emuLator, and exec_w "hello.exe" now works as expected. Cool!

So, back to you to complete the file I/O and other parts of the RTL, I guess?

BTW, just as a heads up, it looks like I'll be very busy in the coming 1-2 weeks, so my reaction/code integration time might be significantly longer the next time around. Just saying. But we'll get there.

[mk79]

That is correct. There was a FillChar() call in the original startup code by me for this, which zeroed out the .bss.

Ouch, sorry about that Back then it was still my understanding that the memory areas were disjunct...

There's a quick & dirty hack in the compiler for register-relative addressing of the .bss area, which I used in the PalmOS port, but it limited the .bss size to 32K, at least on a standard '000. You still needed relocations, but because no relocations pointed into the .bss are, you were good. It would be possible to use that, maybe even improve on it. Something for later maybe, I'd say.

Interesting, but yes, that would be very far down the line...

I figured it out. There are two new platform specific arguments, as documented here. They're also shown in the compiler help text on the command line. Commit here. The integrated patch to write the headers (based on your work, but modified) is here.

Cool, I like the way you did it.

I wonder if it would make sense to add a "NONE" style as well, which would avoid writing any style. In case someone needs that.

That is very cool I don't think NONE is needed as the 8-byte XTcc trailer doesn't harm in any way. Looking over my QL hard drive there are quite a few XTcc binaries that were cross-compiled at one time

So, back to you to complete the file I/O and other parts of the RTL, I guess?

There is certainly more to come, but I have to slow down as well. I was overworked even before this thing happened

Cheers! Marcel

[NormanDunbar]

Another QL Rabbit Hole! I disappeared down this one for a happy hour!

Following the instructions at https://wiki.freepascal.org/Sinclair_QL I built VASM, VLINK and Free Pascal for the QL. The instructions are excellent and everything was compiled, renamed as appropriate and installed.

I have found a couple of foibles that others here might come across, it seems Marcel didn't suffer these.


I've installed fpc into /home/norman/fpc-ql, so this is my /home/norman/fpc-ql/lib/fpc/etc/fpc.cfg file:

Code: Select all

#IFDEF CPUM68K
-Fu/home/norman/fpc-ql/lib/fpc/3.3.1/units/m68k-sinclairql
-Fu/home/norman/fpc-ql/lib/fpc/3.3.1/units/m68k-sinclairql/*
#IFDEF SINCLAIRQL
-FD/home/norman/bin
-XPm68k-sinclairql-
-WQxtcc
#ENDIF
#ENDIF

I've added the option -WQxtcc to get the XTcc trailer record for use with my XTcc utility (https://github.com/NormanDunbar/QLAssem ... ag/Issue_6)

I've had to expand manually, $fpcversion and $FPCTARGET in the original contents listed on the above URL, as I constantly get an error about not being able to find the units used by my test "Hello World" program:

Code: Select all

$ ppcross68k -Tsinclairql HelloWorld.pas 

Fatal: Can't find unit system used by test
Fatal: Compilation aborted

I'd love to add the -Tsinclairql option to the config file, but it seems to be ignored. Bummer.

I searched for details on the error messages and it seems to be a problem on Ubuntu based Linux systems. I'm on Linux Mint 19.3 which is Ubuntu based, which is Debian based. The fix is listed at https://stackoverflow.com/a/64637024/1020550 but that fix appears to have a bug in it as well. It advised creating a file fp.cfg in the source directory containing the two "-Fu..." lines from the main fpc.cfg file. It still won't work as the file needs to be called fpc.cfg and even when it is correctly named, it still doesn't expand the two variables to get the correct path to the units. I found that just expanding t he affected lines in the master config file and deleting the one in the source directory worked fine.

Hopefully, this will help someone.


Cheers,
Norm.

[mk79]

I have found a couple of foibles that others here might come across, it seems Marcel didn't suffer these.

Au contraire, I had exactly the same problems, but I thought I leave something for you to figure out, too So well done

Actually I had to tackle so many problems at first I just couldn't be bothered to investigate this further at that point and forgot about it later.

Cheers, Marcel

[Chain-Q]

I've installed fpc into /home/norman/fpc-ql, so this is my /home/norman/fpc-ql/lib/fpc/etc/fpc.cfg file:

Code: Select all

#IFDEF CPUM68K
-Fu/home/norman/fpc-ql/lib/fpc/3.3.1/units/m68k-sinclairql
-Fu/home/norman/fpc-ql/lib/fpc/3.3.1/units/m68k-sinclairql/*
#IFDEF SINCLAIRQL
-FD/home/norman/bin
-XPm68k-sinclairql-
-WQxtcc
#ENDIF
#ENDIF

You can actually extend environment variables inside fpc.cfg. So you can replace /home/norman with $HOME$ (note the trailing $) and it will work. Minor thing, but if you move your config files around in several systems, it might help.

I've had to expand manually, $fpcversion and $FPCTARGET in the original contents listed on the above URL, as I constantly get an error about not being able to find the units used by my test "Hello World" program:

Ooops... So yeah. The reason for this is because 8+3 filenames support is enabled for the Sinclair QL, because I based the system descriptor table inside the compiler on the Atari TOS version... And this leads to some inconsistencies if the compiler and the external build process prefers "m68k-sinclairql" or just "sinclairql" as the full name of the target. As a workaround, I changed this in the wiki now, to recommend $fpccpu-$fpcos combination, instead of $fpctarget. This works for now. (I actually tested it now... )

Having said this, what do you think? Should the compiler/build system stick to the shorter version of "sinclairql" or use "m68k-sinclairql" everywhere? The later would require switching the 8+3 short filenames support, but it doesn't really make a difference for now, because the native compiler is still quite far away (if it ever happens), but it would be good to make it consistent at least. I'd prefer the longer version, for consistency with other systems (where no 8+3 unless required), but I also don't have a really strong preference.

Another hint: Debugging config file and path issues is quite easy using the -vv compiler argument. This will write a file called "fpcdebug.txt" into the current directory, which contains human readable verbose information about the config file parsing, telling you what it expands those $foobar templates into, and if it can find the path or not. Among a lot of other internal information.

I'd love to add the -Tsinclairql option to the config file, but it seems to be ignored. Bummer.

You have to add inside the #ifdef CPUM68K section, not inside the #ifdef SINCLAIRQL section. Then it works. Since the #ifdef SINCLAIRQL section is only parsed, if the compiler is already in Sinclair QL target mode. Again using the -vv argument and reading fpcdebug.txt makes this quite obvious.

I searched for details on the error messages and it seems to be a problem on Ubuntu based Linux systems (...) It advised creating a file fp.cfg in the source directory

fp.cfg is the config file used by the FPC text mode IDE, but *NOT* by the command line compiler. That always uses fpc.cfg. Also, see above for the solution. It wasn't related to any Ubuntu Bug.

Great that you tried FPC. Thanks for testing and reporting back.