Minerva, C68 and memory mysteries

Anything QL Software or Programming Related.
User avatar
tofro
QL Wafer Drive
Posts: 1795
Joined: Sun Feb 13, 2011 10:53 pm
Location: SW Germany

Minerva, C68 and memory mysteries

Postby tofro » Sat Dec 14, 2019 2:38 pm

Hi,

today I have somewhat of a tricky C problem:

Up to now, I never saw a need to use the _mneed variable (which, to my understanding, hints the runtimes to the initial need for heap memory of the program) in one of my (many) C68 programs. I normally leave it at its default of 8k. (I must admit I haven't written programs for BBQLs or Minerva a lot recently, so most of these programs run on SMSQ/E)

Now I have one program which is fairly large, about 70k pure program size, 32k of stack and data space, which is allocating (and holding) rather large amounts of memory via malloc at sizes of about 400-500k. It doesn't set _mneed.

That works perfectly fine on a SGC QL running SMSQ/E, no hickups whatsoever.

It doesn't work at all on the very same QL running Minerva only. Even with a newly started machine that has no other programs running, one of the first allocations (a 32k chunk to save screen memory in) is returning NULL, which is puzzling me. At this point in time, FREE_MEM returns nearly the full 4MB. No good reason for malloc to fail.

When I set _mneed, though, the allocations work fine, and the program works, even on Minerva. (I do, however, not really like to set _mneed: The memory requirements of the program are mainly determined by data files loaded at run time and thus not at all constant or deterministic)

Does anyone have a clue what the mysterious difference between SMSQ/e and Minnie is when executing that program? I would assume the C runtimes uses mt.alchp to fetch the memory in both systems and simply fetches more, when needed, up to the limit set by _memmax - But there must be subtile differences why Minerva wants to see the memory pre-allocated at startup. Even if Minnie would be so crazy and put the program to the middle of the TPA, there should be enough contiguous space to fulfill the request)

Tobias


ʎɐqǝ ɯoɹɟ ǝq oʇ ƃuᴉoƃ ʇou sᴉ pɹɐoqʎǝʞ ʇxǝu ʎɯ 'ɹɐǝp ɥO
User avatar
janbredenbeek
Gold Card
Posts: 355
Joined: Wed Jan 21, 2015 4:54 pm
Location: Hilversum, The Netherlands

Re: Minerva, C68 and memory mysteries

Postby janbredenbeek » Sun Dec 15, 2019 12:59 am

There is a huge difference between the memory management of Minerva and SMSQ/E.
SMSQ/E does not serve memory allocations from the Common Heap but from the TPA, which lies above 1MB in memory. Memory below 1MB is reserved for slave blocks.

This by itself does not explain why allocations by C68 fail on Minerva, we have to look at the actual code for this.

Jan.


User avatar
Peter
Aurora
Posts: 960
Joined: Sat Jan 22, 2011 8:47 am

Re: Minerva, C68 and memory mysteries

Postby Peter » Sun Dec 15, 2019 9:50 pm

What happens if you run the same program under Q68 Minerva?


User avatar
mk79
Gold Card
Posts: 459
Joined: Sun Feb 02, 2014 10:54 am

Re: Minerva, C68 and memory mysteries

Postby mk79 » Sun Dec 15, 2019 11:35 pm

tofro wrote:Now I have one program which is fairly large, about 70k pure program size, 32k of stack and data space, which is allocating (and holding) rather large amounts of memory via malloc at sizes of about 400-500k. It doesn't set _mneed.
I don't know where your problems originate either, but the way the C runtime sets up its heap and then even a new stack is a little bit weird:

Code: Select all

;========================================================================
;  Allocate Dynamic memory (including stack)
;  ~~~~~~~~~~~~~~~~~~~~~~~
;  It is time to set up our initial data area
;  The following code acts as follows:
;      a)  Allocate stack + data as a single area
;      b)  Release the area allocated
;      c)  Allocate the data area
;      d)  Allocate the stack area
;      e)  deallocate the data area
;  This apparently convoluted process is to get the stack to the
;  end of the initial data area.   This means that if the stack
;  outgrows its allocated space is will corrupt this programs data
;  area before other items on the common heap.   This MAY stop
;  QDOS from crashing if we are very lucky!
;-------------------------------------------------------------------------
It somehow feels like it could have something to do with it.

Apart from that malloc depends on the result of mt_free(), what does that return before the malloc fails?

Cheers, Marcel


User avatar
tofro
QL Wafer Drive
Posts: 1795
Joined: Sun Feb 13, 2011 10:53 pm
Location: SW Germany

Re: Minerva, C68 and memory mysteries

Postby tofro » Mon Dec 16, 2019 10:04 pm

Hmm. Embarassing.

Actually, the fault was in an entirely different place. Actually with pointers. And their environment (*).

My program had some lines like

Code: Select all

int pePresent = 0;

if (iop_pinf (channel, -1, &peVersion) > 0) {
   pePresent = 1
   // And more stuff that should only be done when the Pointer Interface is loaded
   ;
}


to check whether the PE is loaded. (iop_pinf is just a C wrapper around the PE trap with the same name, but with a catch as you'll see below)

Well, and that's just wrong. The documentation says

Get Window Manager vector and version information. On success returns the address of the Window Manager vector. On failure returns the QDOS/SMS error code (which is negative).


That should have hinted me (especially the second sentence) that this function actually returns a (void*), i.e. a pointer. Pointers are by definition always positive(**) - that function never detected a "no PE there" case, and code that afterwards was supposed to patch a channel definition block patched, well, something else. Didn't crash, though, but apparently messed up something in the memory management. Me setting _mneed simply moved memory around to higher places and caused the code to patch "another something else" that maybe wasn't that important.

The proper version of this line is actually

Code: Select all

int pePresent = 0;

if ((int)iop_pinf (channel, -1, &peVersion) > 0) {
   pePresent = 1
   // And more stuff that should only be done when the Pointer Interface is loaded
   ;
}


C is a bloody bastard...

(This is actually the first time that I would disagree with the makers of the C68 library that defining this function to return a pointer and/or a negative error was good design.)

Tobias

(*) hope you got that pun
(**) It's actually a bit more complicated than that. But in effect boils down to that statement.


ʎɐqǝ ɯoɹɟ ǝq oʇ ƃuᴉoƃ ʇou sᴉ pɹɐoqʎǝʞ ʇxǝu ʎɯ 'ɹɐǝp ɥO
User avatar
XorA
Super Gold Card
Posts: 608
Joined: Thu Jun 02, 2011 11:31 am
Location: Shotts, North Lanarkshire, Scotland, UK

Re: Minerva, C68 and memory mysteries

Postby XorA » Sun Dec 29, 2019 11:48 pm

(This is actually the first time that I would disagree with the makers of the C68 library that defining this function to return a pointer and/or a negative error was good design.)


That design pattern is all over linux and caused many security flaws over the years. There is now a PTR_ERR macro to prevent false tests.



Who is online

Users browsing this forum: No registered users and 5 guests