RE: [MLton] Crash fread(...) failed (only read 0) (Cannot al…

Top Page
Attachments:
Message as email
+ (text/plain)
+ card-cross-maps.patch (text/plain)
+ (text/plain)
Delete this message
Reply to this message
Author: Matthew Fluet
Date:  
To: Matthew Fluet
CC: mlton@mlton.org
Subject: RE: [MLton] Crash fread(...) failed (only read 0) (Cannot allocate memory) during deepFlatten with MLton 20070826
On Tue, 22 Apr 2008, Matthew Fluet wrote:
> On Tue, 22 Apr 2008, Nicolas Bertolotti wrote:
>>  There was actually another bug in heap.c. Here is an additional patch to
>>  fix it:
>>  --- mlton/runtime/gc/heap.c   2008-04-11 19:12:58.000000000 +0200
>>  +++ mltonp1/runtime/gc/heap.c 2008-04-22 16:24:50.000000000 +0200
>>  @@ -337,6 +337,7 @@
>>        }
>>        GC_diskBack_read (data, curHeapp->start, size);
>>        GC_diskBack_close (data);
>>  +      curHeapp->oldGenSize = size;
>>      } else {
>>        GC_diskBack_close (data);
>>        if (s->controls.messages)

>
> Yes, I was just coming to that exact conclusion.
>
>>  Unfortunately, now, I get the following error:
>>             deepFlatten starting
>>  Out of memory.  Unable to allocate 1,686,393,408 bytes.

>>
>> ... which, I guess, is a different problem.
>
> It is a different problem, in the sense that you succeeded (at least once) in
> writing the heap to disk, allocating a larger heap, and reading the heap from
> disk. Getting an out of memory error suggests that even after writing the
> heap to disk and releasing it, the operating system was unable to mmap 1.6G.
> That seems strange, even on a 32bit system, where the virtual address space
> is 4G, though I suppose it could happen with sufficient fragmentation.
>
> If you compile with @MLton gc-messages --, you should get a more detailed
> memory listing of what memory is reserved, which might indicate whether there
> is some extra memory being held.


One thing I notice is that when writing the heap to disk, we do not
release the card/cross maps. Although with a 1.6G heap, these are only
12.8M, if they happen to fall in the middle of the virtual address space,
then it may not be possible to find the requested 1.6G heap.

You could try the attached patch, which releases the card/cross maps after
writing the heap to disk.