The garbage collector refuses to collect garbage. MacOS X 10.7.2, DMD v2.056
import core.memory;
void main(){
long[] arr;
foreach(i;0..100){
arr = new long[10_000_000];
GC.collect();
GC.minimize();
}
}
$ rdmd gcrun.d
core.exception.OutOfMemoryError
$ _
Comment #1 by dsimcha — 2011-12-11T09:47:56Z
D's GC isn't fully precise yet. In other words, it can't always tell the difference between a pointer and a non-pointer. When you allocate an 80-megabyte array, the probability that some non-pointer will look like it points into the array if interpreted as a pointer is pretty high.
This is a fundamental limitation of the current garbage collector design, rather than a bug. As a workaround, it's best to use C's malloc for very large arrays.
Comment #2 by soul8o8 — 2011-12-11T10:01:25Z
I understand. Thanks for the reply!
(With risk of polluting the bug tracker here, but perhaps to clarify the workaround: Could GC.malloc/free be used as well? Also, I assume if a large array of structs that in turn contain references to other heap objects, those would not get scanned had the array been C-alloced risking active objects to get collected?)
Comment #3 by dsimcha — 2011-12-11T10:12:27Z
(In reply to comment #2)
> I understand. Thanks for the reply!
>
> (With risk of polluting the bug tracker here, but perhaps to clarify the
> workaround: Could GC.malloc/free be used as well?
You could use GC.free(). It's somewhat dangerous, though, because you risk corrupting the GC if you free something incorrectly.
> Also, I assume if a large
> array of structs that in turn contain references to other heap objects, those
> would not get scanned had the array been C-alloced risking active objects to
> get collected?)
Use GC.addRange() to get around this.
Comment #4 by deadalnix — 2011-12-11T10:27:18Z
(In reply to comment #1)
> D's GC isn't fully precise yet. In other words, it can't always tell the
> difference between a pointer and a non-pointer. When you allocate an
> 80-megabyte array, the probability that some non-pointer will look like it
> points into the array if interpreted as a pointer is pretty high.
>
> This is a fundamental limitation of the current garbage collector design,
> rather than a bug. As a workaround, it's best to use C's malloc for very large
> arrays.
Isn't GC has a flag that says if something may contains pointer or not ?
Why this flag is set when allocation an array of long ?
Comment #5 by dsimcha — 2011-12-11T10:38:41Z
(In reply to comment #4)
> Isn't GC has a flag that says if something may contains pointer or not ?
>
> Why this flag is set when allocation an array of long ?
It's not. If a bit pattern in `arr` would point into GC memory if interpreted as a pointer, then the pattern is ignored. The GC has some level of precision in that it knows whether a heap-allocated block contains any pointers or not. However, the issue is that pointers **from** somewhere else point **at** arr, keeping it alive incorrectly.
There are two sources for false pointers:
1. The stack, where the GC has no knowledge whatsoever of what's a pointer and what isn't. I think this also applies to the static data segment.
2. Heap allocated data structures that contain both pointer and non-pointer data, e.g.:
struct Foo {
void* ptr;
size_t num;
}
If you allocate a Foo on the heap, the whole block will be marked as potentially containing pointers. The GC won't know what parts of it aren't pointers. `num` could have a bit pattern that would be a valid pointer and will keep some objects alive unnecessarily.