Bug 18900 – GC doesn't collect large arrays on 32-bit Windows

I was loading into memory an processing a bunch of large (~130 Mb) files sequentially and got an out of memory error after about 10. I reduced it to this test code: ``` import std.stdio; import core.memory: GC; enum Mb = 1024*1024; void main(string[] args) { foreach(i; 0..10) { auto data = new ubyte[12*Mb]; writefln("GC used: %d Mb / %d Mb", GC.stats.usedSize/Mb, (GC.stats.freeSize+GC.stats.usedSize)/Mb); } writeln("Collecting"); GC.collect(); writefln("GC used: %d Mb / %d Mb", GC.stats.usedSize/Mb, (GC.stats.freeSize+GC.stats.usedSize)/Mb); } ``` When compiling on Windows with `dmd -m32` it results in: ``` GC used: 100 Mb / 150 Mb GC used: 200 Mb / 300 Mb GC used: 300 Mb / 450 Mb GC used: 400 Mb / 600 Mb GC used: 500 Mb / 750 Mb GC used: 600 Mb / 900 Mb GC used: 700 Mb / 1050 Mb GC used: 800 Mb / 1200 Mb GC used: 900 Mb / 1350 Mb GC used: 1000 Mb / 1500 Mb Collecting GC used: 1000 Mb / 1500 Mb ``` With -m64 it collects just fine and doesn't go over 200 Mb. When the array is 10 Mb, it also collects just fine on 32-bit leaving 0 Mb used in the end, but with 11 Mb arrays it ends at 22 Mb used and with 12 Mb arrays it ends at 132 Mb used. Looking around if someone else had this problem, I found this thread: https://forum.dlang.org/post/[email protected] Mentioning "If your programm is a 32-bit application the GC most likely leaks memory because it thinks its still referenced.". I don't know exactly why 32-bit applications complicates checking whether something is referenced, but if this can be fixed that would be nice.

The reason people talk about 32-bit applications likely leaking memory is because D's current GC sometimes can't distinguish an integer from a pointer (so-called false pointers). The GC will scan all global variables, every thread stack, and every allocation (probably only every allocation of a type that might include pointers, but it's been a while since I looked into it). It's possible that there is no 12MB section of memory that does not have a false pointer to it, but that's not likely.

*** This issue has been marked as a duplicate of issue 15723 ***