Bug 2927 – Ignore Interior GC attribute

Status
RESOLVED
Resolution
FIXED
Severity
enhancement
Priority
P2
Component
dmd
Product
D
Version
D2
Platform
x86
OS
Windows
Creation time
2009-05-03T11:19:00Z
Last change time
2015-06-09T01:26:25Z
Keywords
patch
Assigned to
bugzilla
Creator
dsimcha

Attachments

IDFilenameSummaryContent-TypeSize
344interior.patchAdds interior pointer ignoring feature to gcx.dtext/plain1104
345coreMemory.patchMakes core.memory interface with modified gcx.d.text/plain286
347interior.patchFix a few things I missed in the first patch.text/plain1308

Comments

Comment #0 by dsimcha — 2009-05-03T11:19:23Z
A very easy, but often very effective, way to deal with false pointer issues in the GC, would be to include an attribute in GC.BlkAttr that allows for interior pointers to a block of memory to be ignored and only pointers to the root to be able to keep the object alive. An example use case is when a class or struct object stores a large array that never escapes the scope of the object. The array will therefore always have a pointer to its root if it is referenced. Counting only pointers to the root as references to the object would drastically lessen the probability that large arrays are kept around due to false pointers.
Comment #1 by dsimcha — 2009-05-03T12:48:21Z
Created attachment 344 Adds interior pointer ignoring feature to gcx.d
Comment #2 by dsimcha — 2009-05-03T12:49:11Z
Created attachment 345 Makes core.memory interface with modified gcx.d.
Comment #3 by dsimcha — 2009-05-03T12:50:41Z
I've submitted patches that implement this feature. If the BlkAttr.NO_INTERIOR flag is set, then only pointers to the base address of a GC-allocated block can keep that block alive. Here's a small test program demonstrating how this works: import std.stdio, core.memory; void main() { // Test for < PAGESIZE test(32, cast(GC.BlkAttr) 0, 0); test(32, cast(GC.BlkAttr) 0, 1); test(32, GC.BlkAttr.NO_INTERIOR, 0); test(32, GC.BlkAttr.NO_INTERIOR, 1); // Test for > PAGESIZE enum megabyte = 1024 * 1024; test(megabyte, cast(GC.BlkAttr) 0, 0); test(megabyte, cast(GC.BlkAttr) 0, 1); test(megabyte, GC.BlkAttr.NO_INTERIOR, 0); test(megabyte, GC.BlkAttr.NO_INTERIOR, 1); } // In this test case, we expect the GC to allocate the same address more than // once only when NO_INTERIOR is set and we store an interior pointer in ptrs // instead of a pointer to the root address. void test(size_t size, GC.BlkAttr attr, uint toAdd) { uint[] oldPtrs = new uint[10]; void*[10] ptrs; foreach(i; 0..10) { auto ptr = GC.malloc(size, attr); ptrs[i] = ptr + toAdd; foreach(oldPtr; oldPtrs) { if(oldPtr == cast(uint) ptr) { writeln("GC re-allocated address ", oldPtr, ". (Parameters: ", size, " ", attr, " ", toAdd, ")"); return; } } oldPtrs[i] = cast(uint) ptr; GC.collect(); } }
Comment #4 by dsimcha — 2009-05-03T16:57:22Z
Created attachment 347 Fix a few things I missed in the first patch.
Comment #5 by bugzilla — 2009-11-02T23:44:29Z
The problem is, I don't see how an interior pointer could be prevented. You can always take a slice to the interior array, so there is an interior pointer. I think it's a big risk to assume or tell the user not to slice the array, because they will, and then will suffer from mysterious memory errors.
Comment #6 by dsimcha — 2009-11-03T05:51:22Z
I know this isn't completely safe, but neither is any alternative, such as using the C heap, using the GC but deleting stuff manually, or having your program leak memory like crazy. In some cases, it may be the least bad solution.
Comment #7 by leandro.lucarella — 2009-11-03T06:57:17Z
(In reply to comment #5) > The problem is, I don't see how an interior pointer could be prevented. You can > always take a slice to the interior array, so there is an interior pointer. I > think it's a big risk to assume or tell the user not to slice the array, > because they will, and then will suffer from mysterious memory errors. But this shouldn't be set on any array, only on some private data where you can ensure that no interior pointers exist. Example: ---- class A { private void* losts_of_pointers; this() { lots_of_pointers = new void*[100_000_000]; } } It would be nice to be able to tell the GC "trust me, nobody will ever store an interior pointer to this".
Comment #8 by dsimcha — 2009-11-03T07:06:24Z
Yes, and the killer use case would be associative arrays, which leak memory like a seive. They are currently (and probably quite often) implemented as something like: struct Node { Node* left; Node* right; hash_t hash; K key; V value; } struct AA { Node*[] table; // Can be huge, will always have a pointer to head as // long as the AA is alive, fully owned by the AA // instance. Perfect candidate for NO_INTERIOR. }
Comment #9 by dsimcha — 2010-11-04T07:19:16Z
Now that we have the concept of sealed containers, I think it's time to reconsider this patch. If necessary I'll redo it so it works with the current GC. One of the advantages of sealed containers is that, in some cases, they could safely use ignore interior on their internal data structures.
Comment #10 by dsimcha — 2011-08-27T07:25:57Z