Bug 649 – concatenation hangs in threaded program

The following thread program hangs upon calling format() from a thread. The call to sleep() is to make sure that 't' is deleted before the thread exits. import std.thread; import std.string; extern(C) uint sleep(uint secs); class Test { Thread thr; int printStats() { sleep(1); format(1); // Program hangs at this point return 0; } this() { thr = new Thread(&printStats); thr.start(); } ~this() { thr.wait(); } } void main() { Test t = new Test(); delete t; } The following things will prevent the bug from occuring: - remove the call to format() - make the thread finish before 't' is deleted - remove the call to thr.wait() from the destructor, or call it from somewhere else Nick

After some digging it turns out that the problem is actually in string concatination. So a simpler example is to exchange the call format(1); with char[] r; r ~= "a"; Nick

Nick wrote: > After some digging it turns out that the problem is actually in string > concatination. So a simpler example is to exchange the call > > format(1); > > with > > char[] r; > r ~= "a"; What's happening is this: The compiler runtime relies on gc_free to finalize t, which in turn waits on the child thread. At the same time, the child thread attempts to allocate memory for the string concatenation and is forced to wait for the gc_free call to finish. This is a classic deadlock situation. I think the best fix for this would be for the compiler runtime code (gc.d: _d_delclass) to explicitly call the object's finalizer and for gc_free to simply free memory. So the GC would only be responsible for finalizing objects whose lifetime ends during a collection, not for those destroyed as the result of a delete operation. The consequence of this would be that a call to gc_free for an arbitrary block of memory will not call the finalizer for that block, even if one is known to exist, but this seems a clear separation of responsibilities IMO. If the user really wants the finalizer called he can cast the pointer to Object and delete it. A possible compromise would be for _d_delclass to explicitly finalize the object and then set a flag indicating that gc_free should not finalize the block, and for gc_free to finalize so long as the 'finalize' flag is still set. This may be a bit slower depending on how it's implemented, but it would allow gc_free to finalize tagged blocks when appropriate. But again, I think it is probably more appropriate for the GC to only finalize on collections, and assume that if gc_free is called at other times, then the user does not intend for finalization to occur. Sean

Sean Kelly wrote: > Nick wrote: >> After some digging it turns out that the problem is actually in string >> concatination. So a simpler example is to exchange the call >> >> format(1); >> >> with >> >> char[] r; >> r ~= "a"; > > What's happening is this: > > The compiler runtime relies on gc_free to finalize t, which in turn > waits on the child thread. At the same time, the child thread attempts > to allocate memory for the string concatenation and is forced to wait > for the gc_free call to finish. This is a classic deadlock situation. > > I think the best fix for this would be for the compiler runtime code > (gc.d: _d_delclass) to explicitly call the object's finalizer and for > gc_free to simply free memory. So the GC would only be responsible for > finalizing objects whose lifetime ends during a collection, not for > those destroyed as the result of a delete operation. The consequence of > this would be that a call to gc_free for an arbitrary block of memory > will not call the finalizer for that block, even if one is known to > exist, but this seems a clear separation of responsibilities IMO. If > the user really wants the finalizer called he can cast the pointer to > Object and delete it. > > A possible compromise would be for _d_delclass to explicitly finalize > the object and then set a flag indicating that gc_free should not > finalize the block, and for gc_free to finalize so long as the > 'finalize' flag is still set. This may be a bit slower depending on how > it's implemented, but it would allow gc_free to finalize tagged blocks > when appropriate. But again, I think it is probably more appropriate > for the GC to only finalize on collections, and assume that if gc_free > is called at other times, then the user does not intend for finalization > to occur. I don't think I like the idea of the behavior of gc_free depending on whether or not it's called during GC. What would happen if the 'delete t' in the program was in a destructor, called by the GC? I think it'd be the same thing as in the original code: deadlock. (note: the below assumes each GC memory block has a pointer to its finalizer. I'm not sure if that's how it's implemented, but that seems to me to be the logical implementation of finalizers) Wouldn't this be more safely fixed by having explicit deletion first explicitly call the finalizer and then either: - null out the finalizer pointer and tell the GC to free the memory. Since the finalizer pointer is null, no finalizer is run. Presumably the GC checks for this anyway since e.g. a char[] doesn't need a finalizer. Or: - tell the GC to free the memory, but pass an (optional?) parameter to ignore the finalizer. The main advantage of the second method is encapsulation (_d_delclass wouldn't need to know how to find the finalizer pointer).

Tried the code on DMD 1.033 and it doesn't hang. Has this bug been resolved?

Still hangs on my end, DMD 1.036 on Ubuntu Linux. Possibly OS-dependent?

The program exits normally on Arch Linux (DMD 2.067.0-b1) I have updated the test case: import core.thread; import std.string; extern(C) uint sleep(uint secs); class Test { Thread thr; void printStats() { sleep(1); char[] r; r ~= "a"; } this() { thr = new Thread(&printStats); thr.start(); } ~this() { thr.join(); } } void main() { Test t = new Test(); destroy(t); }