After some test, the bug is likely to be that, an Array instantiated by make!(Array!T) WITHOUT ANY ARGUMENT, cannot be used as a value of AA.
following code fail to run:
alias Array!int A;
A[int] m;
m[0] = make!A;
Comment #2 by elvis.x.zhou — 2013-09-05T05:48:14Z
Further test shows that, Array just cann't be used as an value of AA in dmd2.063(on dpaste), while under git-head version of dmd, segfault only occured when an array is created by make!Array without any argument.
Comment #3 by andrej.mitrovich — 2013-09-05T19:34:24Z
I think this is a 64-bit only bug. Could you try compiling with -m32 to confirm?
Comment #4 by andrej.mitrovich — 2013-09-05T19:35:48Z
(In reply to comment #3)
> I think this is a 64-bit only bug. Could you try compiling with -m32 to
> confirm?
Ok nevermind, I was testing your http://d.puremagic.com/issues/show_bug.cgi?id=10970#c1 instead of the dpaste one (try not to use dpaste for short examples).
Your test-case:
-----
import std.stdio;
import std.container;
import std.process;
class A
{
this(string name)
{
m[name] = make!Arr;
}
alias Array!A Arr;
Arr[string] m;
}
int main()
{
A a = new A("test");
return 0;
}
-----
Crashes on win32 as well.
Comment #5 by elvis.x.zhou — 2013-09-05T20:14:47Z
(In reply to comment #4)
> (In reply to comment #3)
> > I think this is a 64-bit only bug. Could you try compiling with -m32 to
> > confirm?
>
> Ok nevermind, I was testing your
> http://d.puremagic.com/issues/show_bug.cgi?id=10970#c1 instead of the dpaste
> one (try not to use dpaste for short examples).
>
> Your test-case:
>
> -----
> import std.stdio;
> import std.container;
> import std.process;
>
> class A
> {
> this(string name)
> {
> m[name] = make!Arr;
> }
>
> alias Array!A Arr;
> Arr[string] m;
> }
>
> int main()
> {
> A a = new A("test");
> return 0;
> }
> -----
>
> Crashes on win32 as well.
Hope it helps and the issue be fixed asap, thank you!
Comment #6 by monarchdodra — 2013-09-06T02:10:45Z
(In reply to comment #5)
> Crashes on win32 as well.
If I recall correctly, I think the problem is "simply" that AA has *very* bad support for types with elaborate CC, and destruction. Take this "simple" use case:
//####
import std.stdio;
int count;
struct S
{
this(int){++count; writeln("construction");}
this(this){assert(0);} //never called
~this(){--count; writeln("destruction");}
}
void main()
{
S[int] aa;
writeln("----");
aa[5] = S(3);
writeln("----");
aa.remove(5);
writeln("----");
assert(aa.length == 0);
assert(count >= 0); //Fails here
}
//####
----
construction
destruction
destruction
----
----
//####
In just this little snippet, there are *all* kinds of wrong:
1. insertion: when inserting a new item, CC ("this(this)") is never called for some strange reason. Furthermore, destruction happens *twice* (again, for some strange reason).
2. removal: Where's the destructor?
These use cases are probably reported already, you are experiencing the results of these defects.
Long story short, at the end of the day, your array is "over-destroyed". Given that AA is a *strongly* relies on reference counting to provide deterministic behavior, things simply don't go well.
A lot of work is being done on fixing AAs (AFAIK), but don't expect it to be fixed any time soon.
As a workaround, you could try using slices instead of AA's?
//----
import std.stdio;
class A
{
this(string name)
{
m[name] = Arr.init;
}
alias A[] Arr;
Arr[string] m;
}
int main()
{
A a = new A("test");
return 0;
}
//----
In particular, "A" is a class, so it can't have a destructor (in the sense that a *reference* doesn't get destroyed). As such, there is minimal use to keeping an Array over a slice.
Comment #7 by andrej.mitrovich — 2013-09-06T07:18:43Z
Yeah it's very unfortunate the AA situation we have in D today, it's really giving D a bad rep.
Comment #8 by elvis.x.zhou — 2013-09-06T07:22:25Z
(In reply to comment #6)
> (In reply to comment #5)
> > Crashes on win32 as well.
>
> If I recall correctly, I think the problem is "simply" that AA has *very* bad
> support for types with elaborate CC, and destruction. Take this "simple" use
> case:
>
> //####
> import std.stdio;
>
> int count;
>
> struct S
> {
> this(int){++count; writeln("construction");}
> this(this){assert(0);} //never called
> ~this(){--count; writeln("destruction");}
> }
>
> void main()
> {
> S[int] aa;
> writeln("----");
> aa[5] = S(3);
> writeln("----");
> aa.remove(5);
> writeln("----");
> assert(aa.length == 0);
> assert(count >= 0); //Fails here
> }
> //####
> ----
> construction
> destruction
> destruction
> ----
> ----
> //####
>
> In just this little snippet, there are *all* kinds of wrong:
> 1. insertion: when inserting a new item, CC ("this(this)") is never called for
> some strange reason. Furthermore, destruction happens *twice* (again, for some
> strange reason).
> 2. removal: Where's the destructor?
>
> These use cases are probably reported already, you are experiencing the results
> of these defects.
>
> Long story short, at the end of the day, your array is "over-destroyed". Given
> that AA is a *strongly* relies on reference counting to provide deterministic
> behavior, things simply don't go well.
>
> A lot of work is being done on fixing AAs (AFAIK), but don't expect it to be
> fixed any time soon.
>
> As a workaround, you could try using slices instead of AA's?
>
> //----
> import std.stdio;
>
> class A
> {
> this(string name)
> {
> m[name] = Arr.init;
> }
>
> alias A[] Arr;
> Arr[string] m;
> }
>
> int main()
> {
> A a = new A("test");
> return 0;
> }
> //----
>
> In particular, "A" is a class, so it can't have a destructor (in the sense that
> a *reference* doesn't get destroyed). As such, there is minimal use to keeping
> an Array over a slice.
Thank you, I can work around it by changing the design, however, this really need to be fixed in the near future!
Comment #9 by elvis.x.zhou — 2013-09-06T07:29:53Z
(In reply to comment #7)
> Yeah it's very unfortunate the AA situation we have in D today, it's really
> giving D a bad rep.
Absolutely, after hanging around D community for days, I decide to start a private project in D, however, the first class I implemented broke by this issue!
FYI, I'm a game programmer and GC is an another trouble-maker for me, hope it can be improved in the near futher too.
Comment #10 by hsteoh — 2013-09-06T16:56:43Z
(In reply to comment #6)
[...]
> In just this little snippet, there are *all* kinds of wrong:
> 1. insertion: when inserting a new item, CC ("this(this)") is never called for
> some strange reason. Furthermore, destruction happens *twice* (again, for some
> strange reason).
> 2. removal: Where's the destructor?
[...]
Ugh. I looked at the code, and it's just one big ugly mess.
For starters, _aaGetX, which implements "aa[key] = value", takes only a *size* parameter to the value to be stored. If the Slot for that key doesn't already exist, it creates one for it, and then proceeds to memset the value part of the Slot to binary zero. So already, we have a problem: now there's a Slot for which the value isn't properly initialized (e.g., if the value type has a non-trivial ctor that sets things up).
Next, when you call remove(), it ultimately goes to _aaDelX, which calls GC.free on the Slot, which, according to the docs, explicitly does NOT finalize the block. So, the key and value originally in that slot, if they have dtors, won't have their dtors called.
To top things off, I added writeln's before and after inserting the element into the AA, and discovered that the ctor call and BOTH of the dtor calls happen *during insertion into the AA*. So it looks like after _aaGetX created an invalid instance of S (bypassing the ctor), the compiler is attempting to perform an assignment to it from a temporary copy of S it created from the RHS of the statement. Since the compiler believes (wrongly) that the AA Slot already contains a value of type S, it calls S's dtor in order to delete the old value, then copies the new value into it. Afterwards, it destructs the temporary copy of S (so this accounts for the 1 ctor call and 2 dtor calls). But here, the mess is topped off with yet another bug: the postblit isn't being called after this assignment!!
Ugh! This code is an embarrassment!! Makes me want to fork dmd, rip out every last bit of the existing AA implementation, and redo it from scratch. It'll probably be easier than trying to sort out this rabbit warren of AA bugs.
Comment #11 by andrej.mitrovich — 2013-09-06T17:03:34Z
Only 50? I have 78 on my list, and that's not including bugs that don't say AA or hash in the subject line, like this one.
Comment #13 by andrej.mitrovich — 2013-09-06T17:10:41Z
(In reply to comment #10)
> Ugh! This code is an embarrassment!! Makes me want to fork dmd, rip out every
> last bit of the existing AA implementation, and redo it from scratch. It'll
> probably be easier than trying to sort out this rabbit warren of AA bugs.
If you do that, we will build a statue in your honor in Menlo Park.
Comment #14 by andrej.mitrovich — 2013-09-06T17:11:24Z
(In reply to comment #12)
> Only 50? I have 78 on my list, and that's not including bugs that don't say AA
> or hash in the subject line, like this one.
My bugzilla query string must be inadequate, what's yours?