Bug 4130 – DMD crashes if it has to compile a project which is too complex

I am trying to compile a medium sized project of roughly 380 files (using phobos). As long as I keep the total code length (number of lines) under a limit (yet unknown) dmd creates the executeables sucessfully. However, if I add some functions to some classes or if I add some new files, it gives a segmentation fault after roughly 5 minutes of compiling which consumes 350 MiB of memory. If I run dmd with -v, the last lines it prints are: function equals function notEquals function greater function lesser function greaterOrEqual function lesserOrEqual function like function notIn function In function this function saveToDB function saveToDB function loadFromDBAsString Segmentation fault As far as I can see, the gcc for linking is not called yet. I have the same problem with older versions of dmd, however I cannot try dmd2.

Is it possible you could run it under gdb and see where it is dying?

(In reply to comment #1) > Is it possible you could run it under gdb and see where it is dying? I tried this but there are no debug symbols. I wounder if the memory address can help you (gdb 6.8): (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) Program received signal SIGSEGV, Segmentation fault. 0x081427b7 in ?? () (gdb) quit The program is running. Exit anyway? (y or n) [answered Y; input not from terminal]

Not really. Can you try recompiling dmd with debug symbols on?

I have managed to compile dmd :) Here is the output i got from gdb: Program received signal SIGSEGV, Segmentation fault. 0x08180c2d in obj_comdat (s=0x1e40c464) at backend/elfobj.c:1489 1489 SegData[s->Sseg]->SDsym = s; Current language: auto; currently c++ (gdb) quit The program is running. Exit anyway? (y or n) [answered Y; input not from terminal]

Hmm. Sseg is a short. I wonder if it's overflowing? (Each function is put into its own seg.)

In backend/cc.h, try changing it from a short to an int.

See changeset 458 with the short => int fix.

If the fix Walter has mentioned didn't work, try typing 'bt full' instead of quitting straight away, that will give us a backtrace and the contents of the variables at each stage. It might be a big output, it should help figure out what's going wrong, if it isn't what Walter suspected.

I can confirm it did not work. It took a long time too; this time. Unfortunately I did not use gdb so I am doing it again for some stack trace.

I got the same error again. I put "bt full" to my run script but I got no further output. I do not know much about gdb so I do not know how to extract more information. GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-pc-linux-gnu"... (gdb) Signal Stop Print Pass to program Description SIGUSR1 No No Yes User defined signal 1 (gdb) Signal Stop Print Pass to program Description SIGUSR2 No No Yes User defined signal 2 (gdb) Starting program: (very long line) Program received signal SIGSEGV, Segmentation fault. 0x08180c4d in obj_comdat (s=0x1e04c464) at backend/elfobj.c:1489 1489 SegData[s->Sseg]->SDsym = s; Current language: auto; currently c++ (gdb) quit The program is running. Exit anyway? (y or n) [answered Y; input not from terminal]

GDB isn't picking up the bt full, try invoking it with: gdb --args dmd $OPTIONS $FILES Where $OPTIONS and $FILES are the usual arguments you pass to dmd when compiling. When gdb loads, type 'r' to run dmd, and when it segfaults type 'bt full', followed by 'quit'. All of these are, of course, without the quotes.

Now a got some more information: (gdb) p s->Sseg $2 = -32768 You can see that p s->Sseg is negative. Either I made an error when I recompiled dmd or there is another place where short has to be changed to int. Just like in changeset 456 my line in cc.h is: #if TX86 int Sseg; // segment index The complete output: Program received signal SIGSEGV, Segmentation fault. 0x08180c4d in obj_comdat (s=0x1d141464) at backend/elfobj.c:1489 1489 SegData[s->Sseg]->SDsym = s; Current language: auto; currently c++ (gdb) bt full #0 0x08180c4d in obj_comdat (s=0x1d141464) at backend/elfobj.c:1489 prefix = 0x81b078f ".data." type = 1 flags = 3 #1 0x0812cd85 in outdata (s=0x1d141464) at backend/out.c:302 dt = (dt_t *) 0x0 datasize = 76 a = 3220481112 offset = 3220481064 flags = 0 p = 0x2 <Address 0x2 out of bounds> tls = 0 dtstart = (dt_t *) 0x17961d74 seg = 134522191 ty = 35 #2 0x08168dd4 in ClassDeclaration::toObjFile (this=0xbf61720, multiobj=0) at toobj.c:636 offset = 76 dt = (dt_t *) 0x17961d74 flags = 44 sinit = (Symbol *) 0x1d1c8884 scclass = 7 '\a' __PRETTY_FUNCTION__ = "virtual void ClassDeclaration::toObjFile(int)" cd = (class ClassDeclaration *) 0x0 bvtbl = {<Object> = {_vptr.Object = 0x81837a8}, dim = 0, allocdim = 0, data = 0x0} i = 0 name = 0x1d1d3230 "framework.ag.db.DFForeignColumn.DFForeignColumn!(AGGoodsCost,AGGoodsCost).DFForeignColumn" namelen = 89 #3 0x0815afff in TemplateInstance::toObjFile (this=0x9f5afd8, multiobj=0) at template.c:4274 s = (class Dsymbol *) 0xbf61720 i = 0 #4 0x08168311 in ClassDeclaration::toObjFile (this=0x9f598d0, multiobj=0) at toobj.c:241 member = (class Dsymbol *) 0x9f5afd8 offset = 487285552 dt = (dt_t *) 0xbff4a5b8 flags = 12 sinit = (Symbol *) 0x9 scclass = 6 '\006' __PRETTY_FUNCTION__ = "virtual void ClassDeclaration::toObjFile(int)" cd = (class ClassDeclaration *) 0x1 bvtbl = {<Object> = {_vptr.Object = 0x27}, dim = 144787360, allocdim = 0, data = 0x0} i = 102 name = 0x10 <Address 0x10 out of bounds> namelen = 166954312 #5 0x080f9087 in Module::genobjfile (this=0x8a11c00, multiobj=0) at glue.c:267 member = (class Dsymbol *) 0x9f598d0 i = 20 __PRETTY_FUNCTION__ = "void Module::genobjfile(int)" #6 0x081160dd in main (argc=385, argv=0x89d8120) at mars.c:1234 firstmodule = 0 aw = (AsyncRead *) 0x8a14988 anydocfiles = 0 library = (Library *) 0x0 files = {<Object> = {_vptr.Object = 0x81837a8}, dim = 371, allocdim = 382, data = 0x89d77d8} p = 0xbff50502 "DSocket.d" m = (class Module *) 0x8a11c00 argcstart = 383 status = 0 ---Type <return> to continue, or q <return> to quit--- setdebuglib = 0 modules = {<Object> = {_vptr.Object = 0x81837a8}, dim = 371, allocdim = 371, data = 0x89e0ab8} __PRETTY_FUNCTION__ = "int main(int, char**)" i = 350 libmodules = {<Object> = {_vptr.Object = 0x81837a8}, dim = 0, allocdim = 0, data = 0x0} (gdb) p s $1 = (Symbol *) 0x1d141464 (gdb) p s->Sseg $2 = -32768

I made a mistake while compiling dmd: I did not call "make -f linux.mak clean" so the old object files have been used. Now dmd crashes at a different position ("seg" is still negative): Program received signal SIGSEGV, Segmentation fault. 0x081824bf in reftoident (seg=-32768, offset=28, s=0x1def161c, val=0, flags=16) at backend/elfobj.c:2399 2399 int segtyp = MAP_SEG2TYP(seg); Current language: auto; currently c++ (gdb) bt all No symbol "all" in current context. (gdb) bt full #0 0x081824bf in reftoident (seg=-32768, offset=28, s=0x1def161c, val=0, flags=16) at backend/elfobj.c:2399 buf = (Outbuffer *) 0x2fd94d relinfo = 3214229704 refseg = 4075508 segtyp = 1 ty = 502252264 external = true #1 0x08095a29 in searchfixlist (s=0x1def161c) at backend/cod3.c:4232 lp = (fixlist **) 0x81c1f04 p = (fixlist *) 0x1defc7c8 #2 0x08180c5d in obj_comdat (s=0x1def161c) at backend/elfobj.c:1493 prefix = 0x81b070f ".data." type = 1 flags = 3 #3 0x0812cd6a in outdata (s=0x1def161c) at backend/out.c:302 dt = (dt_t *) 0x0 datasize = 92 a = 0 offset = 76 flags = 16 p = 0x2 <Address 0x2 out of bounds> tls = 0 dtstart = (dt_t *) 0x1901f9f8 seg = 32768 ty = 90 #4 0x08168f2d in ClassDeclaration::toObjFile (this=0xcc8a720, multiobj=0) at toobj.c:701 offset = 76 dt = (dt_t *) 0x1901f9f8 flags = 44 sinit = (Symbol *) 0x1def1884 scclass = 7 '\a' __PRETTY_FUNCTION__ = "virtual void ClassDeclaration::toObjFile(int)" cd = (class ClassDeclaration *) 0x0 bvtbl = {<Object> = {_vptr.Object = 0x8183728}, dim = 0, allocdim = 0, data = 0x0} i = 23 name = 0x1defc230 "framework.ag.db.DFForeignColumn.DFForeignColumn!(AGGoodsCost,AGGoodsCost).DFForeignColumn" namelen = 89 #5 0x0815afd3 in TemplateInstance::toObjFile (this=0xac83fd8, multiobj=0) at template.c:4274 s = (class Dsymbol *) 0xcc8a720 i = 0 #6 0x081682e5 in ClassDeclaration::toObjFile (this=0xac828d0, multiobj=0) at toobj.c:241 member = (class Dsymbol *) 0xac83fd8 offset = 501084976 dt = (dt_t *) 0xbf954358 flags = 12 sinit = (Symbol *) 0x9 scclass = 6 '\006' __PRETTY_FUNCTION__ = "virtual void ClassDeclaration::toObjFile(int)" cd = (class ClassDeclaration *) 0x1 bvtbl = {<Object> = {_vptr.Object = 0x27}, dim = 158586784, allocdim = 0, data = 0x0} i = 102 name = 0x10 <Address 0x10 out of bounds> namelen = 180753736 #7 0x080f9073 in Module::genobjfile (this=0x973ac00, multiobj=0) at glue.c:267 member = (class Dsymbol *) 0xac828d0 i = 20 __PRETTY_FUNCTION__ = "void Module::genobjfile(int)" ---Type <return> to continue, or q <return> to quit--- #8 0x081160c9 in main (argc=385, argv=0x9701120) at mars.c:1234 firstmodule = 0 aw = (AsyncRead *) 0x973d988 anydocfiles = 0 library = (Library *) 0x0 files = {<Object> = {_vptr.Object = 0x8183728}, dim = 371, allocdim = 382, data = 0x97007d8} p = 0xbf959502 "DSocket.d" m = (class Module *) 0x973ac00 argcstart = 383 status = 0 setdebuglib = 0 modules = {<Object> = {_vptr.Object = 0x8183728}, dim = 371, allocdim = 371, data = 0x9709ab8} __PRETTY_FUNCTION__ = "int main(int, char**)" i = 350 libmodules = {<Object> = {_vptr.Object = 0x8183728}, dim = 0, allocdim = 0, data = 0x0} (gdb) (gdb) whatis seg type = int (gdb) p seg $1 = -32768 (gdb) p s $2 = (Symbol *) 0x1def161c (gdb) p s->Sseg $3 = 32769 (gdb) whatis s->Sseg type = int (gdb) p p->Lseg No symbol "p" in current context.

After I have modified the file cod3.c changing struct fixlist { symbol *Lsymbol; // symbol we don't know about short Lseg; // where the fixup is going (CODE or DATA, never UDATA) ... }; to struct fixlist { symbol *Lsymbol; // symbol we don't know about int Lseg; // where the fixup is going (CODE or DATA, never UDATA) ... }; dmd is able to compile successfully. However, now gcc(ld) failes with /usr/bin/ld: warning: cannot find entry symbol _start; defaulting to 000000000804a8b0 Program received signal SIGSEGV, Segmentation fault. 0x00bff5f7 in ?? () from /usr/lib/libbfd-2.20.so (gdb) bt #0 0x00bff5f7 in ?? () from /usr/lib/libbfd-2.20.so #1 0x00c9039c in ?? () from /usr/lib/libbfd-2.20.so #2 0x089a7258 in ?? () #3 0x089a95f0 in ?? () #4 0xb5e57008 in ?? () #5 0x0000175c in ?? () #6 0x00000000 in ?? () So it seems there was a reason the data type was short and not int Any ideas?

Sorry I missed the Lseg, that should be int too. Another legacy of it being a 16 bit compiler. I don't know why it fails in ld, I'm trying out the test suite on Linux now.

We can't repro this. OK to close?

I can't repro this even with > 2^^16 comdat symbols.

I take that back, this crashes dmd (although it might be a different bug) void fun(size_t a)() {} string gen(size_t limit) { string s; foreach(i; 0..limit) { s ~= "fun!__LINE__();\n"; } return s; } void main() { mixin(gen(2^^16)); }

(In reply to yebblies from comment #18) > I take that back, this crashes dmd (although it might be a different bug) > Using 2^^15 it takes 8.75 GiB and ~15s to compile. 2^^16 causes swapping.

(In reply to safety0ff.bugz from comment #19) > Using 2^^15 it takes 8.75 GiB and ~15s to compile. > 2^^16 causes swapping. It still ices for me on 2^^16, linux64. I'm guessing it's an OOM that isn't handled properly.

I can't reproduce this. The OS kills the program when it starts to use too much memory, however, there is no ice.