Bug 18026 – Stack overflow in ddmd/dtemplate.d:6241, TemplateInstance::needsCodegen()

Arch/Manjaro 64-bit, dmd v2.077.0 and ldc 1.6.0 (2.076.1) from official repositories. Tested on two machines with similar setups. Could not reproduce in Windows (32-bit), and I didn't seem to have the libraries for 32-bit linux builds so could not test that. I've run into this on more than once now. In cases that are difficult to isolate and reason about, dmd and ldc will both fail to compile with dub -b plain and dub -b release, exiting with code -11. -b debug and -b unittest work fine. Things like in what order files are specified when compiling matters. I've even managed to get it to a point where it builds sometimes (!) and other times errors out. > zorael@xps ~/src/kameloso [overflow] $ dub build -b plain --force > Performing "plain" build using dmd for x86_64. > kameloso ~overflow: building configuration "application"... > Linking... > > zorael@xps ~/src/kameloso [overflow] $ dub build -b plain --force > Performing "plain" build using dmd for x86_64. > kameloso ~overflow: building configuration "application"... > dmd failed with exit code -11. In https://forum.dlang.org/post/[email protected] I managed to find one line to change that fixed one issue (replaced size_t with uint). I now ran into a different case where it breaks if I remove a particular function call line in a function. I wanted to move it to a different module, so what I end up having to do is renaming the original function and keeping it with just that one critical line in it. > void removeMeWhenPossible() > { > import kameloso.debugging : formatEventAssertBlock; > import std.array : Appender; > > Appender!(char[]) sink; > sink.formatEventAssertBlock(IRCEvent.init); // <-- stack overflow without this line > assert(0); > } As mentioned on the forum page with the issue there, gdb on a debug dmd showed a stack overflow in TemplateInstance::needsCodegen(). > 6233| // Determine necessity of tinst before tnext. > 6234| if (tinst && tinst.needsCodegen()) > 6235| { > 6236| minst = tinst.minst; // cache result > 6237| assert(minst); > 6238| assert(minst.isRoot() || minst.rootImports()); > 6239| return true; > 6240| } > 6241|----------> if (tnext && (tnext.needsCodegen() || tnext.minst)) > 6242| { > 6243| minst = tnext.minst; // cache result > 6244| assert(minst); > 6245| return minst.isRoot() || minst.rootImports(); 6246 The ldc output seems to agree; > /usr/lib/libLLVM-5.0.so(_ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamE+0x2b)[0x7fd8b08329bb] > /usr/lib/libLLVM-5.0.so(_ZN4llvm3sys17RunSignalHandlersEv+0x56)[0x7fd8b0830806] > /usr/lib/libLLVM-5.0.so(+0x808953)[0x7fd8b0830953] > /usr/lib/libpthread.so.0(+0x11da0)[0x7fd8afe1bda0] > ldc(_ZN16TemplateInstance12needsCodegenEv+0xfe)[0x56173bcef2c2] > ldc(_ZN16TemplateInstance12needsCodegenEv+0x103)[0x56173bcef2c7] > ldc(_ZN16TemplateInstance12needsCodegenEv+0x103)[0x56173bcef2c7] > ldc(_ZN16TemplateInstance12needsCodegenEv+0x103)[0x56173bcef2c7] > ldc(_ZN16TemplateInstance12needsCodegenEv+0x103)[0x56173bcef2c7] > [...] I managed to produce a smaller test case by just gutting my project and seeing how small I could get a broken sample. There were a lot of places where commenting something out fixed it. It still weighs in at just less than 1k sloc (ignoring arsd modules) and it's full of stubs and noops, but it's easy to compile. > $ git clone -b overflow https://github.com/zorael/kameloso.git The latest commit in that overflow branch (00f375) builds *sometimes*, while the commit previous to it (546cea or HEAD~) reliably fails every time. I stopped trying to reduce there. I'm not sure if any of this helps, but Petar Kirov [ZombineDev] suggested I report it in either case.

I am now at a stage where with 2.079.0 I can no longer trick dmd on linux into compiling my project in release mode (or dub -b plain) at all; only debug works. dmd 2.079.0 fails with exit code -11, with gdb backtraces listing needsCodegen(). I can still coerce dmd 2.078.3 and ldc 1.8.0 into compiling, partly. I expect ldc will similarly stop working when a new version of it based on 2.079 is released. When I reduce the problem with dustmite I end up with nonsense that apparently does still trigger the behaviour, but isn't something you can reason about. Things like function calls that *need to be there* for everything to work, and without which I get the overflow. The commit that didn't work in the previously linked overflow branch now builds in 2.079.0, but instead now all my other tricks don't for the current state of the codebase. The issue is an emergent property of a hundred little things. Is there anything I can do to help diagnose? > $ git clone https://github.com/zorael/kameloso > $ cd kameloso > $ git checkout 8b05c0f # unreduced until interest expressed, expect build times > $ dub build -c vanilla -b plain > /usr/bin/dmd failed with exit code -11.

> When I reduce the problem with dustmite I end up with nonsense that apparently does still trigger the behaviour, but isn't something you can reason about. Things like function calls that *need to be there* for everything to work, and without which I get the overflow. Could you still post the reduced example? > Is there anything I can do to help diagnose? Well, the smaller the reduced example, the easier it will be to dive through the DMD codebase and fix the issue. Apart from that maybe also post the issue on the #dbugfix campaign, s.t. it gets more exposure: https://dlang.org/blog/2018/02/03/the-dbugfix-campaign/

I reduced it to 58 lines using dustmite, which is incidentally amazing. Under 2.079.0, it compiles in -debug mode but doesn't outside of it. 2.078.3 can however compile it in both. The code is nonsense, but it appears to trigger the behaviour. > $ git clone https://github.com/zorael/kameloso -b 2.079-overflow > $ cd kameloso > $ dub build -b debug # works > > $ dub build -b plain > /usr/bin/dmd failed with exit code -11. > > $ gdb --batch -ex "run" --args dmd source/kameloso/*.d source/kameloso/plugins/*.d > Program received signal SIGSEGV, Segmentation fault. > 0x0000555555825982 in TemplateInstance::needsCodegen() () I tested it on Travis and CircleCI. Both run 2.079.0, and Travis could not reproduce the segfault. CircleCI however could! Please see https://circleci.com/gh/zorael/kameloso/151. Again, this seems to only affect 2.079.0, and only on *some* distributions. Unless your linux installation sees the behaviour, I imagine the only way to observe it is to download a Manjaro .iso and run it in a virtual machine. Apologies. I wish I could produce something more tangible but this is how ephemeral the issue is.

JR: Cool! I can reproduce it on my machine too :) For reference, here's what's necessary for reproducing this without dub: /home/seb/dlang/dmd/generated/linux/release/64/dmd -c -offoo.o -w -Isource/ source/kameloso/common.d source/kameloso/irc.d source/kameloso/ircdefs.d source/kameloso/main.d source/kameloso/plugins/notes.d -vcolumns | head -n20 Program received signal SIGSEGV, Segmentation fault. 0x00005555557f4187 in TemplateInstance::needsCodegen() () #0 0x00005555557f4187 in TemplateInstance::needsCodegen() () #1 0x00005555557f42c6 in TemplateInstance::needsCodegen() () #2 0x00005555557f42c6 in TemplateInstance::needsCodegen() () #3 0x00005555557f42c6 in TemplateInstance::needsCodegen() () #4 0x00005555557f42c6 in TemplateInstance::needsCodegen() () #5 0x00005555557f42c6 in TemplateInstance::needsCodegen() () #6 0x00005555557f42c6 in TemplateInstance::needsCodegen() () #7 0x00005555557f42c6 in TemplateInstance::needsCodegen() () #8 0x00005555557f42c6 in TemplateInstance::needsCodegen() () I will look at reducing this even further tomorrow.

(In reply to Seb from comment #4) > JR: Cool! I can reproduce it on my machine too :) [...] > I will look at reducing this even further tomorrow. Awesome, thanks!

I can no longer reproduce with 2.079.1 nor 2.080.0, on the same machines that previously exhibited the overflow. Was something changed, or is it merely being tricky again?

It hit again on dmd 2.080.0 and ldc 1.9.0. The commit that made it sneak back in was literally just this: > - plugin.setEvent = event; > + plugin.currentPeekType = AdminPlugin.PeekType.set; > + IRCEvent mutEvent = event; > plugin.state.mainThread.send(ThreadMessage.PeekPlugins(), > - cast(shared IRCPlugin)plugin); > + cast(shared IRCPlugin)plugin, mutEvent); > } Before that it works, after that it segfaults. I'll have to change my CI scripts so they only test debug mode again. > zorael@xps ~/src/kameloso [master] $ dub build -b plain > Performing "plain" build using /usr/bin/dmd for x86_64. > requests 0.7.4: target for configuration "std" is up to date. > kameloso 1.0.0-rc.1+commit.13.g9bb032c: building configuration "posix"... > /usr/bin/dmd failed with exit code -11. I haven't made a new attempt at reducing it yet, but it's not like I can produce something that looks like a reduced bug. All I can do is lessen the number of lines needed to compile and still get it to surface. Do bounties work? I'd file one for this but not sure if people read those.

You could search for the "dbugfix campaign". Bounties generally don't work too well, but you could still try to post on the NG for visibility - this one typically does help.

To update, I did a new dustmite run and managed to produce something that segfaults on all my machines running dmd 2.080, ldc 1.7.0 and 1.9.0. This includes local Windows, but also CircleCI and Travis. dscanner --sloc reports it as 108 lines. Mind that it is still nonsense! But it is compiler-segfaulting nonsense. > git clone https://github.com/zorael/kameloso -b 2080-overflow1 > cd kameloso > dub build -b plain > dmd -c source/kameloso/*.d source/kameloso/plugins/*.d > ldc source/kameloso/*.d source/kameloso/plugins/*.d I'll write up a #dbugfix post for the General forum next and ask for more help there.

(In reply to JR from comment #9) > > git clone https://github.com/zorael/kameloso -b 2080-overflow1 Reduced further to a single file (but still with Phobos dependencies): ---- enum Type { x000,x001,x002,x003,x004,x005,x006,x007,x008,x009, x010,x011,x012,x013,x014,x015,x016,x017,x018,x019, x020,x021,x022,x023,x024,x025,x026,x027,x028,x029, x030,x031,x032,x033,x034,x035,x036,x037,x038,x039, x040,x041,x042,x043,x044,x045,x046,x047,x048,x049, x050,x051,x052,x053,x054,x055,x056,x057,x058,x059, x060,x061,x062,x063,x064,x065,x066,x067,x068,x069, x070,x071,x072,x073,x074,x075,x076,x077,x078,x079, x080,x081,x082,x083,x084,x085,x086,x087,x088,x089, x090,x091,x092,x093,x094,x095,x096,x097,x098,x099, x100,x101,x102,x103,x104,x105,x106,x107,x108,x109, x110,x111,x112,x113,x114,x115,x116,x117,x118,x119, x120,x121,x122,x123,x124,x125,x126,x127,x128,x129, x130,x131,x132,x133,x134,x135,x136,x137,x138,x139, x140,x141,x142,x143,x144,x145,x146,x147,x148,x149, x150,x151,x152,x153,x154,x155,x156,x157,x158,x159, x160,x161,x162,x163,x164,x165,x166,x167,x168,x169, x170,x171,x172,x173,x174,x175,x176,x177,x178,x179, x180,x181,x182,x183,x184,x185,x186,x187,x188,x189, x190,x191,x192,x193,x194,x195,x196,x197,x198,x199, x200,x201,x202,x203,x204,x205,x206,x207,x208,x209, x210,x211,x212,x213,x214,x215,x216,x217,x218,x219, x220,x221,x222,x223,x224,x225,x226,x227,x228,x229, x230,x231,x232,x233,x234,x235,x236,x237,x238,x239, x240,x241,x242,x243,x244,x245,x246,x247,x248,x249, x250,x251,x252,x253,x254,x255,x256,x257,x258,x259, x260,x261,x262,x263,x264,x265,x266,x267,x268,x269, x270,x271,x272,x273,x274,x275,x276,x277,x278,x279, x280,x281,x282,x283,x284,x285,x286,x287,x288,x289, x290,x291,x292,x293,x294,x295,x296,x297,x298,x299, x300,x301,x302,x303,x304,x305,x306,x307,x308,x309, x310,x311,x312,x313,x314,x315,x316,x317,x318,x319, x320,x321,x322,x323,x324,x325,x326,x327,x328,x329, x330,x331,x332,x333,x334,x335,x336,x337,x338,x339, x340,x341,x342,x343,x344,x345,x346,x347,x348,x349, x350,x351,x352,x353,x354,x355,x356,x357,x358,x359, x360,x361,x362,x363,x364,x365,x366,x367,x368,x369, x370,x371,x372,x373,x374,x375,x376,x377,x378,x379, x380,x381,x382,x383,x384,x385,x386,x387,x388,x389, x390,x391,x392,x393,x394,x395,x396,x397,x398,x399, x400,x401,x402,x403,x404,x405,x406,x407,x408,x409, x410,x411,x412,x413,x414,x415,x416,x417,x418,x419, x420,x421,x422,x423,x424,x425,x426,x427,x428,x429, x430,x431,x432,x433,x434,x435,x436,x437,x438,x439, x440,x441,x442,x443,x444,x445,x446,x447,x448,x449, x450,x451,x452,x453,x454,x455,x456,x457,x458,x459, x460,x461,x462,x463,x464,x465,x466,x467,x468,x469, x470,x471,x472,x473,x474,x475,x476,x477,x478,x479, x480,x481,x482,x483,x484,x485,x486,x487,x488,x489, x490,x491,x492,x493,x494,x495,x496,x497,x498,x499, x500,x501,x502,x503,x504,x505,x506,x507,x508,x509, x510,x511,x512,x513,x514,x515,x516,x517,x518,x519, x520,x521,x522,x523,x524,x525,x526,x527,x528,x529, x530,x531,x532,x533,x534,x535,x536,x537,x538,x539, x540,x541,x542,x543,x544,x545,x546,x547,x548,x549, x550,x551, } import std.concurrency: send; import std.format: format; struct PeekPlugins {} struct IRCEvent { Type type; byte sender; string channel; string content; string aux; uint num; long time; } @Type void onSetCommand() { alias s = send!(PeekPlugins, shared IRCPlugin, IRCEvent); } interface IRCPlugin {} class AdminPlugin : IRCPlugin {} class WHOISRequestImpl { void foo() { IRCEvent event; "[%s] @ %s".format(event); } } ---- Fails a bit less reliably, but still 7 out of 10 times.

(In reply to ag0aep6g from comment #10) > Fails a bit less reliably, but still 7 out of 10 times. I can't reproduce your snippet, which confirms that it's inconsistent on different setups on different machines. Updated my best dustmite to also segfault ldc on non-Windows (66 lines): > git clone https://github.com/zorael/kameloso.git -b 2080-overflow1-ldc Posted as https://forum.dlang.org/post/[email protected].

This is caused by a giant chain of templates. I will take a look.

(In reply to Stefan Koch from comment #12) > This is caused by a giant chain of templates. > > I will take a look. Thank you! If you need more dustmite examples I can generate those on demand.

Removed Phobos from the test case from comment 10, and made sure that it fails reliably (for me) by raising the number of enum members again: ---- enum Type { x000,x001,x002,x003,x004,x005,x006,x007,x008,x009,x010,x011,x012,x013,x014, x015,x016,x017,x018,x019,x020,x021,x022,x023,x024,x025,x026,x027,x028,x029, x030,x031,x032,x033,x034,x035,x036,x037,x038,x039,x040,x041,x042,x043,x044, x045,x046,x047,x048,x049,x050,x051,x052,x053,x054,x055,x056,x057,x058,x059, x060,x061,x062,x063,x064,x065,x066,x067,x068,x069,x070,x071,x072,x073,x074, x075,x076,x077,x078,x079,x080,x081,x082,x083,x084,x085,x086,x087,x088,x089, x090,x091,x092,x093,x094,x095,x096,x097,x098,x099,x100,x101,x102,x103,x104, x105,x106,x107,x108,x109,x110,x111,x112,x113,x114,x115,x116,x117,x118,x119, x120,x121,x122,x123,x124,x125,x126,x127,x128,x129,x130,x131,x132,x133,x134, x135,x136,x137,x138,x139,x140,x141,x142,x143,x144,x145,x146,x147,x148,x149, x150,x151,x152,x153,x154,x155,x156,x157,x158,x159,x160,x161,x162,x163,x164, x165,x166,x167,x168,x169,x170,x171,x172,x173,x174,x175,x176,x177,x178,x179, x180,x181,x182,x183,x184,x185,x186,x187,x188,x189,x190,x191,x192,x193,x194, x195,x196,x197,x198,x199,x200,x201,x202,x203,x204,x205,x206,x207,x208,x209, x210,x211,x212,x213,x214,x215,x216,x217,x218,x219,x220,x221,x222,x223,x224, x225,x226,x227,x228,x229,x230,x231,x232,x233,x234,x235,x236,x237,x238,x239, x240,x241,x242,x243,x244,x245,x246,x247,x248,x249,x250,x251,x252,x253,x254, x255,x256,x257,x258,x259,x260,x261,x262,x263,x264,x265,x266,x267,x268,x269, x270,x271,x272,x273,x274,x275,x276,x277,x278,x279,x280,x281,x282,x283,x284, x285,x286,x287,x288,x289,x290,x291,x292,x293,x294,x295,x296,x297,x298,x299, x300,x301,x302,x303,x304,x305,x306,x307,x308,x309,x310,x311,x312,x313,x314, x315,x316,x317,x318,x319,x320,x321,x322,x323,x324,x325,x326,x327,x328,x329, x330,x331,x332,x333,x334,x335,x336,x337,x338,x339,x340,x341,x342,x343,x344, x345,x346,x347,x348,x349,x350,x351,x352,x353,x354,x355,x356,x357,x358,x359, x360,x361,x362,x363,x364,x365,x366,x367,x368,x369,x370,x371,x372,x373,x374, x375,x376,x377,x378,x379,x380,x381,x382,x383,x384,x385,x386,x387,x388,x389, x390,x391,x392,x393,x394,x395,x396,x397,x398,x399,x400,x401,x402,x403,x404, x405,x406,x407,x408,x409,x410,x411,x412,x413,x414,x415,x416,x417,x418,x419, x420,x421,x422,x423,x424,x425,x426,x427,x428,x429,x430,x431,x432,x433,x434, x435,x436,x437,x438,x439,x440,x441,x442,x443,x444,x445,x446,x447,x448,x449, x450,x451,x452,x453,x454,x455,x456,x457,x458,x459,x460,x461,x462,x463,x464, x465,x466,x467,x468,x469,x470,x471,x472,x473,x474,x475,x476,x477,x478,x479, x480,x481,x482,x483,x484,x485,x486,x487,x488,x489,x490,x491,x492,x493,x494, x495,x496,x497,x498,x499,x500,x501,x502,x503,x504,x505,x506,x507,x508,x509, x510,x511,x512,x513,x514,x515,x516,x517,x518,x519,x520,x521,x522,x523,x524, x525,x526,x527,x528,x529,x530,x531,x532,x533,x534,x535,x536,x537,x538,x539, x540,x541,x542,x543,x544,x545,x546,x547,x548,x549,x550,x551,x552,x553,x554, x555,x556,x557,x558,x559,x560,x561,x562,x563,x564,x565,x566,x567,x568,x569, x570,x571,x572,x573,x574,x575,x576,x577,x578,x579,x580,x581,x582,x583,x584, x585,x586,x587,x588,x589,x590,x591,x592,x593,x594,x595,x596,x597,x598,x599, x600,x601,x602,x603,x604,x605,x606,x607,x608,x609,x610,x611,x612,x613,x614, x615,x616,x617,x618,x619,x620,x621,x622,x623,x624,x625,x626,x627,x628,x629, x630,x631,x632,x633,x634,x635,x636,x637,x638,x639,x640,x641,x642,x643,x644, x645,x646,x647,x648,x649,x650,x651,x652,x653,x654,x655,x656,x657,x658,x659, x660,x661,x662,x663,x664,x665,x666,x667,x668,x669,x670,x671,x672,x673,x674, x675,x676,x677,x678,x679,x680,x681,x682,x683,x684,x685,x686,x687,x688,x689, x690,x691,x692,x693,x694,x695,x696,x697,x698,x699,x700,x701,x702,x703,x704, x705,x706,x707,x708,x709,x710,x711,x712,x713,x714,x715,x716,x717,x718,x719, x720,x721,x722,x723,x724,x725,x726,x727,x728,x729,x730,x731,x732,x733,x734, x735,x736,x737,x738,x739,x740,x741,x742,x743,x744,x745,x746,x747,x748,x749, x750,x751,x752,x753,x754,x755,x756,x757,x758,x759,x760,x761,x762,x763,x764, x765,x766,x767,x768,x769,x770,x771,x772,x773,x774,x775,x776,x777,x778,x779, x780,x781,x782,x783,x784,x785,x786,x787,x788,x789,x790,x791,x792,x793,x794, x795,x796,x797,x798,x799,x800,x801,x802,x803,x804,x805,x806,x807,x808,x809, } template Unqual() {} struct FormatSpec() if (is(Unqual!()) || !is(Unqual!())) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (is(T == string)) {} void formatValueImpl(W, T)(W w, T val, FormatSpec!() f) if (is(T == Type)) { foreach (i, e; __traits(allMembers, T)) { formatValueImpl!(W, string)(w, e, f); } } void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T val, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} alias fvi_Type_1 = formatValueImpl!(void function(char c), Type); alias fvi_Type_2 = formatValueImpl!(void function(const(char)[] s), Type); alias fvi_Type_3 = formatValueImpl!(void delegate(char c), Type); alias fvi_Type_4 = formatValueImpl!(void delegate(const char[]), Type); alias fvi_Type_5 = formatValueImpl!(void delegate(const(char)[] s), Type); ---- If this happens to not crash on other systems, adding more enum members and/or "false" formatValueImpl overloads and/or fvi_Type_* variants might do the trick.

Shorter repro: module test; struct FormatSpec() {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (is(T == string)) { } void formatValueImpl(W, T)(W w, T val, FormatSpec!() f) if (is(T == int)) { import std.range : iota; static foreach (_; iota(1000)) { formatValueImpl!(W, string)(w, "", f); } } void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} void formatValueImpl(W, T)(W w, T obj, FormatSpec!() f) if (false) {} alias fvi_Type_1 = formatValueImpl!(void function(char c), int); alias fvi_Type_2 = formatValueImpl!(void function(const(char)[] s), int); alias fvi_Type_3 = formatValueImpl!(void delegate(char c), int); alias fvi_Type_4 = formatValueImpl!(void delegate(const char[]), int); alias fvi_Type_5 = formatValueImpl!(void delegate(const(char)[] s), int); void main() {} Seems to happen more reliably in gdb. Since it's a stack overflow, recommended to limit your stack size with ulimit -s 128 to test.

I've been looking into fixing this. The problem is this block of code: https://github.com/dlang/dmd/blob/750f0245c2882887cdb830c171e9221719e0da28/src/dmd/dtemplate.d#L6291-L6320 --- if (!minst) { // If this is a speculative instantiation, // 1. do codegen if ancestors really needs codegen. // 2. become non-speculative if siblings are not speculative TemplateInstance tnext = this.tnext; TemplateInstance tinst = this.tinst; // At first, disconnect chain first to prevent infinite recursion. this.tnext = null; this.tinst = null; // Determine necessity of tinst before tnext. if (tinst && tinst.needsCodegen()) { minst = tinst.minst; // cache result assert(minst); assert(minst.isRoot() || minst.rootImports()); return true; } // Issue 18026 - Stack overerflow due to recursive call to needsCodegen if (tnext && (tnext.needsCodegen() || tnext.minst)) { minst = tnext.minst; // cache result assert(minst); return minst.isRoot() || minst.rootImports(); } // Elide codegen because this is really speculative. return false; } --- Specifically, the stack overflow occurs at... --- if (tnext && (tnext.needsCodegen() || tnext.minst)) { minst = tnext.minst; // cache result assert(minst); return minst.isRoot() || minst.rootImports(); } --- ... due to the recursive call to `needsCodegen` The whole function `needsCodegen` looks like a mess. There's probably a way to implement the algorithm as a loop instead of recursive function calls, but I'm not yet sure what to do to fix this.

(In reply to Mike Franklin from comment #16) > I've been looking into fixing this. The problem is this block of code: > https://github.com/dlang/dmd/blob/750f0245c2882887cdb830c171e9221719e0da28/ > src/dmd/dtemplate.d#L6291-L6320 > > The whole function `needsCodegen` looks like a mess. There's probably a way > to implement the algorithm as a loop instead of recursive function calls, > but I'm not yet sure what to do to fix this. Ah, this part of code is indeed source of other issues (speculative sometimes fails). Original op can use `-allinst` as a workaround, waiting for a fix.

On CircleCI build servers it has started to segfault even in debug mode[1]. Travis is still okay and I cannot reproduce it locally. > [...] > Compiling source/kameloso/objmanip.d... > Compiling source/kameloso/plugins/admin.d... > Compiling source/kameloso/plugins/automode.d... > Compiling source/kameloso/plugins/bashquotes.d... > Compiling source/kameloso/plugins/chanqueries.d... > Compiling source/kameloso/plugins/chatbot.d... > Segmentation fault > dmd failed with exit code 139. > Exited with code 2 (It has to be built with --mode=singleFile there, or else the process gets killed due to high memory usage.) gdb showed it to be TemplateInstance::needsCodegen(), as before. Modifying dub.json to make it compile with -allinst lets it successfully build, but it will then instead fail to link. This may be Issue 19123 - -allinst gives undefined reference linker errors[2]. [1]: https://circleci.com/gh/zorael/kameloso/547 [2]: https://issues.dlang.org/show_bug.cgi?id=19123

Small update. I no longer experience this with the dmd and ldc from the Arch/Manjaro repository, even when checking out the old `overflow` branch used in the original report. I do experience it when I build dmd myself (with Digger). It is not possible for me to build and use my own dmd. > [...] > #58123 0x00000000006344e2 in TemplateInstance::needsCodegen() (this=0x7fff5d89b910) at src/dmd/dtemplate.d:6265 > #58124 0x00000000006344e2 in TemplateInstance::needsCodegen() (this=0x7fff5d89c2b0) at src/dmd/dtemplate.d:6265 > #58125 0x00000000006347ca in TemplateInstance::needsCodegen() (this=0x7ffff2acf3b0) at src/dmd/dtemplate.d:6352 > #58126 0x000000000078be89 in toObjFile::ToObjFile::visit(TemplateInstance*) (this=0x7fffffffa838, ti=0x7ffff2acf3b0) at src/dmd/toobj.d:829 > #58127 0x00000000006377ba in TemplateInstance::accept(Visitor*) (this=0x7ffff2acf3b0, v=0x7fffffffa838) at src/dmd/dtemplate.d:7488 > #58128 0x000000000078ac13 in toObjFile(Dsymbol*, bool) (ds=0x7ffff2acf3b0, multiobj=false) at src/dmd/toobj.d:1027 > #58129 0x0000000000777352 in genObjFile(Module*, bool) (m=0x7ffff7b331f0, multiobj=false) at src/dmd/glue.d:420 > #58130 0x00000000006d85aa in dmd.mars.tryMain(ulong, const(char)**, ref dmd.globals.Param) (params=..., argv=0x7fffffffcef8, argc=112) at src/dmd/mars.d:696 > #58131 0x00000000006d9571 in D main (_param_0=...) at src/dmd/mars.d:972 Has there been any changes since last? Having a compiler that does work is comforting, but also discomforting to know it may break when it feels like it.

I have also seen a stack overflow with TemplateInstance.needsCodegen and reduced it to the following code: bool f(T)(T x) { return false; } static foreach(i; 0..60000) { static if(f(i)) { } } The failure happens on Windows. It can also be reproduced on Linux by increasing the number of iterations or reducing the stack size first with the command "ulimit -s 1024"

@ibuclaw created dlang/dmd pull request #14787 "fix Issue 18026 - Stack overflow in dmd/dtemplate.d:6248, TemplateInstance::needsCodegen()" fixing this issue: - fix Issue 18026 - Stack overflow in dmd/dtemplate.d:6248, TemplateInstance::needsCodegen() https://github.com/dlang/dmd/pull/14787

dlang/dmd pull request #14787 "fix Issue 18026 - Stack overflow in dmd/dtemplate.d:6248, TemplateInstance::needsCodegen()" was merged into stable: - ce7c5df44f4ac6aee1671ecbc54bb08c2a8f3e31 by Iain Buclaw: fix Issue 18026 - Stack overflow in dmd/dtemplate.d:6248, TemplateInstance::needsCodegen() https://github.com/dlang/dmd/pull/14787