Bug 9044 – dmd sometimes produces object files with multiple definitions

Created attachment 1162 Testcase Not sure if it is a dmd regression or dmd changes discovered new optlink bug. Looks like a buffer overflow bug somewhere as removing almost any definition of any file makes testcase linkable. Testcase building output (launching "build.bat"): --- OPTLINK (R) for Win32 Release 8.00.12 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html xlib.lib(object) Offset FFD10H Record Type 0091 Error 1: Previous Definition Different : _D64D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.5412__ModuleInfoZ xlib.lib(object) Offset FFDECH Record Type 00C3 Error 1: Previous Definition Different : _D64D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.547__arrayZ xlib.lib(object) Offset FFE1EH Record Type 00C3 Error 1: Previous Definition Different : _D64D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.548__assertFiZv xlib.lib(object) Offset FFE50H Record Type 00C3 Error 1: Previous Definition Different : _D64D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.5415__unittest_failFiZv ---

"previous definition different" means you have the same name defined more than once. http://www.digitalmars.com/ctg/OptlinkErrorMessages.html#previous_definition_different I suspect the problem you are having stems from having a file named "object.d" in your xlib.

(In reply to comment #1) > I suspect the problem you are having stems from having a file named "object.d" > in your xlib. Are you writing it seriously? I'm not a newbie, I know linker error messages and I think of what I'm reporting. Have you looked at testcase? It consists of a.d, b.d, c.d, d.d, and main.d. Have you read the description? The error disappear if almost any definition of any file is removed. Probably you are very tired and need some rest...

Attempting to extract your test case: ------------------------------ 7z l linker-error.7z 7-Zip 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 Error: 7-Zip cannot find the code that works with archives. ------------------------------ Please use ordinary zip files or just upload the files themselves.

(In reply to comment #3) > Attempting to extract your test case: > > ------------------------------ > 7z l linker-error.7z > > 7-Zip 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 > > Error: > 7-Zip cannot find the code that works with archives. > ------------------------------ > > Please use ordinary zip files or just upload the files themselves. Common! 7-Zip has [one of] the best compression ratio, small compression time, good GUI and explorer integration, so why not to use it? Also why not to use the installer? If you really want to do everything manually, why are you complaining about errors? It is you decision to not use installer and user-friendly GUI. But if you for some reason need to use 7z.exe, you also need 7z.dll for "code that works with archives".

(In reply to comment #4) > Common! 7-Zip has [one of] the best compression ratio, small compression time, > good GUI and explorer integration, so why not to use it? Who cares? File size simply no longer matters, especially not 5K. I've already spend magnitudes more time trying to get it uncompressed than any microseconds it supposedly saves. I'm not spending any more time on it. > Also why not to use the installer? If you really want to do everything > manually, why are you complaining about errors? It is you decision to not use > installer and user-friendly GUI. If it's my fault that 7z.exe does not work with 7z files, that makes me even less interested in 7z. > But if you for some reason need to use 7z.exe, you also need 7z.dll for "code > that works with archives". That's exactly why I don't care to use it. It does not work without mucking about with it. .zip works everywhere, no issues. Please upload the files in a usable format. I'm interested in fixing the bug in this issue. I couldn't be less interested in wrestling with 7z.

That's not funny any more, really. Walter, is it really hard to install 7-zip extractor? It is ubiquitous and free. There is hardly a single non-lame programmer who doesn't have it installed. Why irritate an already irritated user that cares to file bug reports for your half-working thing?

Created attachment 1172 The same files in zip archive (In reply to comment #6) > That's not funny any more, really. Walter, is it really hard to install 7-zip > extractor? It is ubiquitous and free. There is hardly a single non-lame > programmer who doesn't have it installed. Why irritate an already irritated > user that cares to file bug reports for your half-working thing? I agree with Walter that Bugzilla entries should have attachments in the most common and user-friendly formats, like zip, unless the file is huge or there are special situations. So a 7 kb archive does not justify the usage of a less common archiver. On the other hand 7z is an archiver better than zip, and progress in computer science requires a little of extra work to phase out older technologies to replace them with newer and better. So today installing a working and free version of 7z should be considered expected by serious computer users.

Please let's not go off-topic. I can't recreate the linker failure with git-head. I get the obj, lib, map and exe file with no errors. This might also be related to Issue 3094, which doesn't have source test-cases but only object file test-cases. In 3094 linking fails with optlink but it works with Unilink (minus the missing entry function error which is expected as none were provided). Denis do you know which DMD commit you were on when testing this?

(In reply to comment #8) > Please let's not go off-topic. > > I can't recreate the linker failure with git-head. I get the obj, lib, map and > exe file with no errors. > > This might also be related to Issue 3094, which doesn't have source test-cases > but only object file test-cases. In 3094 linking fails with optlink but it > works with Unilink (minus the missing entry function error which is expected as > none were provided). > > Denis do you know which DMD commit you were on when testing this? I confirm that test-case links fine with 04cbe5d8a76186d8327c50eb46a02eb5633d7835 but it is only because this is a "random" failure. Original code (from my hooking project) still fails to link with the same error. I don't want to reduce my sources every time dmd changes so you can use e01eb59f842dfe7a5275d96c420691c4a64f57f4 where provided test-case triggers the bug or find later one using `git bisect`.

Thank you, bearophile, for uploading the zip. I can't reproduce a problem, either.

(In reply to comment #10) > Thank you, bearophile, for uploading the zip. > > I can't reproduce a problem, either. Does it mean that it is a problem for you to work with e01eb59f842dfe7a5275d96c420691c4a64f57f4 instead of HEAD?

(In reply to comment #11) > Does it mean that it is a problem for you to work with > e01eb59f842dfe7a5275d96c420691c4a64f57f4 instead of HEAD? It works with both HEAD and e01eb59f842dfe7a5275d96c420691c4a64f57f4. I cannot reproduce this problem.

(In reply to comment #12) > It works with both HEAD and e01eb59f842dfe7a5275d96c420691c4a64f57f4. > > I cannot reproduce this problem. Confirm. Looks like the problem is in my phobos.lib and I can't remember exact dmd/druntime/phobos commits used to build it.

Here is the full testcase (with phobos library file): http://deoma-cmd.ru/files/other/linker-error-full.zip (can not upload here because of its size (~1.2 MiB))

To disable the issue and make code linkable, you can e.g. remove any line from `c.d`.

(In reply to comment #14) > Here is the full testcase (with phobos library file): > http://deoma-cmd.ru/files/other/linker-error-full.zip > (can not upload here because of its size (~1.2 MiB)) Phobos of which commit?

(In reply to comment #16) > Phobos of which commit? As I wrote in comment 13, I can't remember exact dmd/druntime/phobos commits used to build it.

Created attachment 1189 Almost unreduced sources Fails to link on every dmd/druntime/phobos commit I tried. It fails e.g.: dmd: 8f9ee7b112c644c4c2b830a4f9c79974196e37d5 druntime: 778d91803e49bc1e829f87014e745e75df84d17d phobos: 6a3ffa5e136d22b31529e6a3688cb8ce3a5508a0

Can somebody confirm link failure in the last test case?

(In reply to comment #19) > Can somebody confirm link failure in the last test case? I can. For me it fails with stock dmd2.062. Here is the complete log: Compiling xlib.lib... Compiling x.obj... Linking... OPTLINK (R) for Win32 Release 8.00.12 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html psapi.lib Warning 2: File Not Found psapi.lib xlib.lib(object) Offset 0B1ACH Record Type 0091 Error 1: Previous Definition Different : _D58C:\dmd2\windows\bin\..\..\src\druntime\import\object.di.4612__ModuleInfoZ xlib.lib(object) Offset 0B239H Record Type 00C3 Error 1: Previous Definition Different : _D58C:\dmd2\windows\bin\..\..\src\druntime\import\object.di.467__arrayZ xlib.lib(object) Offset 0B266H Record Type 00C3 Error 1: Previous Definition Different : _D58C:\dmd2\windows\bin\..\..\src\druntime\import\object.di.468__assertFiZv xlib.lib(object) Offset 0B293H Record Type 00C3 Error 1: Previous Definition Different : _D58C:\dmd2\windows\bin\..\..\src\druntime\import\object.di.4615__unittest_failFiZv Failed.

This is definitely a dmd bug, not OPTLINK as it is also triggered when building a library (when dmd doesn't call linker). `f` is a member function of a class: --- void f() { auto a = asJSON(["type": asJSON(to!string(type))]); auto b = asJSON(cycleData.map!(d => toJSON(d.height))().array()); auto c = asJSON(zip(cycleData, sequence!`a[0] + n`(cast() creationCycle)).filter!`!a[0].available`().map!(a => asJSON(a[1]))().array()); } --- If any of 3 `f`'s lines are commented out the library compiles fine into `.lib` file. Otherwise dmd fails with: --- xxx.lib: Error: multiple definition of object_32e_f8d: _D65D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.81412__ModuleInfoZ and object: _D65D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.81412__ModuleInfoZ xxx.lib: Error: multiple definition of object_32e_f8d: _D65D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.8147__arrayZ and object: _D65D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.8147__arrayZ xxx.lib: Error: multiple definition of object_32e_f8d: _D65D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.8148__assertFiZv and object: _D65D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.8148__assertFiZv xxx.lib: Error: multiple definition of object_32e_f8d: _D65D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.81415__unittest_failFiZv and object: _D65D:\D\dmd2head\windows\bin\..\..\src\druntime\import\object.di.81415__unittest_failFiZv ---

Looks like this is not a regression, this bug is triggered from time to time on the same code because of druntime/phobos changes.

Ping. This is a dmd issue, not OPTLINK one. And the fact it is not a REGRESSION doesn't mean we should continue releasing dmd with it included as it even worse than a regression as it is triggered randomly. Personally I already suffer enough from OPTLINK and the fact dmd also produces corrupted object files looks too cruel. With all this words about language stability we have this issue which may accidentally brake build of every code base on every platform because of a any small code change. I worked on only two non-toy D executables for the last 6 month and both are [partially] blocked because of this. For now I abandoned the first project and have managed to juggle with files to "detrigger" this issue in second one. So I'm completely missing the point where are these issues with higher priority than this, abandoned, unvoted one? Probably I'm the most luckless D user who always suffer from these linking failures...

Any plans on fixing this? In my unlucky hands even VisualD's cpp2d fails to build in debug mode because of the issue...

I very much suspect that this is caused by the generated internal object file names of a library or other generated identifiers that are assumed to be unique, but clearly this is not guaranteed when building libraries in separate build steps. I tried to generate a small test on that suspicion some time ago, but it didn't trigger a linker error then. I have figured it out now: //////////////////////// module ab; struct AB { int a; int len() { return (new AB).a; } } unittest { } //////////////////////// module ba; struct BA { int b; int len() { return (new BA).b; } } unittest { } //////////////////////// import ab; import ba; int main() { AB ab; BA ba; return ab.len() + ba.len(); } //////////////////////// now building with dmd 2.062: dmd -lib -of"alib.lib" ab.d dmd -lib -of"blib.lib" ba.d dmd -c main.d dmd main.obj alib.lib blib.lib yields: OPTLINK (R) for Win32 Release 8.00.12 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html blib.lib(object) Offset 013AEH Record Type 0091 Error 1: Previous Definition Different : _D59c:\l\dmd2\windows\bin\..\..\src\druntime\import\object.di.412__ModuleInfoZ blib.lib(object) Offset 0143CH Record Type 00C3 Error 1: Previous Definition Different : _D59c:\l\dmd2\windows\bin\..\..\src\druntime\import\object.di.47__arrayZ blib.lib(object) Offset 01469H Record Type 00C3 Error 1: Previous Definition Different : _D59c:\l\dmd2\windows\bin\..\..\src\druntime\import\object.di.48__assertFiZv blib.lib(object) Offset 01496H Record Type 00C3 Error 1: Previous Definition Different : _D59c:\l\dmd2\windows\bin\..\..\src\druntime\import\object.di.415__unittest_failFiZv blib.lib(object) Offset 00F6EH Record Type 0091 Error 1: Previous Definition Different : _D59c:\l\dmd2\windows\bin\..\..\src\druntime\import\object.di.312__ModuleInfoZ blib.lib(object) Offset 0103BH Record Type 00C3 Error 1: Previous Definition Different : _D59c:\l\dmd2\windows\bin\..\..\src\druntime\import\object.di.37__arrayZ blib.lib(object) Offset 01068H Record Type 00C3 Error 1: Previous Definition Different : _D59c:\l\dmd2\windows\bin\..\..\src\druntime\import\object.di.38__assertFiZv blib.lib(object) Offset 01095H Record Type 00C3 Error 1: Previous Definition Different : _D59c:\l\dmd2\windows\bin\..\..\src\druntime\import\object.di.315__unittest_failFiZv also happens with current git-head.

(In reply to comment #25) > I very much suspect that this is caused by the generated internal object file > names of a library or other generated identifiers that are assumed to be > unique, but clearly this is not guaranteed when building libraries in separate > build steps. It also fails in dmd (not OPTLINK) when a library is generated using a single `dmd -lib ...` call (see Comment 21).

*** This issue has been marked as a duplicate of issue 6461 ***