Bug 7044 – Missing a way to control the order of arguments passed to the linker makes impossible to link some programs

There are some cases where linker flags order do matter (for example for linking libraries). In those cases DMD provide no mechanism for altering the order. This patches tries to minimize this problem by adding an option (-sL) to add flags at the end of the linker command line. This is specially useful to include libraries after the standard library passed with -debuglib or -defaultlib. Patch available as a pull request: https://github.com/D-Programming-Language/dmd/pull/497

By the way, I don't find the solution elegant at all, but it's the only solution I could think of. Maybe it would be better to use the same order of the command line switches when passing them to the linker, even for the -defaultlib and -debuglib. Examples: dmd -L-L/opt/lib -defaultlib=blah -L-lfoo -> gcc -L/opt/lib -lblah -lfoo dmd -L-lbar -defaultlib=blah -> gcc -lbar -lblah The problem is what to do then with the switches that are automatically added by DMD. Maybe those should be just moved to dmd.conf (except for -Xlinker of course), because I think they are not really needed by the the generated code but by the runtime, which can be changed via -defaultlib.

Changed the title to describe the problem more generally instead of describing a possible solution. I think the way to go is the one described in comment 1. I'm willing to implement it if there is consensus on moving all the linker option out of the compiler to the configuration file.

See the (deprecated) pull request for more discussion on the topic. Here I propose a solution that's simple, backwards compatible, and works the same way DMD already does in terms of environment variables and switches. We just need to extend -defaultlib and -debuglib to accept not only the name of the standard library but any other linker flag needed. A simple heuristic should be enough, like if the argument starts with "-" or "/" it should be considered raw arguments to pass to the linker instead of just a library name. This way you could use just -defaultlib=phobos2 or a more complex -defaultlib="-lphobos2 -lrt -ldl". When using the (deprecated) old way, we can still add the needed hardcoded linker flags until at some point it can be not supported anymore and just pass it as raw linker arguments wihout any heuristic to detect the old behaviour. This way you can control which runtime to use either in the command line, in the dmd.conf or in the DFLAGS enviroment variable, with the usual overriding rules (you can specify a system default in the dmd.conf file, override it in your user's setup or Makefile using the DFLAGS environment variable and even override that using regular command line arguments when invoking dmd.

The problem is the order of -defaultlib and -L is fixed as: 1. -L flags 2. libraries on the command line 3. libraries specified by pragma(lib) 4. standard libraries (also -debuglib, -defaultlib libraries) I suggest the straightforward approach of the order of (1), (2), and (4) be determined by the order in which they appear on the command line. That leaves (3), and I suggest that be inserted immediately before (4). This may break existing build systems, though.

(In reply to comment #4) > The problem is the order of -defaultlib and -L is fixed as: > > 1. -L flags > 2. libraries on the command line > 3. libraries specified by pragma(lib) > 4. standard libraries (also -debuglib, -defaultlib libraries) > > I suggest the straightforward approach of the order of (1), (2), and (4) be > determined by the order in which they appear on the command line. This is good enough to change the runtime in the dmd.conf file but is not ideal for overriding a default later, because the -L flags are not replaced as the -defaultlib option is, just appended. For example, I have a default runtime which needs to link to librt, so I have this in dmd.conf: DFLAGS=-defaultlib=runtime1 -L-lrt Then I want to compile some program using another runtime that depends on libz, if I write: dmd -defaultlib=runtime2 -Llz The best we can do is to get these link options: -lrt -lruntime2 -lz Which includes -lrt, which is not needed by runtime2 (and it might not even be available in the environment runtime2 is compiled). I think we need a flag to override the default runtime library *and* all its dependencies in one go. > That leaves (3), and I suggest that be inserted immediately before (4). I guess it makes sense, if you want complete control on the ordering then you have to switch to using -L instead of pragma(lib). > This may break existing build systems, though. This is one good point about the solution I proposed before, is completely backwards compatible.

Full (slightly updated) proposal: * Change -defaultlib and -debuglib to accept an arbitrary number of linker flags. Example: -defaultlib="-lphobos2 -lrt -ldl". * Remove all linker flags from DMD that are not strictly needed by the compiler generated code (i.e. all the linker flags needed only by the runtime). * Update the sample configuration file to set -defaultlib and -debuglib appropriately to make the runtime work. * Change the order in which options are passed to the linker like this (from left to right): * Options passed with -L through the command line * Options passed with -L through environment variables * Libraries specified by pragma(lib) * Options passed with -defaultlib/-debuglib (command line option overrides environment variables and environment variables override config files) * Options needed for the compiler generated code to work (if any)

This is getting complex enough that I have to go back to basics and point out that all dmd is doing is building a command line that is sent to gcc to do the actual work. So I suggest, why not, when your needs for the command line are this complex, simply use gcc to do the link step instead of passing through dmd as an intermediary? To see what command dmd formulates and passes to gcc, compile with -verbose.

(In reply to comment #7) > This is getting complex enough that I have to go back to basics and point out > that all dmd is doing is building a command line that is sent to gcc to do the > actual work. > > So I suggest, why not, when your needs for the command line are this complex, > simply use gcc to do the link step instead of passing through dmd as an > intermediary? > > To see what command dmd formulates and passes to gcc, compile with -verbose. Because is not an option to ask to do that to all your users when you're providing an alternative runtime. The -defaultlib as it is, is useless. And I don't see the complexity in this, is just changing the -defaultlib flag really, and making the compiler really agnostic about the implementation details of the runtime.

BTW, as things are now, you can't compile a program using curl wrapper in phobos, so this doesn't even apply only to the runtime: http://stackoverflow.com/questions/10095150/std-net-curl-linker-errors-in-linux To fix this, you'll have to change the compiler now, while if we implement this proposal, it will enough to update the sample config file. curl.d import std.net.curl, std.stdio; void main() { writeln(get("dlang.org")); } dmd -L-lcurl curl.d /home/luca/dmd/dmd2-git/src/../../phobos/generated/linux/release/64/libphobos2.a(curl.o): In function `_D3std3net4curl4Curl19_sharedStaticCtor28FZv': std/net/curl.d:(.text._D3std3net4curl4Curl19_sharedStaticCtor28FZv+0xf): undefined reference to `curl_global_init' (... more errors ...)

I think link flags should not be hardcoded in the compiler. They are platform dependent and belong into a configuration file so that package maintainers can easily patch them. I don't see why we need defaultlib/debuglib if one can specify link flags.

(In reply to comment #10) > I think link flags should not be hardcoded in the compiler. They are platform > dependent and belong into a configuration file so that package maintainers can > easily patch them. > > I don't see why we need defaultlib/debuglib if one can specify link flags. Is need to allow changing easily between a "normal" and a "debug" build with -debug or -g (I don't remember which one triggers switching to debuglib instead of using defaultlib). Projects can't do that by themselves because they don't need to know where the runtime is installed and how can they switch between a library compiled with debug symbols (and maybe assertions, contracts, etc.) and the normal one. I think what I proposed is general enough to support all that, even having the debug library need to link to different libraries than the normal library.

(In reply to comment #11) > Is need to allow changing easily between a "normal" and a "debug" build with > -debug or -g (I don't remember which one triggers switching to debuglib instead > of using defaultlib). You can do by supporting configurations in the config file, e.g. Environment32/Environment64 [1] or dub packages [2][3]. One could also use different config files and introduce a -conf argument. [1]: https://github.com/D-Programming-Language/dmd/pull/1220 [2]: http://registry.vibed.org/package-format#build-settings [3]: http://registry.vibed.org/package-format#configurations

(In reply to comment #12) > (In reply to comment #11) > > Is need to allow changing easily between a "normal" and a "debug" build with > > -debug or -g (I don't remember which one triggers switching to debuglib instead > > of using defaultlib). > > You can do by supporting configurations in the config file, e.g. > Environment32/Environment64 [1] or dub packages [2][3]. > One could also use different config files and introduce a -conf argument. > > [1]: https://github.com/D-Programming-Language/dmd/pull/1220 > [2]: http://registry.vibed.org/package-format#build-settings > [3]: http://registry.vibed.org/package-format#configurations I still can see how that could help changing the runtime library and all its dependencies while linking with -debug (or -gc). Could you provide a concrete example on how would you achieve that?

Since the fixup to the link command in this case is quite simple, how about a flag in dmd that performs a dry run and just prints out the link command?

(In reply to comment #14) > Since the fixup to the link command in this case is quite simple, how about a > flag in dmd that performs a dry run and just prints out the link command? I think that's just an ugly hack. What's wrong with the solution I proposed which I think solves the problem from the root?

(In reply to comment #15) > I think that's just an ugly hack. What's wrong with the solution I proposed > which I think solves the problem from the root? It would be the fourth compiler flag that alters the link flags. There are two problems: The runtime dependencies are hardcoded in the compiler. The flags -debuglib and -defaultlib make linking more complicated even though they are almost redundant and can be replaced with -L-l. What we could do IMHO is to allow sections in dmd.conf for certain build settings. [Environment] DFLAGS=-w -g [Environment-X86] DFLAGS+=-L-L/usr/lib32 [Environment-X86_64] DFLAGS+=-L-L/usr/lib64 [Environment-release] DFLAGS+=-L-lphobos [Environment-debug] DFLAGS+=-L-lphobos_d [Environment-linux] DFLAGS+=-L-lrt -L-lpthread -L-ldl [Environment-myversion] DFLAGS=

(In reply to comment #16) > (In reply to comment #15) > > I think that's just an ugly hack. What's wrong with the solution I proposed > > which I think solves the problem from the root? > > It would be the fourth compiler flag that alters the link flags. > > There are two problems: > The runtime dependencies are hardcoded in the compiler. Agreed > The flags -debuglib and -defaultlib make linking more complicated even though > they are almost redundant and can be replaced with -L-l. I don't agree with this, because you need to change a whole set of flags depending on which runtime you are using. -L-l is just not good enough. > What we could do IMHO is to allow sections in dmd.conf for certain build > settings. > > [Environment] > DFLAGS=-w -g > > [Environment-X86] > DFLAGS+=-L-L/usr/lib32 > > [Environment-X86_64] > DFLAGS+=-L-L/usr/lib64 > > [Environment-release] > DFLAGS+=-L-lphobos > > [Environment-debug] > DFLAGS+=-L-lphobos_d > > [Environment-linux] > DFLAGS+=-L-lrt -L-lpthread -L-ldl > > [Environment-myversion] > DFLAGS= This will fix the debug vs. normal runtime, true, but it doesn't help to *easily* use a different runtime library, you have to mess with the config file, but I want a way to do it from the command line, so it can be integrated in the build system.

(In reply to comment #17) > I don't agree with this, because you need to change a whole set of flags > depending on which runtime you are using. -L-l is just not good enough. > True, but that's not what the compiler does at the moment. > This will fix the debug vs. normal runtime, true, but it doesn't help to > *easily* use a different runtime library, you have to mess with the config > file, but I want a way to do it from the command line, so it can be integrated > in the build system. [Environment-tango-X86_64] ... dmd -m64 -version=tango ...

(In reply to comment #18) > (In reply to comment #17) > > I don't agree with this, because you need to change a whole set of flags > > depending on which runtime you are using. -L-l is just not good enough. > > > True, but that's not what the compiler does at the moment. > > > This will fix the debug vs. normal runtime, true, but it doesn't help to > > *easily* use a different runtime library, you have to mess with the config > > file, but I want a way to do it from the command line, so it can be integrated > > in the build system. > > [Environment-tango-X86_64] > ... > > dmd -m64 -version=tango ... Mmmm, I don't like the idea of having to mess with the config file, but since you can actually create one even in your home, I think I ran out of technical arguments, so my only objection about that solution is purely aesthetic. Summary: I'm sold.

(In reply to comment #19) > (In reply to comment #18) > > (In reply to comment #17) > > > I don't agree with this, because you need to change a whole set of flags > > > depending on which runtime you are using. -L-l is just not good enough. > > > > > True, but that's not what the compiler does at the moment. > > > > > This will fix the debug vs. normal runtime, true, but it doesn't help to > > > *easily* use a different runtime library, you have to mess with the config > > > file, but I want a way to do it from the command line, so it can be integrated > > > in the build system. > > > > [Environment-tango-X86_64] > > ... > > > > dmd -m64 -version=tango ... > > Mmmm, I don't like the idea of having to mess with the config file, but since > you can actually create one even in your home, I think I ran out of technical > arguments, so my only objection about that solution is purely aesthetic. > > Summary: I'm sold. Well, not a 100% really. How are you parsing/planning to parse this? --- [Environment] DFLAGS=-w -g [Environment-X86] DFLAGS+=-L-L/usr/lib32 [Environment-X86_64] DFLAGS+=-L-L/usr/lib64 [Environment-release] DFLAGS+=-L-lphobos [Environment-debug] DFLAGS+=-L-lphobos_d [Environment-linux] DFLAGS+=-L-lrt -L-lpthread -L-ldl [Environment-myversion] DFLAGS= --- What do I get if I do: dmd file.d dmd -release file.d dmd -debug file.d ? I think we either need a new section: --- [Environment-default] DFLAGS+=-L-lphobos --- or something (that can't be used in a -version), or a way to *replace* a flag (the former seems more reasonable). Or maybe I'm not getting this right.

Another option would be to just eliminate the -release (or the -debug) flag. Having 3 modes to compile stuff (release, normal, debug) might be too much. But that's even a bigger and more disruptive change, I think.

(In reply to comment #19) I think the matching scheme should be similar to http://registry.vibed.org/package-format#build-settings except that it uses os-versions-debug/release-compiler. > dmd file.d matches [Environment] and [Environment-X86_64] or [Environment-X86] > dmd -release file.d matches [Environment] and [Environment-X86_64] or [Environment-X86] and [Environment-release] but depending on your system it might also match [Environment-linux-X86_64-D_SIMD-release-dmd] > so my only objection about that solution is purely aesthetic. Yeah, it's probably not the most beautiful solution but it's flexible enough for our needs and still reasonably simple.

(In reply to comment #22) > (In reply to comment #19) > I think the matching scheme should be similar to > http://registry.vibed.org/package-format#build-settings except that it uses > os-versions-debug/release-compiler. > > > dmd file.d > > matches [Environment] > and [Environment-X86_64] or [Environment-X86] > > > dmd -release file.d > > matches [Environment] > and [Environment-X86_64] or [Environment-X86] > and [Environment-release] > but depending on your system it might also match > [Environment-linux-X86_64-D_SIMD-release-dmd] Then, this is still no good enough. If I put: [Environment] DFLAGS+=-L-lphobos2 [Environment-debug] DFLAGS+=-L-lphobos2-dbg Then when compiling with -debug I get 2 different libraries linked. If I leave [Environment] empty, then I can't link a program without using -debug.

(In reply to comment #23) > Then when compiling with -debug I get 2 different libraries linked. If I leave > [Environment] empty, then I can't link a program without using -debug. Mmh, the simplest solutions I can think of would be these. [Environment-debug] DFLAGS=-L-lphobos-dbg or [Environment] LIB=phobos2 [Environment-debug] LIB=phobos2-dbg [Environment] DFLAGS+=-L-l%LIB% (In reply to comment #22) > os-versions-debug/release-compiler. Actually os and compiler is already handled by versions, so we'd just need to match versions and debug/release.

(In reply to comment #24) > (In reply to comment #23) > > Then when compiling with -debug I get 2 different libraries linked. If I leave > > [Environment] empty, then I can't link a program without using -debug. > > Mmh, the simplest solutions I can think of would be these. > > [Environment-debug] > DFLAGS=-L-lphobos-dbg > > or > > [Environment] > LIB=phobos2 > > [Environment-debug] > LIB=phobos2-dbg > > [Environment] > DFLAGS+=-L-l%LIB% Interesting, so the file is interpreted in order and you can override variables like that and specify the same [Section] several times. Then again I'm out of technical arguments :)

I haven't yet got around to do this though the implementation is fairly simple and similar to the Environment64 change. https://github.com/D-Programming-Language/dmd/commit/190702057ff01819a55e8823fef7d3b5b2686cf2 It requires two passes over the command line and env arguments and should be something like this. The first scans any argument that affects version identifiers. Then we can process the inifile. Now the second pass processes the updated DFLAGS and any remaining argument.

Added industry keyword, it's starting to impact us seriously at Sociomantic.

(In reply to comment #27) > Added industry keyword, it's starting to impact us seriously at Sociomantic. Have you considered building a new runtime lib that includes the libs it depends on? It's not ideal, but it should solve the case where the runtime relies on eg librt and therefore needs it later on the command line.

(In reply to comment #28) > (In reply to comment #27) > > Added industry keyword, it's starting to impact us seriously at Sociomantic. > > Have you considered building a new runtime lib that includes the libs it > depends on? It's not ideal, but it should solve the case where the runtime > relies on eg librt and therefore needs it later on the command line. This is not actually the best option, because the runtime is statically compiled so we'd have to include the whole dependencies in the .a, and we lose dynamic linking, which mean we don't get security updates from the distribution anymore unless we recompile our stuff with a new version. I think at least we reach an agreement for the solution to adopt, right? If that the case I might give it another shot to the implementation. But I don't want to come up with another PR that gets rejected.

This is necessary to move the hardcoded link flags into the config file so that dmd package maintainers can adjust them to different platforms. Therefor I'm absolutely supporting the proposed implementation. Matching version flags already allows fine grained yet simple configurations in dub. http://code.dlang.org/package-format#build-settings It should work out well for dmd too.

(In reply to comment #29) > > This is not actually the best option, Yes, just a workaround for you until we get a real fix. > because the runtime is statically > compiled so we'd have to include the whole dependencies in the .a, and we lose > dynamic linking, which mean we don't get security updates from the distribution > anymore unless we recompile our stuff with a new version. > This is not true. You still get dynamic linking. The dmd-produced command line is something like this: g++ prog.obj -lstuffprogneeds -lphobos2 But you want g++ prog.obj -lstuffprogneeds -lphobos2 -lstuffphobosneeds A little bit of 'ar' magic will produce a new static lib that contains everything in libphobos2.a and everything in libstuffphobosneeds.a You can this pass this to dmd via -debuglib/-defaultlib and get (almost) the same thing as the above command line. > I think at least we reach an agreement for the solution to adopt, right? If > that the case I might give it another shot to the implementation. But I don't > want to come up with another PR that gets rejected. Sure. I was just giving a workaround that doesn't require changing the compiler.

(In reply to yebblies from comment #31) > Yes, just a workaround for you until we get a real fix. Possibly a better workaround is a custom CC which prepends --no-as-needed onto gcc's command line: https://issues.dlang.org/show_bug.cgi?id=12572#c5

A start: https://github.com/D-Programming-Language/dmd/pull/4646

BTW, it is possible now to select different configuration files entirely with the -conf=filename switch.

(In reply to Walter Bright from comment #34) > BTW, it is possible now to select different configuration files entirely > with the -conf=filename switch. How is the PR and this addressing this issue? I might be missing something because I don't see the relation :-/

(In reply to Leandro Lucarella from comment #35) > (In reply to Walter Bright from comment #34) > > BTW, it is possible now to select different configuration files entirely > > with the -conf=filename switch. > > How is the PR and this addressing this issue? I might be missing something > because I don't see the relation :-/ It's a step towards implementing comment #16

(In reply to yebblies from comment #36) > (In reply to Leandro Lucarella from comment #35) > > (In reply to Walter Bright from comment #34) > > > BTW, it is possible now to select different configuration files entirely > > > with the -conf=filename switch. > > > > How is the PR and this addressing this issue? I might be missing something > > because I don't see the relation :-/ > > It's a step towards implementing comment #16 OK, there was still one problem with that approach, but I guess, fair enough.

Also see issue 13324 – dynamically load libcurl at runtime.

Here's my evil plan: Add a switch -confsection=sectionname which will cause [sectionname] in the config file to be parsed. There can be multiple such switches and sections, they'll be parsed in lexical order. Note that: DFLAGS += string can be handled without adding new features with: DFLAGS = %DFLAGS% string The current way the config file is read twice wreaks havoc with the order in which sections are processed and things like appending to existing environment variables. This all has to be redone, but I'd like a green light on it being an acceptable resolution of this bugzilla issue before investing the effort. (Currently order of evaluation is undocumented.) I decided not to have -version=identifier do this, because inevitably someone will have a problem with it and want another switch to be able to do them independently. Calling it -confsection will place it next to the -conf switch in the documentation, making the connection easily discoverable.

(In reply to Walter Bright from comment #39) > Calling it -confsection will place it next to the -conf switch in the > documentation, making the connection easily discoverable. Will -confsection=debug be automatically added when -debug is specified? If yes (and there will be no more -llib flags automatically added at a certain place hardcoded in dmd), then I think this solves the problem at hand. For cross-compiling it might be useful to also automatically enable other predefined sections, similar to -version, like linux, x86_64, etc., which is what I think Marting was suggesting. But that would be an extra feature.

(In reply to Leandro Lucarella from comment #40) > Will -confsection=debug be automatically added when -debug is specified? No. You'll have to add -confsection=debug to have a [debug] section parsed. We can always add such a feature later, but it would be very hard to subtract it if it turns out to not be right. > If yes (and there will be no more -llib flags automatically added at a certain > place hardcoded in dmd), then I think this solves the problem at hand. I don't see any reason to add more. > For cross-compiling it might be useful to also automatically enable other > predefined sections, similar to -version, like linux, x86_64, etc., which is > what I think Marting was suggesting. But that would be an extra feature. I'd rather do without such, preferring to keep things simple and easy to explain.

(In reply to Walter Bright from comment #41) > (In reply to Leandro Lucarella from comment #40) > > Will -confsection=debug be automatically added when -debug is specified? > > No. You'll have to add -confsection=debug to have a [debug] section parsed. > We can always add such a feature later, but it would be very hard to > subtract it if it turns out to not be right. How is this then different from just specifying a whole new config file? > > For cross-compiling it might be useful to also automatically enable other > > predefined sections, similar to -version, like linux, x86_64, etc., which is > > what I think Marting was suggesting. But that would be an extra feature. > > I'd rather do without such, preferring to keep things simple and easy to > explain. Same question as above. The whole point of this issue, IMHO, is to select different config options (specifically link flags) automatically depending on how you are building your project. The user shouldn't care about the runtime needing different flags when using -debug or not. The user should be required to just use -debug when he wants to use the debug version of the library, not -debug -confsection=debug, that's overly cryptic IMHO. And not (much) better than what we have now (-debug -config=/whatever/config.debug.ini).

I see two distinct issues here: 1) getting rid from automatic linker flags inserted by compiler and make it fully controllable by packager (preferrably via dmd.conf) instead. This is actual issue that needs to be solved 2) allowing to specify distinct debug library in dmd.conf - this is only needed to enable interface compatibility after (1) is fixed Last proposal seems to address (2) perfectly but not (1) which was the main issue here (because manual intervention with adding compiler flags to build system is required)

> How is this then different from just specifying a whole new config file? Because multiple sections can be selected, rather than having a separate config file for each combination of sections, which would be a combinatorial explosion. > The user should be required to just use -debug when he wants to use the debug version of the library, I agree, but I'd like to try this out first to see what the common usage patterns will be, and then adding those later. If we add them now, we may wind up with a bunch of useless clutter that we cannot remove because we don't want to break legacy makefiles. > getting rid from automatic linker flags inserted by compiler and make it fully controllable by packager (preferrably via dmd.conf) instead. This is actual issue that needs to be solved Linker flags can already be specified with the -L switch. The trouble is the order of them. I just do not have a good idea for that, and yet still have the compiler conveniently generate them. It's also good to keep in mind that all dmd does is generate a command line for gcc - that can always be done by the user. Pull 497 does add a switch to add linker switches in front, but I feel this is simply doomed - what happens when a linker switch is needed in the middle?

Leandro is currently on vacation. I suggest to pause this until he is back.

(In reply to Dicebot from comment #45) > Leandro is currently on vacation. I suggest to pause this until he is back. Sure, and meanwhile there's a pull request we can talk about whenever everyone is ready.

(In reply to Walter Bright from comment #44) > > How is this then different from just specifying a whole new config file? > > Because multiple sections can be selected, rather than having a separate > config file for each combination of sections, which would be a combinatorial > explosion. I don't see how this could explode, as you only need 2: normal and debug. But maybe I'm just looking at my limited use case... > > The user should be required to just use -debug when he wants to use the debug version of the library, > > I agree, but I'd like to try this out first to see what the common usage > patterns will be, and then adding those later. If we add them now, we may > wind up with a bunch of useless clutter that we cannot remove because we > don't want to break legacy makefiles. OK, I can see your concern and conservative approach to the problem. In our particular case, is not something we can test really, to test it you are breaking all our makefiles anyway, as we need to add the new extra flags for the case where we want to build a debug version, so the "do not break the users build system" is not working very well with this approach :) > > getting rid from automatic linker flags inserted by compiler and make it fully controllable by packager (preferrably via dmd.conf) instead. This is actual issue that needs to be solved > > Linker flags can already be specified with the -L switch. The trouble is the > order of them. I just do not have a good idea for that, and yet still have > the compiler conveniently generate them. It's also good to keep in mind that > all dmd does is generate a command line for gcc - that can always be done by > the user. Again, this is not a satisfactory solution. We are not talking about users code here, we are using about the runtime. You can't ask the user to link manually just because the compiler can't let the runtime pick the right linker flags to compile a simple hello world. > Pull 497 does add a switch to add linker switches in front, but I feel this > is simply doomed - what happens when a linker switch is needed in the middle? Yeah, I agree the current approach suggested here is superior, is just for me the solution is not that useful unless you have a section automatically selected when -debug is used. Let's recap the current problems/facts: * The compiler automatically adds some -llib to the linker command-line, hardcoded with a particular order. This is not good because it highly depends on the runtime implementation, so it shouldn't be done by the compiler. * The compiler links against 2 different libraries depending on whether -debug was specified or not (and according to -defaultlib and -debuglib). So if the runtime uses different libraries in debug and default mode, it might need to pass different linker flags. About the solution, I think the requirements are: * There should be no user intervention to be able to compile a program only depending on the runtime/base library, either using -debug or not (all the linker flags configuration should be set by the runtime developer/packager via configuration files). Do we at least agree on these basic problems and requirements for a solution?

Great to see some momentum for this. We also need something like this to select different libs based on m32coff or the installed VC version, see https://github.com/D-Programming-Language/druntime/pull/1341.

Leandro is on (quite long) sick leave right now. I will start poking Walter again about this issue once he is back.

I think requiring a `-confsection` switch limits the usefulness a lot. One reason for the extended ini format is that we can make the linking configurable for certain targets, e.g. m32coff or msvc14 (see [here](https://github.com/D-Programming-Language/druntime/pull/1341#issuecomment-129163526)) without hardcoding those rules in the compiler. This gives packagers the necessary control to adapt dmd to every platform and provide some extras, such as a debug phobos library. Currently it's not possible to select a debug phobos lib solely by `-debug`. We have similar requirements for `m32coff` and VS2015. The other reason is similar but focused on letting the user select different configurations. This is already solved by the `-conf=` switch, but having the compiler match sections would reduce the necessary number of dmd.conf files. We already have a proved implementation to select configurations based on compiler flags and targets. http://code.dlang.org/package-format?lang=json#build-settings Matching "version" identifiers is a generic solution that's both simple and flexible, e.g. it readily allows to match `[Environment-D_HardFloat-ARM_Thumb]` or `[Environment-linux-D_NoBoundsChecks-D_InlineAsm_X86_64]`. Maybe we don't need that much flexibility but `Environment32/Environment64/Environment32mscoff` already don't scale with a adding a second orthogonal switch for [MSVC14](https://github.com/D-Programming-Language/druntime/pull/1341#issuecomment-129168177). > I agree, but I'd like to try this out first to see what the common usage patterns will be, and then adding those later. If we add them now, we may wind up with a bunch of useless clutter that we cannot remove because we don't want to break legacy makefiles. That's a noble effort, but it's already clear that user selected config sections aren't the right solution to ship a debug/profile/msvc14 phobos variant. Also trying to find out usage patterns by providing an inferior feature is flawed, because you don't necessarily get feedback and it might get less or different acceptance as the real feature. In this case the requirements are already know, so if you want to know the usage patterns just ask the packagers that need to deal with dmd.conf limitations.

Sorry for taking over this bug report, none of the dmd.conf enhancements address the original issue. Linking is already fully configurable since we have both the `-defaultlib=` and the `-conf=` switch. Problem is that this discards all configuration, so you need to pass import and library paths to dmd, making this unportable and cumbersome. Regarding the linker order for curl, this is a bug on the side of dmd/phobos to not specify the correct dependencies and shouldn't require a user side flag. We could solve this by always adding -L--as-needed -L-lcurl (which requires libcurl-dev packages) or by dynamically loading libcurl (see issue 13324). The later would also solve the annoying issue with versioned libcurl symbols in a shared libphobos2.so.

(In reply to Martin Nowak from comment #52) > We could solve this by always adding -L--as-needed -L-lcurl (which requires > libcurl-dev packages) In the meantime LDC took this approach. https://github.com/ldc-developers/ldc/pull/977

*** Issue 12572 has been marked as a duplicate of this issue. ***

THIS ISSUE HAS BEEN MOVED TO GITHUB https://github.com/dlang/dmd/issues/18385 DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB