Bug 7016 – local import does not create -deps dependency

a.d : import b; void main() { f(); } b.d : int f() { import c; return i; } c.d : int i = 42; The command $ dmd -deps=a.deps -o- a.d creates: a (a.d) : public : object (C:\\Soft\\dmd2d\\windows\\bin\\..\\..\\import\\druntime\\object.di) b (b.d) : public : object (C:\\Soft\\dmd2d\\windows\\bin\\..\\..\\import\\druntime\\object.di) a (a.d) : public : b (b.d)

*** Issue 8858 has been marked as a duplicate of this issue. ***

*** Issue 8856 has been marked as a duplicate of this issue. ***

Apparently it's because semantic isn't run on the dependencies. It's in Import::semantic where the import is printed. You can verify this with that -rb feature.

Right, semantic3 for imported functions is only run during the inline pass.

I am not sure this really needs to be fixed: b.d either needs to be built at some point, then the dependency is printed and can be gathered by tools or it does not get built, in this case it does not matter either, because any use of the function would need to be inlined, causing the dependency to be printed. If the function is not used at all, then the dependency also does not matter.

jfanatiker, the compiler is supposed to print the dependencies between all modules when given just the starting point. This is the purpose of the -deps switch, it is not intended to be run once for every module in the program. The same goes for the verbose output (-v).

(In reply to comment #6) > jfanatiker, the compiler is supposed to print the dependencies between all > modules when given just the starting point. This is the purpose of the -deps > switch, it is not intended to be run once for every module in the program. The > same goes for the verbose output (-v). I see. You are right of course, I viewed it from an incremental build tool perspective, where it would not matter, but for rdmd this bug is of course a problem.

see http://forum.dlang.org/post/[email protected] [fix for enumerating local import dependencies, resolving all rdmd link errors]

(In reply to comment #0) > a.d : import b; void main() { f(); } > b.d : int f() { import c; return i; } > c.d : int i = 42; > > The command > $ dmd -deps=a.deps -o- a.d > creates: > a (a.d) : public : object > (C:\\Soft\\dmd2d\\windows\\bin\\..\\..\\import\\druntime\\object.di) > b (b.d) : public : object > (C:\\Soft\\dmd2d\\windows\\bin\\..\\..\\import\\druntime\\object.di) > a (a.d) : public : b (b.d) Problem is still here, with 2.065alpha. The output now looks like: a (a.d) : private : object (/home/dmitry/dmd2/linux/bin64/../../src/druntime/import/object.di) b (b.d) : private : object (/home/dmitry/dmd2/linux/bin64/../../src/druntime/import/object.di) a (a.d) : private : b (b.d)

Added $25 bounty: https://www.bountysource.com/issues/3785375-local-import-does-not-create-deps-dependency

*** Issue 14118 has been marked as a duplicate of this issue. ***

DMD 2.070.0-b1 Windows, problem still here :( RDMD affected by this bug.

So is this on Windows only? Just confirming.

It happens on all platforms.

all platforms. This is the #1 problem with rdmd

Thanks. Razvan will be on this.

Cannot reproduce on Ubuntu 16.04, latest version of compiler. Hopefully I will try on a windows machine when I get the chance

(In reply to RazvanN from comment #17) > Cannot reproduce on Ubuntu 16.04, latest version of compiler. Hopefully I > will try on a windows machine when I get the chance What have you tried? I can reproduce it exactly as per the original description from 2011.

I ran the exact example as the one in 2011 with the exact command and the output is: (a.d) : private : object (..) (b.d) : private : object (..) (a.d) : private : b (b.d) (c.d) : private : object (..) (a.d) : private : c (c.d) It seems to me that it prints all the dependencies of a.d

(In reply to RazvanN from comment #19) > I ran the exact example as the one in 2011 with the exact command and the > output is: > > (a.d) : private : object (..) > > (b.d) : private : object (..) > (a.d) : private : b (b.d) > (c.d) : private : object (..) > (a.d) : private : c (c.d) > > It seems to me that it prints all the dependencies of a.d Strange... I can't get that result with DMD 2.072.1 or latest git master (on Linux). What compiler version did you use, exactly?

(In reply to Vladimir Panteleev from comment #20) > (In reply to RazvanN from comment #19) > > I ran the exact example as the one in 2011 with the exact command and the > > output is: > > > > (a.d) : private : object (..) > > > > (b.d) : private : object (..) > > (a.d) : private : b (b.d) > > (c.d) : private : object (..) > > (a.d) : private : c (c.d) > > > > It seems to me that it prints all the dependencies of a.d > > Strange... I can't get that result with DMD 2.072.1 or latest git master (on > Linux). What compiler version did you use, exactly? DMD64 D Compiler v2.073.0-devel-878b882

>dmd --version DMD32 D Compiler v2.072.0-master-386ef6d Copyright (c) 1999-2016 by Digital Mars written by Walter Bright Windows 7 32 bit exact example from first post produce a.deps: a (a.d) : private : object (R:\\dmd2\\windows\\bin\\..\\..\\src\\druntime\\import\\object.d) b (b.d) : private : object (R:\\dmd2\\windows\\bin\\..\\..\\src\\druntime\\import\\object.d) a (a.d) : private : b (b.d)

Sorry, loaded latest >dmd --version DMD32 D Compiler v2.072.1 Copyright (c) 1999-2016 by Digital Mars written by Walter Bright Windows 7 32 bit exact example from first post produce a.deps: a (a.d) : private : object (R:\\dmd2\\windows\\bin\\..\\..\\src\\druntime\\import\\object.d) b (b.d) : private : object (R:\\dmd2\\windows\\bin\\..\\..\\src\\druntime\\import\\object.d) a (a.d) : private : b (b.d)

same result on nightly DMD32 D Compiler v2.073.0-master-cb7f8fe

(In reply to RazvanN from comment #17) > Cannot reproduce on Ubuntu 16.04, latest version of compiler. I can reproduce this on a brand new Ubuntu 16.04 amd64 installation. Please see the log below. Something on your system must be causing you to get a different result, please try in a new chroot or virtual machine. -------------------------------------------------------------------------- ~/tmp/2016-12-30b $ sudo debootstrap --arch amd64 xenial xenial http://archive.ubuntu.com/ubuntu/ [...] sudo debootstrap --arch amd64 xenial xenial http://archive.ubuntu.com/ubuntu/ 24.10s user 3.41s system 35% cpu 1:17.96 total ~/tmp/2016-12-30b $ sudo systemd-nspawn -D xenial Spawning container xenial on /home/vladimir/tmp/2016-12-30b/xenial. Press ^] three times within 1s to kill container. root@xenial:~# apt install curl xz-utils [...] root@xenial:~# curl -fsS https://dlang.org/install.sh | bash -s dmd ######################################################################## 100.0% The latest version of this script was installed as ~/dlang/install.sh. It can be used it to install further D compilers. Run `~/dlang/install.sh --help` for usage information. Downloading and unpacking http://downloads.dlang.org/releases/2.x/2.072.1/dmd.2.072.1.linux.tar.xz ######################################################################## 100.0% Downloading https://dlang.org/d-keyring.gpg ######################################################################## 100.0% Downloading and unpacking http://code.dlang.org/files/dub-1.1.1-linux-x86_64.tar.gz ######################################################################## 100.0% Run `source ~/dlang/dmd-2.072.1/activate` in your shell to use dmd-2.072.1. This will setup PATH, LIBRARY_PATH, LD_LIBRARY_PATH, DMD, DC, and PS1. Run `deactivate` later on to restore your environment. root@xenial:~# source ~/dlang/dmd-2.072.1/activate (dmd-2.072.1)root@xenial:~# echo 'import b; void main() { f(); }' > a.d (dmd-2.072.1)root@xenial:~# echo 'int f() { import c; return i; }' > b.d (dmd-2.072.1)root@xenial:~# echo 'int i = 42;' > c.d (dmd-2.072.1)root@xenial:~# dmd -deps=a.deps -o- a.d (dmd-2.072.1)root@xenial:~# cat a.deps a (a.d) : private : object (/root/dlang/dmd-2.072.1/linux/bin64/../../src/druntime/import/object.d) b (b.d) : private : object (/root/dlang/dmd-2.072.1/linux/bin64/../../src/druntime/import/object.d) a (a.d) : private : b (b.d) (dmd-2.072.1)root@xenial:~#

I confirm this is still broken with dmd HEAD DMD64 D Compiler v2.073.0-devel-a2b772f RazvanN, are you sure you're not using rdmd ? Please re-run using -v and show your entire log. rdmd -deps=a.deps --force -c a.d && cat a.deps would indeed produce a line "c (c.d) : private : object" dmd -deps=a.deps --force -c a.d && cat a.deps would currently NOT produce this line, which is what this bug is about. the reason rdmd produces the correct -deps here is because a.d contains top level "import b" so it'll also compile b.d and hence show c.d, but that won't work with more complex cases (eg if c.d contains: void test2(){import c2;}, it won't list c2.d) This is the biggest bug for d's build system.

Running semantic3 recursively on dependencies will end up in an infinite loop when there are circular imports. So, I propose the following solution: -> the current -deps implementation should remain untouched and this will print all the direct dependencies. -> add another compiler flag: "-rdeps" (or maybe a more inspired name) which will recurs through all the dependencies, excluding druntime or phobos imports. The rationale behind this is that both druntime and phobos have circular imports which will cause the recursive implementation to fail; also having just 1 phobos import will lead to a cascade of imports which will be hard to follow. Any better ideas? RazvanN

(In reply to RazvanN from comment #27) > -> add another compiler flag: "-rdeps" (or maybe a more inspired name) which > will recurs through all the dependencies, excluding druntime or phobos > imports. > The rationale behind this is that both druntime and phobos have circular > imports > which will cause the recursive implementation to fail; also having just 1 > phobos > import will lead to a cascade of imports which will be hard to follow. Hard-coding blacklists for druntime and phobos into the compiler sounds wrong. (People may need to build D against other libraries with the same issues, or build against Phobos from source - i.e. without linking to libphobos2.) If this is done, there would probably need to be ways to add to or remove from that list. Either way, it sounds like this will also result is a huge slowdown... > Any better ideas? Would it be a far stretch from this to add a switch to just do code generation whenever an import is done, whether it's in semantic3 or earlier? Meaning, move rdmd into dmd itself. I think all we need is a switch to enable this behavior (e.g. -r, for recursive compilation), and switches to add or remove from the list of packages that will be excluded from compilation (default being std.*, etc.* and core.*). This will make rdmd much faster, too, since it will no longer need to ask the compiler to perform semantics just to get a list of modules to compile and do it all over again during the real compilation.

Commit pushed to master at https://github.com/dlang/phobos https://github.com/dlang/phobos/commit/29273f261c94e1bbe1042ec58a362d70cb344188 remove .deps file generation - for development, people can use the much faster std/algorithm.test targets - for batch unittest recompiling isn't of much interest as all the testers just start from a clean slate anyhow - furthermore the .deps generation is broken anyhow, see Issue 7016

Commits pushed to master at https://github.com/dlang/dmd https://github.com/dlang/dmd/commit/6be32270b411d13f49ffccaa055b2cb13060c495 Fix Issue 7016 https://github.com/dlang/dmd/commit/afebe0c2ba89594b434d676fbe0c050389a5b48c Merge pull request #6748 from RazvanN7/Fix_Issue_7016 Fix Issue 7016 - local import does not create -deps dependency merged-on-behalf-of: Andrei Alexandrescu <[email protected]>

Commits pushed to stable at https://github.com/dlang/dmd https://github.com/dlang/dmd/commit/6be32270b411d13f49ffccaa055b2cb13060c495 Fix Issue 7016 https://github.com/dlang/dmd/commit/afebe0c2ba89594b434d676fbe0c050389a5b48c Merge pull request #6748 from RazvanN7/Fix_Issue_7016

Commit pushed to stable at https://github.com/dlang/phobos https://github.com/dlang/phobos/commit/29273f261c94e1bbe1042ec58a362d70cb344188 remove .deps file generation

Commits pushed to newCTFE at https://github.com/dlang/dmd https://github.com/dlang/dmd/commit/6be32270b411d13f49ffccaa055b2cb13060c495 Fix Issue 7016 https://github.com/dlang/dmd/commit/afebe0c2ba89594b434d676fbe0c050389a5b48c Merge pull request #6748 from RazvanN7/Fix_Issue_7016

Commit pushed to dmd-cxx at https://github.com/dlang/phobos https://github.com/dlang/phobos/commit/29273f261c94e1bbe1042ec58a362d70cb344188 remove .deps file generation

Nobody thought about incremental rebuilds with other build tools like make, cmake, or ninja, despite the argument being made in comment 5. Incremental rebuilds in those tools are now slower as the fix implies running semantic3 on all imports when collecting -deps, even though only shallow dependencies are needed. Furthermore the deps options isn't (and apparently never has been) used by rdmd. So leaving only a recursive dependency option in the compiler hasn't served any purpose. Furthermore the "fix" is broken since inception, see Issue 17898.