Bug 15459 – [REG2.065.0] stdin.byLine.each!(map!somefunc) compiles, fails to link with ld

http://dpaste.dzfl.pl/ce501c212d32 This code calls byLine, each, map!somefunc. It compiles fine, but fails to link. Tested on my Linux machine with DMD 2.069.2 and on dpaste with DMD 2.069.1. Expected: Either an error during compilation, or a well-linked executable. Code copied here for convenience: #!/usr/bin/rdmd import std.stdio; import std.algorithm; char somefunc(in char c) { return c; } void main() { stdin .byLine .each!(map!somefunc); } dmd outputs: /d773/f942.o: In function `_D3std9algorithm9iteration32__T3mapS21_D4f9428somefuncFxaZaZ11__T3mapTAaZ3mapFNaNbNiNfAaZS3std9algorithm9iteration41__T9MapResultS21_D4f9428somefuncFxaZaTAaZ9MapResult': /d773/f942.d:(.text._D3std9algorithm9iteration32__T3mapS21_D4f9428somefuncFxaZaZ11__T3mapTAaZ3mapFNaNbNiNfAaZS3std9algorithm9iteration41__T9MapResultS21_D4f9428somefuncFxaZaTAaZ9MapResult+0x30): undefined reference to `_D3std9algorithm9iteration41__T9MapResultS21_D4f9428somefuncFxaZaTAaZ9MapResult6__ctorMFNaNbNcNiNfAaZS3std9algorithm9iteration41__T9MapResultS21_D4f9428somefuncFxaZaTAaZ9MapResult' collect2: error: ld returned 1 exit status --- errorlevel 1

When you're iterating any kind of string (char, wchar or dchar based) the element type is always dchar. This special case for narrow strings (UTF-8 and UTF-16) is also known as UTF autodecoding. You can see this for yourself if you change `somefunc` into a template and add a pragma statement to make the compiler emit an information message about the type of `c` for each instantiation of the function: auto somefunc(C)(C c) { pragma (msg, C); return c; } See also: http://dlang.org/phobos/std_range_primitives.html#.ElementType, http://dlang.org/phobos/std_range_primitives.html#.front and http://dlang.org/glossary.html#narrow strings This is still a valid bug report, because the compiler should refuse to compile this code.

Thanks for the thorough explanation!

The example presented is invalid, as the parameter of each is not expected to return anything, and map is lazy and by itself does not mutate global state (is pure if its predicate is). It also doesn't compile any more in the latest version of D. I'm closing this for now, but please reopen if you can provide a working practical example.

Vladimir, shouldn't this bug be closed as "WORKSFORME", as the bug was that the compiler accepted the invalid example in the first place?

(In reply to ZombineDev from comment #4) > Vladimir, shouldn't this bug be closed as "WORKSFORME", as the bug was that > the compiler accepted the invalid example in the first place? Well, the code wasn't technically invalid, just nonsensical. It stopped working after this PR: https://github.com/dlang/phobos/pull/5148 I'm not sure if that behavior change was intentional either, hence my comment there. There is also no practical difference between INVALID and WORKSFORME in practice, I think.

(In reply to Vladimir Panteleev from comment #3) > I'm closing this for now, but please reopen if you can provide a working > practical example. Reduced accepts-invalid: ---- enum e = is(typeof(map())); struct MapResult() { this(char[] a) {} char front() { dchar d; return d; /* should fail compilation */ } } void map()() { auto mr = MapResult!()([]); } void main() { map(); }

(In reply to ag0aep6g from comment #6) > (In reply to Vladimir Panteleev from comment #3) > > I'm closing this for now, but please reopen if you can provide a working > > practical example. > > Reduced accepts-invalid: This is not a valid reduction because .front is never called. The compiler seems to be skipping semantic analysis of uncalled functions here to avoid forward reference issues: https://github.com/dlang/dmd/pull/2851

(In reply to Vladimir Panteleev from comment #7) > This is not a valid reduction because .front is never called. front is never called in the original code either. > The compiler > seems to be skipping semantic analysis of uncalled functions here to avoid > forward reference issues: https://github.com/dlang/dmd/pull/2851 So the invalid front is supposed to be accepted (if it's never called)? Then I suppose the accepts-invalid aspect of this issue is WONTFIX. But it should link then, no?

(In reply to ag0aep6g from comment #8) > So the invalid front is supposed to be accepted (if it's never called)? Then > I suppose the accepts-invalid aspect of this issue is WONTFIX. Forward references and generally order of semantic analysis is a tricky subject and I'm not qualified to answer with authority whether the behaviour here can actually be called a bug or not, however I do know that we generally want the compiler to be as lazy as possible/reasonable for performance reasons. For example, currently unit tests are not even parsed without unittest, so you can put any series of tokens in an unittest block as long as the curly braces are balanced. So, even if there is a bug here, fixing it would not align with long-term goals. > But it should link then, no? Oh right, I missed that the test case you posted also exhibits the link failure.

Regression (changing the compilation error into a link error + possible accepts-invalid) introduced in https://github.com/dlang/dmd/pull/2851

(In reply to Simon Naarmann from comment #0) > http://dpaste.dzfl.pl/ce501c212d32 > > This code calls byLine, each, map!somefunc. It compiles fine, but fails to > link. Tested on my Linux machine with DMD 2.069.2 and on dpaste with DMD > 2.069.1. > > Expected: Either an error during compilation, or a well-linked executable. > > Code copied here for convenience: > > #!/usr/bin/rdmd > > import std.stdio; > import std.algorithm; > > char somefunc(in char c) > { > return c; > } > > void main() > { > stdin > .byLine > .each!(map!somefunc); > } > > dmd outputs: I get: test.d(13): Error: template `std.algorithm.iteration.each!(map).each` cannot deduce function from argument types `!()(ByLine!(char, char))`, candidates are: c:\cbx\mars\phobos\std\algorithm\iteration.d(890): `std.algorithm.iteration.each!(map).each(Range)(Range r) if (!isForeachIterable!Range && (isRangeIterable!Range || __traits(compiles, typeof(r.front).length)))` c:\cbx\mars\phobos\std\algorithm\iteration.d(925): `std.algorithm.iteration.each!(map).each(Iterable)(auto ref Iterable r) if (isForeachIterable!Iterable || __traits(compiles, Parameters!(Parameters!(r.opApply))))` So it works for me.

(In reply to Walter Bright from comment #11) > So it works for me. The code from the opening post now fails to compile in DMD v2.083.1 and the linker won't be called. That's good. But ag0aep6g's reduced case from Comment 6 -- https://issues.dlang.org/show_bug.cgi?id=15459#c6 -- still compiles with DMD v2.083.1 on 64-bit Linux, then fails to link. Linker error below. Expected instead: That code should fail compilation, as annotated via /* should fail compilation */. /usr/bin/ld: a.o: in function `_D1a__T3mapZQfFNaNbNiNfZv': a.d:(.text._D1a__T3mapZQfFNaNbNiNfZv[_D1a__T3mapZQfFNaNbNiNfZv]+0x27): undefined reference to `_D1a__T9MapResultZQl6__ctorMFNaNbNcNiNfAaZSQBp__TQBqZQBu' collect2: error: ld returned 1 exit status Error: linker exited with status 1 Or should we file a separate bug for that?

The Comment 6 example still fails to compile. It looks like the speculative compilation in the `is(typeof(map))` wound up hiding the error in somefunc().

@RazvanN7 created dlang/dmd pull request #13359 "Fix Issue 15459 - [REG2.065.0] stdin.byLine.each!(map!somefunc) compiles, fails to link with ld" fixing this issue: - Fix Issue 15459 - [REG2.065.0] stdin.byLine.each\!(map\!somefunc) compiles, fails to link with ld https://github.com/dlang/dmd/pull/13359

*** Issue 16481 has been marked as a duplicate of this issue. ***

*** Issue 19545 has been marked as a duplicate of this issue. ***

*** Issue 20237 has been marked as a duplicate of this issue. ***

*** Issue 24282 has been marked as a duplicate of this issue. ***

THIS ISSUE HAS BEEN MOVED TO GITHUB https://github.com/dlang/dmd/issues/19078 DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB