Bug 19097 – Extend Return Scope Semantics

Extend return semantics so that a 'return' parameter for void or constructor functions applies to the first parameter, if it is a ref or out parameter. For a member function, the first parameter is 'this'. Currently, a 'return' parameter transfers its lifetime to the function return value: ``` int* frank(return scope int* p) { return p; } int* p; int i; int* q; q = frank(&i); // ok p = frank(&i); // error ``` However, the following just issues errors: ``` @safe void betty(ref scope int* r, return scope int* p) { r = p; } // (1) error int* p; int i; int* q; betty(q, &i); // (2) ok betty(p, &i); // (3) should be error ``` Marking `betty` as `@trusted` resolves (1), but does not detect the difference between (2) and (3). Hence, the callers of `betty` would have to be marked `@trusted` as well. This situation comes up repeatedly with: 1. constructors 2. property setters 3. put(dest, source) functions Annotating a parameter with `return` has been quite successful at tracking scope dependencies from the parameter to the function return value. Extending this in cases where there is no return value to the first parameter resolves the issue identified above. The scope check for: ``` void betty(ref scope int* r, return scope int* p); betty(q, &i); ``` is as if `q = &i;` was encountered. This is not new syntax, but new semantics for cases that were compile-time errors before (i.e. annotating parameters with `return` when there was no return value).

Trying to dissect Walter's back-of-the-napkin description: Example 1 --------- @safe: int* frank(return scope int* p) { return p; } void main() { // lifetimes end in reverse order from which they are declared int* p; // `p`'s lifetime is longer than `i`'s int i; // `i`'s lifetime is longer than `q`'s int* q; // `q`'s lifetime is the shortest q = frank(&i); // ok because `i`'s lifetime is longer than `q`'s p = frank(&i); // error because `i`'s lifetime is shorter than `p`'s } This works in the compiler today if both functions are declared @safe and if compiled with -dip1000: https://run.dlang.io/is/CZ3YuU

Example 2 --------- @safe: void betty(ref scope int* r, return scope int* p) { r = p; // (1) Error: scope variable `p` assigned to `r` with longer lifetime } void main() { int* p; int i; int* q; betty(q, &i); // (2) ok betty(p, &i); // (3) should be error } Compile with -dip1000: https://run.dlang.io/is/t6wj71 So, I'm assuming (1) should not be an error because it depends on the lifetimes of the arguments supplied by the caller. Correct?

> This situation comes up repeatedly with: > 1. constructors > 2. property setters > 3. put(dest, source) functions In other words 2 and 3 are functions that return `void`. Constructors are like `static` functions that return an object instance, so I'm not sure how the problem at hand applies to constructors. > Annotating a parameter with `return` has been quite successful at tracking scope dependencies from the parameter to the function return value. The problem with that is we don't have sufficient documentation describing the semantics of `return` parameters and their relationship with `scope`. So, I can't understand how `return` has been used in the past to solve such issues. So `betty` returns `void`, but a `return` parameter transfers its lifetime to the function return value. So I guess that's the nature of this issue. How does `return` apply to a function that has no `return` type?

(In reply to Mike Franklin from comment #4) > How does `return` apply to a function that has no `return` type? If the return type is 'void', and the first parameter is by 'ref', the 'return' applies to the first parameter. That's the whole thing in one sentence. (For member functions, the first parameter is the implicit 'this' reference.)

As I work to understand this proposal and evaluate it I've noticed it bears resemblance to Problem Case #3 in Rust's Non-Lexical Lifetimes design in 2016: http://smallcultfollowing.com/babysteps/blog/2016/04/27/non-lexical-lifetimes-introduction/#problem-case-3-conditional-control-flow-across-functions http://smallcultfollowing.com/babysteps/blog/2016/05/09/non-lexical-lifetimes-adding-the-outlives-relation/#problem-case-3-revisited

@Mike It applies to constructors in the same way it applies to `void` functions whose first argument is `ref` or `out`. The hidden first parameter to the constructor is a `ref` parameter: `this`. I think a better way to describe this issue is that first parameters that are `ref` or `out` (including `this` for constructors) should be considered and treated the same as return values for other functions.

Just throwing this out there, why do we need it to be the first parameter? Why not ALL ref parameters? It makes sense for a constructor, for which there is an obvious return expectation of the `this` parameter, but anyone can devise some calling scheme by which any arguments are transferred to any other ref arguments. Just because Phobos follows the convention of putting the "return" parameter as the first parameter, does that mean the language should require it?

(In reply to Steven Schveighoffer from comment #8) > Just because Phobos follows the convention of putting the "return" parameter > as the first parameter, does that mean the language should require it? I don't think that's necessarily true. Take a look at https://dlang.org/phobos/std_algorithm_mutation.html#copy `TargetRange copy(SourceRange, TargetRange)(SourceRange source, TargetRange target)` Is copy an exception? I recently asked about this convention at https://github.com/dlang/druntime/pull/2264#pullrequestreview-143076892

Some comments from the forum worth visiting with regard to this proposal: https://forum.dlang.org/post/[email protected] https://forum.dlang.org/post/[email protected]

@safe void betty(bool b, ref scope int* r, return scope int* p) { if (b) r = p; } betty("", q, &i); This is not going to work, I take it? Is that desirable?

I'm wondering if it might be possible to do something like this: ``` void betty(ref scope int* r, return(r) scope int* p) { r = p; } ``` `return(r)` explicitly ties p's lifetime to r's. So, the `return` attribute would take an argument specifying which "output" parameter to tie the input lifetime to, and the semantics no longer rely on convention or the order of arguments.

For constructors, one could use `return(this)` to the same effect. Perhaps for constructors, if there are no other "output" parameters in the signature, `return(this)` could be inferred by simply typing add `return` to the input parameters. If there is more than one "output" parameter, the user would have to explicitly disambiguate by adding an argument to the `return` attribute. struct S { // does `return` apply to `this` or `r`? this(ref scope int* r, return scope int* p); // disambiguate by explicitly adding an argument to `return` this(ref scope int* r, return(this) scope int* p); // or this(ref scope int* r, return(r) scope int* p); }

Here's a little more about what I'm thinking: struct S { // Error: `return` must be disambiguated between `this` and `r` this(ref scope int* r, return scope int* p); // OK, `return` has been disambiguated this(ref scope int* r, return(this) scope int* p); this(ref scope int* r, return(r) scope int* p); // OK, only one possible *output* int* f(return scope int* p); // OK, only one possible *output* void f(ref scope int* r, return scope int* p); // Error: `return` must be disambiguated between `this` and `r` void f(ref scope int* r1, ref scope int* r2, return scope int* p); // OK, `return` has been disambiguated void f(ref scope int* r1, ref scope int* r2, return(r1) scope int* p); void f(ref scope int* r1, ref scope int* r2, return(r2) scope int* p); // Error: `return` must be disambiguated between the return value // and the `r` output parameter. int* f(ref scope int* r, return scope int* p); // OK, `return` has been disambiguated int* f(ref scope int* r, return(r) scope int* p); // OK, `return` has been disambiguated // `return(return)` is pretty yucky, but I can't think of anything // else right now. int* f(ref scope int* r, return(return) scope int* p); }

I think in my prior example, I may have overlooked the implicit `this` argument for member functions. Here I attempt to compensate. //------------------------------------------------ // OK, only one possible *output* int* f(return scope int* p); // OK, only one possible *output* void f(ref scope int* r, return scope int* p); //------------------------------------------------ // Error: `return` must be disambiguated between `this` and `r` void f(ref scope int* r1, ref scope int* r2, return scope int* p); // OK, `return` has been disambiguated void f(ref scope int* r1, ref scope int* r2, return(r1) scope int* p); void f(ref scope int* r1, ref scope int* r2, return(r2) scope int* p); //------------------------------------------------ // Error: `return` must be disambiguated between the return value // and the `r` output parameter. int* f(ref scope int* r, return scope int* p); // OK, `return` has been disambiguated int* f(ref scope int* r, return(r) scope int* p); // OK, `return` has been disambiguated // `return(return)` is pretty yucky, but I can't think of anything // else right now. int* f(ref scope int* r, return(return) scope int* p); struct S { // Error: `return` must be disambiguated between `this` and `r` this(ref scope int* r, return scope int* p); // OK, `return` has been disambiguated this(ref scope int* r, return(this) scope int* p); this(ref scope int* r, return(r) scope int* p); //------------------------------------------------ // Error: `return` must be disambiguated between `this` and `r` void f(ref scope int* r, return scope int* p); // OK, `return` has been disambiguated void f(ref scope int* r, return(this) scope int* p); // OK, `return` has been disambiguated void f(ref scope int* r, return(r) scope int* p); //------------------------------------------------ // Error: `return` must be disambiguated between `this`, `r1`, and `r2` void f(ref scope int* r1, ref scope int* r2, return scope int* p); // OK, `return` has been disambiguated void f(ref scope int* r1, ref scope int* r2, return(r1) scope int* p); void f(ref scope int* r1, ref scope int* r2, return(r2) scope int* p); //------------------------------------------------ // Error: `return` must be disambiguated between `this` and the return value int* f(return scope int* p); // OK, `return` has been disambiguated int* f(return(this) scope int* p); // OK, `return` has been disambiguated - tied to return value int* f(return(return) scope int* p); } Hopefully, you get the idea.

Partial: https://github.com/dlang/dlang.org/pull/2535

Commits pushed to master at https://github.com/dlang/dlang.org https://github.com/dlang/dlang.org/commit/6299e9193986cb7f9accf1a3059115145b8a938d add initial documentation for issue 19097 https://github.com/dlang/dlang.org/commit/140267d3cf3b6ea4977853a0c017f5f81b3ce287 Merge pull request #2535 from WalterBright/fix19097-1 add initial documentation for issue 19097 merged-on-behalf-of: Nicholas Wilson <[email protected]>

Commits pushed to master at https://github.com/dlang/dmd https://github.com/dlang/dmd/commit/326e50b04a69e3eaebab54229ae44c4ac60579f2 fix Issue 19097 - Extend Return Scope Semantics https://github.com/dlang/dmd/commit/4719804f54d67e51a4fb8cb413c5aa9506126f9f Merge pull request #8504 from WalterBright/fix19097 fix Issue 19097 - Extend Return Scope Semantics