Bug 7543 – inout opApply should work properly

After fixing issue 7542, inout opApply should work on foreach. class C { int[] arr; this(int[] a){ arr = a; } int opApply(int delegate(ref inout(int)) dg) inout { foreach (ref e; arr) if (auto r = dg(e)) return r; return 0; } } void main() { size_t i; i = 0; foreach (e; new C([1,2,3])) { static assert(is(typeof(e) == int)); assert(e == ++i); e = 10; } assert(i == 3); i = 0; foreach (e; new const(C)([1,2,3])) { static assert(is(typeof(e) == const int)); assert(e == ++i); static assert(!__traits(compiles, e = 10)); } assert(i == 3); i = 0; foreach (e; new immutable(C)([1,2,3])) { static assert(is(typeof(e) == immutable int)); assert(e == ++i); static assert(!__traits(compiles, e = 10)); } assert(i == 3); }

Sorry, my thought in bug 7543 is not correct, so my pull is also bad.

We'll need a clear spec of what it means to have inout at two nesting levels of a function signature. digitalmars.D "inout and function/delegate parameters" On web interface: http://forum.dlang.org/thread/[email protected]

(In reply to comment #3) > We'll need a clear spec of what it means to have inout at two nesting levels of > a function signature. This follows naturally from variance rules: In-arguments are contravariant, so you can pass anything whose type is a subtype of the argument type. Then the usual rules of function subtyping apply (e.g. see bug 7676) So, if you have a function of type "R1 function(R2 delegate(inout(T1)) f)" you can pass anything for f that is a subtype of "R2 delegate(inout(T1))". For argument types to the delegate, the subtyping rules apply again. Since in-arguments are contravariant (and return types (and types of out-args) are covariant), any value of type "R delegate(T)" where (R <: R2 and inout(T1) <: T) is a subtype of "R2 delegate(inout(T1))" (<: is the subtype relation). Also the following subtyping rules for inout apply (R2 <: R1): R2 delegate(inout(T)) <: R1 delegate(immutable(T)) R2 delegate(inout(T)) <: R1 delegate(const(T)) R2 delegate(inout(T)) <: R1 delegate(T) And of course: R2 delegate(inout(T)) <: R1 delegate(inout(T)) From that follows (given two types A,B and B <: A): B delegate(immutable(A)) <: A delegate(inout(A)) Which is exactly what we want. (Analogous for functions. Btw, functions should be a subtype of delegates). Example: class A {} class B : A {} void foo(A function(A function(inout(A))) dg); B bar(A function(inout(A)) f); B baz(inout(A) a); foo(bar(baz(new immutable(A)))); foo(bar(baz(new const(A)))); foo(bar(baz(new A))); so it all works out nicely... To make this work in DMD, it has to support function subtyping according to variance rules. It's important that functions with an inout arg are subtypes of functions with an immutable/const/mutable arg (all other arg types / return type being equal) because the former can be substituted for one of the latter. (This issue is related to bug 7542.)

(In reply to comment #4) > (In reply to comment #3) > > We'll need a clear spec of what it means to have inout at two nesting levels of > > a function signature. > > This follows naturally from variance rules: > [snip.] I think you may misunderstand this issue. > Also the following subtyping rules for inout apply (R2 <: R1): > R2 delegate(inout(T)) <: R1 delegate(immutable(T)) > R2 delegate(inout(T)) <: R1 delegate(const(T)) > R2 delegate(inout(T)) <: R1 delegate(T) > And of course: > R2 delegate(inout(T)) <: R1 delegate(inout(T)) > > From that follows (given two types A,B and B <: A): > B delegate(immutable(A)) <: A delegate(inout(A)) > No, it does not. This is not sound. class A{immutable(A) get(){return null;}} class B:A{ immutable(A) other; this(immutable(A) other){this.other = other;} override immutable(A) get(){return other;} } A delegate(inout(A)) dg = (immutable(A) a) => new B(a); A a = new A; A x=dg(a); immutable(A) b = x.get(); // now a is mutable, b is immutable and (a is b).

(In reply to comment #5) > (In reply to comment #4) > > (In reply to comment #3) > > > We'll need a clear spec of what it means to have inout at two nesting levels of > > > a function signature. > > > > This follows naturally from variance rules: > > [snip.] > > I think you may misunderstand this issue. > > > Also the following subtyping rules for inout apply (R2 <: R1): > > R2 delegate(inout(T)) <: R1 delegate(immutable(T)) > > R2 delegate(inout(T)) <: R1 delegate(const(T)) > > R2 delegate(inout(T)) <: R1 delegate(T) > > And of course: > > R2 delegate(inout(T)) <: R1 delegate(inout(T)) > > > > From that follows (given two types A,B and B <: A): > > B delegate(immutable(A)) <: A delegate(inout(A)) > > > > No, it does not. This is not sound. > > class A{immutable(A) get(){return null;}} > class B:A{ > immutable(A) other; > this(immutable(A) other){this.other = other;} > override immutable(A) get(){return other;} > } > > A delegate(inout(A)) dg = (immutable(A) a) => new B(a); > A a = new A; > A x=dg(a); > immutable(A) b = x.get(); > > // now a is mutable, b is immutable and (a is b). Your example is unsound (no offense :) You can't do: A delegate(inout(A)) dg = (immutable(A) a) => new B(a); and neither: void function(inout(int)*) wfp = function(int*)(*p = 1;} // as you did in comment 2 of bug 7542 1. as I wrote in bug 7542: Given (R2 <: R1) R2 delegate(inout(T)) <: R1 delegate(immutable(T)) (because of argument contravariance and immutable(T) <: inout(T)). You can't assign an instance of a supertype to an instance of a subtype. 2. What do you expect from that func ptr? That you can assign a function that takes mutable, const or immutable values as args? You should not be allowed to assign a function that takes an immutable arg because it assumes it won't be modified outside (and you could be passing in mutable/const args)! And you can't assign a function that takes a mutable because it could change the const arg! So you are left with being able to assign functions that take a const arg and those that take an inout arg, because they can be substituted for the former. Thats why you'd use a pointer to a function taking a const arg! (see bug 7542) void function(const(int)*) wfp; and then you can also assign a void function(inout(int)*) You can call wfp with mutable, const and immutable args because as I wrote in comment 4 of bug 7542: T <: const(T) immutable(T) <: const(T) 3. Aside from all the above, inout already has different semantics inside function bodies, so it would be ambiguous to have another instance of inout in a variable declaration because you can only refer to one constness (that of the args of the current function). Consider: void main() { class A {} inout(A) foo(inout(A) a) { inout(A) b = a; // inout already depends on a's constness // can't introduce a different inout like you want here inout(A) delegate(inout(A)) dg = (immutable(A) a) => new immutable(A); return b; } foo(new A); foo(new const(A)); foo(new immutable(A)); }

(In reply to comment #6) > (In reply to comment #5) > > (In reply to comment #4) > > > From that follows (given two types A,B and B <: A): > > > B delegate(immutable(A)) <: A delegate(inout(A)) > > > > > > > No, it does not. This is not sound. > > > > class A{immutable(A) get(){return null;}} > > class B:A{ > > immutable(A) other; > > this(immutable(A) other){this.other = other;} > > override immutable(A) get(){return other;} > > } > > > > A delegate(inout(A)) dg = (immutable(A) a) => new B(a); > > A a = new A; > > A x=dg(a); > > immutable(A) b = x.get(); > > > > // now a is mutable, b is immutable and (a is b). > > Your example is unsound (no offense :) Obviously it is unsound. That is the point. > > You can't do: > A delegate(inout(A)) dg = (immutable(A) a) => new B(a); You claimed I could. Boskop wrote: > B delegate(immutable(A)) <: A delegate(inout(A)) > and neither: > void function(inout(int)*) wfp = function(int*)(*p = 1;} // as you did in > comment 2 of bug 7542 > I know that both of those should not compile. They were specifically constructed to demonstrate why certain subtyping relationships do not hold. I suggest you to carefully read the relevant posts again.

I think the claim that the reporter is actually making is that the inout constancy of the delegate parameter should vary with that of this. In other words, int opApply(int delegate(ref inout(int)) dg) inout should be callable as if it's any one of these: int opApply(int delegate(ref int) dg) int opApply(int delegate(ref const(int)) dg) const int opApply(int delegate(ref immutable(int)) dg) immutable I was trying to do this myself before this issue was filed. (In reply to comment #6) > 3. Aside from all the above, inout already has different semantics > inside function bodies, so it would be ambiguous to have another > instance of inout in a variable declaration because you can only > refer to one constness (that of the args of the current function). Indeed. The point is that there are two possible interpretations of the opApply signature, and the spec isn't clear on which applies: (a) the constancy is passed through to the delegate (b) the delegate has an inout parameter in its own right See the newsgroup thread I've already linked to for a more detailed discussion of this.

(In reply to comment #8) > > Indeed. The point is that there are two possible interpretations of the > opApply signature, and the spec isn't clear on which applies: > > (a) the constancy is passed through to the delegate > (b) the delegate has an inout parameter in its own right > > See the newsgroup thread I've already linked to for a more detailed discussion > of this. The main issue is that both (a) and (b) are useful in different contexts. Otherwise this would be a no-brainer.

(In reply to comment #8) > Indeed. The point is that there are two possible interpretations of the > opApply signature, and the spec isn't clear on which applies: > > (a) the constancy is passed through to the delegate > (b) the delegate has an inout parameter in its own right I think it must be (b). Consider you don't know when the delegate was constructed: void foo(inout(int)* x, inout(int)* delegate(inout(int)* x) dg) { inout(int)* bar(inout(int)* m) { return m;} auto dg2 = &bar; assert(typeof(dg2) == typeof(dg)); // ??? immutable int y = 5; dg2(&y); // ok dg(&y); // must fail! } If the assert doesn't pass, then what is the type of dg vs. dg2? If it passes, then dg and dg2 are interchangeable, and you will violate const (what if x is mutable?). Even if the assert fails, it's going to be way *way* too confusing to have two types that are identical in syntax be actually different types under the hood. We *absolutely* need a new syntax if case (a) is to be included.

(In reply to comment #10) > We *absolutely* need a new syntax if case (a) is to be included. I agree.

(In reply to comment #7) > (In reply to comment #6) > > (In reply to comment #5) > > > (In reply to comment #4) > > > > You can't do: > > A delegate(inout(A)) dg = (immutable(A) a) => new B(a); > > You claimed I could. > > Boskop wrote: > > B delegate(immutable(A)) <: A delegate(inout(A)) This is wrong, I didn't realize that I accidentally wrote that, sorry. What I wrote above this (and in comment 6) was right: Given R2 <: R1, R2 delegate(inout(T)) <: R1 delegate(immutable(T)) (because of argument contravariance and immutable(T) <: inout(T)). I didn't know at first that you wanted to point this out (probably was too distracted by your code example). (In reply to comment #8) > I think the claim that the reporter is actually making is that the inout > constancy of the delegate parameter should vary with that of this. In other > words, > > int opApply(int delegate(ref inout(int)) dg) inout > > should be callable as if it's any one of these: > > int opApply(int delegate(ref int) dg) > int opApply(int delegate(ref const(int)) dg) const > int opApply(int delegate(ref immutable(int)) dg) immutable The problem is more deep-rooted. You can only pass delegates to opApply that take a supertype of "ref inout(int)" as argument, but immutable(T) <: inout(T) and const(T) <: inout(T), and T <: inout(T). If it were possible to use inout that way, you would have to enable implicit casts from T to immutable(T) (T <: immutable(T)) but you can't do that because functions taking an immutable arg assume that it's not changed outside! There is no way to have inout delegates like you want without enabling implicit casts from T to immutable(T)! But we don't need to enable implicit casts from T to immutable(T). We can have our cake and eat it, too! Now this is the important thing to notice: In the case with opApply you actually want to transfer the constness of the this object to the argument to dg! foreach (e; new immutable(C)([1,2,3])) opApply takes ONLY a "int opApply(int delegate(ref immutable(int)) dg) immutable" foreach (e; new const(C)([1,2,3])) opApply takes ONLY a "int opApply(int delegate(ref const(int)) dg) const" foreach (e; new C([1,2,3])) opApply takes ONLY a "int opApply(int delegate(ref int) dg)" So we need a way to transfer the constness of the this object to declarations within the class. This is basically inout for types (inout_t). (It's related to the inout that is written after method signatures, but can now be be transferred to arbitrary declarations). So, with inout_t you would write: class C { int[] arr; this(int[] a){arr = a;} int opApply(int delegate(ref inout_t(int)) dg) inout { foreach(ref e; arr) if(auto r = dg(e)) return r; return 0; } } BTW: If D didn't have transitive const, you could implement it with inout_t like this: struct S{inout_t(S)* next;}

(In reply to comment #12) > > I didn't know at first that you wanted to point this out (probably was too > distracted by your code example). > It seems to me in general that it could be beneficial to the quality of your contributions if you would spend more time reading and less time writing. ;) > > So, with inout_t you would write: > class C { > int[] arr; > this(int[] a){arr = a;} > int opApply(int delegate(ref inout_t(int)) dg) inout { > foreach(ref e; arr) > if(auto r = dg(e)) return r; > return 0; > } > } > This is an interesting suggestion. There are some other ideas discussed here: http://forum.dlang.org/post/[email protected] > BTW: If D didn't have transitive const, you could implement it with inout_t > like this: > struct S{inout_t(S)* next;} This is moot.

(In reply to comment #10) > void foo(inout(int)* x, inout(int)* delegate(inout(int)* x) dg) > { > inout(int)* bar(inout(int)* m) { return m;} > auto dg2 = &bar; > assert(typeof(dg2) == typeof(dg)); // ??? Whether we pick (a) or (b), it should apply equally to declarations in the body of foo as to parameters of foo itself. So this assert would pass either way.

(In reply to comment #10) > Even if the assert fails, it's going to be way *way* too confusing > to have two types that are identical in syntax be actually different types > under the hood. Following on from my last comment, I see now that it reinforces why (b) is better as the default behaviour. If we went with (a), removing the inouts from the signature of foo would completely change the meaning of the inouts in bar. What (b) is saying is that inout is treated as referencing the constancy level with which the function is called _only_ if it would be illegal in the absence of an enclosing inout context. > We *absolutely* need a new syntax if case (a) is to be included. Would we allow it in the body of bar as well, for when we want to reference the constancy level with which foo was called?

THIS ISSUE HAS BEEN MOVED TO GITHUB https://github.com/dlang/dmd/issues/18417 DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB