Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add __rvalue(expression) builtin #17050

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

WalterBright
Copy link
Member

This adds the __rvalue(expression) builtin, which causes expression to be treated as an rvalue, even if it is an lvalue.

@WalterBright WalterBright added Severity:Enhancement Review:WIP Work In Progress - not ready for review or pulling labels Nov 3, 2024
@WalterBright WalterBright requested a review from ibuclaw as a code owner November 3, 2024 06:03
@dlang-bot
Copy link
Contributor

Thanks for your pull request, @WalterBright!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#17050"

@WalterBright WalterBright force-pushed the __rvalue branch 4 times, most recently from 0f383fd to 60c0f9e Compare November 3, 2024 07:24
@thewilsonator thewilsonator added Review:Needs Changelog A changelog entry needs to be added to /changelog Review:Needs Spec PR A PR updating the language specification needs to be submitted to dlang.org labels Nov 3, 2024
@WalterBright WalterBright force-pushed the __rvalue branch 5 times, most recently from 082c48d to 6e0e44f Compare November 4, 2024 08:07
@nordlow
Copy link
Contributor

nordlow commented Nov 4, 2024

Can this __rvalue(...) be used in place of core.lifetime.move(...)?

@nordlow
Copy link
Contributor

nordlow commented Nov 4, 2024

Will the call to use in

S s;
__rvalue(s)
use(s);

be either

  1. a defined behavior where s is set to S.init by __rvalue(s)
  2. a compiler error (optionally in @safe code) at least in the case where S has indirections or destructors, or
  3. an undefined (@system) behavior when S has indirections or destructors

?

I prefer option 2 and is the closest to what Rust does. Can it be implemented with constant-time overhead by adding a status bit to the (parameter) variable declaration indicating that its contents has been invalidated by a move?

I recently saw a https://youtu.be/08gvuBC-MIE?t=1839 in which Jon Kalb admits that the standard committe made a mistake by forcing moved from object to in a so called "fully formed state" instead of a so called "partially formed state". As this prevents certain kinds of optimizations. For details see https://youtu.be/08gvuBC-MIE?t=1839.

@WalterBright
Copy link
Member Author

@nordlow I'll write a proper document for this after I figure out just what the end result will be!

@WalterBright WalterBright force-pushed the __rvalue branch 2 times, most recently from 86b02a7 to 8ccedfb Compare November 5, 2024 06:41
@nordlow
Copy link
Contributor

nordlow commented Nov 5, 2024

Thanks. Please see updates to my comment at #17050 (comment).

@WalterBright
Copy link
Member Author

My opinion on move semantics is that once you've moved s to t, then s's lifetime is over, and it should be in the default initialized state (a concept C++ doesn't have).

@nordlow
Copy link
Contributor

nordlow commented Nov 5, 2024

My opinion on move semantics is that once you've moved s to t, then s's lifetime is over, and it should be in the default initialized state (a concept C++ doesn't have).

Is this realized by

  1. applying, if present, move constructor of S moving s to t and
  2. resetting all the bytes at s to S.init?

@nordlow
Copy link
Contributor

nordlow commented Nov 5, 2024

Have you considered it making it a compiler error to access s after it has been moved? If not, why? I'm asking because this would lead to slight better performance in debug mode at least. And this is one of the reasons why Rust has this behavior.

@WalterBright
Copy link
Member Author

@WalterBright
Copy link
Member Author

Have you considered it making it a compiler error to access s after it has been moved?

Yes, but it requires Data Flow Analysis, which is slow.

@nordlow
Copy link
Contributor

nordlow commented Nov 6, 2024

Have you considered it making it a compiler error to access s after it has been moved?

Yes, but it requires Data Flow Analysis, which is slow.

Ok, thanks.

Afaict, the complexity of supporting Rust-style r-value semantics depends on the context in which __rvalue would be used. For instance, in

S use(S);
S x;
auto y = __rvalue(x)
use(y); // allowed
use(x);  // disallowed

such a analysis could be implemented in the compiler with negligible overhead using an extra status bit in the Declaration node.

But in the general case I realize now that the compiler needs to recurse into all function calls that are passed l-values by reference as arguments.

Do you have a good reference to which data flow analysis in general and its applications such as this one?

@nordlow
Copy link
Contributor

nordlow commented Nov 6, 2024

I currently experimenting with using __rvalue defined in the branch of this MR in my code. Is __rvalue currently supposed to be wrapped in core.lifetime.move? If so, is

static if (__traits(compiles, { int x; const y = __rvalue(x); })) {
	import core.stdc.string : memcpy;
	T move(T)(return scope ref T source) @trusted {
		scope(exit) {
			static immutable init = T.init;
			memcpy(&source, &init, T.sizeof);
		}
		return __rvalue(source);
	}
	void move(T)(ref T source, ref T destination) @trusted {
		scope(exit) {
			static immutable init = T.init;
			memcpy(&source, &init, T.sizeof);
		}
		destination = __rvalue(source);
	}
} else
	public import core.lifetime : move;

/// unary move()
pure nothrow @nogc @safe unittest {
	auto x = S(42);
	assert(x == S(42));
	const y = move(x);
	assert(y == S(42));
	assert(x == S.init);
}

/// binary move()
pure nothrow @nogc @safe unittest {
	auto x = S(42);
	assert(x == S(42));
	S y;
	move(x, y);
	assert(y == S(42));
	assert(x == S.init);
}

version(unittest) {
	struct S { @disable this(this); int x; }
}

a suitable rewrite of the core.lifetime.move overloads?

@TurkeyMan
Copy link
Contributor

__rvalue() is a bad name for a move() intrinsic, the answer you seek is yes, __rvalue is exactly a replacement for move(), and it should be named move(). I'll argue for this before it's merged, but we're making very good progress here! :)

@TurkeyMan
Copy link
Contributor

Also no, this can't be 'wrapped', it's an intrinsic; it needs to be renamed move, you can't wrap it in a function named move.

@TurkeyMan
Copy link
Contributor

We're playing around with deep foundation level initialisation code; this should be expected... it's a tough transformation to fight through no doubt, but it'll be the most valuable thing that can happen to the language in a long time.
I'm surprised it hasn't gone worse, and the diff is still fairly compact, surprisingly... considering what we're trying to achieve.

@WalterBright WalterBright force-pushed the __rvalue branch 3 times, most recently from 0dc145b to d081468 Compare December 8, 2024 05:23
@WalterBright
Copy link
Member Author

The only error message reported is:

make: *** [Makefile:371: generated/linux/release/64/libphobos2.so.0.110.0] Error 1

How useless.

@WalterBright
Copy link
Member Author

Unfortunately, this PR has become too big and complex. I made some changes, and it is inexplicably broken. I'm going to have to start over. This is why I strongly dislike large PRs.

@WalterBright
Copy link
Member Author

Finally discovered why this is failing.

@WalterBright WalterBright force-pushed the __rvalue branch 2 times, most recently from a4fa8a0 to 244c56b Compare December 11, 2024 04:58
@WalterBright
Copy link
Member Author

I redesigned the code that deals with overload resolution of copy and move constructors. Seems to be much better.

@WalterBright
Copy link
Member Author

The buildkite error is:

/buildkite/builds/geod24-main-5/dlang/dmd/buildkite-ci-build/distribution/bin/../imports/core/atomic.d(297,64): Error: template `atomicCompareExchangeStrongNoResult` is not callable using argument types `!(MemoryOrder.seq, MemoryOrder.seq)(Task*, const(Task), Task)`
--
  | /buildkite/builds/geod24-main-5/dlang/dmd/buildkite-ci-build/distribution/bin/../imports/core/internal/atomic.d(426,10):        Candidate is: `atomicCompareExchangeStrongNoResult(MemoryOrder succ = MemoryOrder.seq, MemoryOrder fail = MemoryOrder.seq, T)(T* dest, const T compare, T value)`
  | with `succ = succ,
  | fail = fail,
  | T = Task`
  | must satisfy the following constraint:
  | `       CanCAS!T`

Anybody have a clue?

@WalterBright
Copy link
Member Author

The vibe.d error seems to be the result of regarding a struct as "not POD" if it contains a move constructor, something it did not do before.

@WalterBright
Copy link
Member Author

To get it to pass buildkite tests, dstruct.d isPOD() is changed:

            /* This is commented out because otherwise buildkite vibe.d:
               `canCAS!Task` fails to compile
             */
            //hasMoveCtorLocal               || // has move constructor

@WalterBright
Copy link
Member Author

@TurkeyMan ready for another try

@TurkeyMan
Copy link
Contributor

Well I guess isPOD probably needs to be tweaked... but probably the definition of POD is actually up for question... I'm not even sure what POD means in D?

C++ has a similar moment where they realised they didn't know what POD even meant anymore, and then redefined it to mean something useful; I think the final rules were:

  1. Support static initialisation (ie, default initialisation)
  2. Has the same struct layout as a C struct with the same members (no vtable, etc)

I mean, the error you're seeing must have been introduced by something that already has a move constructor (formerly "rvalue constructor"), which previously was seen as POD, and now not seen as POD? There's no change in semantics... so why the change in POD-ness?
It's either that it was considered POD before, and so it's still POD now (so the POD check needs an update); or that it was not actually POD before but it was determined POD in error; which is a bug in that code, and it can be fixed there.

Probably worth working out what POD actually means though, and then tightening the definition around that and correcting any broken code.

I'll get back on with testing this out...

@TurkeyMan
Copy link
Contributor

So, now there's 2 branches __rvalue and __rvalue2, both have been rebased recently. Which one should I be testing?

@TurkeyMan
Copy link
Contributor

...I went with __rvalue. Seems promising, although I found another crash bug, but I haven't reduced it yet. In simple tests it doesn't happen, but when I add a move constructor to a container that's used everywhere, something somewhere goes wrong.

@TurkeyMan
Copy link
Contributor

I think I've reduced this as much as I can; it's weirdly elaborate!

struct StringTest
{
    alias toString this; // this MUST exist; I guess this affects the return expression in `fun` somehow...
    const(char)[] toString() // can return anything other than `int`
        => null;

    // must have both copy and move ctors
    this(StringTest rhs) {}
    this(ref StringTest rhs) {}

    // UNCOMMENT THIS TO DO CRASH
//    this(T)(int x) {}   // crash! function has a template, param type (`int`) unrelated to construction

    this(int x) {}              // doesn't crash: no tempalte arg
    this(T)(const(char)[] x) {} // doesn't crash: type matches alias this

    ~this() {} // only if there is a destructor! comment this: crash goes away!
}

StringTest fun(ref StringTest arg)
{
    return arg; // I guess this crashes searching for the copy ctor and getting confused
}

void main()
{
    StringTest st;
    fun(st); // must CALL this function, even though it's not a template!
}

You must uncomment the commented line to see the crash.
Every other line is necessary; comment any single line out, and the crash goes away.

The compiler shouldn't crash when un-commenting that line. Nothing calls that function, it should just be ignored in this program.
It seems the return expression in fun is the source of the crash, but fun is not a template, so it should be always compiled... but what's weird is that this crash only happens if fun is CALLED... (there in main())

@WalterBright
Copy link
Member Author

The __rvalue branch is the main line now.

@WalterBright
Copy link
Member Author

It's another infinite recursion bug:

#963 0x0000000000783569 in dmd.funcsem.resolveFuncCall() (flags=0 '\000', argumentList=..., tthis=0x7ffff7ee1dc0, tiargs=0x0, s=0x7ffff7ee04a0,
    sc=0x7ffff625feb0, loc=...) at src/dmd/templatesem.d:1889
#964 0x00000000007556c2 in ExpressionSemanticVisitor::visit(CallExp*) (this=0x7fffff85f2c8, exp=0x7ffff6260140) at src/dmd/expressionsem.d:6401
#965 0x000000000073ddba in CallExp::accept(Visitor*) (this=0x7ffff6260140, v=0x7fffff85f2c8) at src/dmd/expression.d:3554
#966 0x0000000000770594 in dmd.expressionsem.expressionSemantic() (sc=0x7ffff625feb0, e=0x7ffff6260140) at src/dmd/expressionsem.d:14221
#967 0x0000000000770426 in dmd.expressionsem.binSemantic() (sc=0x7ffff625feb0, e=0x7ffff62601b0) at src/dmd/expressionsem.d:14185
#968 0x00000000007704cd in dmd.expressionsem.binSemanticProp() (sc=0x7ffff625feb0, e=0x7ffff62601b0) at src/dmd/expressionsem.d:14204
#969 0x000000000076200a in ExpressionSemanticVisitor::visit(CommaExp*) (this=0x7fffff85f3f8, e=0x7ffff62601b0) at src/dmd/expressionsem.d:9910
#970 0x000000000073ec42 in CommaExp::accept(Visitor*) (this=0x7ffff62601b0, v=0x7fffff85f3f8) at src/dmd/expression.d:4025
#971 0x0000000000770594 in dmd.expressionsem.expressionSemantic() (sc=0x7ffff625feb0, e=0x7ffff62601b0) at src/dmd/expressionsem.d:14221
#972 0x0000000000765542 in ExpressionSemanticVisitor::visit(AssignExp*) (this=0x7fffff8601e8, exp=0x7ffff6260030) at src/dmd/expressionsem.d:10948
#973 0x000000000084b94e in Visitor::visit(ConstructExp*) (this=0x7fffff8601e8, e=0x7ffff6260030) at src/dmd/visitor/package.d:84
#974 0x000000000073f46a in ConstructExp::accept(Visitor*) (this=0x7ffff6260030, v=0x7fffff8601e8) at src/dmd/expression.d:4322
#975 0x0000000000770594 in dmd.expressionsem.expressionSemantic() (sc=0x7ffff625feb0, e=0x7ffff6260030) at src/dmd/expressionsem.d:14221
#976 0x0000000000701923 in dmd.dsymbolsem.DsymbolSemanticVisitor.visit() (__capture=0x7fffff860cf0, isBlit=false) at src/dmd/dsymbolsem.d:1355
#977 0x000000000070118a in DsymbolSemanticVisitor::visit(VarDeclaration*) (this=0x7fffff860d40, dsym=0x7ffff625fcc0) at src/dmd/dsymbolsem.d:1398
#978 0x00000000006be6b2 in VarDeclaration::accept(Visitor*) (this=0x7ffff625fcc0, v=0x7fffff860d40) at src/dmd/declaration.d:1231
#979 0x00000000006fca7e in dmd.dsymbolsem.dsymbolSemantic() (sc=0x7ffff625f120, dsym=0x7ffff625fcc0) at src/dmd/dsymbolsem.d:96
#980 0x0000000000758c1c in ExpressionSemanticVisitor::visit(DeclarationExp*) (this=0x7fffff860ee8, e=0x7ffff625fdf0) at src/dmd/expressionsem.d:7014
#981 0x000000000073c7c2 in DeclarationExp::accept(Visitor*) (this=0x7ffff625fdf0, v=0x7fffff860ee8) at src/dmd/expression.d:2874
#982 0x0000000000770594 in dmd.expressionsem.expressionSemantic() (sc=0x7ffff625f120, e=0x7ffff625fdf0) at src/dmd/expressionsem.d:14221
#983 0x0000000000770411 in dmd.expressionsem.binSemantic() (sc=0x7ffff625f120, e=0x7ffff625fe60) at src/dmd/expressionsem.d:14184
#984 0x00000000007704cd in dmd.expressionsem.binSemanticProp() (sc=0x7ffff625f120, e=0x7ffff625fe60) at src/dmd/expressionsem.d:14204
#985 0x000000000076200a in ExpressionSemanticVisitor::visit(CommaExp*) (this=0x7fffff861018, e=0x7ffff625fe60) at src/dmd/expressionsem.d:9910
#986 0x000000000073ec42 in CommaExp::accept(Visitor*) (this=0x7ffff625fe60, v=0x7fffff861018) at src/dmd/expression.d:4025
#987 0x0000000000770594 in dmd.expressionsem.expressionSemantic() (sc=0x7ffff625f120, e=0x7ffff625fe60) at src/dmd/expressionsem.d:14221
#988 0x00000000007752a8 in dmd.expressionsem.addDtorHook() (__capture=0x7fffff861100, exp=0x7ffff7ee2930) at src/dmd/expressionsem.d:15573
#989 0x0000000000774e18 in dmd.expressionsem.addDtorHook() (sc=0x7ffff625f120, e=0x7ffff7ee2930) at src/dmd/expressionsem.d:15599
#990 0x000000000075d440 in ExpressionSemanticVisitor::visit(DotVarExp*) (this=0x7fffff8612d8, exp=0x7ffff625fc50) at src/dmd/expressionsem.d:8358
#991 0x000000000073d4aa in DotVarExp::accept(Visitor*) (this=0x7ffff625fc50, v=0x7fffff8612d8) at src/dmd/expression.d:3317
#992 0x0000000000770594 in dmd.expressionsem.expressionSemantic() (sc=0x7ffff625f120, e=0x7ffff625fc50) at src/dmd/expressionsem.d:14221
#993 0x0000000000835096 in dmd.typesem.dotExp() (__capture=0x7fffff8615b0, mt=0x7ffff7ee1dc0) at src/dmd/typesem.d:5241
#994 0x0000000000832579 in dmd.typesem.dotExp() (flag=<incomplete type>, ident=0x7ffff7edde00, e=0x7ffff625fc50, sc=0x7ffff625f120, mt=0x7ffff7ee1dc0)
    at src/dmd/typesem.d:5766
#995 0x00000000006fd2f3 in dmd.dsymbolsem.resolveAliasThis() (findOnly=false, gag=true, e=0x7ffff7ee2930, sc=0x7ffff625f120) at src/dmd/dsymbolsem.d:322
#996 0x000000000081959a in dmd.templatesem.deduceFunctionTemplateMatch() (argumentList=..., tthis=0x7ffff7ee1dc0, fd=@0x7fffff861fb0: 0x7ffff7ee0cf0,
    sc=0x7ffff625f120, ti=0x7ffff625f6b0, td=0x7ffff7ee0f90) at src/dmd/templatesem.d:1346
#997 0x000000000081c460 in dmd.templatesem.functionResolve() (__capture=0x7fffff8622b0, td=0x7ffff7ee0f90) at src/dmd/templatesem.d:2248
#998 0x000000000081c9f4 in dmd.templatesem.functionResolve() (__capture=0x7fffff8622b0, s=0x7ffff7ee0f90) at src/dmd/templatesem.d:2362
#999 0x0000000000789260 in dmd.funcsem.overloadApply() (__capture=0x7fffff862200, sc=0x7ffff625f120, dg=..., fstart=0x7ffff7ee04a0) at src/dmd/funcsem.d:3519
#1000 0x0000000000788ec9 in dmd.funcsem.overloadApply() (sc=0x7ffff625f120, dg=..., fstart=0x7ffff7ee04a0) at src/dmd/funcsem.d:3544
#1001 0x000000000081aef1 in dmd.templatesem.functionResolve() (errorHelper=..., argumentList=..., tthis=0x7ffff7ee1dc0, tiargs=0x0, sc=0x7ffff625f120, loc=...,
    dstart=0x7ffff7ee04a0, m=...) at src/dmd/templatesem.d:2364
#1002 0x0000000000783569 in dmd.funcsem.resolveFuncCall() (flags=0 '\000', argumentList=..., tthis=0x7ffff7ee1dc0, tiargs=0x0, s=0x7ffff7ee04a0,
    sc=0x7ffff625f120, loc=...) at src/dmd/templatesem.d:1889

@TurkeyMan
Copy link
Contributor

Hmmm, well it's very specific... the conditions that lead to it are super fragile, but it did appear in the wild instantly.

@WalterBright
Copy link
Member Author

Reduced example:

struct S {
    alias TT this;
    long TT();
    this(T)(int x) {}   // works if `int` is `long`

    this(S);
    this(ref S);

    ~this();
}

S fun(ref S arg);

void test() {
    S st;
    fun(st);
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Review:Needs Changelog A changelog entry needs to be added to /changelog Review:Needs Spec PR A PR updating the language specification needs to be submitted to dlang.org Review:WIP Work In Progress - not ready for review or pulling Severity:Enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants