Tracking Issue for breakpoint feature (core::arch::breakpoint)

[joshtriplett]

Feature gate: #![feature(breakpoint)]

This is a tracking issue for the breakpoint feature, which gates the core::arch::breakpoint function. This feature was approved in ACP 491.

Public API

// core::arch

/// Compiles to a target-specific software breakpoint instruction or equivalent.
///
/// (Long description elided.)
#[inline(always)]
pub fn breakpoint() {
    unsafe {
        core::intrinsics::breakpoint();
    }
}

Steps / History

• ACP: ACP: core::arch::breakpoint libs-team#491
• Implementation: Add core::arch::breakpoint and test #133726
• Final comment period (FCP)¹
• Stabilization PR

Unresolved Questions

• None yet.

Footnotes

1. https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

[clarfonthey]

Is there a reason why this is in core::arch instead of core::hint? I feel like it would be logical to say that a perfectly acceptable implementation of this would be to do nothing, and I'd imagine this is what a lot of embedded targets will do.

Plus, core::arch is generally reserved for arch-specific stuff, and this is specifically independent of architecture.

[Scripter17]

Does it really fit in std::hint? Everything else in there is, well, hints to the compiler. Either for lints or for optimization

It fits better in there but it doesn't feel like the obvious place to look for a breakpoint function

[joshtriplett]

@clarfonthey I don't think "do nothing" should be a valid implementation of this. "abort" is a valid implementation. Given that, I don't think this belongs in hint.

That said, that doesn't necessarily make core::arch the best place. It seemed like a good fit at the time, in part because its behavior is arch-specific.

I don't see any existing module that this logically fits in other than arch.

We could put it in a new core::debug or similar, but I'm not sure it's worth a new module for this, and debug could be confused as being related to Debug.

[clarfonthey]

I mean, sure, but we have other things in hint which emit special instructions, like spin_loop. Just because the definition of a "trivial" implementation changes, or that the result is architecture-specific, doesn't mean it doesn't fit into that module.

[joshtriplett]

@clarfonthey AFAICT, every function in core::hint can be replaced by a no-op/identity for a correct (if suboptimal) implementation. I think that's a property worth not losing.

A breakpoint isn't a "hint"; it can't be replaced with a no-op.

Is there any other module in core that you think this could fit into? Or, do you think it's worth creating a new module for this and potential related items?

[kpreid]

JavaScript has the debugger statement which causes the debugger to pause the program if a debugger is available (in current browsers, this means “if the developer tools are open”), and otherwise continues execution. This is a useful behavior (it allows executing an identical program with and without pauses for debugging, without needing to remove the breakpoint and recompile) and if Rust offered it, it would fit in core::hint.

[joshtriplett]

@kpreid I'd love to have that operation, but unfortunately, that's a much more complex operation that isn't as simple as emitting an instruction, it'd be more error-prone (it can erroneously detect a debugger), it'd be less portable (as it's OS-specific rather than CPU-specific), and it wouldn't be available on all targets.

[BrainBacon]

@joshtriplett Maybe an ACP for debugger presence detection? C++ 26 is getting that.

I recently added debugger presence detection to Unbug using the dbg_breakpoint crate. The bulk of that crate was previously accepted as a panic hook in the Rust standard library, but then reverted later.

[joshtriplett]

Stabilization Report

Implementation History

Approved in ACP rust-lang/libs-team#491 , implemented in #133726 , and not modified since. Includes tests and documentation.

Some discussion about what the correct module for this function should be, but no conclusion about a specific better place than core::arch.

API Summary

In core::arch (and thus also std::arch):

pub fn breakpoint();

Experience Report

The ACP was originally inspired by the unbug crate, which was promptly able to switch to this (and will be able to drop some architecture-specific code when this becomes stable, and run on stable Rust on more architectures). https://github.com/microsoft/edit is also using this.

[Amanieu]

@rfcbot merge

[rfcbot]

Team member @Amanieu has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[traviscross]

@rustbot labels +I-lang-nominated +P-lang-drag-1

Generally we lang FCP the first stable use of an intrinsic. This is an intrinsic, and this is the first stable use. At the same time, as we discussed in the libs-api meeting today, perhaps what we're meaning to lang FCP are new capabilities of the language, and this one could be seen as equivalent to some inline assembly. So I don't know. It's worth us having a look in any case, so let's nominate.

cc @RalfJung @workingjubilee @rust-lang/lang

[RalfJung]

This doesn't really have semantics so I don't think there's much to say here from the opsem side.

[Amanieu]

The semantics are essentially:

if debugger is attached and chooses to skip over the breakpoint {
    continue execution
} else {
    trap
}

[hanna-kruppe]

The "trap" part seems like a somewhat "new capability" for no_std programs. There's lots of ways to diverge, of course, but all the ones I can think of right now are either std-exclusive (std::process::abort), platform-specific (e.g., inline assembly or functions like core::arch::wasm32::unreachable), diverge by entering an infinite loop, or end up in the #[panic_handler] which has to use one of the aforementioned options to diverge. The kind of "portable trap" that breakpoint does is more like core::intrinsics::abort, which is unstable and has no stable equivalent as far as I know.

[clarfonthey]

I'm still not 100% convinced on this being in core::arch, since again, it's not an architecture-specific extension. (Inline assembly is technically callable by all architectures, but still inherently architecture-specific.)

That said, the closest analogue that seems to exist is std::process, and I'm not sure whether we'd want this to be core::process::breakpoint. It would go along with std::process::abort and std::process::exit.

Perhaps std::process::breakpoint could be the advanced version of breakpoint checking that truly does nothing when a debugger is not attached, and core::process::trap could be this version which is explicitly simpler and has a more descriptive name.

[hanna-kruppe]

Also, regarding these arguments for core::arch over core::hint from last December:

AFAICT, every function in core::hint can be replaced by a no-op/identity for a correct (if suboptimal) implementation. I think that's a property worth not losing.

A breakpoint isn't a "hint"; it can't be replaced with a no-op.

Now that core::hint::select_unpredictable has been stabilized in beta, it's no longer true that every "hint" function is just a no-op or identity (though the correct-but-suboptimal implementation of select_unpredictable is pretty trivial). The quality of implementation lever to tune here is "whether a debugger can detect it and resume execution" and so the "suboptimal but valid" implementation would be unconditionally aborting. Indeed that's the implementation you get on targets where there is no standard mechanism that debuggers recognize as breakpoint distinct from any other program abort (e.g., I think this is the case for wasm).

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[clarfonthey]

Actually, that being the case now means that it probably is best to put this in core::hint for those reasons.

[joshtriplett]

I'm still not 100% convinced on this being in core::arch, since again, it's not an architecture-specific extension. (Inline assembly is technically callable by all architectures, but still inherently architecture-specific.)

That said, the closest analogue that seems to exist is std::process, and I'm not sure whether we'd want this to be core::process::breakpoint. It would go along with std::process::abort and std::process::exit.

We did discuss core::process in the meeting, and we weren't sure either. It would be the only thing in core::process if we did that.

Perhaps std::process::breakpoint could be the advanced version of breakpoint checking that truly does nothing when a debugger is not attached, and core::process::trap could be this version which is explicitly simpler and has a more descriptive name.

trap is a different operation; this would be debugtrap. But breakpoint seems more descriptive.

[RalfJung]

Are users expected to rely on this “by default” aborting the program, and if so, how?

I'd still love to see an answer to this, I don't think there has been one?

But yeah it seems too late... this feels a bit rushed given the amount of concerns and questions raised here.

given the desired semantics

Is there a writeup somewhere of why these semantics are desired? "stop if debugger is attached, ignore otherwise" seems like a much more desirable semantics to me.

[ChrisDenton]

A higher level if debugger_present wrapper would need to build on top of the lower level SIGTRAP behaviour.

And the debug or abort behaviour is useful even if we had a function with different semantics.

[clarfonthey]

The primary motivation, at least according to the ACP, is to be able to have this available on core, and while you're right that "stop if debugger is attached" behaviour is more desirable, this is the best that can be done without relying on OS help, AFAIK.

I've mentioned my arguments for why I think such a method should not be given a friendly name like breakpoint given how its undesirable semantics are mostly only suited for targets which are forced to use them, however, libs disagrees and I don't think that I'm going to change their mind.

[RalfJung]

A higher level if debugger_present wrapper would need to build on top of the lower level SIGTRAP behaviour.

So the argument is "because that's what the system API provides, we don't have a choice here"?

And the debug or abort behaviour is useful even if we had a function with different semantics.

That's an assertion, not an argument.

[ChrisDenton]

That's an assertion, not an argument.

Sure? You asserted that "stop if debugger is attached, ignore otherwise" is "more desirable". I did not take from that that you were requiring justification for any other behaviours being available in the standard library.

[joshtriplett]

Are users expected to rely on this “by default” aborting the program, and if so, how?

I'd still love to see an answer to this, I don't think there has been one?

It emits a debug breakpoint instruction, or a trapping instruction if the target doesn't have a breakpoint instruction. Either way, that will have the behavior of aborting the program if not handled. On Linux, for instance, it'll result in the program receiving a SIGTRAP signal. The description of the function as aborting the program by default if not handled is very much part of the definition, and intended to be normative. Users can rely on it. Does that answer your question?

[joshtriplett]

So the argument is "because that's what the system API provides, we don't have a choice here"?

No, some of the arguments are:

• It's helpful, at the lowest level, to have the underlying mechanism to emit a breakpoint, which can then be inserted along a path the user wants to debug or get a core dump from. It can also be wrapped in any kind of conditional the user wants (e.g. if weird_error_condition { breakpoint(); }). This mechanism is always available, and doesn't require any OS services, so it can live in core.
• We could also consider adding a function to detect if there's a debugger attached. That is a complex and target-specific operation, which may not be possible on all targets or in all environments, and which has some potential side effects. For instance, on Linux, this might require having /proc mounted and making several system calls to look for TracerPid, or attempting to attach a debugger to your own process and assuming that a failure indicates that you're being debugged. Also, this would need to live in std since it uses the OS.
• The "break if debugger attached" operation still isn't something you'd want to leave in the program when not debugging, without some other conditional attached to it. Otherwise, for instance, it would trigger any time you run the program under strace, and it would also interfere with other unrelated debugging. So, typically I'd expect these operations to either be added when debugging and removed when done, or to be added underneath some conditional that has to be enabled. Once you have such a conditional, that reduces the value of also checking if a debugger is attached.

All that aside, it's a matter of debugging style and preference whether you want the "breakpoint if debugger attached" operation or the "breakpoint unconditionally" operation. The former can be built atop the latter, but the latter can't be built atop the former, and the latter is a useful operation to have.

[hanna-kruppe]

You’ve stated what the function’s definition is and that “users can rely on it” but that doesn’t get to the heart of the question. Why is the definition like that, why and how is this definition useful to users? Would users complain or write their programs differently if breakpoint was changed to, e.g.,

• Describe the “trap if not under a debugger” as best effort quality of implementation aspect (QoI)? Since we agree you wouldn’t leave this in a binary shipped to customers/production, does it matter if it occasionally doesn’t trap when you’re not actively debugging it?
• Describe the “this is not a normal trap because a debugger can continue execution” aspect as QoI? (Indeed there are target platforms which don’t have such a concept, like wasm)
• Give more or less detail about what “under a debugger” means? (i.e., if a user files a bug report “I used breakpoint and ran under the foo debugger but the debugger doesn’t recognize it as a breakpoint” would this be considered a bug in Rust if the foo debugger only recognizes a different instruction than the one Rust chose?)
• Give more or less detail about how it aborts execution? For example, is the SIGTRAP on Linux guaranteed? Is entering an infinite loop (which counts as ! in Rust’s type system) a legal implementation?
• Not exist at all, replaced with the advice “just panic and read these docs about how to place a breakpoint on panic“
• Not exist at all, replaced with the advice “just define an inline(never) no_mangle function, call it where you want a breakpoint, and instruct your program to set a breakpoint on it”

Note that these questions are not coming from a preference for a different API (like “breakpoint under debugger, nop otherwise”) but from a language spec perspective: how and why is this “guarantee” you’re describing different from what the functions in std::hint get? Why is it important to users that this function is defined in a way that attracts T-lang attention? How are miri and formal verification tools supposed to model it?

[joshtriplett]

• Since we agree you wouldn’t leave this in a binary shipped to customers/production

That's not what I said. "breakpoint if debugger attached" isn't sufficient to leave the operation in place in production without some other conditional attached to it. In some production environments, it might make sense to deploy a binary that does have a breakpoint with a conditional wrapped around it, controlled by (for instance) a special option you can pass. But in the absence of such an additional conditional, a binary with a "breakpoint if debugger attached" such a binary would break if you tried to strace it, which people also need to do in production sometimes.

does it matter if it occasionally doesn’t trap when you’re not actively debugging it?

Yes, that would break user expectations. As one of many potential examples, you may want to run it without a debugger, expecting it to produce a coredump when it hits the breakpoint.

Describe the “this is not a normal trap because a debugger can continue execution” aspect as QoI?

That part of the behavior is target-specific, and if a target didn't have any mechanism for resuming execution, that wouldn't prevent aborting.

Give more or less detail about what “under a debugger” means? (i.e., if a user files a bug report “I used breakpoint and ran under the foo debugger but the debugger doesn’t recognize it as a breakpoint” would this be considered a bug in Rust if the foo debugger only recognizes a different instruction than the one Rust chose?)

That would depend on the nature of the issue, the debugger, and what the standard conventions on the target are. If it's the standard breakpoint of the platform and the debugger doesn't recognize it, that sounds like an issue in the debugger. But if there's some convention we failed to follow, and a proposed modification still works with other debuggers, it might be something we'd want to fix.

Not exist at all, replaced with the advice “just panic and read these docs about how to place a breakpoint on panic“

That would be less useful, for multiple reasons. Among them: you might not want to panic, because that's harder to step "around" than a breakpoint instruction. Also, a panic errors in a different way,

Not exist at all, replaced with the advice “just define an inline(never) no_mangle function, call it where you want a breakpoint, and instruct your program to set a breakpoint on it”

That would be less convenient, and additionally would not abort the program if hit without a debugger attached, and additionally

[RalfJung]

@ChrisDenton

Sure? You asserted that "stop if debugger is attached, ignore otherwise" is "more desirable". I did not take from that that you were requiring justification for any other behaviours being available in the standard library.

Josh claimed that what core does is the "desired semantics", implying that other semantics are not desired. I asked for justification, since I can totally see other semantics being "more desirable". I'm not the one trying to add something to our stable API surface here, so it is not me who has the burden of justification. If it was just about one gut feeling for what is desirable vs another, surely we should wait a bit longer and see whether we can find any actual arguments.

@joshtriplett

This mechanism is always available, and doesn't require any OS services, so it can live in core.

We could also consider adding a function to detect if there's a debugger attached. That is a complex and target-specific operation, which may not be possible on all targets or in all environments, and which has some potential side effects.

That very much sounds like "because that's what the system API provides, we don't have a choice here" to me. (I infer that the semantics of "only break under debugger" cannot live in core.)

To be clear, that's a reasonable argument as far as I am concerned.

I think it's just the name that is confusing for people not familiar with the details of assembly instructions, people that only ever used breakpoints in an IDE.

[hanna-kruppe]

@joshtriplett Answering specific illustrative examples unfortunately doesn’t tell me directly what you think about the nature of this “guarantee users rely on” and I don’t want to assume. It feels unproductive to repeat it several times but I still haven’t gotten a straight answer how the points you made are not just QoI concerns. Is the (lack of) “guarantee” about what breakpoint does more like “your programs’ correctness can rely on <[T]>::sort actually sorting the slice, provided the comparison is well-behaved” or more like “you can use select_unpredictable to ask for branch-free codegen but it’s best effort, so don’t rely on it for trying to prevent timing side channels”? The concrete answers you’ve given (focusing on what’s practically useful) sound more like the latter, but again, I’d rather get a straight answer than having to guess at what you’re thinking.

[joshtriplett]

@RalfJung wrote:

That very much sounds like "because that's what the system API provides, we don't have a choice here" to me.

It seems like you're starting from the baseline assumption that if we had a choice, we'd necessarily choose to have the "breakpoint only if debugging", and thus it must be that "we don't have a choice".

Even leaving aside that the "if debugging" check isn't as portable, and adds complexity that has side effects that might interfere with debugging: if you acknowledge that the underlying breakpoint operation low-level operation and high-level operation are both useful, the intention here was to expose the low-level operation, and if someone wants to propose building the high-level operation on top of that, they can.

[RalfJung]

It seems like you're starting from the baseline assumption that if we had a choice, we'd necessarily choose to have the "breakpoint only if debugging", and thus it must be that "we don't have a choice".

I wasn't trying to do that, though maybe I wasn't separating my opinions sufficiently from the argument-gathering.
I was just trying to make sense of your argument, which was focused on what is technically available in OSes and CPUs, not on the user experience or language guarantees.

[workingjubilee]

Anyone interested in discussing the hypothetical high-level operation of "trigger the (architectural-specific breakpoint handling) if a debugger is detached" might be entertained by reading the story of our attempt to implement exactly that as an internal detail of std, to make debugging more convenient. It is actually surprisingly possible to implement, but it has unintended consequences: #142325 (comment)

[joshtriplett]

So, to attempt to more precisely call out the documented answer to the question of what the language semantics are:

The doc comment on the breakpoint function specifically says:

The precise behavior and the precise instruction generated are not guaranteed, except that in normal execution with no debug tooling involved this will not continue executing.

The world in which you're using debugging tooling, much like the world in which you open /proc/self/mem, is one where you could be arbitrarily modifying the program semantics, and the behavior of the program in that environment is best-effort and implementation-defined.

However, I don't think the entirety of the specification should be "breakpoint is implementation defined". Based on conversations with @nikomatsakis, I'm suggesting that it should instead be something like this:

If no debugging tool or other implementation-defined breakpoint-handling mechanism (e.g. signal handler) is in use, execution will not continue; the manner in which it does not continue is target-defined (e.g. abort, trapping instruction, infinite loop). If a debugging tool or other implementation-defined breakpoint-handling mechanism is in use, the behavior is implementation-defined, and may be able to continue execution. Users of breakpoint can, normally, assume that in the absence of a debugging tool or other implementation-defined breakpoint-handling mechanism (e.g. catching and handling a signal), execution does not continue; however, because debugging tools and other breakpoint-handling mechanisms may allow continuing execution, the compiler must allow for the possibility that execution continues, and cannot treat breakpoint as a noreturn (-> !) function.

@hanna-kruppe, @RalfJung, does that answer the question you were trying to answer?

Also, we might want to consolidate the discussion into one place rather than two. I'd suggest here, rather than on the stabilization PR.

[RalfJung]

If no debugging tools are in use, execution will not continue,

This seems to contradict the later part that even in the absence of debugging tools, execution may sometimes continue through target-specific means?

[the8472]

I think resuming from a breakpoint is in a weird space somewhere between implementation-defined behavior and erroneous behavior.

A normal program execution should not return from it, which means implementations also should not do this. But optimizations may not assume that this never happens.

#142325 (comment):

I suppose in a sense, then, the semantics here is similar to

if platform_specific_volatile_read() {
    abort()
}

and it needs to be treated as similarly immovable as volatile operations?

Instead of volatile we could also model it as a shared atomic that is initialized to zero at program startup and may not be written to by the program itself, but since it's shared the environment may write to it.
The consequence is that if only the program is acting on it then the program will always abort and only when an input from the environment happens it may continue.

[joshtriplett]

If no debugging tools are in use, execution will not continue,

This seems to contradict the later part that even in the absence of debugging tools, execution may sometimes continue through target-specific means?

I've expanded the text to consistently use "debugging tool or other implementation-defined breakpoint-handling mechanism".

Does the new text look reasonable to you?

[hanna-kruppe]

I think it would be a category mistake to bring notions like "if a debugging tool is used" or "normally, unless impl-defined mechanisms are used" into normative language spec contents. Everything written there may need to be reflected in (for example) a pen-and-paper mathematical proof, some code in Miri, or a verification tool like Kani. At least for those purposes, this intrinsics has to be boiled down into (at most) a conditional abort depending on some non-deterministic choice or some escape hatch into "interaction with outside environment we're not modeling here" (as for volatile loads and stores). This is already saying more than just "breakpoint() has entirely implementation-defined effects" (for good reasons) but it's still a far cry from how @joshtriplett phrases it! If we can agree this is effectively what the spec means for non-rustc consumers, then I think couching it in more suggestive language like "debugger" just obscures what the spec actually says.

Consequently, there's not much wiggle room left, certainly no way to normatively nail down things like "normally, if there's no debugger attached [...]" because none of that has any meaning in the terms of the AM. I think all that intent about how this intrinsic should be useful to flesh and blood programmers dealing with silicon belong in non-normative notes and/or rustc's documentation of how it implements the language spec. The spec can be far removed from that, it just has to be compatible with any reasonable implementation strategy. To me this implies:

• It shouldn't be "unconditional trap" because we don't want code to be compiled and optimized under this assumption, nor do we want to make its signature () -> ! because then even the source program won't be compatible with resuming execution.
• It can't be "just a no-op" either, instead it should involve something that ensures it's only actually executed when intended by the programmer, ordered w.r.t. other AM-acknowledged side effects like println!() or volatile accesses. We probably want to count "evaluating the condition" as an observable side effect like volatile reads are, so that they can't be legally "optimized out" or coalesced.
• Going in the weeds about different ways of what may or may not cause execution to proceed ("debugger" vs. "other impl-defined breakpoint handling mechanism") seems out of scope for normative text. It can all be shoved into a single binary choice per execution of the intrinsic.
• Both "always abort" and "always no-op" should be allowed by the spec because both may be occasionally useful for formal verification, e.g., if you want to know something about the program behavior compiled with rustc and run in a particular environment.
• Not specifying any details about how the program execution is stopped (e.g., allowing loop {} as well as actually stopping) seems like a good idea to me.

Coming back to the physical realm for a minute: I don't think rustc should document/support any other "breakpoint handling mechanisms" like catching SIGTRAP from within the program as a means of automatically "handling" breakpoints. To me that's in the same category as "handling" SIGFPE to let the program proceed after a division by zero. Technically the physical machine and OS let you do it, but practically speaking you can't know if the program is running off a cliff and hoping for the signal to stop it. The intended use case for this feature is interactive debugging; other ways of continuing execution should be unsupported so we don't have to worry about weird hacks people come up for it while evolving the implementation strategy to best fit the intended use case.

[workingjubilee]

Maybe we should flip the question around.

@joshtriplett If an architecture, by default, masks most faults/interrupts unless you do something equivalent to "attach a debugger", perhaps because the architecture is oriented around parallel execution (like a GPU, say) and does not have abundant reserve state to both execute quickly and relatively-easily halt forward flow of the program upon hitting a condition, then should this intrinsic not also emit an instruction that would, by default, be masked by the architecture's default interrupt-handling of "I would prefer not to"?

I believe most such architectures do support "unconditional" faults that happen even if debuggers are not attached, as they still have erroneous conditions like memory violations. But for a parallel execution of shaders, that might not even kill the entire "program", just that wave's execution. Meanwhile, this intrinsic seems to clearly want to emit the instruction that the architecture and attendant tools would recognize as part of the debugging protocol, not "definitely kill forward momentum".

I understand that you're trying to produce a guarantee that on x86 and x86_64, an actual ud2 is inserted in the binary, here, I'm just not sure why. If we want to produce a reliable way to halt forward execution, then we can simply produce a reliable way to halt forward execution. As you have observed, debuggers can do what's necessary. They can even simply move the instruction pointer.

I know there's the problem of hypothetical psycho-compilers, but defending against them seems to introduce more complications than it solves. Programmers can and do rely, for instance, on most of the core::arch SIMD-related "intrinsic functions" producing useful behavior... emitting SIMD instructions, namely... but it's actually convenient that we do not have a hard-and-fast spec that they must be implemented with certain instructions, as that makes it much easier for e.g. Cranelift or MIRI to support that code.

[joshtriplett]

Coming back to the physical realm for a minute: I don't think rustc should document/support any other "breakpoint handling mechanisms" like catching SIGTRAP from within the program as a means of automatically "handling" breakpoints. To me that's in the same category as "handling" SIGFPE to let the program proceed after a division by zero. Technically the physical machine and OS let you do it, but practically speaking you can't know if the program is running off a cliff and hoping for the signal to stop it. The intended use case for this feature is interactive debugging; other ways of continuing execution should be unsupported so we don't have to worry about weird hacks people come up for it while evolving the implementation strategy to best fit the intended use case.

They can and will be used, and they should not be undefined behavior. They could be implementation-defined behavior, for instance, but the compiler still isn't allowed to eat your laundry if you do it (as it could if the function were defined as -> ! and you tried to continue from it).

[joshtriplett]

If an architecture, by default, masks most faults/interrupts unless you do something equivalent to "attach a debugger", perhaps because the architecture is oriented around parallel execution (like a GPU, say) and does not have abundant reserve state to both execute quickly and relatively-easily halt forward flow of the program upon hitting a condition, then should this intrinsic not also emit an instruction that would, by default, be masked by the architecture's default interrupt-handling of "I would prefer not to"?

That sounds really painful to work with, but that's a reasonable point. If the conventional behavior of breakpoints on the target is not to fault, and the target doesn't have a mechanism for "breakpoint unconditionally" that is still useful with a debugger attached (e.g. if the only two options are "halt in an unrecoverable way" or "breakpoint in a resumable way but only if a debugger is attached"), then it's worth at least considering whether breakpoint should follow the target convention. That seems likely to end up producing an effect comparable to the history you gave about the standard library doing that in panic, though.

The thing I primarily want to avoid is, given a target that has a conventional "breakpoint unconditionally" operation and a conventional "breakpoint if debugger attached" operation, mapping breakpoint to the latter rather than the former. But it's not clear what the best answer is if the target has no "breakpoint unconditionally" operation but does have a "breakpoint if debugger attached" operation.

I understand that you're trying to produce a guarantee that on x86 and x86_64, an actual ud2 is inserted in the binary, here

int3, not ud2. (I mentioned ud2 in the context of explaining what "undefined instruction" means, but that's not what breakpoint() should do on platforms that have a breakpoint instruction.)

I'm just not sure why

In part because I'm trying to guard against any target mapping this to a "do something if debugger is attached" primitive or a no-op, for reasons such as those demonstrated in your story about panics in the standard library.

Programmers can and do rely, for instance, on most of the core::arch SIMD-related "intrinsic functions" producing useful behavior... emitting SIMD instructions, namely... but it's actually convenient that we do not have a hard-and-fast spec that they must be implemented with certain instructions

That much seems fine, but I wouldn't want to specify a SIMD intrinsic as "this does an implementation-defined thing", to the point that it'd be a correct implementation to not do the operation. e.g. a SIMD add intrinsic doesn't necessarily have to use one specific add instruction, but it should behave as-if it did, and in particular, should perform an addition.

[ChrisDenton]

I worry we're continually in danger of mixing libs concerns in with lang concerns. What matters for lang is the abstract machine, right? And in that sense is breakpoint any different from an FFI call that may or may not return?

Of course on the libs side (and perhaps compiler side) there are some interesting things but lang doesn't need to worry about any of that because it doesn't concretely affect the AM, no?

[hanna-kruppe]

It’s probably useful for reasoning about programs if breakpoint is more constrained than an arbitrary FFI call (it either aborts or does nothing, e.g., it may not modify a static and then return), but otherwise I agree.

[clarfonthey]

I worry we're continually in danger of mixing libs concerns in with lang concerns. What matters for lang is the abstract machine, right? And in that sense is breakpoint any different from an FFI call that may or may not return?

I would disagree since lang still cares about those: FFI is a landmine of undefined behaviour and we still would like to put up guard rails to define what is and isn't allowed.

The abort path is self-explanatory because the program behaviour after is explicitly nothing: no program, no behaviour. But it's desirable to say that the continue path hasn't affected the rest of the program in a way that would cause UB, assuming that the program hasn't been altered in any way.

Since it's intended for debugging, it's highly undesirable if the state of the program is affected at all by introducing the breakpoint, but at the same time, you'd also expect the breakpoint to act similarly to an atomic fence, since shouldn't everything before the breakpoint be executed before the breakpoint occurs? These are competing concerns: one says the breakpoint shouldn't affect anything, but that is an example of something it should affect. And these are very language-domain decisions to be made.

[hanna-kruppe]

Giving breakpoint any special powers in the spec to support inspection of the physical machine state is risky. Doing it carelessly can accidentally rule out desirable compiler optimizations just because of the possibility that some function call that the optimizer can’t crack open may invoke the intrinsic. So realistically, “whatever any ordinary FFI function with this signature could do” is actually one of the easiest options here because that’s something that spec and compilers have to deal with anyway and breakpoint can just piggy-back. But saying it’s a no-op if it doesn’t abort is also fine, since it leaves the “how to compile this in a way that is implementable and useful for debugging” question to the implementation defined libs/compiler realm where it arguably belongs.

[clarfonthey]

I guess that the distinction here on what qualifies as lang's realm is also extra confusing since this function's usefulness is directly tied to how it's compiled. I was mostly basing my understanding of it being a lang issue based upon the fact that inline assembly is also a lang issue, even though it's also heavily tied to how the program is compiled. Since the idea is that two different compiler backends would have to support the same behaviour on the same target, even though individual targets might differ.

[joshtriplett]

I think it would be a category mistake to bring notions like "if a debugging tool is used" or "normally, unless impl-defined mechanisms are used" into normative language spec contents.

I disagree; normative language can include things like SHOULD, and it can also include things like "if X, MUST, if not X, implementation-defined". The latter is an improvement over "always implementation-defined".

Everything written there may need to be reflected in (for example) a pen-and-paper mathematical proof, some code in Miri, or a verification tool like Kani. At least for those purposes, this intrinsics has to be boiled down into (at most) a conditional abort depending on some non-deterministic choice or some escape hatch into "interaction with outside environment we're not modeling here" (as for volatile loads and stores).
[...]
If we can agree this is effectively what the spec means for non-rustc consumers, then I think couching it in more suggestive language like "debugger" just obscures what the spec actually says.
[...]
Consequently, there's not much wiggle room left, certainly no way to normatively nail down things like "normally, if there's no debugger attached [...]" because none of that has any meaning in the terms of the AM.

I wouldn't object to a formal proof modeling it as if coinflip() { abort() } or similar, since from a correctness point of view the compiler has to allow for either outcome, but that shouldn't stop us from e.g. normatively saying an implementation that always does nothing is not semantically correct.

I'm all for giving sufficient information to also attempt a formal proof, but I think that should not stop us from giving people some clarity on what the implementation-defined behavior might likely be, for instance.

It seems like you're equating "normative" with exclusively things that are meaningful in the AM. If you want to say that the normative language should include guidance to model this as if coinflip() { abort(); }, that seems fine. It sounds like you're also saying the normative language shouldn't include anything else, and that's the part I'd disagree with.

The spec can be far removed from that, it just has to be compatible with any reasonable implementation strategy. To me this implies:

* It shouldn't be "unconditional trap" because we don't want code to be compiled and optimized under this assumption, nor do we want to make its signature `() -> !` because then even the source program won't be compatible with resuming execution.

Agreed.

* It can't be "just a no-op" either,

Agreed.

instead it should involve something that ensures it's only actually executed when intended by the programmer, ordered w.r.t. other AM-acknowledged side effects like println!() or volatile accesses. We probably want to count "evaluating the condition" as an observable side effect like volatile reads are, so that they can't be legally "optimized out" or coalesced.

Agreed. And from an implementation perspective, we should ensure that it includes a compiler barrier. I would hope that LLVM's debugtrap provides that, but if it doesn't, we may need to add one.

* Going in the weeds about different ways of what may or may not cause execution to proceed ("debugger" vs. "other impl-defined breakpoint handling mechanism") seems out of scope for normative text. It can all be shoved into a single binary choice per execution of the intrinsic.

See above; "formal verification" is not the only use of a spec, or of other normative language such as guarantees in documentation.

That said, would "implementation-defined breakpoint-handling mechanism (such as a debugger)" be more clear than "debugger or other implementation-defined breakpoint-handling mechanism"?

* Both "always abort" and "always no-op" should be allowed by the spec because both may be occasionally useful for formal verification, e.g., if you want to know something about the program behavior compiled with rustc and run in a particular environment.

That depends on perspective. From the perspective of formal verification, sure, verification has to allow for those possibilities. From the perspective of constraining implementations of compiler backends, targets, etc, "always no-op" isn't a valid impl, and I continue to argue that that guidance should be normative (e.g. what's written in the doc comment on the function, which should probably be expanded).

* Not specifying any details about how the program execution is stopped (e.g., allowing `loop {}` as well as actually stopping) seems like a good idea to me.

I'm glad to hear that.

[workingjubilee]

That sounds really painful to work with, but that's a reasonable point. If the conventional behavior of breakpoints on the target is not to fault, and the target doesn't have a mechanism for "breakpoint unconditionally" that is still useful with a debugger attached (e.g. if the only two options are "halt in an unrecoverable way" or "breakpoint in a resumable way but only if a debugger is attached"), then it's worth at least considering whether breakpoint should follow the target convention. That seems likely to end up producing an effect comparable to the history you gave about the standard library doing that in panic, though.

My understanding is that on architectures that have this semantic, it isn't that much of a problem, because the software people write to debug, trace, and profile the architecture must handle this inversion relative to more classical debugger protocols, so it simply does.

I would think in the case of breakpoint, we would want the SHOULD vs. SHALL to be "whatever your debugger will be looking out for, this SHOULD emit, and this SHALL either crash the program or do nothing".

And because we aren't, like the C spec, saying a SHOULD about something as simultaneously vague and important as "well, if you take a pointer, and cast it to an integer, and then do math on that integer, and then cast it back to a pointer, sure, it can be a trap representation, but it SHOULD be whatever would happen on the architecture's address space" ...which went notoriously poorly because implementing that line entirely faithfully stands in the way of a zillion optimizations, whereas this is more like a compiler fence at worst... I think we can expect reasonable everyday tittering about quality-of-implementation to press folks into handling that correctly.

[RalfJung]

I worry we're continually in danger of mixing libs concerns in with lang concerns. What matters for lang is the abstract machine, right? And in that sense is breakpoint any different from an FFI call that may or may not return?

Of course on the libs side (and perhaps compiler side) there are some interesting things but lang doesn't need to worry about any of that because it doesn't concretely affect the AM, no?

This is an intrinsic, i.e. a new language primitive. So yes, it is very different from an FFI call. I don't think saying "it behaves like calling an arbitrary function" is a useful spec. In particular, that spec would make it unsound to expose the intrinsic in a safe function, since you can't possibly justify the soundness of calling an arbitrary FFI function.

The question we are trying to get an answer to boils down to: if I want to verify correctness of my code, what do I have to prove and what can I assume? Even if I don't use formal verification -- if I want to be sure that my unsafe code is sound, and there's a breakpoint in the middle of my code, what do I have to argue and what do I get to assume around this call? Saying it is an arbitrary function would imply I cannot verify correctness of code using the intrinsic (since I cannot know the function), so that's pretty much useless. @hanna-kruppe and me have been arguing for "non-deterministically aborts or returns [or loops infinitely I guess?]" as that seems to capture things well. But there is a strong push towards making the function guarantee something about always aborting when no debugger is attached, without it being clear what I would do with that e.g. as an unsafe code author verifying my code. This is what Hanna asked about here.

I think there has been a lot of miscommunication and different people having different ideas of what it means for a "user to rely on something": 'rely' for the purpose of verification vs 'rely' on a more quality-of-life sense? (And, again, when I say "verification" this doesn't mean "using a formal verification tool", this includes people that want to argue systematically for the correctness of their unsafe code and it includes tools like Miri.)

The thing I primarily want to avoid is, given a target that has a conventional "breakpoint unconditionally" operation and a conventional "breakpoint if debugger attached" operation, mapping breakpoint to the latter rather than the former.

This is very useful, thank you! However, that does sound like a QoI concern to me. For instance, we'd also want to avoid actually making black_box a NOP (it'd be terrible for benchmarking), and yet we document it as a NOP and everything else is QoI.

This is why there have been arguments (that I agree with) that breakpoint should be in hint: to make it clear that there's nothing unsafe code can rely on here. This is a debugging aid, not an operation unsafe code can use to achieve any particular semantics.

That depends on perspective. From the perspective of formal verification, sure, verification has to allow for those possibilities. From the perspective of constraining implementations of compiler backends, targets, etc, "always no-op" isn't a valid impl, and I continue to argue that that guidance should be normative (e.g. what's written in the doc comment on the function, which should probably be expanded).

What makes this operation different from black_box in this regard?