Stream: wg-ffi-unwind

Topic: noexcept-like feature


gnzlbg (Oct 23 2019 at 10:11, on Zulip):

It would be unfortunate to end up with both a noexcept-like feature and ABIs of the form "X" and "X nounwind/unwind" requiring user code to handle both

Amanieu (Oct 23 2019 at 10:18, on Zulip):

I would argue that we do not need the full functionality of noexcept in Rust. In C++, some functions need to work differently if a move constructor might throw an exception since you could end up in a situation with a partially moved value that you can't fully move back without triggering a second exception. The noexcept detection is used to fall back to copy constructors (i.e. Clone in Rust) in such cases. This is the only legitimate use of noexcept detection in C++ that I know of, and it doesn't apply to Rust because moving an object is just a memcpy and is guaranteed not to panic.

Amanieu (Oct 23 2019 at 10:24, on Zulip):

However in Rust I can see a use for #[no_unwind] for utilities such as take_mut where unwinding would leave things in an unsound state.

gnzlbg (Oct 23 2019 at 13:53, on Zulip):

@Amanieu another legitimate use of noexcept in C++ is some algorithmic optimizations

gnzlbg (Oct 23 2019 at 13:54, on Zulip):

maybe that's what you meant above, but if you know that, e.g., a move constructor cannot panic, then you can just avoid doing certain work

gnzlbg (Oct 23 2019 at 13:55, on Zulip):

In Rust, catch_unwind(never_panics) is not free (on the contrary, it is super expensive)

gnzlbg (Oct 23 2019 at 13:56, on Zulip):

so an implementation of catch_unwind that is able to use something like noexcept(never_panics()) could be much better, but for that, such a noexcept feature is not really required

gnzlbg (Oct 23 2019 at 13:56, on Zulip):

(we could have something like needs_drop, that doesn't need to return a reliable answer, and that could be able to do more than just looking at the functions type)

gnzlbg (Oct 23 2019 at 13:58, on Zulip):

@Amanieu

However in Rust I can see a use for #[no_unwind] for utilities such as take_mut where unwinding would leave things in an unsound state.

Isn't that the problem that UnwindSafe solved ? (I'm not sure I understood that part of Rust though)

gnzlbg (Oct 23 2019 at 14:00, on Zulip):

Another legitimate use of noexcept is to just say "this operation never panics and that's part of its API". That's important when violating unsafe invariants.

gnzlbg (Oct 23 2019 at 14:00, on Zulip):

e.g.

vec.set_len(x); // vec invariants violated
// do stuff
ptr::write(...to the vec...);  // vec invariants restored
gnzlbg (Oct 23 2019 at 14:01, on Zulip):

you can defend against "do stuff" panicking, e.g., by having some DropGuard on the stack that restores the vec invariants before letting "do stuff" panic escape

gnzlbg (Oct 23 2019 at 14:03, on Zulip):

but you don't have to do that if "do stuff" does not panic, and you might prefer to simplify your code by not allowing "do stuff" to panic at all

gnzlbg (Oct 23 2019 at 14:04, on Zulip):

with something like needs_drop but for panic, e.g., might_panic, you can just test if a user-provided Fn type might panic, and just abort, but you might want to lift that to a type-error, to let the user know that Fn types that can panic cannot be provided to that particular operation

gnzlbg (Oct 23 2019 at 14:11, on Zulip):

So I think there are two major use cases:

Amanieu (Oct 23 2019 at 14:38, on Zulip):

@gnzlbg My point is, noexcept optimizations in C++ are almost exclusively based on move constructors, and Rust doesn't have that problem since move can never panic in Rust.

gnzlbg (Oct 23 2019 at 14:40, on Zulip):

Is that true?

gnzlbg (Oct 23 2019 at 14:41, on Zulip):

I see noexcept being used for optimizations that require exception safety

gnzlbg (Oct 23 2019 at 14:41, on Zulip):

For example, consider sort in the standard library, which uses merge_sort

gnzlbg (Oct 23 2019 at 14:42, on Zulip):

The input slice is always kept in a valid state because the user operations can panic

gnzlbg (Oct 23 2019 at 14:42, on Zulip):

That is, the elements of the input slice are only swapped

gnzlbg (Oct 23 2019 at 14:43, on Zulip):

If we would be able to tell that the user-provided operation does not panic, we could maybe do a bit better there

gnzlbg (Oct 23 2019 at 14:43, on Zulip):

e.g. leaving "holes" in the slice while operating on it

gnzlbg (Oct 23 2019 at 14:44, on Zulip):

we often deal with such things by using "DropGuard"s, that on panic, restore all broken invariants

gnzlbg (Oct 23 2019 at 14:44, on Zulip):

but those DropGuards are tricky to write correctly, and we have had a couple of issues with them, e.g., in the Iterators of the collections, etc.

gnzlbg (Oct 23 2019 at 14:46, on Zulip):

All use cases of DropGuards in libstd that I know about are optimizations that violate safety invariants that, in case of a panic, need to be restored

Amanieu (Oct 23 2019 at 15:31, on Zulip):

But DropGuard are the best way to solve this problem. They are optimized away when the compiler inlines the user callback and can see that it doesn't panic.

Amanieu (Oct 23 2019 at 15:31, on Zulip):

We are able to restore the structure to a safe state with DropGuard because Rust doesn't have move constructors.

Amanieu (Oct 23 2019 at 15:32, on Zulip):

In C++ you would need to take into account the possibility that even moving objects around can throw exceptions.

gnzlbg (Oct 23 2019 at 17:37, on Zulip):

But DropGuard are the best way to solve this problem. They are optimized away when the compiler inlines the user callback and can see that it doesn't panic.

If the compiler inlines the callback. With noexcept you don't need inlining for this to happen.

Kyle Strand (Oct 23 2019 at 18:11, on Zulip):

It seems the fundamental consideration here is that no_except is strictly an optimization, is never required for correctness, and, regardless of the final design, can be added in a fully backwards-compatible way

Kyle Strand (Oct 23 2019 at 18:12, on Zulip):

So my current view is: it does (probably) fall within the scope of the WG, but does not need to be resolved in order to stabilize cross-FFI-unwinding

Kyle Strand (Oct 23 2019 at 18:13, on Zulip):

Does the first sentence seem accurate and the second reasonable?

Kyle Strand (Oct 23 2019 at 18:13, on Zulip):

If so, I'm happy for continued discussion to take place here, but I would like it to be "officially" considered a low priority for now

Kyle Strand (Oct 23 2019 at 18:14, on Zulip):

(But definitely something that our first technical RFC should mention in the "future possibilities" section)

centril (Oct 23 2019 at 22:04, on Zulip):

I think nopanic fn or some such should be viewed in the same light as const fn -- it is a restriction to the base effect fn has which may 1) diverge, 2) panic, and 3) cause side-effects.

So I think there are two major use cases:

I think there's interest in embedded to prevent panics.
Fundamentally, restrictions like const (interpreted as "pure") and "cannot panic" also improve code quality beyond usefulness for soundness.
It's a way to be sure that you've handled more cases, enhancing strong typing. I think this is desirable, but way out of scope for this WG.

centril (Oct 23 2019 at 22:04, on Zulip):

(I also think provable convergence (e.g. total fn) is desirable, but that's more difficult to make ergonomic)

Amanieu (Oct 23 2019 at 22:06, on Zulip):

Note that what we proposed for a nopanic attribute is not what you described. Instead, if unwinding reaches a nopanic, it aborts the process instead of continuing to unwind.

centril (Oct 23 2019 at 22:12, on Zulip):

Seems strange to me, especially when combined with a closure

centril (Oct 23 2019 at 22:13, on Zulip):
pub fn take<T, F>(mut_ref: &mut T, closure: impl nopanic FnOnce(T) -> T);
//~^ You cannot pass a closure that will panic; the type system will prevent it
Kyle Strand (Oct 23 2019 at 22:16, on Zulip):

The attribute being discussed here is for optimization, not an effects system. The effects system "can't panic" type does sound interesting, but isn't what #[no_unwind] would be.

centril (Oct 23 2019 at 22:21, on Zulip):

When take_mut is being discussed that is for soundness; not optimization.

This #[no_unwind]... is it attached just to function definitions? function pointers? closures? what exactly?

Amanieu (Oct 23 2019 at 22:32, on Zulip):

I think there's a bit of confusion here because there are actually two similar but different attributes being discussed here:
- First, an attribute that you can place on (FFI) declarations which (unsafely) tells the compiler that this function will never unwind. This is basically the LLVM nounwind attribute.
- Second, an attribute on a function definition which forces unwinding to abort the process if it reaches this function. Incidentally, this function can also be marked with LLVM's nounwind since no exceptions will ever unwind out of it (they'll abort first).

centril (Oct 23 2019 at 22:33, on Zulip):

This seems to be #[unwind(aborts)]?

Kyle Strand (Oct 23 2019 at 23:26, on Zulip):

Yes, I believe that's the main thing being discussed that we believe would be in scope for this project

gnzlbg (Oct 24 2019 at 07:17, on Zulip):

It seems the fundamental consideration here is that no_except is strictly an optimization, is never required for correctness,

@Kyle Strand How do you arrive at this conclusion? The example I showed is only correct if some code cannot ever unwind, i.e., without noexcept, such unsafe code is unsound.

gnzlbg (Oct 24 2019 at 07:18, on Zulip):

That's not an optimization, but a requirement for correctness.

gnzlbg (Oct 24 2019 at 07:25, on Zulip):

@centril

I think there's interest in embedded to prevent panics.

There are many users of dtolnay's #[no_panic] crate, which has a proc-macro that wraps a function, in such a way that it only compiles if it cannot panic. It does so by wrapping the function in another function, that contains a type in the stack with a destructor that's only called if the original function can panic. That destructor is implemented to call an external symbol that does not exist. So if the destructor call is optimized away because the function is proven not to panic, then your program compiles. Otherwise, your program fails to link.

The existence of users that are willing to go to such hacky extends to make sure that a function cannot panic shows that there is more than interest, there is an active need.

gnzlbg (Oct 24 2019 at 07:29, on Zulip):

The attribute being discussed here is for optimization, not an effects system.

@Kyle Strand This is unclear to me, since @Amanieu also claimed that they would be good with a "C nounwind" ABI string that makes this part of the type system.

gnzlbg (Oct 24 2019 at 07:30, on Zulip):

If the only goal is optimizations, then making it part of the type system would be a bad idea.

gnzlbg (Oct 24 2019 at 07:31, on Zulip):

If it makes sense to make this part of the type system, then designing the feature only for optimizations and neglecting how it impacts the type system as a whole is probably a bad idea as well.

gnzlbg (Oct 24 2019 at 07:32, on Zulip):

I think a #[no_unwind] attribute for FFI declarations only would be ok

gnzlbg (Oct 24 2019 at 07:32, on Zulip):

For extern "C" fn foo() { ... } definitions in Rust, the attribute should not be necessary

gnzlbg (Oct 24 2019 at 07:32, on Zulip):

Since the compiler should be able to properly deduce whether that function can or cannot be nounwind

Kyle Strand (Oct 28 2019 at 20:34, on Zulip):

It seems the fundamental consideration here is that no_except is strictly an optimization, is never required for correctness,

Kyle Strand How do you arrive at this conclusion? The example I showed is only correct if some code cannot ever unwind, i.e., without noexcept, such unsafe code is unsound.

Looking back, I have to admit I'm not entirely sure what I meant by that. But if ensuring a function can't panic is required for correctness, you can just put a catch_panic at the top with an abort statement. So I think what I was trying to say is that there is no case in which users would be unable to write some particular functionality _without_ noexcept that would be enabled by noexcept. Is that true?

gnzlbg (Oct 29 2019 at 07:46, on Zulip):

No.

gnzlbg (Oct 29 2019 at 07:48, on Zulip):

If catch_unwind were a zero-cost abstraction, then maybe. But a catch_unwind is extremely expensive, and if you are targeting a constrained environment, you can't use it.

gnzlbg (Oct 29 2019 at 07:48, on Zulip):

noexcept removes the need for the catch_unwind, enabling that

Kyle Strand (Oct 29 2019 at 14:22, on Zulip):

Hm. How would it remove that need? I am imagining here that noexcept would have the abort-on-unwind behavior that we want for extern "C". Why is catch_unwind more expensive than that would be?

gnzlbg (Oct 29 2019 at 15:59, on Zulip):

noexcept makes it a compiler error instead, removing any run-time cost

gnzlbg (Oct 29 2019 at 16:00, on Zulip):

catch_unwind is probably just implemented "poorly"

gnzlbg (Oct 29 2019 at 16:01, on Zulip):

it's a library feature, not a language construct, and whatever LLVM-IR we generate doesn't get optimized, probably because of calls to unknown functions

Kyle Strand (Oct 29 2019 at 16:07, on Zulip):

We discussed two variants of noexcept. If it's an effect annotation, that's out of scope. If it's like C++'s feature (which std::terminates when an exception is thrown), that's probably in scope.

gnzlbg (Oct 29 2019 at 16:08, on Zulip):

Did you understood the code example that was shown above ?

Kyle Strand (Oct 29 2019 at 16:09, on Zulip):

Centril's?

gnzlbg (Oct 29 2019 at 16:09, on Zulip):

All of them

gnzlbg (Oct 29 2019 at 16:22, on Zulip):

(there are 3 examples I think)

gnzlbg (Oct 29 2019 at 16:22, on Zulip):

The second one shows the issue

Kyle Strand (Oct 29 2019 at 16:35, on Zulip):

The one about temporarily violating invariants e.g. w/ vec.set_len?

gnzlbg (Oct 29 2019 at 17:42, on Zulip):

Yep, noexcept makes such examples sound

Kyle Strand (Oct 29 2019 at 17:53, on Zulip):

As does catch_unwind + abort. If there's an implementation reason why catch_unwind can't be used somewhere that noexcept (the version that causes an abort) would be usable, that seems separate from the consideration of whether noexcept is "required for correctness".

gnzlbg (Oct 29 2019 at 18:02, on Zulip):

_Again_, what you are proposing is for the error to be reported at run-time by aborting.

gnzlbg (Oct 29 2019 at 18:03, on Zulip):

If you want to ignore that, that's ok.

gnzlbg (Oct 29 2019 at 18:03, on Zulip):

I prefer code like let x: u32 = 0_f32; to error at compile-time due to a type error

gnzlbg (Oct 29 2019 at 18:03, on Zulip):

Sure, we could also make let x: u32 = 0_f32; "sound" by aborting the program at run-time

gnzlbg (Oct 29 2019 at 18:04, on Zulip):

but if I wanted a language that does that I would be using Python, and not Rust

Kyle Strand (Oct 29 2019 at 18:05, on Zulip):

I'm not disagreeing that a compile-time error is preferable or even necessarily "proposing" something as such. I'm just saying that there are two "noexcept-like" features discussed above, and I don't believe that the "effects system" one is within the scope of this project group.

gnzlbg (Oct 29 2019 at 18:07, on Zulip):

I thought you were asking "what problems does noexcept solve", which is something that has been discussed above

Kyle Strand (Oct 29 2019 at 18:07, on Zulip):

I am also trying to respond to your comment about unsoundness. The "effects system" version is a way to strengthen the type system to remove some of the soundness proof-burden from the user. The "abort on unwind" version is a way to ensure soundness at runtime via a mechanism that users could in theory implement themselves. Neither is "required" for soundness, though.

gnzlbg (Oct 29 2019 at 18:08, on Zulip):

You are forgetting the guarantee that noexcept provides for callers

gnzlbg (Oct 29 2019 at 18:08, on Zulip):

which is that something won't unwind, ever

gnzlbg (Oct 29 2019 at 18:09, on Zulip):

this means, they don't even need to add a test to verify the behavior of the program if that something were to unwind, because it is something that, if the program compiles, is guaranteed not to happen

gnzlbg (Oct 29 2019 at 18:09, on Zulip):

and that's the property of noexcept that the example above requires

Kyle Strand (Oct 29 2019 at 18:10, on Zulip):

Without a built-in effects system, callers are responsible for knowing whether function effects can cause unsoundness. That's why an effects system would be nice, but that doesn't mean it's in scope for this project group.

gnzlbg (Oct 29 2019 at 18:10, on Zulip):

C++ doesn't have an effect system, but it has this feature

gnzlbg (Oct 29 2019 at 18:10, on Zulip):

adding "C unwind" "C nounwind" is equivalent to adding noexcept

Kyle Strand (Oct 29 2019 at 18:10, on Zulip):

not...really....

Kyle Strand (Oct 29 2019 at 18:10, on Zulip):

Do you mean "C nounwind"?

gnzlbg (Oct 29 2019 at 18:10, on Zulip):

Do you mean "C nounwind"?

Yes, or equivalently adding "C unwind" and making "C" nounwind

Kyle Strand (Oct 29 2019 at 18:11, on Zulip):

Okay. Again, I'm not trying to debate the merits of having nounwind in the type system, or anything like that.

Kyle Strand (Oct 29 2019 at 18:12, on Zulip):

I agree that "C nounwind" and/or guaranteeing the abort behavior of "C" would both provide useful guarantees that aren't available in the type system today.

gnzlbg (Oct 29 2019 at 18:12, on Zulip):

Adding "C unwind" makes "C" nounwind and adds noexcept to Rust in some way

Kyle Strand (Oct 29 2019 at 18:12, on Zulip):

Right

gnzlbg (Oct 29 2019 at 18:13, on Zulip):

Doing that, and then also adding another noexcept feature, would be adding two subtly different features to do the same thing to the language

Kyle Strand (Oct 29 2019 at 18:13, on Zulip):

Yes, but they're semantically different in important ways, and I don't think it would be inappropriate to do so.

gnzlbg (Oct 29 2019 at 18:13, on Zulip):

So it is in scope for this working group to say why two different features instead of just one.

gnzlbg (Oct 29 2019 at 18:13, on Zulip):

They are identical

gnzlbg (Oct 29 2019 at 18:14, on Zulip):

"C unwind" nounwind == "C"

Kyle Strand (Oct 29 2019 at 18:14, on Zulip):

Um....not really. Because the extra noexcept feature wouldn't fall within the scope of this group, so the burden to explain why it's non-redundant would be on whoever proposes it.

Kyle Strand (Oct 29 2019 at 18:14, on Zulip):

Oh, wait, you're saying that "C" and "C nounwind" would be subtly different?

gnzlbg (Oct 29 2019 at 18:15, on Zulip):

Um....not really. Because the extra noexcept feature wouldn't fall within the scope of this group, so the burden to explain why it's non-redundant would be on whoever proposes it.

I strongly disagree. This process would result in thousand of small incoherent features being added to the language.

Kyle Strand (Oct 29 2019 at 18:15, on Zulip):

I don't agree with that. "C nounwind" would just be an explicit way of writing "C", just like extern "Rust" is merely an explicit way of specifying the default ABI.

gnzlbg (Oct 29 2019 at 18:15, on Zulip):

Because it would always be somebodyelses job to see the big picture.

gnzlbg (Oct 29 2019 at 18:15, on Zulip):

Oh, wait, you're saying that "C" and "C nounwind" would be subtly different?

No, I precisely meant what I said.

Kyle Strand (Oct 29 2019 at 18:16, on Zulip):

I'm not accusing you of misspeaking; I'm trying to understand this comment:

"C unwind" nounwind == "C"

gnzlbg (Oct 29 2019 at 18:16, on Zulip):

nounwind extern "C unwind" fn foo() {} would be semantically identical to extern "C" fn foo() {}

gnzlbg (Oct 29 2019 at 18:17, on Zulip):

therefore by adding the "C" and "C unwind" ABIs distinction to the language one is actually adding nounwind to the language

gnzlbg (Oct 29 2019 at 18:17, on Zulip):

just with less explicit syntax, less powerful, and probably in such a way that actually making it more powerful would require a different feature that would subsume it and deprecate it

gnzlbg (Oct 29 2019 at 18:18, on Zulip):

that's the opposite of "future proofing" a feature

Kyle Strand (Oct 29 2019 at 18:19, on Zulip):

When you write nounwind outside of an extern ABI specifier, is that your hypothetical syntax for what I'm calling an "effects system" nounwind, i.e., purely a compile-time guarantee, with no runtime abort?

Kyle Strand (Oct 29 2019 at 18:36, on Zulip):

If so, I agree that those are semantically identical from the perspective of the caller, though specifying both "C unwind" and nounwind seems...confused at best.

Kyle Strand (Oct 29 2019 at 18:37, on Zulip):

But they have very different impacts on the author of the function, of course.

Kyle Strand (Oct 29 2019 at 18:39, on Zulip):

And, since "C" already exists in the language, we can't introduce a "safe-to-unwind" ABI and stabilize the desired abort-on-unwind behavior for "C" without having such an "implicit nounwind". That is maybe a good argument to add to Niko's document about the default behavior of "C", but I don't really see it as a reason not to introduce "C unwind".

Amanieu (Oct 29 2019 at 18:49, on Zulip):

Note that we already have an equivalent of noexcept in (unstable) Rust: #[unwind(aborts)]. Literally the only difference is that it calls intrinsics::trap (Rust) instead of std::terminate (C++).

gnzlbg (Oct 30 2019 at 08:41, on Zulip):

@Amanieu the other difference is that #[unwind(aborts)] is not part of a function type

gnzlbg (Oct 30 2019 at 08:41, on Zulip):

So users can't really do any kind of type level reasoning about this

gnzlbg (Oct 30 2019 at 08:42, on Zulip):

@Kyle Strand i agree that the situation might be confusing, which is why I think that we should at least recognize the impact of "C" vs "C unwind" at that level, and be able to talk about it in this project group and its RFCs

gnzlbg (Oct 30 2019 at 08:47, on Zulip):

If you take a look at my first example here: https://rust-lang.zulipchat.com/#narrow/stream/210922-wg-ffi-unwind/topic/noexcept-like.20feature/near/178839302, if we add "C unwind" and guarantee that "C" never unwinds I can use that trick to write code like this:

trait NeverUnwinds {}
impl<Ret, Args...> NeverUnwinds for extern "C" fn(Args...) -> Ret {}

/*safe */ fn duplicate_map<F>(x: &mut Vec<i32>, map: F)
where F: Fn(i32)->i32 + NeverUnwinds
{
      unsafe {
       // safe because `map`is safe and it  never unwinds
       // no dropgurad, no catch_unwind
       let new_len = x.len() * 2;
       x.reserve(new_len);
       x.set_len(new_len);
       for ... { ptr.write(..., map(ptr.read(...)) }
    }
}

that's a very poor's man noexcept feature, with the exact same power (except for noexcept(expr)), but worse ergonomics.

gnzlbg (Oct 30 2019 at 08:50, on Zulip):

While we are not adding this feature to enable this use case, it turns out that "by accident" this is a very powerful thing that the feature enables.

gnzlbg (Oct 30 2019 at 08:51, on Zulip):

I think that these super-powerful use-cases by accident are worth studying, e.g., C++ templates were not intended for meta-programming, yet here we are.

Kyle Strand (Oct 30 2019 at 14:34, on Zulip):

How is that more powerful than map: extern "C" fn(i32) -> i32? I don't see how the NeverUnwindd trait could be extended in any compile-checked way to other function-like types. For instance, duplicate_map itself could be made to not unwind by aborting if the reserve triggers OOM, but I don't see how to encode that in any sort of compile-checked way.

gnzlbg (Oct 30 2019 at 14:39, on Zulip):

You are right, the reserve should be a: assert!(capacity() >= 2 * len())

gnzlbg (Oct 30 2019 at 14:39, on Zulip):

Ah wait, no, not necessarily, it is ok for reserve to panic

gnzlbg (Oct 30 2019 at 14:40, on Zulip):

so that is not a problem sorry

gnzlbg (Oct 30 2019 at 14:40, on Zulip):

the problem is map panicking after the x.set_len has been called

gnzlbg (Oct 30 2019 at 14:40, on Zulip):

that's the only thing that's required not to panic for correctness

gnzlbg (Oct 30 2019 at 14:40, on Zulip):

I don't see how the NeverUnwindd trait could be extended in any compile-checked way to other function-like types.

What do you mean ?

Kyle Strand (Oct 30 2019 at 15:10, on Zulip):

It's not unsafe for it to panic in your example, but I'm talking about giving duplicate_map the NeverUnwind trait, in which case it would be unsafe for it to panic

Kyle Strand (Oct 30 2019 at 15:11, on Zulip):

My point is that I don't see how this "poor man's noexcept" does anything except alias the extern "C" fn types

gnzlbg (Oct 30 2019 at 15:35, on Zulip):

@Kyle Strand the way to give duplicate_map the trait is just make it an extern "C" function

gnzlbg (Oct 30 2019 at 15:36, on Zulip):

in which case, I don't understand why you say that it would be unsafe for it to panic: if it panics, the program aborts, and the destructor of the vector is never run, but that doesn't matter because the invariant of the vector was not violated when reserve is called

gnzlbg (Oct 30 2019 at 15:37, on Zulip):
/* safe */ extern "C" fn duplicate_map_nounwind<F>(x: &mut Vec<i32>, map: F)
where F: Fn(i32)->i32 + NeverUnwinds
{
      unsafe {
       // safe because `map`is safe and it  never unwinds
       // no dropgurad, no catch_unwind
       let new_len = x.len() * 2;
       // this can panic, and that's ok
       x.reserve(new_len);
       x.set_len(new_len);
       for ... { ptr.write(..., map(ptr.read(...)) }
    }
}
gnzlbg (Oct 30 2019 at 15:39, on Zulip):

This means that extern "C" is now a "performance" feature.

gnzlbg (Oct 30 2019 at 15:39, on Zulip):

Just like people stamp noexcept everywhere in C++, people might want to stamp extern "C" everywhere in Rust

Kyle Strand (Oct 30 2019 at 15:49, on Zulip):

If the NeverUnwind trait can _only ever_ apply to extern "C" fn, then no, we are not accidentally introducing a new "powerful" feature.

Kyle Strand (Oct 30 2019 at 15:49, on Zulip):

If people want to modify many functions at once to abort instead of panic, the right way to do that is, of course, panic = abort.

Kyle Strand (Oct 30 2019 at 15:54, on Zulip):

Conversely, if you're just saying that people will take applications that need to have unwinding somewhere in them and optimize them by adding extern "C" elsewhere, then we're back to my statement that this is "strictly an optimization".

Kyle Strand (Oct 30 2019 at 15:58, on Zulip):

I will say that this all reminds me of something I was told by the LLVM dev I've mentioned before: "FFI isn't a sandbox"

Kyle Strand (Oct 30 2019 at 15:58, on Zulip):

The full quote:

If the callee can unwind, then it can unwind. FFI isn't a sandbox, it's not supposed to change behaviour or protect the caller. At the extreme I can recognize that some mapping of types is done in some systems (strings are a common one). On Linux, exceptions are primitive and well-defined much like 32-bit integers, despite the lack of support in C[1].

[1] - You can both throw and capture exceptions in C. The language doesn't define a syntax, but it doesn't need to. It doesn't define a syntax for networking or for threading either. Just include the header file and use the appropriate API.

Kyle Strand (Oct 30 2019 at 16:00, on Zulip):

....which would of course be an argument for allowing extern "C" to unwind by default (as @Amanieu suggested), and _not_ introducing "C unwind" or "C nounwind".

Kyle Strand (Oct 30 2019 at 16:02, on Zulip):

My main concern there is that the claim is so Linux-specific, though it's true that SEH on Windows is also always useable even in C code.

gnzlbg (Oct 31 2019 at 08:21, on Zulip):

If the NeverUnwind trait can _only ever_ apply to extern "C" fn, then no, we are not accidentally introducing a new "powerful" feature

I disagree, since without "C unwind" people cannot write the NeverUnwind trait, but if we add "C unwind" to the language, they can.

I will say that this all reminds me of something I was told by the LLVM dev I've mentioned before: "FFI isn't a sandbox"

That quote is about FFI, while we are here talking about noexcept or LLVM's nounwind attribute.

gnzlbg (Oct 31 2019 at 11:00, on Zulip):

Thinking more about how to use it, another cool way to use this feature is:

trait NoUnwindCall {
    type Arg;
}
Kyle Strand (Oct 31 2019 at 14:13, on Zulip):

Without "C unwind" people cannot write the NeverUnwind trait, but if we add "C unwind" to the language, they can.

My point is that NeverUnwind is not particularly powerful, since there's no way to extend it to other types of functions, so it doesn't expose any abilities beyond what you'd get from using extern "C" directly. But, also, this has nothing to do with "C unwind" except insofar as the already planned change in behavior to make extern "C" abort-on-unwind hasn't yet been successfully stabilized because there was no way to opt out (which is what unwind would provide).

Kyle Strand (Oct 31 2019 at 14:15, on Zulip):

any abilities beyond what you'd get from using extern "C" directly...

This is why I said above that duplicate_map could just take map as an extern "C" fn directly, since there's no other possible type that can implement that trait.

Kyle Strand (Oct 31 2019 at 14:16, on Zulip):

Re: NoUnwindCall, that's just the equivalent of making an extern "C" lambda, but with a more confusing syntax.

Kyle Strand (Oct 31 2019 at 14:19, on Zulip):

That quote is about FFI, while we are here talking about noexcept or LLVM's nounwind attribute.

Right, but your "ABI dance" is just abusing a feature intended for FFI to create such a "sandbox".

Amanieu (Oct 31 2019 at 14:26, on Zulip):

I'll just repeat what I said before: NeverUnwind trait isn't that useful in practice because you can always use a drop guard instead, which allows you to accept any kind of callback, not just ones that never unwind. I don't think we should be basing our design decisions on a hypothetical trait which is almost never going to be used in practice.

gnzlbg (Oct 31 2019 at 15:40, on Zulip):

@Amanieu DropGuards do not let you tell the optimizer that a call does not unwind

gnzlbg (Oct 31 2019 at 15:40, on Zulip):

The problem NeverUnwind solves is the same problem that the #[no_panic] crate solves

gnzlbg (Oct 31 2019 at 15:40, on Zulip):

DropGuards have nothing to do with that

gnzlbg (Oct 31 2019 at 15:41, on Zulip):

@Kyle Strand

But, also, this has nothing to do with "C unwind" except insofar as the already planned change in behavior to make extern "C" abort-on-unwind hasn't yet been successfully stabilized because there was no way to opt out (which is what unwind would provide).

Right now one cannot assume that an extern "C" function never unwinds. The "C unwind" proposal _guarantees_ that extern "C" functions never unwind.

Kyle Strand (Oct 31 2019 at 15:42, on Zulip):

No, those are separate. The "C unwind" proposal is partly _motivated_ by the fact that we've previously been unable to specify the already-planned never-unwind behavior.

Kyle Strand (Oct 31 2019 at 15:42, on Zulip):

But the "C unwind" proposal does not _introduce_ that idea.

gnzlbg (Oct 31 2019 at 15:42, on Zulip):

Not in @nikomatsakis RFC, which starts by saying that we insert abort-on-panic shims for extern "C" functions (that's the first thing that RFC changes in the language, and what enables NeverUnwind above).

gnzlbg (Oct 31 2019 at 15:44, on Zulip):

A different RFC might say that extern "C" functions can unwind just fine, and do not provide any way to make a function nounwind, making it impossible for NeverUnwind to be written in Rust.

gnzlbg (Oct 31 2019 at 15:44, on Zulip):

Right now both options are on the table, the "C unwind" RFC picks one of them.

Kyle Strand (Oct 31 2019 at 15:45, on Zulip):

...right, with a link to a rust issue stating that the behavior was originally introduced in Rust 1.24.

gnzlbg (Oct 31 2019 at 15:46, on Zulip):

If that was really the case, allowing extern "C" to unwind would be a breaking change

Kyle Strand (Oct 31 2019 at 15:46, on Zulip):

I.e., that is not a change to the intended defined-behavior of extern "C"; it just implements that behavior and makes it no longer undefined.

Kyle Strand (Oct 31 2019 at 15:46, on Zulip):

Yes, Rust 1.24.1 has a breaking change from 1.24.0

gnzlbg (Oct 31 2019 at 15:47, on Zulip):

IIUC, Rust 1.24.0 still said that unwinding from an extern "C" function is UB

gnzlbg (Oct 31 2019 at 15:47, on Zulip):

it did not provide any guarantees

gnzlbg (Oct 31 2019 at 15:47, on Zulip):

it just implemented that UB to abort

Kyle Strand (Oct 31 2019 at 15:48, on Zulip):

https://blog.rust-lang.org/2018/02/15/Rust-1.24.html#other-good-stuff

Kyle Strand (Oct 31 2019 at 15:48, on Zulip):

In Rust 1.24, this code will now abort instead of producing undefined behavior.

gnzlbg (Oct 31 2019 at 15:48, on Zulip):

Indeed. Well then this feature is already in the language, and allowing extern "C" to unwind isn't on the table

Kyle Strand (Oct 31 2019 at 15:48, on Zulip):

....

Kyle Strand (Oct 31 2019 at 15:49, on Zulip):

the unwind behavior was immediately restored in 1.24.1

gnzlbg (Oct 31 2019 at 15:49, on Zulip):

so... then the "C unwind" rfc introduces this feature in the language

Kyle Strand (Oct 31 2019 at 15:49, on Zulip):

because 1.24.0 broke actual existing code

Kyle Strand (Oct 31 2019 at 15:50, on Zulip):

I believe my original wording was "permits stabilizing" that behavior. We can restore that verbiage if it's clearer.

gnzlbg (Oct 31 2019 at 15:50, on Zulip):

You are confusing me. Either this feature is in the language already or not. And if it isn't, either "C unwind" introduces it or not.

gnzlbg (Oct 31 2019 at 15:50, on Zulip):

My original claim is that right now it isn't in the language, and the "C unwind" RFC introduces it.

Kyle Strand (Oct 31 2019 at 15:51, on Zulip):

The plan has been in the language for a long, long, long time. Because there's only one implementation and we don't have a formal spec, whether or not a planned feature is "in the language" is a bit of a hazy question, I think.

Kyle Strand (Oct 31 2019 at 15:52, on Zulip):

But, yes, if the "C unwind" RFC (in its final form, not its draft form) says something like "we stabilize the unwind-on-abort behavior", then yes, once the RFC spec is actually implemented, that will be a breaking change.

gnzlbg (Oct 31 2019 at 15:52, on Zulip):

it's not a breaking change

gnzlbg (Oct 31 2019 at 15:52, on Zulip):

right now the behavior is undefined

gnzlbg (Oct 31 2019 at 15:52, on Zulip):

defining undefined behavior is not a breaking change

Kyle Strand (Oct 31 2019 at 15:52, on Zulip):

I disagree. Wasmer and Lucet have relied on that behavior.

gnzlbg (Oct 31 2019 at 15:53, on Zulip):

They know about it, and they can upgrade to "C unwind" afterwards

Kyle Strand (Oct 31 2019 at 15:53, on Zulip):

So, changing it breaks the functionality of their previously-working code, regardless of whether their previously-working code was "guaranteed" by the Rust Reference to work.

Kyle Strand (Oct 31 2019 at 15:53, on Zulip):

RIght.

Kyle Strand (Oct 31 2019 at 15:53, on Zulip):

Because we'd be changing the behavior of extern "C" in the implementation.

gnzlbg (Oct 31 2019 at 15:54, on Zulip):

I disagree, that code is already broken.

gnzlbg (Oct 31 2019 at 15:54, on Zulip):

And has always been.

Kyle Strand (Oct 31 2019 at 15:54, on Zulip):

I'm not going to re-hash that argument here; it was pretty played out in the original thread about mozjpeg.

gnzlbg (Oct 31 2019 at 15:55, on Zulip):

Which is why i don't know why you are bringing it up again.

Kyle Strand (Oct 31 2019 at 15:55, on Zulip):

The point is: the design for extern "C unwind" does not necessarily imply the behavior of extern "C". For the last several years, the lang team has been consistent in stating that the desired behavior for extern "C" to have "someday" is abort-on-unwind.

gnzlbg (Oct 31 2019 at 15:55, on Zulip):

You claim that defining undefined behavior is a breaking change. The logical consequence of that is that we cannot define undefined behavior, and therefore, we can't change extern "C" to anything (neither allow unwinding, nor disallow it).

gnzlbg (Oct 31 2019 at 15:57, on Zulip):

I disagree with the claim, and the logical consequence of that is that we can define the behavior of extern "C" to whatever we want.

Amanieu (Oct 31 2019 at 15:57, on Zulip):

The main issue was that #[unwind(allowed)] was still unstable, which mean there was no way for people to fix their code to no longer make it UB.

gnzlbg (Oct 31 2019 at 15:58, on Zulip):

Sure, but that's a practical issue orthogonal to whether "defining UB to something" is a breaking change or not.

gnzlbg (Oct 31 2019 at 15:59, on Zulip):

Just because we can define the behavior of extern "C" from undefined to something else does not mean that we should do it, e.g., until a different feature is added to the language.

Kyle Strand (Oct 31 2019 at 15:59, on Zulip):

I think there are a couple logical leaps there, but in any case that's beside the point. All I'm trying to say is that, until an RFC saying otherwise lands, we can still decide not to stabilize extern "C"'s abort-on-unwind behavior, independently of introducing the extern "C unwind" feature.

gnzlbg (Oct 31 2019 at 16:00, on Zulip):

Sure, we could change the "C unwind" RFC to not guarantee that extern "C" panics on abort, that prevents NeverUnwinds from working

Kyle Strand (Oct 31 2019 at 16:00, on Zulip):

Therefore, it is not true the extern "C unwind" introduces the "noexcept" concept; it is true that the currently planned guarantee for extern "C" would introduce the "noexcept" concept (though, again, I think it's much weaker than you've stated above).

gnzlbg (Oct 31 2019 at 16:01, on Zulip):

I'm not sure where you are going. Do you want to remove the abort-on-panic guarantee from the "C unwind" RFC?

Kyle Strand (Oct 31 2019 at 16:02, on Zulip):

This is why I want to know what Centril thinks of the "FFI is not a sandbox" quote. The lang team has been planning (since long before I got involved with this effort) to make extern "C" abort-on-unwind, which, as you're now pointing out, creates a less-powerful noexcept-like feature. If the lang team thinks this is an okay thing to introduce this way, well, okay. If this is something they hadn't considered before and makes them reconsider the abort-on-unwind issue, that's also okay.

gnzlbg (Oct 31 2019 at 16:03, on Zulip):

again, I think it's much weaker than you've stated above

What doesnoexcept allow that extern "C" does not ?

Kyle Strand (Oct 31 2019 at 16:03, on Zulip):

You can apply it to Fns, not just fns, for one

Kyle Strand (Oct 31 2019 at 16:03, on Zulip):

(Or at least, I would expect it to be designed in such a way)

gnzlbg (Oct 31 2019 at 16:04, on Zulip):

extern "C" fn foo<F: Fn(...) -> ...>(f: F) -> ... { f(...) }

gnzlbg (Oct 31 2019 at 16:04, on Zulip):

You can trivially wrap Fn traits in extern "C" functions to apply it.

Kyle Strand (Oct 31 2019 at 16:04, on Zulip):

Again, that might as well be a lambda

gnzlbg (Oct 31 2019 at 16:04, on Zulip):

That's what https://rust-lang.zulipchat.com/#narrow/stream/210922-wg-ffi-unwind/topic/noexcept-like.20feature/near/179520449 does

Kyle Strand (Oct 31 2019 at 16:04, on Zulip):

But, yes, it allows such wrapping

Kyle Strand (Oct 31 2019 at 16:05, on Zulip):

That's what https://rust-lang.zulipchat.com/#narrow/stream/210922-wg-ffi-unwind/topic/noexcept-like.20feature/near/179520449 does

Yes, I remember, and I responded there!

gnzlbg (Oct 31 2019 at 16:05, on Zulip):

Again, that might as well be a lambda

Do you have a playground link? extern "C" | ... | ... is invalid syntax.

gnzlbg (Oct 31 2019 at 16:06, on Zulip):

Or do you mean that the lambda can coerce to an extern "C" fn type if it captures no environment ?

gnzlbg (Oct 31 2019 at 16:07, on Zulip):

(notice that the trait works for closures with an environment)

Kyle Strand (Oct 31 2019 at 16:08, on Zulip):

I was thinking of something else from the RFC draft, about converting different ABIs to each other:

It is
252-not possible, for example, to cast a `extern "C" fn()` value into a `extern "C
253-unwind" fn()` value. It is however possible to convert between ABIs by using a
254-closure expression (which is then coerced into a standalone function):
255-
256-```rust
257-fn convert(f: extern "C" fn()) -> extern "C unwind" fn() {
258:    return || f();
259-}
260-```
gnzlbg (Oct 31 2019 at 16:10, on Zulip):

Yes, that works, but requires the lambda to have no environment.

Kyle Strand (Oct 31 2019 at 16:10, on Zulip):

Obviously you'd change the signature, from extern "C" fn -> extern "C unwind fn to Fn -> extern "C" fn, but it's the same idea

Kyle Strand (Oct 31 2019 at 16:10, on Zulip):

Fair enough.

gnzlbg (Oct 31 2019 at 16:10, on Zulip):

The trait works even if the lambda captures an environment, it works for all Fn traits in general

gnzlbg (Oct 31 2019 at 16:11, on Zulip):

The problem I mentioned is that both the trait and the coercion need to "adapt" the ABIs

gnzlbg (Oct 31 2019 at 16:11, on Zulip):

fn convert(f: extern "Rust" fn()) -> extern "C unwind" fn() { || f() } needs to adapt the Rust ABI (or any other) to the "C unwind" ABI

Kyle Strand (Oct 31 2019 at 16:11, on Zulip):

Right...

gnzlbg (Oct 31 2019 at 16:12, on Zulip):

so you end up with an ABI dance that hopefully LLVM removes, but its at least in the language

gnzlbg (Oct 31 2019 at 16:12, on Zulip):

A proper noexcept feature removes this, but that's the only thing I can think of in which they differ.

gnzlbg (Oct 31 2019 at 16:13, on Zulip):

That makes extern "C" only slightly less powerful than noexcept in my eyes, unless I'm missing other differences. A lot less ergonomic though.

Kyle Strand (Oct 31 2019 at 16:13, on Zulip):

Anyway: I don't really have an opinion on whether introducing the panic-on-abort logic in extern "C" is a great idea. Again, that's why I want the lang team to weigh in on it, since they've been planning it for a long time. If they're committed to it, I don't think it matters one way or the other whether it stabilizes as part of the extern "C unwind" RFC or separately.

gnzlbg (Oct 31 2019 at 16:14, on Zulip):

It kind of needs to either stabilize at the same time as "C unwind" or afterwards, because otherwise Lucene breaks without an easy migration path.

Kyle Strand (Oct 31 2019 at 16:14, on Zulip):

Right, it's the "or afterwards" on which I have no opinion

gnzlbg (Oct 31 2019 at 16:15, on Zulip):

It's kind of weird to add a "C unwind" ABI while reserving the right to allow "C" to unwind

Kyle Strand (Oct 31 2019 at 16:16, on Zulip):

:shrug:

Kyle Strand (Oct 31 2019 at 16:16, on Zulip):

I don't think it's that weird, because "C unwind" is more explicit and will have well-defined behavior.

gnzlbg (Oct 31 2019 at 16:17, on Zulip):

If "C" is defined to allow unwinding, then it has defined behavior as well, and current code works correctly without changes.

Kyle Strand (Oct 31 2019 at 16:18, on Zulip):

....right

gnzlbg (Oct 31 2019 at 16:18, on Zulip):

So I think it does make sense to do both as part of the same RFC.

Kyle Strand (Oct 31 2019 at 16:18, on Zulip):

that's...fine

Kyle Strand (Oct 31 2019 at 16:18, on Zulip):

again, I have no opinion

gnzlbg (Oct 31 2019 at 16:18, on Zulip):

Otherwise its kind of a bit of a big unresolved question.

Kyle Strand (Oct 31 2019 at 16:19, on Zulip):

Leaving questions unresolved in the language spec but promising to resolve them eventually is very nearly the entire purpose of this group!

Kyle Strand (Oct 31 2019 at 16:20, on Zulip):

I think C++'s approach of letting extern "C" functions unwind is entirely reasonable. If the mozjpeg discussion had led to the language team agreeing with this position, and deciding to let extern "C" functions unwind and not consider them UB, then yeah, we wouldn't need "C unwind". But that's not where we are right now.

Kyle Strand (Oct 31 2019 at 16:21, on Zulip):

Again, that's why I added the "FFI is not a sandbox" quote to Niko's HackMD doc, and why I'm saying that this is a discussion that needs to be had with the lang team.

gnzlbg (Oct 31 2019 at 16:24, on Zulip):

This WG is a sub-group of the lang team, so its kind of natural to discuss these here and prepare summaries for lang team discussion

gnzlbg (Oct 31 2019 at 16:25, on Zulip):

I'm not aware of anyone raising the issue that we are technically adding 99% of a noexcept feature to the language, just limited to the "C" abi, and less ergonomic to use, but adding the same kind of power

Kyle Strand (Oct 31 2019 at 16:29, on Zulip):

Okay, would you like to add that to the HackMD issue yourself?

gnzlbg (Oct 31 2019 at 16:29, on Zulip):

Sure, where is it ?

Amanieu (Oct 31 2019 at 16:29, on Zulip):

https://hackmd.io/ymsEL6OpR6OSMoFr1As1rw?view

gnzlbg (Oct 31 2019 at 16:30, on Zulip):

I'll let you know when I add it so that you can check if I presented your arguments accurately

Kyle Strand (Oct 31 2019 at 16:30, on Zulip):

AFAIK, the only lang team member who is somewhat active in this group who (if I remember correctly) was strongly against letting extern "C" unwind is Centril. I think he would be the one to talk to about whether he thinks this is a compelling reason to change course on the planned extern "C" behavior.

Kyle Strand (Oct 31 2019 at 16:31, on Zulip):

@gnzlbg Don't worry about presenting my arguments accurately; just explain why extern "C" is an implicit noexcept, and why you think that's a concern.

Kyle Strand (Oct 31 2019 at 16:31, on Zulip):

I still think that's essentially the same argument being made in the section I added at the bottom, about FFI being a "sandbox", so feel free to add it there, or add a new section, as you see fit.

Amanieu (Oct 31 2019 at 16:36, on Zulip):

My personal feeling is ambivalent: I'll just update my code according to whatever is decided.

gnzlbg (Oct 31 2019 at 17:02, on Zulip):

Don't worry about presenting my arguments accurately; just explain why extern "C" is an implicit noexcept, and why you think that's a concern.

@Kyle Strand ok I;ve done that in the last subsection, as an argument against making extern "C" never unwind by default

Kyle Strand (Oct 31 2019 at 17:05, on Zulip):

Thanks!

centril (Nov 13 2019 at 06:54, on Zulip):

I've finally caught up with this great. :slight_smile:

@Kyle Strand I see that I've been mentioned a few times. Feel free to ping me if you want my input.

centril (Nov 13 2019 at 06:54, on Zulip):

I've left some comments in the HackMD; so far it has enhanced my belief that nounwind for extern "C" is the right default

centril (Nov 13 2019 at 07:04, on Zulip):

Some other points:

- I agree that NeverUnwinds adds no more expressive power than extern "C" already affords one way or the other. It is indeed an alias.

- I cannot tell whether @gnzlbg is arguing for or against nounwind by default by discussing NeverUnwinds. To me, however, it would be an advantage if I can use extern "C" to encode higher order functions where I know that the passed-in function cannot unwind. The fact that there might be desirable generalizations in terms of impl nopanic Fn(..) does not, in my view, negate that advantage. A generalization having short-hands for the common case isn't a bad thing in language design.

- Further to ^-, we should, in my view, clarify whether we allow negative reasoning wrt. extern "C" having nounwind being UB. I think it would be useful to allow such negative reasoning in this case.

- The use of "breaking change" in this thread has been ambiguous on both @gnzlbg's and @Kyle Strand's ends. We can think of "breaking" in the sense of RFC 1122 or in "breaking in practice". In the former sense, we all agree that unwinding through extern "C" is UB. In the latter sense, despite explicit documentation in the reference to the contrary and LLVM UB, some people rely on the UB not to cause mis-compilations. It's clear to me that we are allowed to break those clients, but this WG exists because to resolve the soundness issue of extern "C" fn foo(...) { ... } we need to offer an alternative before we re-add the abort shim.

- Further to ^--, we must add the abort shim for extern "C" fn foo() { ... } or remove nounwind in LLVM. For the purposes of soundness, this is a binary choice.

gnzlbg (Nov 13 2019 at 07:12, on Zulip):

@centril I’m not arguing against or in favor of nounwind by default, just trying to point out that no matter which syntax we use and how we try to limit it, we probably end up adding a NeverUnwinds kind of capability to the language that doesn’t exist today. Instead of doing that „by accident“ it might be worth it to step back and consider doing that „by design“.

centril (Nov 13 2019 at 07:13, on Zulip):

@gnzlbg Ok, so you are neutral and would like more deliberation on this; very fair. But what does "by design" entail?

gnzlbg (Nov 13 2019 at 07:13, on Zulip):

There was also the possibility of allowing all extern “C” functions to unwind, and figure out later how to specify that some do not unwind. That solution doesn’t add a NeverUnwinds capability to the language.

centril (Nov 13 2019 at 07:13, on Zulip):

it could mean designing an effect system, but that's is comparatively hugely more complicated than a mere ABI string from the POV of the type system

gnzlbg (Nov 13 2019 at 07:14, on Zulip):

We already have const, async, target feature, ...

gnzlbg (Nov 13 2019 at 07:14, on Zulip):

We could add another nounwind qualifier

gnzlbg (Nov 13 2019 at 07:14, on Zulip):

But there are many trade offs involved

centril (Nov 13 2019 at 07:15, on Zulip):

@gnzlbg yes, but note that const functions cannot be polymorphic yet, and if they are, we haven't decided to add e.g. arg: impl const Fn() which isn't just polymorphic but "must" be passed-in a const fn

centril (Nov 13 2019 at 07:15, on Zulip):

async higher order functions aren't a thing either

gnzlbg (Nov 13 2019 at 07:16, on Zulip):

These trade offs might not be worth solving right now. We could just allow all extern “C” to unwind and provide an FFI only attribute to specify that some unknown ones are known to not unwind

centril (Nov 13 2019 at 07:16, on Zulip):

I agree that abstracting over effects would be useful, but I'm not the only one to convince

gnzlbg (Nov 13 2019 at 07:16, on Zulip):

But don’t reflect that on the ABI

gnzlbg (Nov 13 2019 at 07:16, on Zulip):

Avoiding NeverUnwinds

gnzlbg (Nov 13 2019 at 07:16, on Zulip):

In Rust, LLVM automatically infers nounwind anyways

centril (Nov 13 2019 at 07:17, on Zulip):

@gnzlbg I don't see much gain in delaying solving that; it's not like preventing NeverUnwinds is a useful goal (imo)

gnzlbg (Nov 13 2019 at 07:17, on Zulip):

So for an Exeter “C” defined in Rust this attribute isn’t necessary

centril (Nov 13 2019 at 07:17, on Zulip):

and I have not been convinced that changing the defaults is the right move

gnzlbg (Nov 13 2019 at 07:17, on Zulip):

I think that having NeverUnwinds only work for extern C is bad

gnzlbg (Nov 13 2019 at 07:17, on Zulip):

It requires juggling between the Rust and the C ABI all the time

gnzlbg (Nov 13 2019 at 07:17, on Zulip):

For no reason

gnzlbg (Nov 13 2019 at 07:18, on Zulip):

I agree as well it would be useful, but it is a big hack

centril (Nov 13 2019 at 07:18, on Zulip):

I don't think users would be encouraged to somehow sprinkle extern "C" fn in HoFs all of a sudden

gnzlbg (Nov 13 2019 at 07:18, on Zulip):

A proper solution might be easier if all ABI strings would behave similarly wrt unwinding

centril (Nov 13 2019 at 07:19, on Zulip):

we could also split things into extern "C", "unwind" and now allow extern "Rust", "unwind" -- that's still a cheap solution wrt. spec and compiler internals

centril (Nov 13 2019 at 07:19, on Zulip):

very cheap in fact

gnzlbg (Nov 13 2019 at 07:20, on Zulip):

Rust is unwind by default, if C is nounwind by default, we’d need both “unwind” and “nounwind”

centril (Nov 13 2019 at 07:20, on Zulip):

sure

centril (Nov 13 2019 at 07:20, on Zulip):

not a problem imo

gnzlbg (Nov 13 2019 at 07:20, on Zulip):

Imagine if extern “C” functions were const fn by default

centril (Nov 13 2019 at 07:20, on Zulip):

still very cheap

gnzlbg (Nov 13 2019 at 07:20, on Zulip):

We’d be in a bit more trouble with traits

gnzlbg (Nov 13 2019 at 07:21, on Zulip):

E.g need to implement !const trait impls for extern “C”

gnzlbg (Nov 13 2019 at 07:21, on Zulip):

We can also say that extern “C” functions don’t unwind by default

centril (Nov 13 2019 at 07:21, on Zulip):

Imagine if extern “C” functions were const fn by default

That might have made sense if const fn was the default and we had io fn for adding the effect instead of stripping away from the base effect

gnzlbg (Nov 13 2019 at 07:21, on Zulip):

Unknown ones

gnzlbg (Nov 13 2019 at 07:22, on Zulip):

@centril but for Rust it would be different than for extern C

gnzlbg (Nov 13 2019 at 07:22, on Zulip):

The point isn’t what default makes sense, but having different defaults for const depending on the ABI

centril (Nov 13 2019 at 07:22, on Zulip):

I don't think there's anything inherently wrong with having different defaults

gnzlbg (Nov 13 2019 at 07:22, on Zulip):

I think that it is possible for extern “C” to allow unwinding

gnzlbg (Nov 13 2019 at 07:23, on Zulip):

And for unknown extern “C” functions to be nounwind by default

gnzlbg (Nov 13 2019 at 07:23, on Zulip):

Those aren’t incompatible choices

centril (Nov 13 2019 at 07:23, on Zulip):

Possible, yes, desirable, that's debatable (I think not personally)

gnzlbg (Nov 13 2019 at 07:23, on Zulip):

What’s the downside ?

gnzlbg (Nov 13 2019 at 07:23, on Zulip):

It’s just an optimization attribute

gnzlbg (Nov 13 2019 at 07:24, on Zulip):

If someone wants to avoid it they can use an attribute

centril (Nov 13 2019 at 07:24, on Zulip):

Having differences between "unknown" and known feels even more inconsistent so to speak

centril (Nov 13 2019 at 07:24, on Zulip):

as for nounwind being the default, I think the hackmd elaborates

centril (Nov 13 2019 at 07:24, on Zulip):

so I won't repeat those arguments

gnzlbg (Nov 13 2019 at 07:24, on Zulip):

It’s not a difference between known and unknown

gnzlbg (Nov 13 2019 at 07:25, on Zulip):

Known functions can be nounwind as well

centril (Nov 13 2019 at 07:25, on Zulip):

then I don't understand what you are saying

centril (Nov 13 2019 at 07:25, on Zulip):

we have 3 things: imports, exports, pointers

gnzlbg (Nov 13 2019 at 07:25, on Zulip):

nounwind is an optimizer attribute

centril (Nov 13 2019 at 07:25, on Zulip):

well unless you allow negative reasoning, in which case you can do type based reasoning on it

gnzlbg (Nov 13 2019 at 07:26, on Zulip):

We can apply nounwind to declarations (imports) by default and say that if they unwind the behavior is undefined, and that you can use an attribute to allow that

gnzlbg (Nov 13 2019 at 07:26, on Zulip):

The type doesn’t encode that attribute

gnzlbg (Nov 13 2019 at 07:26, on Zulip):

So you can’t do any kind of type based reasoning with it

gnzlbg (Nov 13 2019 at 07:27, on Zulip):

You cannot express that a function pointer does not unwind, etc

centril (Nov 13 2019 at 07:27, on Zulip):

I know that we can, but it seems strictly less powerful but more costly in terms of spec & compiler internals

gnzlbg (Nov 13 2019 at 07:27, on Zulip):

Why more costly ?

gnzlbg (Nov 13 2019 at 07:27, on Zulip):

“If it’s a extern C decl, add nounwind I’d attribute not present” we already have this logic

gnzlbg (Nov 13 2019 at 07:28, on Zulip):

For the #[unwinds(...)] attribute

centril (Nov 13 2019 at 07:28, on Zulip):

ABI strings are an existing machinery whereas an attribute adds something different in the pipeline

centril (Nov 13 2019 at 07:28, on Zulip):

it's a new sort of category

centril (Nov 13 2019 at 07:28, on Zulip):

and new categories cost more

gnzlbg (Nov 13 2019 at 07:28, on Zulip):

We need the attribute any ways

centril (Nov 13 2019 at 07:29, on Zulip):

why that?

gnzlbg (Nov 13 2019 at 07:29, on Zulip):

For unknown function, we can’t prove that they don’t unwind

gnzlbg (Nov 13 2019 at 07:29, on Zulip):

Even for an extern “Rust”

gnzlbg (Nov 13 2019 at 07:29, on Zulip):

Unless we add a more powerful mechanism, like an effect, that we can use instead

gnzlbg (Nov 13 2019 at 07:30, on Zulip):

But if we ever do that we can deprecate the FFI Import-only attribute

centril (Nov 13 2019 at 07:30, on Zulip):

I'm not a fan of "can deprecate" reasoning in language design

gnzlbg (Nov 13 2019 at 07:31, on Zulip):

Me neither, but adding a new powerful effect by accident feels worse

centril (Nov 13 2019 at 07:31, on Zulip):

(And C++ is a cautionary example wrt. "can deprecate")

centril (Nov 13 2019 at 07:31, on Zulip):

"by accident" suggests we are not aware of what is enabled

centril (Nov 13 2019 at 07:32, on Zulip):

This discussion proves otherwise ^^

gnzlbg (Nov 13 2019 at 07:32, on Zulip):

I think that NeverUnwinds is so useful that we should support it without a hack

gnzlbg (Nov 13 2019 at 07:32, on Zulip):

You argue that people won’t use it much, but the existence of #[ni_panic] shows otherwise

gnzlbg (Nov 13 2019 at 07:32, on Zulip):

People will go long ways to hack themselves into being able to use this feature

gnzlbg (Nov 13 2019 at 07:33, on Zulip):

And NeverUnwinds is much better than #[no_panic] so it would at least replace it.

centril (Nov 13 2019 at 07:33, on Zulip):

https://crates.io/crates/no-panic/reverse_dependencies -- I'm not convinced by 6 rev deps.

For an unknown function (e.g. via a function pointer), we can assume it doesn't unwind (in the spec, if not in LLVM); we don't need to prove

centril (Nov 13 2019 at 07:34, on Zulip):

But again, there are other reasons for C to be nounwind by default as in the HackMD

centril (Nov 13 2019 at 07:34, on Zulip):

I'm not troubled by having a low-cost extension to the type system being an enabler; you can call it a hack, but it doesn't feel that way to me

gnzlbg (Nov 13 2019 at 07:34, on Zulip):

Depends on the reverse deps

centril (Nov 13 2019 at 07:34, on Zulip):

I agree that it's not as general as it could be

gnzlbg (Nov 13 2019 at 07:34, on Zulip):

Libm is used by compiler builtins

gnzlbg (Nov 13 2019 at 07:35, on Zulip):

Which means is part of the rust runtime

gnzlbg (Nov 13 2019 at 07:35, on Zulip):

On WASM at least

centril (Nov 13 2019 at 07:35, on Zulip):

I mean in terms of unique uses of no_panic -- not transitive benefits throughout the ecosystem

gnzlbg (Nov 13 2019 at 07:36, on Zulip):

Ah yes, i don’t know how many people directly use the crate

gnzlbg (Nov 13 2019 at 07:36, on Zulip):

Just that some low level libraries used by everybody so

centril (Nov 13 2019 at 07:36, on Zulip):

ryu and libm seem big indeed

gnzlbg (Nov 13 2019 at 07:37, on Zulip):

I think it might be worth exploring how hard would it be to provide good support for this feature

gnzlbg (Nov 13 2019 at 07:37, on Zulip):

And see if that would solve the FFI case

gnzlbg (Nov 13 2019 at 07:37, on Zulip):

I don’t think these crates need generics

centril (Nov 13 2019 at 07:37, on Zulip):

you mean like impl nounwind Fn?

gnzlbg (Nov 13 2019 at 07:37, on Zulip):

Libm doesn’t at least

gnzlbg (Nov 13 2019 at 07:38, on Zulip):

Nounwind fn foo As a start

gnzlbg (Nov 13 2019 at 07:38, on Zulip):

Being generic over it could come later

centril (Nov 13 2019 at 07:38, on Zulip):

probably not a whole lot difficult to implement in the compiler, but actually agreeing to a design and bringing the community on-board would be

centril (Nov 13 2019 at 07:39, on Zulip):

the effects polymorphism and as a modifier on bounds should be designed as a general effects thing, which years of proofing and testing

gnzlbg (Nov 13 2019 at 07:40, on Zulip):

I think C++ shows that a lot of code just stamps nounwind all over the place and tries to be generic over it for little benefit. So doing better might be desirable, but can be hard.

centril (Nov 13 2019 at 07:40, on Zulip):

does C++ allow genericity over noexcept?

gnzlbg (Nov 13 2019 at 07:40, on Zulip):

Yes

gnzlbg (Nov 13 2019 at 07:41, on Zulip):

But I think the const Trait impl solution is better

centril (Nov 13 2019 at 07:41, on Zulip):

can you provide an example for my edification?

gnzlbg (Nov 13 2019 at 07:41, on Zulip):

Let me get out of bed

centril (Nov 13 2019 at 07:42, on Zulip):

But I think the const Trait impl solution is better

Yea, as with most of Rust and generics (Thanks Haskell & ML!)

gnzlbg (Nov 13 2019 at 07:44, on Zulip):

So in C++, noexcept(expr) is a constant expression that returns true if expr cannot unwind, and false otherwise. In C++, noexcept(bool-expr) is also a function qualifier, that sets whether a function can or cannot unwind.

So this code:

template<typename T>
T generic_add(T a, T b) noexcept(noexcept(a + b)) { return a + b; }

is generic over noexcept.

centril (Nov 13 2019 at 07:45, on Zulip):

Now that's a hack :D

gnzlbg (Nov 13 2019 at 07:45, on Zulip):

In Rust, instead, we'll write generic_add as follows:

fn generic_add<T: Add>(a: T, b: T) -> T { a + b }

and whether it is nounwind or not depends on the Add impl, e.g., for Foo:

impl nounwind Add for Foo {
    fn add(...) -> ... { }
}
centril (Nov 13 2019 at 07:46, on Zulip):

sure

gnzlbg (Nov 13 2019 at 07:46, on Zulip):

So people won't need to stamp nounwind all over the place, and that problem is "solved"

gnzlbg (Nov 13 2019 at 07:46, on Zulip):

The main problem with this approach is that its an "all methods of the trait or nothing"

gnzlbg (Nov 13 2019 at 07:47, on Zulip):

but that's a problem we already considered acceptable for const fn

gnzlbg (Nov 13 2019 at 07:47, on Zulip):

I don't think nounwind changes anything here

centril (Nov 13 2019 at 07:47, on Zulip):

can always generalize later

centril (Nov 13 2019 at 07:47, on Zulip):

I agree it doesn't

gnzlbg (Nov 13 2019 at 07:48, on Zulip):

So for me I kind of prefer to explore how hard would a minimal solution for this problem be, and if that solution solves the FFI issue or not.

centril (Nov 13 2019 at 07:48, on Zulip):

but arg: impl nounwind Add and arg: impl const Add would be something more

centril (Nov 13 2019 at 07:48, on Zulip):

@gnzlbg I understand your point

gnzlbg (Nov 13 2019 at 07:49, on Zulip):

I'm not against adding a #[unwind(aborts/allowed)] attribute that's only usable inside extern "C" { imports } as a temporary solution. I would prefer not to do that, but we need to balance shipping with language evolution.

centril (Nov 13 2019 at 07:50, on Zulip):

Personally I think that keeping "C" as nounwind and adding "C unwind" is the most low-cost solution that has the best defaults and disrupts the ecosystem least

gnzlbg (Nov 13 2019 at 07:50, on Zulip):

It might also well be that doing that makes also sense even if we had an impl nounwind Trait feature

gnzlbg (Nov 13 2019 at 07:51, on Zulip):

I don't know.

centril (Nov 13 2019 at 07:51, on Zulip):

Seems like we've explored the options at least :slight_smile:

gnzlbg (Nov 13 2019 at 07:52, on Zulip):

Yes

gnzlbg (Nov 13 2019 at 07:52, on Zulip):

Might be worth summarizing in a hackmd

nikomatsakis (Nov 13 2019 at 15:03, on Zulip):

If you think there's a new alternative worth discussing, you could add it to the meaning of C hackmd -- I think an "effect system" variation probably qualifies. I myself am pretty unconvinced right now that we need genericity over this, and I also don't have nearly as many qualms about deprecation and evolution, I suspect. =) I feel like a #[unwind(never)] sort of attribute might well be reasonable. The big question to me though remains how much it matters which we default to, in terms of the interaction with -Cpanic=abort. I don't feel that's resolved yet, it requires some measurement and impl work.

nikomatsakis (Nov 13 2019 at 15:04, on Zulip):

skimming over the thread, I'm not sure I saw much "reasoning" that didn't appear in there, but it certainly doesn't discuss effect systems

nikomatsakis (Nov 13 2019 at 15:04, on Zulip):

worth noting that using the ABI also doesn't permit genericity

nikomatsakis (Nov 13 2019 at 15:08, on Zulip):

to clarify about deprecation: I wouldn't want to plan on deprecating, but I think in this case that (a) generalization is kind of "YAGNI", (b) this is somewhat niche. So, if we do wind up with some kind of effect system we would like to generalize, I would be ok with transitioning this feature over to a new syntax so it can use it.

centril (Nov 13 2019 at 16:17, on Zulip):

@nikomatsakis By "you" do you mean me or @gnzlbg ? I'm personally pretty happy with extern "C unwind" and I agree that this discussion didn't add anything novel

centril (Nov 13 2019 at 16:17, on Zulip):

(Also agree re. "plan on deprecating")

nikomatsakis (Nov 13 2019 at 16:19, on Zulip):

I meant @gnzlbg, but then I got nerd-swiped and added in some discussion of generics / effects, though I didn't go so far as to add a full blown alternative, because I don't think anybody really wants that

centril (Nov 13 2019 at 16:20, on Zulip):

I definitely want an effect system, but I also think that can wait ;)

nikomatsakis (Nov 13 2019 at 16:23, on Zulip):

Yes, I meant "I don't think anybody wants that to be done by this group"

gnzlbg (Nov 14 2019 at 10:44, on Zulip):

Yes, an effect system is out of scope for this group. But the alternative I was mentioning is adding a new effect to Rust, not adding a new effect system.

gnzlbg (Nov 14 2019 at 10:45, on Zulip):

By adding this feature we are adding a new effect to Rust.

gnzlbg (Nov 14 2019 at 10:46, on Zulip):

I'm not suggesting that we should constrain the syntax to be compatible with a future effect system, only that it might maybe be worth it to consider a syntax that does not make it too hard to do so.

gnzlbg (Nov 14 2019 at 10:48, on Zulip):

I'm not sure how important this particular effect is, whether it will ever matter to be generic over it, and therefore, I don't know whether more "general purpose" syntax for it (as opposed to "C unwind") would be worth it.

gnzlbg (Nov 14 2019 at 10:49, on Zulip):

What I know is that some crates are already making "this function never unwinds" part of their API contract via a quite hacky macro / linker approach

gnzlbg (Nov 14 2019 at 10:50, on Zulip):

And some of those APIs do not use the C ABI today

gnzlbg (Nov 14 2019 at 10:51, on Zulip):

I would find it a bit weird if with this feature those people would start using the C ABI instead of the Rust ABI to expres "never unwinds"

gnzlbg (Nov 14 2019 at 10:53, on Zulip):

I'm not suggesting that we should constrain the syntax to be compatible with a future effect system, only that it might maybe be worth it to consider a syntax that does not make it too hard to do so.

To add to this, we have const which "removes" an effect, async which adds it, attributes like #[target_feature(enable)] which also add effects, etc.

Right now all functions can be made unwinds, so we could add a nounwind syntax to remove that effect. That becomes a bit more complicated if whether a function unwinds or not is part of its "ABI" string, and we have to add both unwinds to add the effect to ABIs that are nounwind by default, and nounwind to remove it (or somehow support !nounwind or !unwinds or similar).

Last update: Nov 15 2019 at 09:45UTC