Stream: t-lang/wg-unsafe-code-guidelines

Topic: UB in the nomicon


RalfJ (Jul 20 2019 at 11:07, on Zulip):

I am updating what the nomicon calls UB: https://github.com/rust-lang-nursery/nomicon/pull/149
Is there anything else that should be added to that list?

RalfJ (Jul 20 2019 at 11:21, on Zulip):

I'd also appreciate some extra eyes on https://github.com/rust-lang/rust/pull/62822

Lokathor (Jul 20 2019 at 14:34, on Zulip):

"function pointers are pointers that point to code, not data"

[Lisp programmers screaming in the distance]

Lokathor (Jul 20 2019 at 14:35, on Zulip):

Nomicon i left a comment, the main rust docs look good with no changes

Nick12 (Aug 24 2019 at 23:53, on Zulip):

Should the reference also be updated?
https://doc.rust-lang.org/reference/behavior-considered-undefined.html

Nick12 (Aug 24 2019 at 23:54, on Zulip):

I'm particularly thinking of Reads of undef (uninitialized) memory.

Nick12 (Aug 24 2019 at 23:54, on Zulip):

Regardless of if that is technically correct or not, it gives a very different picture than what the nomicon currently says I think

RalfJ (Aug 25 2019 at 08:36, on Zulip):

Should the reference also be updated?
https://doc.rust-lang.org/reference/behavior-considered-undefined.html

yeah I have updating that on my list

gnzlbg (Aug 25 2019 at 09:24, on Zulip):

has the same issue, no #[no_mangle] involved

RalfJ (Aug 25 2019 at 09:24, on Zulip):

(wrong topic)

RalfJ (Aug 25 2019 at 10:53, on Zulip):

here's the Reference UB PR: https://github.com/rust-lang-nursery/reference/pull/659

gnzlbg (Aug 25 2019 at 11:40, on Zulip):

If we wanted to say that "Unwinding from Rust into other languages is UB", we'd have to police other languages to never define the behavior of calling a Rust function that unwinds.

gnzlbg (Aug 25 2019 at 11:41, on Zulip):

I don't see how we could achieve that, in particular, given that anybody can write an ASM snippet that calls a Rust function, and handles its unwinding correctly.

gnzlbg (Aug 25 2019 at 11:41, on Zulip):

They'd tell you that in assembly the behavior of doing that is perfectly defined, and they would be right :laughter_tears:

RalfJ (Aug 25 2019 at 11:58, on Zulip):

If we wanted to say that "Unwinding from Rust into other languages is UB", we'd have to police other languages to never define the behavior of calling a Rust function that unwinds.

no. it's never wrong to call too many things UB. it might just be a bad idea.

RalfJ (Aug 25 2019 at 11:59, on Zulip):

OTOH, given all the discussion around unwinding into extern fn, I think it is crucial that that is reflected in this list somehow. if you have suggestions for improving the wording, please share them. :)

gnzlbg (Aug 25 2019 at 13:09, on Zulip):

The problem with this is that such UB is unverifiable

gnzlbg (Aug 25 2019 at 13:10, on Zulip):

And adding it serves no purpose, because its not up to us to say what's UB and what isn't in other programming languages

gnzlbg (Aug 25 2019 at 13:13, on Zulip):

The moment we unwind from a function, we are not in that function anymore

gnzlbg (Aug 25 2019 at 13:14, on Zulip):

So if we unwind from Rust code into another programming language, we are in that other programming language

gnzlbg (Aug 25 2019 at 13:15, on Zulip):

If you want to say that unwinding from Rust into that language is UB, you should do so in the spec of that language

gnzlbg (Aug 25 2019 at 13:15, on Zulip):

The Rust spec does not hold there

RalfJ (Aug 25 2019 at 16:09, on Zulip):

And adding it serves no purpose, because its not up to us to say what's UB and what isn't in other programming languages

this is on the interface between us and other languages, so it is as much our territory as theirs.

RalfJ (Aug 25 2019 at 16:10, on Zulip):

The problem with this is that such UB is unverifiable

just like UB caused by mismatching FFI with C. we'd need a cross-language abstract machine or so to make this precise and verifiable.

gnzlbg (Aug 25 2019 at 16:29, on Zulip):

to me this feels like writing a german law that says that once you cross the border to poland, you can't have potatoes in the car

gnzlbg (Aug 25 2019 at 16:32, on Zulip):

with the twist that germany wouldn't know where the border to austria is and wouldn't have any way to check it

gnzlbg (Aug 25 2019 at 16:33, on Zulip):

and that crossing border with potatos in the car is otherwise fine, and that having potatoes in the car in austria is also potentially fine

gnzlbg (Aug 25 2019 at 16:33, on Zulip):

and well with the obvious consequence that if you happen to actually cross to austria with potatoes in the care, german law doesn't hold there, so you can just eat your potatoes, and come back

gnzlbg (Aug 25 2019 at 16:35, on Zulip):

this creates the problem that it is not sufficient for other languages, of which there are many, that they support Rust functions unwinding, but that we would also need to mention that for every such language

rkruppe (Aug 25 2019 at 16:38, on Zulip):

If we do want to support cross-language unwinding, then listing languages we intend to be compatible with is neither sensible nor necessary. We can just describe our unwinding ABI and others can implement it.

gnzlbg (Aug 25 2019 at 16:39, on Zulip):

Our unwinding ABI is unspecified

gnzlbg (Aug 25 2019 at 16:39, on Zulip):

This means that any language can read our source code, and implement it, and still be compatible with some particular rust toolchain version

gnzlbg (Aug 25 2019 at 16:40, on Zulip):

So even without explicit intent to support cross-language unwinding, it would be "fine" to do so.

rkruppe (Aug 25 2019 at 16:41, on Zulip):

Stating on our end that we consider unwinding out of Rust to be UB is stressing that we really don't want people to do that and won't even try to keep that working. Someone hacking something unsupported together against a specific implementation of a spec is always possible and never makes it "not UB".

gnzlbg (Aug 25 2019 at 16:42, on Zulip):

Sure, but we also say that Rust's unwinding ABI is unspecified, and not undefined

rkruppe (Aug 25 2019 at 16:43, on Zulip):

I don't understand why we shouldn't use the scariest language available that is correct.

rkruppe (Aug 25 2019 at 16:43, on Zulip):

And it is correct to say that the Rust language considers it UB.

gnzlbg (Aug 25 2019 at 16:44, on Zulip):

We can, but it sends a mixed message given that we have been supporting this UB in stable for a year now, while having it panic on nightly in some cases.

RalfJ (Aug 25 2019 at 16:45, on Zulip):

and then not do it only with austria, but with all other countries, and well with the obvious consequence that if you happen to actually cross to another country with potatoes in the care, german law doesn't hold there, so you can just eat your potatoes, and come back

to stretch that stretched analogy further: you cannot come back as German police will prosecute you for breaking German law. The border itself, where the crime happened, is still something that Germany has jurisdiction on.

gnzlbg (Aug 25 2019 at 16:46, on Zulip):

Sure, but germany has no way to tell. Unless it is a goal to, e.g., create a cross-lang abstract machine between Rust, and e.g., assembly

RalfJ (Aug 25 2019 at 16:47, on Zulip):

We can, but it sends a mixed message given that we have been supporting this UB in stable for a year now, while having it panic on nightly in some cases.

that's why I am asking for wording that excludes that case.
cross-langauge FFI is like a puzzle, both sides have to make it fit. so this is where we specify the rules on our side.

gnzlbg (Aug 25 2019 at 16:47, on Zulip):

You also argue that the crime happened at the border, but I think that the crime happens in the other country

RalfJ (Aug 25 2019 at 16:47, on Zulip):

this is true both of exceptions entering and leaving any language's domain

gnzlbg (Aug 25 2019 at 16:47, on Zulip):

if you weren't in another country you couldn't have commited the crime in the first place

RalfJ (Aug 25 2019 at 16:48, on Zulip):

so we could say something like "unwinding out of an #[unwind(allowed)] extern fn into a language with incompatible unwinding ABI", or so

gnzlbg (Aug 25 2019 at 16:48, on Zulip):

we can't use #[unwind(allowed)] in the reference because its unstable :/

RalfJ (Aug 25 2019 at 16:49, on Zulip):

we can add a note saying that this is unstable, and that on stable this is the default attribute

gnzlbg (Aug 25 2019 at 16:49, on Zulip):

in stable Rust the only thing that controls what can unwind is the "ABI" string

RalfJ (Aug 25 2019 at 16:49, on Zulip):

or we can just leave away the attribute

rkruppe (Aug 25 2019 at 16:49, on Zulip):

The lang team position for stable Rust is that it's UB to unwind across any FFI at all, and no amount of hairsplitting will change that

RalfJ (Aug 25 2019 at 16:49, on Zulip):

I dont think we have to mention the ABI string... if some other langauge calls a extern "Rust" fn, the rules are still the same

gnzlbg (Aug 25 2019 at 16:50, on Zulip):

and by some other language, we need to consider also Rust-with-different-toolchain

RalfJ (Aug 25 2019 at 16:50, on Zulip):

sure. I dont think this is the right place to define all FFI terminology, we can leave that open until someone bothers to add an appropriate chapter.

gnzlbg (Aug 25 2019 at 16:51, on Zulip):

@rkruppe stable Rust does not enforce that for Rust-Rust FFI

gnzlbg (Aug 25 2019 at 16:51, on Zulip):

or does it ?

rkruppe (Aug 25 2019 at 16:51, on Zulip):

Since when does UB have to be enforced?

rkruppe (Aug 25 2019 at 16:52, on Zulip):

Look I know how messy the situation is in reality, but de jure it's all UB and UB is UB.

RalfJ (Aug 25 2019 at 16:52, on Zulip):

The lang team position for stable Rust is that it's UB to unwind across any FFI at all, and no amount of hairsplitting will change that

well the lang team is also not fundamentally opposed to an unwind(allowed) attribute, and has agreed to beta-backports for PR that avoid abort-on-unwind-through-FFI

gnzlbg (Aug 25 2019 at 16:52, on Zulip):

So.. in stable Rust the standard library can do Rust-Rust FFI, because it makes use of a special language feature ?

rkruppe (Aug 25 2019 at 16:53, on Zulip):

So.. in stable Rust the standard library can do Rust-Rust FFI, because it makes use of a special language feature ?

You make it sound like that's in any way special. libstd always does lots of magic stuff.

gnzlbg (Aug 25 2019 at 16:53, on Zulip):

I know, but Rust-Rust FFI is used so much, I don't know if it is only used within libstd

RalfJ (Aug 25 2019 at 16:53, on Zulip):

so "unwinding into another language is UB" does not reflect Rust-as-it-is-used and the lang team has agreed to not killing crates that want to unwind out of Rust.

gnzlbg (Aug 25 2019 at 16:53, on Zulip):

or through lang items

rkruppe (Aug 25 2019 at 16:54, on Zulip):

The lang team position for stable Rust is that it's UB to unwind across any FFI at all, and no amount of hairsplitting will change that

well the lang team is also not fundamentally opposed to an unwind(allowed) attribute, and has agreed to beta-backports for PR that avoid abort-on-unwind-through-FFI

The context there was what goes into the reference which mostly (though still non-normatively) defines the rules of stable Rust. That we've not shipped certain changes which are possible under those rules isn't relevant for what the rules are.

RalfJ (Aug 25 2019 at 16:54, on Zulip):

but IMO a reasonable approach would also be to say "unwinding across lang barrier is UB" in this PR, and then open an issue to make this more precise (less UB) to adjust for reality.

gnzlbg (Aug 25 2019 at 16:55, on Zulip):

I think it has become clear that this PR is the wrong place to try to resolve this

gnzlbg (Aug 25 2019 at 16:55, on Zulip):

I'd say, just c&p what the nomicon now says, or what the reference was saying before, and open an issue

RalfJ (Aug 25 2019 at 16:56, on Zulip):

@gnzlbg that issue is https://github.com/rust-lang/rust/issues/58794

gnzlbg (Aug 25 2019 at 16:57, on Zulip):

that issue somehow mixes FFI with extern "C"

gnzlbg (Aug 25 2019 at 16:58, on Zulip):

As in, a lot there, and the RFC, uses FFI to refer exclusively to extern "C".

RalfJ (Aug 25 2019 at 16:58, on Zulip):

hm that issue is more for the abort-on-panic-shim actually

RalfJ (Aug 25 2019 at 16:58, on Zulip):

this would be the place for your concern I think: https://github.com/rust-lang/rfcs/pull/2699

gnzlbg (Aug 25 2019 at 16:59, on Zulip):

I already raised it there, took a bit of convincing and PRs to update the reference

gnzlbg (Aug 25 2019 at 16:59, on Zulip):

There is also the misconception that fn() is different from extern "Rust" fn()

RalfJ (Aug 25 2019 at 17:00, on Zulip):

yeah, the extern keyword here is confusing

gnzlbg (Aug 25 2019 at 17:01, on Zulip):

yes, very

Nick12 (Aug 25 2019 at 17:25, on Zulip):

Is extern "Rust" abi fully defined/documented? I dont think it makes much sense to talk about ffi with it being stable otherwise. Should a possible third party Rust compiler or other languages trying to do ffi with the Rust abi be considered?

RalfJ (Aug 25 2019 at 19:32, on Zulip):

no it is not documented

RalfJ (Aug 25 2019 at 19:33, on Zulip):

so by definition it is incompatible with anything

gnzlbg (Aug 26 2019 at 10:55, on Zulip):

AFAIK the extern "Rust" ABI is _unspecified_

gnzlbg (Aug 26 2019 at 10:56, on Zulip):

Should a possible third party Rust compiler or other languages trying to do ffi with the Rust abi be considered?

That's possible, brittle, but possible.

gnzlbg (Aug 26 2019 at 10:57, on Zulip):

You can use two different Rust toolchains to compile two different Rust static libraries where functions use the Rust ABI, link them, have them call each other, and that often works

gnzlbg (Aug 26 2019 at 10:58, on Zulip):

That's two different compilers, implementing the Rust ABI, and code compiling with those interfacing

gnzlbg (Aug 26 2019 at 10:58, on Zulip):

The same applies to GDB, LLDB, and other debugging tools that need to understand the Rust ABI to be useful

gnzlbg (Aug 26 2019 at 11:05, on Zulip):

Because the ABI is unspecified, and most of it can change any time, all of this can break any time

gnzlbg (Aug 26 2019 at 11:28, on Zulip):

@RalfJ @rkruppe shall i open a rust-lang/rust issue for the declaration/definition xLTO thingy ?

RalfJ (Aug 26 2019 at 11:53, on Zulip):

gnzlbg: @RalfJ @rkruppe shall i open a rust-lang/rust issue for the
declaration/definition xLTO thingy ?

I think a UCG issue would make more sense?

gnzlbg (Aug 26 2019 at 12:03, on Zulip):

I'm not sure

gnzlbg (Aug 26 2019 at 12:03, on Zulip):

One thing is whether the code with the incompatible declarations is correct Rust code.

gnzlbg (Aug 26 2019 at 12:04, on Zulip):

I've got the feeling that either it is, or it should be (because otherwise too much code in the wild could be broken).

gnzlbg (Aug 26 2019 at 12:04, on Zulip):

I don't know if that would fit the reference, or rust-lang/rust, or the rfc repo, but UCGs feels too niche for that

gnzlbg (Aug 26 2019 at 12:05, on Zulip):

once that is settled, then what might remain is just the codegen bug that's already filled upstream

Last update: Nov 20 2019 at 12:15UTC