Stream: project-ffi-unwind

Topic: Blog post


Kyle Strand (Dec 24 2019 at 03:16, on Zulip):

@nikomatsakis I started drafting a blog post, but it's pretty far from finished. If you have time, you may be interested in taking a look and seeing if I'm heading in the right direction. https://github.com/BatmanAoD/project-ffi-unwind/blob/BlogPost-announcement/blogposts/inside-rust/01-announcement.md

nikomatsakis (Jan 02 2020 at 18:58, on Zulip):

the post gives a lot of background, I'm wondering if it almost gives too much, but I'm not sure

nikomatsakis (Jan 02 2020 at 18:59, on Zulip):

I might try an alternative outline just to see

Kyle Strand (Jan 02 2020 at 18:59, on Zulip):

I wasn't planning on giving as much background as I ended up writing

Kyle Strand (Jan 02 2020 at 18:59, on Zulip):

But as I went, I realized that I don't think we have one place that explains the background with a balance of detail & brevity

nikomatsakis (Jan 02 2020 at 21:49, on Zulip):

Yeah

nikomatsakis (Jan 02 2020 at 21:49, on Zulip):

There does seem to be value in that

nikomatsakis (Jan 02 2020 at 21:58, on Zulip):

Maybe we even want two blog posts

nikomatsakis (Jan 02 2020 at 23:04, on Zulip):

@Kyle Strand this is WIP, but take a look at what I wrote so far. I'm trying to capture the "essence" of the problem as I see it...

Kyle Strand (Jan 05 2020 at 22:06, on Zulip):

@nikomatsakis One question on the blog post: the second alternative you describe is:

Add a new ABI ("C unwind") that permits unwinding; the "C" ABI is specified as the system ABI but where unwinding is UB

Since the original plan was for Rust functions defined with the "C" ABI to abort-on-panic, I had assumed that this would be the "default" option. Is this what you're referring to here, or would that be a third option?

Kyle Strand (Jan 05 2020 at 22:08, on Zulip):

I.e., unwinding would only be UB coming from foreign functions (or perhaps from an improper use of unsafe, I suppose), since "C" ABI Rust functions could not otherwise expose unwinding.

Kyle Strand (Jan 06 2020 at 01:18, on Zulip):

In any case, I have updated my draft to incorporate a decent amount of the text from yours.

nikomatsakis (Jan 07 2020 at 22:54, on Zulip):

@Kyle Strand I believe the proposal was that unwinding through the "C" ABI was undefined behavior. Given that it is undefined, functions defined in Rust with the C ABI can abort -- but functions not defined in Rust would not abort

nikomatsakis (Jan 07 2020 at 22:54, on Zulip):

from the POV of the caller, you can be sure that unwinding never happens

nikomatsakis (Jan 07 2020 at 22:54, on Zulip):

and some callees (ones defined in Rust) guarantee that by aborting

nikomatsakis (Jan 07 2020 at 22:55, on Zulip):

I'm not sure if that is different from what you said :)

nikomatsakis (Jan 07 2020 at 22:55, on Zulip):

where is your updated blog post now..? in the PR?

nikomatsakis (Jan 07 2020 at 22:56, on Zulip):

I feel like my draft was still missing a certain amount of the arguments either way, but it was already useful in that I kind of shifted my opinion based on what @Amanieu was pointing out

Kyle Strand (Jan 07 2020 at 22:56, on Zulip):

I think I just didn't realize while reading the draft the first time how important it is to emphasize that even with the "abort-on-unwind" logic, there's still UB.

Kyle Strand (Jan 07 2020 at 22:56, on Zulip):

Yes, I've continued updating the PR

nikomatsakis (Jan 07 2020 at 22:56, on Zulip):

it seems like the plan if "UB if you unwind with -Cpanic=abort, but best effort aborts on debug" is the best option

nikomatsakis (Jan 07 2020 at 22:57, on Zulip):

one thing I was wondering about was whether other -C options could induce UB in this way

Kyle Strand (Jan 07 2020 at 22:57, on Zulip):

that's a good question...

nikomatsakis (Jan 07 2020 at 22:57, on Zulip):

I think you can imagine a world where -Cpanic=abort simply refuses to compile code with "C unwind" ABI, but I think that's not really tenable

Kyle Strand (Jan 07 2020 at 22:57, on Zulip):

I think the longjmp situation more or less precludes that, doesn't it?

nikomatsakis (Jan 07 2020 at 22:57, on Zulip):

it's also very interesting that calls like read may unwind and may not, depending on the details of what platform you are on

nikomatsakis (Jan 07 2020 at 22:58, on Zulip):

which means that if we try to separate out in the ABI, we have to either overapproximate (to use "C unwind") or vary by target ("ugh") or just rule out some details of pthread cancelation ("seems like losing capabilities")

Kyle Strand (Jan 07 2020 at 22:58, on Zulip):

since longjmp should "always work", even if it's using unwinding under the covers, and therefore the "C" ABI needs to never interfere with it

nikomatsakis (Jan 07 2020 at 22:59, on Zulip):

well I imagine that -Ctarget-feature can cause UB

nikomatsakis (Jan 07 2020 at 22:59, on Zulip):

i.e., if your target actually lacks those features

nikomatsakis (Jan 07 2020 at 22:59, on Zulip):

similarly things like link-args

nikomatsakis (Jan 07 2020 at 22:59, on Zulip):

who knows what those can do

nikomatsakis (Jan 07 2020 at 23:00, on Zulip):

so it seems like -C is already "best know what you're doing" territory to me, even though we should try hard to remove rough edges where we can

nikomatsakis (Jan 07 2020 at 23:02, on Zulip):

it seems like the plan if "UB if you unwind with -Cpanic=abort, but best effort aborts on debug" is the best option

the good news about this is that it means -Cpanic=abort continues to be fully optimizable no matter what we choose

nikomatsakis (Jan 07 2020 at 23:05, on Zulip):

which means metrics don't matter

Kyle Strand (Jan 07 2020 at 23:12, on Zulip):

The concerning thing about -Cpanic=abort to me is that even if -C "should" mean "best know what you're doing", I think panic=abort is probably not perceived that way.

Kyle Strand (Jan 07 2020 at 23:13, on Zulip):

Especially since it's exposed via Cargo

nikomatsakis (Jan 07 2020 at 23:14, on Zulip):

Yes

nikomatsakis (Jan 07 2020 at 23:14, on Zulip):

This is why I would definitely want "abort in debug mode"

nikomatsakis (Jan 07 2020 at 23:14, on Zulip):

I would even consider panic=ub or something

nikomatsakis (Jan 07 2020 at 23:14, on Zulip):

(and make abort determinstically abort)

nikomatsakis (Jan 07 2020 at 23:15, on Zulip):

I think a big question is how much you care about pthread_exit and these interactions with read -- I'm trying to put my finger on it

Kyle Strand (Jan 07 2020 at 23:16, on Zulip):

Well....

Kyle Strand (Jan 07 2020 at 23:16, on Zulip):

Semantically, there's an argument, I think, that panic=abort "just" means exactly what it says: panic! will trigger an abort.

Kyle Strand (Jan 07 2020 at 23:17, on Zulip):

To my mind, this means that leaving abort shims around in every function that invokes a "C" function would be entirely permissible.

Kyle Strand (Jan 07 2020 at 23:18, on Zulip):

That would be a concrete difference between panic=ub and panic=abort, and I think it would simplify the implementation of both quite a bit.

Kyle Strand (Jan 07 2020 at 23:18, on Zulip):

Although, come to think of it, going on that semantic argument, it shouldn't be panic=ub, it should be panic=abort _and_ unwind=ub

Kyle Strand (Jan 07 2020 at 23:18, on Zulip):

so a new flag...

nikomatsakis (Jan 07 2020 at 23:20, on Zulip):

To my mind, this means that leaving abort shims around in every function that invokes a "C" function would be entirely permissible.

point is, that wouldn't permit invoking longjmp

nikomatsakis (Jan 07 2020 at 23:21, on Zulip):

er, sorry

nikomatsakis (Jan 07 2020 at 23:21, on Zulip):

longjmp would have to be given "C unwind", presumably, at least on some platforms

nikomatsakis (Jan 07 2020 at 23:21, on Zulip):

I think maybe an appealing configuration would be something like this:

nikomatsakis (Jan 07 2020 at 23:22, on Zulip):
Kyle Strand (Jan 07 2020 at 23:22, on Zulip):

I'm not sure about that; the abort shims already permit longjmp.

nikomatsakis (Jan 07 2020 at 23:22, on Zulip):

basically, if you use foreign unwinding and rely on it to execute dtors, your ;library is just unusable with panic=abort

Kyle Strand (Jan 07 2020 at 23:23, on Zulip):

I don't think that _needs_ to be the case.

nikomatsakis (Jan 07 2020 at 23:23, on Zulip):

well, it kind of does, right?

nikomatsakis (Jan 07 2020 at 23:23, on Zulip):

it's true we could special-case longjmp

Kyle Strand (Jan 07 2020 at 23:23, on Zulip):

Any form of "forced unwind" should never trigger an abort, I think.

nikomatsakis (Jan 07 2020 at 23:24, on Zulip):

well, it is not ok if you fail to run dtors,

Kyle Strand (Jan 07 2020 at 23:24, on Zulip):

Because the definition of a "forced unwind" (from the Itanium spec and from... something else I just read today or yesterday... LLVM manual, I think?)

Kyle Strand (Jan 07 2020 at 23:24, on Zulip):

is that no language can stop the exception

nikomatsakis (Jan 07 2020 at 23:24, on Zulip):

so in those cses an abort is preferred to UB, though we may not be able to amnage one

Kyle Strand (Jan 07 2020 at 23:24, on Zulip):

hmmm

Kyle Strand (Jan 07 2020 at 23:24, on Zulip):

I would think that in the longjmp case, not running dtors would be preferred!

Kyle Strand (Jan 07 2020 at 23:24, on Zulip):

Not that it would be sound

Kyle Strand (Jan 07 2020 at 23:25, on Zulip):

But that it's what users of longjmp would expect.

Kyle Strand (Jan 07 2020 at 23:25, on Zulip):

Possibly-irrelevant question: are C++ destructors expected to run when pthread_exit runs?

nikomatsakis (Jan 07 2020 at 23:27, on Zulip):

it's UB is the point

nikomatsakis (Jan 07 2020 at 23:27, on Zulip):

rust programs are allowed to assume their dtors run

nikomatsakis (Jan 07 2020 at 23:27, on Zulip):

many abstractions rely on this

Kyle Strand (Jan 07 2020 at 23:27, on Zulip):

I know, but it's the same in C++, except moreso.

nikomatsakis (Jan 07 2020 at 23:28, on Zulip):

OK:)

Kyle Strand (Jan 07 2020 at 23:28, on Zulip):

I.e. the C++ community will tell you that "destructors always run"

Kyle Strand (Jan 07 2020 at 23:28, on Zulip):

there is no concept of something like mem::forget in C++

nikomatsakis (Jan 07 2020 at 23:28, on Zulip):

I believe in C++ it is UB to longjmp over a frame w/ dtors

nikomatsakis (Jan 07 2020 at 23:28, on Zulip):

in any case

nikomatsakis (Jan 07 2020 at 23:28, on Zulip):

(but in windows, it is defined to run those dtors)

Kyle Strand (Jan 07 2020 at 23:29, on Zulip):

I think you're right, though Windows says "it may run dtors, it may not, depends on the optimizer"

Kyle Strand (Jan 07 2020 at 23:29, on Zulip):

which is why I think that system users would expect UB rather than abort logic

nikomatsakis (Jan 07 2020 at 23:29, on Zulip):

I guess.. I odn't care?

nikomatsakis (Jan 07 2020 at 23:29, on Zulip):

like, if it is UB

Kyle Strand (Jan 07 2020 at 23:29, on Zulip):

Where "system users" means "programmers used to the way longjmp interacts with other languages that have 'drop'-like features"

nikomatsakis (Jan 07 2020 at 23:29, on Zulip):

then aborting is certain one possible thing :)

Kyle Strand (Jan 07 2020 at 23:29, on Zulip):

That's fair.

nikomatsakis (Jan 07 2020 at 23:30, on Zulip):

Where "system users" means "programmers used to the way longjmp interacts with other languages that have 'drop'-like features"

my point is: they should not be combining it

nikomatsakis (Jan 07 2020 at 23:30, on Zulip):

that is, they may think longjmp should 'just ignore' dtors

nikomatsakis (Jan 07 2020 at 23:30, on Zulip):

but they are wrong :)

nikomatsakis (Jan 07 2020 at 23:30, on Zulip):

even though it might look like this is what happens

nikomatsakis (Jan 07 2020 at 23:31, on Zulip):

but I think this is a bit off topic I guess

nikomatsakis (Jan 07 2020 at 23:31, on Zulip):

and i'm cooking so I should stop :)

nikomatsakis (Jan 07 2020 at 23:31, on Zulip):

main thing I was thinking is:

nikomatsakis (Jan 07 2020 at 23:31, on Zulip):

one advantage of "C unwind" is that it lets you identify call sites where unwinding is "important"

Kyle Strand (Jan 07 2020 at 23:37, on Zulip):

True... it just doesn't seem correct to me to put that information in the ABI.

Kyle Strand (Jan 07 2020 at 23:38, on Zulip):

Backing up to the longjmp-over-Rust question: in panic=abort mode, on Windows, would you expect longjmp to be allowed over "inert" frames (as you termed them in your draft)?

nikomatsakis (Jan 07 2020 at 23:43, on Zulip):

A good question. I was assuming yes

nikomatsakis (Jan 07 2020 at 23:43, on Zulip):

I think that is an explicit goal, in fact, no?

nikomatsakis (Jan 07 2020 at 23:43, on Zulip):

well, maybe not

nikomatsakis (Jan 07 2020 at 23:44, on Zulip):

we were trying to insert abort shims, but I guess those were triggering both with -Cpanic=abort and -Cpanic=unwind?

Kyle Strand (Jan 07 2020 at 23:53, on Zulip):

I was also assuming yes.

nikomatsakis (Jan 07 2020 at 23:58, on Zulip):

somehow i feel more confused than I felt before:)

Kyle Strand (Jan 08 2020 at 20:38, on Zulip):

Okay, so, if -Cpanic=abort only triggers abort on forced-unwind for debug builds when the unwind hits a _non-inert_ frame, I think that's reasonable.

Kyle Strand (Jan 08 2020 at 20:38, on Zulip):

In release builds, UB, but again only for non-inert frames

Kyle Strand (Jan 08 2020 at 20:39, on Zulip):

(Where "forced unwind" is defined by Itanium ABI and by LLVM, and the two most common instances arepthread_exit on NX platforms or longjmp on Windows)

Kyle Strand (Jan 08 2020 at 20:40, on Zulip):

For inert frames, my inclination is to say that any interaction with a forced unwind is a violation of both LLVM's and Itanium's requirements for how language runtimes behave.

Kyle Strand (Jan 08 2020 at 20:40, on Zulip):

@nikomatsakis @Amanieu does that seem correct?

Amanieu (Jan 08 2020 at 20:56, on Zulip):

For inert frames we just let the unwind go through without touching it, yes.

Last update: Jan 28 2020 at 02:00UTC