Stream: wg-async-foundations

Topic: async unsafe fn


nikomatsakis (Jul 08 2019 at 20:15, on Zulip):

Maybe we ought to discuss the async unsafe fn question? cc @Taylor Cramer, @centril, @boats

centril (Jul 08 2019 at 20:16, on Zulip):

Whether to gate or not gate?

centril (Jul 08 2019 at 20:16, on Zulip):

Or the design?

nikomatsakis (Jul 08 2019 at 20:16, on Zulip):

I am referring to @Taylor Cramer's comment

centril (Jul 08 2019 at 20:18, on Zulip):

We did the same for const unsafe fn so I don't think it's surprising and moreover it wasn't mentioned in the RFC or report at all

nikomatsakis (Jul 08 2019 at 20:19, on Zulip):

The basic concern is that async unsafe fn makes it unsafe to produce the closure but not to await it, but almost certainly the unsafety arises because of something the function does and not the mere act of producing the closure.

nikomatsakis (Jul 08 2019 at 20:19, on Zulip):

I suppose you can always "desugar" to

unsafe fn foo() -> impl Future<..> {
    async move { ... }
}
nikomatsakis (Jul 08 2019 at 20:20, on Zulip):

i.e., to get today's semantics

nikomatsakis (Jul 08 2019 at 20:22, on Zulip):

I think the part of @Taylor Cramer's comment that caught my eye was this:

Yet another feature gate will contribute to the impression of async/await as unfinished.

which is perhaps a more basic disagreement.

Still, I feel like async unsafe fn will be a relatively unusual thing..?

centril (Jul 08 2019 at 20:22, on Zulip):

I think the impression is true so there's no need to pretend it is not

centril (Jul 08 2019 at 20:23, on Zulip):

There's an honesty to that; and I don't think gating async unsafe fn changes it particularly

centril (Jul 08 2019 at 20:23, on Zulip):

const fn is also radically more unfinished, yet we shipped it

nikomatsakis (Jul 08 2019 at 20:24, on Zulip):

I think the impression is true so there's no need to pretend it is not

I also feel that way :)

nikomatsakis (Jul 08 2019 at 20:25, on Zulip):

(If nothing else, the fact that it doesn't work in traits feels incomplete, though I understand that this is an oversimplification.)

Taylor Cramer (Jul 08 2019 at 20:38, on Zulip):

I believe that const unsafe fn is significantly different from unsafe async fn in that the interaction between unsafe and const is actually unclear

Taylor Cramer (Jul 08 2019 at 20:38, on Zulip):

there are actually key things about what unsafe code is allowed in const fn that had not yet been determined

Taylor Cramer (Jul 08 2019 at 20:39, on Zulip):

I don't believe the same is true for async unsafe fn-- I don't know of any outstanding questions around the interaction between async and unsafe, and the current behavior is, I believe, easily justified by the desugared understanding of what async fn means.

Taylor Cramer (Jul 08 2019 at 20:40, on Zulip):

I think the impression is true so there's no need to pretend it is not

Taylor Cramer (Jul 08 2019 at 20:41, on Zulip):

I'm not sure what the point you're making here is. We've put a lot of work into making sure that async/await provides a positive end-user experience despite some currently fundamental limitations.

Taylor Cramer (Jul 08 2019 at 20:42, on Zulip):

Those limitations are necessary, understandable and easily justified once users understand the desugared model of async fn.

Taylor Cramer (Jul 08 2019 at 20:45, on Zulip):

Blocking basic things like unsafe from working, however, isn't a fundamental or difficult problem in any way, is not interesting to consider from a user's perspective, the workarounds don't teach the user more about the system than they would have known otherwise (unlike the lessons to be learned from attempting to use async fn in traits), and overall I think this sort of limitation will strike users more as missing polish than interesting open design considerations.

Taylor Cramer (Jul 08 2019 at 20:45, on Zulip):

In that, I think it's more similar to bad error messages than it is to not being able to use async fn in traits.

centril (Jul 08 2019 at 20:49, on Zulip):

I believe that const unsafe fn is significantly different from unsafe async fn in that the interaction between unsafe and const is actually unclear

It wasn't that unclear actually, there was just some hesitation on my end but we didn't change that much aside from not allowing some things to be const.

[...], and the current behavior is, I believe, easily justified by the desugared understanding of what async fn means.

This assumes a certain desugaring and it focuses on explanation by desugaring. One could imagine a different one which would render .await the point at which you have to use unsafe { ... } which at least from my view is saner on initial inspection.

Taylor Cramer (Jul 08 2019 at 20:50, on Zulip):

No, I don't believe that would be correct

Taylor Cramer (Jul 08 2019 at 20:50, on Zulip):

(using unsafe where the future is awaited, rather than constructed)

Taylor Cramer (Jul 08 2019 at 20:51, on Zulip):

For one, await isn't the only way to poll or manipulate a future. The moment at which the future is constructed is the moment at which you must ensure all invariants are satisfied

Taylor Cramer (Jul 08 2019 at 20:51, on Zulip):

this is true today with unsafe fn foo() -> impl Future

nikomatsakis (Jul 08 2019 at 20:53, on Zulip):

For one, await isn't the only way to poll or manipulate a future. The moment at which the future is constructed is the moment at which you must ensure all invariants are satisfied

an interesting observation.

nikomatsakis (Jul 08 2019 at 20:54, on Zulip):

I feel like it happens a plenty that the "contract" on an unsafe fn constraints future actions of what you can do w/ the return value

nikomatsakis (Jul 08 2019 at 20:54, on Zulip):

(and past actions of what you can have done leading up to this)

Taylor Cramer (Jul 08 2019 at 20:54, on Zulip):

Yeah, I do this all the time with things that are e.g. implementations of safe Visitor traits that have unsafe behavior and so are unsafe to construct, yet the trait methods themselves are safe

Taylor Cramer (Jul 08 2019 at 20:55, on Zulip):

it was one of the reasons I pushed back against @RalfJ 's idea way back when that literal unsafe { ... } blocks should have an effect on the way the code inside them was optimized

nikomatsakis (Jul 08 2019 at 20:56, on Zulip):

we'll leave that part aside :P

nikomatsakis (Jul 08 2019 at 20:56, on Zulip):

(there is clearly still some kind of "scope" of unsafety that ought to be observed...)

nikomatsakis (Jul 08 2019 at 20:57, on Zulip):

but it's neither here nor there :) well, it is relevant, in that this is the concern

nikomatsakis (Jul 08 2019 at 20:57, on Zulip):

but you are correct that this same thing arises frequently in Rust

Taylor Cramer (Jul 08 2019 at 20:57, on Zulip):

Right, I don't think the situation with async unsafe fn is unique or interesting, either of which might suggest that we should deliberate

nikomatsakis (Jul 08 2019 at 20:58, on Zulip):

I could vaguely imagine some kind of system where you can do -> impl unsafe Future to indicate that invoke the trait methods is unsafe, which I guess is more in the direction of what @centril is imagining.

Taylor Cramer (Jul 08 2019 at 20:58, on Zulip):

right, with an unsafe poll

nikomatsakis (Jul 08 2019 at 20:58, on Zulip):

But it seems... pretty far off to me to even contemplate. I guess it depends if we get some notion of a const trait bound that can be applied

nikomatsakis (Jul 08 2019 at 20:59, on Zulip):

(the alternative would be a impl UnsafeFuture hack, where that is a distinct trait)

Taylor Cramer (Jul 08 2019 at 20:59, on Zulip):

which requires some kind of abstraction over whether or not a trait impl is unsafe and a way to be generic over unsafety of a trait implementor

nikomatsakis (Jul 08 2019 at 20:59, on Zulip):

I guess it depends if we get some notion of a const trait bound that can be applied

this is specifically the thing I had in mind when I said "seems like it can't hurt to take some time and think about it"

Taylor Cramer (Jul 08 2019 at 20:59, on Zulip):

the same pattern can be applied to the visitor i mentioned above

Taylor Cramer (Jul 08 2019 at 20:59, on Zulip):

This is all astronomically far away from reality, though

centril (Jul 08 2019 at 21:04, on Zulip):

Yeah, I do this all the time with things that are e.g. implementations of safe Visitor traits that have unsafe behavior and so are unsafe to construct, yet the trait methods themselves are safe

Things like that does surprise me; that's not my idea of how scoping of unsafe works but I suppose unsafe is very "build your own" in terms of how you define scoping.

For one, await isn't the only way to poll or manipulate a future.

A good point; ostensibly you can imagine trait UnsafeFuture: Future {} which is checked on calls to .poll through lang items to change this but costs are increasing now... (i.e. "hack" in @nikomatsakis's sense but the trait itself can be forever-unstable)

this is true today with unsafe fn foo() -> impl Future

Well the unsafety could be before the future is returned in that case so it feels different.

The moment at which the future is constructed is the moment at which you must ensure all invariants are satisfied

Any particular examples of when it would be problematic for async unsafe fn otherwise?

This is all astronomically far away from reality, though

That's not my impression from talking with Oliver's re. const fn and trait bounds.

nikomatsakis (Jul 08 2019 at 21:04, on Zulip):

I can go either way I guess. I feel like the current semantics are pretty reasonable given the state of Rust (and, indeed, the desugaring). But I also don't mind waiting a bit to talk over the possible ways we might handle this, it seems like a bit of a corner case.

I also like the strategy of using a "reference desugaring" (whether or not that is how it is implemented) to help handle corner cases in the semantics.

centril (Jul 08 2019 at 21:06, on Zulip):

I also like the strategy of using a "reference desugaring" (whether or not that is how it is implemented) to help handle corner cases in the semantics.

I like this as well when you can actually do it but this can also be a trap... one that I think I laid for myself with if a && let b = c {

boats (Jul 09 2019 at 14:06, on Zulip):

I do not believe there is a viable alternative for async unsafe fn to what is currently implemented and I don't think it should be blocked from stabilization.

boats (Jul 09 2019 at 14:08, on Zulip):

A good point; ostensibly you can imagine trait UnsafeFuture: Future {} which is checked on calls to .poll through lang items to change this but costs are increasing now... (i.e. "hack" in

boats (Jul 09 2019 at 14:09, on Zulip):

This doesn't make sense to me, these types would still implement Future and thus have a safe poll method (unless you meant adding a supertrait to future, rather than what you wrote, but even this is pretty contrived and complicated)

nikomatsakis (Jul 09 2019 at 14:11, on Zulip):

Yeah it has to be a supertrait

nikomatsakis (Jul 09 2019 at 14:11, on Zulip):

But that route feels "not worth it" to me

boats (Jul 09 2019 at 14:15, on Zulip):

yea its a lot of machiner and I agree with taylor that this represents a very normal pattern (imposing invariants on the constructor you expect to be true during the lifetime of the object)

boats (Jul 09 2019 at 14:15, on Zulip):

literally this is how Vec::from_raw_parts works!

boats (Jul 09 2019 at 14:20, on Zulip):

I also don't know what invariants you could be upholding at an await point, either.. I guess if it involved some shared state passed in through the function call.

boats (Jul 09 2019 at 14:22, on Zulip):

Yea after these few minutes of reflection, not only do I think an alternative design is overcomplicated, I think it would be wrong. In practice, the only place to uphold invariants on an async fn is at the call site.

nikomatsakis (Jul 09 2019 at 14:27, on Zulip):

I mean I think you may well have to check things at the await site -- i.e., that a raw pointer is still valid.

nikomatsakis (Jul 09 2019 at 14:28, on Zulip):

But I do think it's "the normal pattern" in Rust today to denote such things primarily at the creation site (from-raw-parts being a good example)

boats (Jul 09 2019 at 14:28, on Zulip):

thats what i mean by shared state passed in through the call

nikomatsakis (Jul 09 2019 at 14:28, on Zulip):

And a lot of times those invariants are solely* enforced at the creation site in any case

nikomatsakis (Jul 09 2019 at 14:28, on Zulip):

e.g., "this must be a raw pointer that represents an owned box" or "this index must be within bounds for that thing" or something

nikomatsakis (Jul 09 2019 at 14:29, on Zulip):

(I guess it'd be interesting to see some examples in practice)

boats (Jul 09 2019 at 14:32, on Zulip):

It's also true for Vec::from_raw_parts for example, that you can't invalidate the pointer you pass to it through another API later on.

boats (Jul 09 2019 at 14:34, on Zulip):

I don't even think this rises to the level of the interaction between Pin and Drop, where we unfortunately have to uphold invariants in a less than ideal place (pin projection) because we can't uphold them in the best place (the drop impl).

nikomatsakis (Jul 09 2019 at 16:57, on Zulip):

It's also true for Vec::from_raw_parts for example, that you can't invalidate the pointer you pass to it through another API later on.

right, it represents a (logical) ownership transfer

RalfJ (Jul 10 2019 at 06:52, on Zulip):

it was one of the reasons I pushed back against RalfJ 's idea way back when that literal unsafe { ... } blocks should have an effect on the way the code inside them was optimized

I wouldnt even say it was my idea, I think I got it from Niko's blog post. ;) But yes I entertained it for a while. Mostly out of necessity. But with Stacked Borrows it does not seem needed any more and then I definitely prefer not to do it (i.e., I prefer to leave unsafe a purely syntactic marker).

Last update: Nov 18 2019 at 01:05UTC