Stream: t-lang/wg-unsafe-code-guidelines

Topic: Clarification: Can you have &T and *mut T point at the sa...


Lokathor (Jun 28 2019 at 18:54, on Zulip):

Assuming that the *mut T is not used to mutate the value, is it legal to have it point to the same location as a live &T points at?

RalfJ (Jun 28 2019 at 18:55, on Zulip):

in general yes, though it also matters how the ptrs got created

RalfJ (Jun 28 2019 at 18:56, on Zulip):

if you created the *mut T from the &T the certainly yes

RalfJ (Jun 28 2019 at 18:57, on Zulip):

(via my_ref as *const _ as *mut T)

RalfJ (Jun 28 2019 at 18:57, on Zulip):

if it's derived from a parent of &T, then other things you did before might have already killed that pointer

RalfJ (Jun 28 2019 at 18:57, on Zulip):

also anything I say is "assuming Stacked Borrows", which is not normative etc ;)

Lokathor (Jun 28 2019 at 18:58, on Zulip):

I think it's the other way around. Ptr all the time, some of the time a &T is made from it temporarily

RalfJ (Jun 28 2019 at 18:59, on Zulip):

you are making the &T directly from the *mut T?

RalfJ (Jun 28 2019 at 18:59, on Zulip):

then your code would even be safe if you replaced *mut T by &mut T, so yes definitely allowed

Lokathor (Jun 28 2019 at 18:59, on Zulip):

uh, i think. They've been updating some old async locking code

RalfJ (Jun 28 2019 at 18:59, on Zulip):

that's like

let x = &mut 42;
let x_shr = &*x;
let _val = *x;
let _val = *x_shr;
let _val = *x;
RalfJ (Jun 28 2019 at 19:00, on Zulip):

as you can see we can use both interchangabky

Lokathor (Jun 28 2019 at 19:00, on Zulip):

They've been guided to use NonNull<T> instead of a raw pointer and then make either &T or &mut T depending of the type of lock the user takes on the data, which seems right

RalfJ (Jun 28 2019 at 19:01, on Zulip):

NonNull is a raw ptr though

RalfJ (Jun 28 2019 at 19:01, on Zulip):

just wrapped

RalfJ (Jun 28 2019 at 19:01, on Zulip):

so that part doesnt change anything

Lokathor (Jun 28 2019 at 19:02, on Zulip):

Yeah that's what I thought :/

Lokathor (Jun 28 2019 at 19:07, on Zulip):

So NonNull has a niche, and *const doesn't allow writing directly to the value, but they're both otherwise effectively the same as *mut. It's just one "raw pointer" concept at the heart of all three?

RalfJ (Jun 28 2019 at 19:12, on Zulip):

basically, yes

RalfJ (Jun 28 2019 at 19:12, on Zulip):

with the one small caveat that &mut foo as *const T implicitly goes through &T

RalfJ (Jun 28 2019 at 19:13, on Zulip):

so, which type you cast to affects which intermediate reference gets created

RalfJ (Jun 28 2019 at 19:13, on Zulip):

and shared references cannot be written to so this makes a difference

RalfJ (Jun 28 2019 at 19:13, on Zulip):

(that's at least what happens currently, I hope we can change that, but that might be hard)

Lokathor (Jun 28 2019 at 19:59, on Zulip):

that's a bit of a footgun technicality for sure

Lokathor (Jun 28 2019 at 20:02, on Zulip):

https://github.com/64/spin-rs/blob/master/src/rw_lock.rs is apparently the full module they're working on, but it's like 700 lines so i don't expect folks to necessarily inspect it all. The crate does seem to get on the order of thousands of downloads a month though, so an eventual careful inspection wouldn't be out of place

Matt Taylor (Jun 28 2019 at 20:04, on Zulip):

i'm the author, would appreciate if you could take a look. i'm fixing an issue with the variance of the RAII guards atm

RalfJ (Jun 28 2019 at 20:06, on Zulip):

makes me wonder if you couldnt use parking_lot with a custom parking implementation that just spins, to avoid having to reimplement all these concurrency primitives?

Matt Taylor (Jun 28 2019 at 20:07, on Zulip):

it's supposed to be no_std, so i'm not sure how easy that would be

RalfJ (Jun 28 2019 at 20:07, on Zulip):

not sure either^^ sorry didnt mean it was obvious or so

RalfJ (Jun 28 2019 at 20:07, on Zulip):

just thinking of ways to maximize sharing with existing well-reviewed well-tested code :D

Shnatsel (Jun 28 2019 at 20:19, on Zulip):

@Matt Taylor someone in #black-magic in Rust Community Discord is combing through it for the past day or so. Perhaps you should chat. https://bit.ly/rust-community

Matt Taylor (Jun 28 2019 at 20:19, on Zulip):

i've successfully managed to impersonate myself, in that case

Matt Taylor (Jun 28 2019 at 20:20, on Zulip):

[that's me :^)]

RalfJ (Jun 28 2019 at 20:33, on Zulip):

now that's some effective identity theft

Matt Taylor (Jun 28 2019 at 22:52, on Zulip):

looks like we're going to need some further clarification: is this UB?

let mut v = 0_i32;
let ptr: *const i32 = &v;
{ let _y = &mut v; }
let _z = unsafe { *ptr };
Matt Taylor (Jun 28 2019 at 22:52, on Zulip):

miri flags the last line as UB

nagisa (Jun 28 2019 at 23:47, on Zulip):

I believe so, yeah. Imagine there were two threads and the pointer was in one and the reference in another. This would allow for a data race (UB).

nagisa (Jun 28 2019 at 23:48, on Zulip):

note that data race is a result of invoking UB, in this case having mutable reference and "live" pointer to the same thing at the same time.

nagisa (Jun 28 2019 at 23:49, on Zulip):

Though that’s just my interpretation and the official answer and exact wording may be different.

gnzlbg (Jun 28 2019 at 23:55, on Zulip):

I think we are going to need some super simple examples like this one to teach stacked borrows.

gnzlbg (Jun 28 2019 at 23:55, on Zulip):

And we don't only need to teach how it works, but also, why it works this way.

gnzlbg (Jun 28 2019 at 23:56, on Zulip):

This example would be better if it would make a potential pitfall clearer

gnzlbg (Jun 29 2019 at 00:00, on Zulip):

Maybe start with:

struct P(*const i32);
unsafe impl Send for P {}
let mut v = 0_i32;
let ptr = P(&v);
let h = std::thread::spawn(|| {
    let y = &mut v;
    *y = 3;
});
let _z = unsafe { *ptr.0 }; // data-race
h.join();
Matt Taylor (Jun 29 2019 at 00:01, on Zulip):

i definitely agree on that, i haven't been doing rust as long as some people here but these things seem are quite subtle and somehow i haven't heard about them before

Matt Taylor (Jun 29 2019 at 00:02, on Zulip):

what confuses me is that you could in theory synchronise access and make sure that the accesses on different threads never happen at the same time (eg by using an atomic)

gnzlbg (Jun 29 2019 at 00:02, on Zulip):

you could, but the code doesn't do that

Matt Taylor (Jun 29 2019 at 00:02, on Zulip):

but the single threaded variant effectively synchronises them

gnzlbg (Jun 29 2019 at 00:12, on Zulip):

weird the spawn code is ok

Matt Taylor (Jun 29 2019 at 00:13, on Zulip):

i didn't think miri could deal with multiple threads yet? (assuming gnzlbg is talking about miri)
https://github.com/rust-lang/miri/issues/789#issuecomment-505201476

Matt Taylor (Jun 29 2019 at 00:14, on Zulip):

'for example, Miri currently does not support concurrency' from the readme

gnzlbg (Jun 29 2019 at 00:16, on Zulip):

@Matt Taylor https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=12c5c1a90341a63f7c5a4d66520321a1

gnzlbg (Jun 29 2019 at 00:16, on Zulip):

that fails under miri

gnzlbg (Jun 29 2019 at 00:17, on Zulip):

(I had to adapt the signature of spawn a bit - I guess for an example something like crossbeam::scope would be better)

Matt Taylor (Jun 29 2019 at 00:20, on Zulip):

yeah, makes sense given that the original example i posted is UB

Lokathor (Jun 29 2019 at 00:57, on Zulip):

it doesn't make sense that the 4 line example is UB given that it's clearly all in a single thread

Lokathor (Jun 29 2019 at 00:57, on Zulip):

i mean there's a lot of code that's UB if you suddenly make it multi-threaded, that doesn't make it UB single threaded

nagisa (Jun 29 2019 at 01:35, on Zulip):

@Lokathor Rust’s mutable references introduce certain requirements to the code which you are not supposed to violate. One of them is no aliasing. This is what ultimately makes even something like this UB, even if you never really cause a data race to happen. The reason why I bought up the data race is because that is the most trivial way to demonstrate issues manifesting due to violating such kind of UB.

nagisa (Jun 29 2019 at 01:36, on Zulip):

Now, whether just having a mutable reference and an aliased, but "dead" (unused) pointer is a problem is another question altogether. However in their case the pointer is live which makes that particular piece of code UB.

Lokathor (Jun 29 2019 at 01:43, on Zulip):

this is unfortunate and terrible

RalfJ (Jun 29 2019 at 09:10, on Zulip):

looks like we're going to need some further clarification: is this UB?

let mut v = 0_i32;
let ptr: *const i32 = &v;
{ let _y = &mut v; }
let _z = unsafe { *ptr };

Miri flags this as UB because then you create a new mutable reference like _y, you are asserting that it is the only pointing to whatever-you-crated-it-from -- that's v in this case. So anything that already exists that points to v is now invalid.

RalfJ (Jun 29 2019 at 09:11, on Zulip):

it's not really about concurrency at all, in my mind

RalfJ (Jun 29 2019 at 09:11, on Zulip):

the point is that we'd like to optimize assuming that mutable references are unique

RalfJ (Jun 29 2019 at 09:11, on Zulip):

and this code violates (some interpretation of) that uniqueness requirement

RalfJ (Jun 29 2019 at 09:12, on Zulip):

however, also see https://github.com/rust-lang/unsafe-code-guidelines/issues/133: I am aware that this asserting-uniqueness-on-creation is very aggressive, and maybe there is a way to do better here, i.e. to allow more code without losing (most) optimizations.

RalfJ (Jun 29 2019 at 09:13, on Zulip):

@Lokathor @Matt Taylor question while I got your attention, how surprised are you by this code being UB? Any variant of Stacked Borrows that I can imagine makes this UB:

let mut v = 0_i32;
let ptr: *const i32 = &v;
{ let y = &mut v; *y = 1; }
let _z = unsafe { *ptr };
RalfJ (Jun 29 2019 at 09:13, on Zulip):

now the mutable reference gets used for an access, and at that point definitely it cannot have any aliases

Lokathor (Jun 29 2019 at 13:25, on Zulip):

Retep998 strongly believes that such code is not supposed to be UB because a pointer has no rules beyond C rules, and in C you could do that, and so _even if_ Rust rules apply while a reference exists (which is fair), basically once the reference stops existing things should "revert" to just being normal pointer-only rules, including a restoration of the *const pointer.

Lokathor (Jun 29 2019 at 13:26, on Zulip):

i tend to agree

gnzlbg (Jun 29 2019 at 13:34, on Zulip):

Can you do that even if you mix it with a restrict pointer to the same memory location ?

gnzlbg (Jun 29 2019 at 13:37, on Zulip):
int v = 0;
int const* ptr = &v;
{ int* restrict y = &v; *y = 1; }
int z = *ptr; // OK ?
gnzlbg (Jun 29 2019 at 13:45, on Zulip):

This is definitely UB in C:

int v = 0;
int const* ptr = &v;
int* restrict y = &v;
int z = *ptr; // UB
*y = 1;

you can't access v via ptr because there is a restrict pointer to v in that scope that modifies it.

gnzlbg (Jun 29 2019 at 13:46, on Zulip):

I suppose that the first example is OK in C, because the access doesn't happen in the scope of the restrict pointer, but this is also UB in C:

int v = 0;
int const* ptr = &v;
int* restrict y = &v; *y = 1;
int z = *ptr; // UB

(just removed the {} of the first example, increasing the scope of restrict)

RalfJ (Jun 29 2019 at 14:44, on Zulip):

Retep998 strongly believes that such code is not supposed to be UB because a pointer has no rules beyond C rules, and in C you could do that, and so _even if_ Rust rules apply while a reference exists (which is fair), basically once the reference stops existing things should "revert" to just being normal pointer-only rules, including a restoration of the *const pointer.

the pointer has no rules, but the reference does

RalfJ (Jun 29 2019 at 14:44, on Zulip):

if you use only pointers, you wont have a problem

RalfJ (Jun 29 2019 at 14:45, on Zulip):

but you are mixing references with pointers. we have to make this UB if we want to get any benefit from our aliasing.

RalfJ (Jun 29 2019 at 14:46, on Zulip):

if I applied your argument, the following code would be fine:

fn foo(x: &mut i32, y: *mut i32) {
// Note that we emit noalias here, telling LLVM these don't alias!
}

let mut local = 4;
let raw = &mut local as *mut _;
foo(&mut local, raw); // *oops*
RalfJ (Jun 29 2019 at 14:47, on Zulip):

so @Lokathor the only way to make such code not UB and do what Retep998 suggests is to remove the aliasing optimizations that we are already performing

RalfJ (Jun 29 2019 at 14:48, on Zulip):

"stops existing" is not something I think we can formalize, and scopes clearly shouldnt matter

RalfJ (Jun 29 2019 at 14:48, on Zulip):

so my code is the same as

let mut v = 0_i32;
let ptr: *const i32 = &v;
let y = &mut v; *y = 1;
let _z = unsafe { *ptr };
RalfJ (Jun 29 2019 at 14:48, on Zulip):

I suppose that the first example is OK in C, because the access doesn't happen in the scope of the restrict pointer, but this is also UB in C:

in C it's not really clear what restrict means when not used as a function argument, I think

Lokathor (Jun 29 2019 at 14:50, on Zulip):

So if you have *mut and then use .as_mut to get &mut and then pass that to a function, and then the function returns... your *mut is dead so you just lost your value?

RalfJ (Jun 29 2019 at 14:51, on Zulip):

I think what you are basically saying is that raw pointers should exist "next to" the tracking that knows which references got "derived from" which other reference and so on... the problem with that is that you can create a reference from a raw pointer, and we have to know where that reference "comes from"

RalfJ (Jun 29 2019 at 14:51, on Zulip):

So if you have *mut and then use .as_mut to get &mut and then pass that to a function, and then the function returns... your *mut is dead so you just lost your value?

if you create the &mut from the *mut you are fine

RalfJ (Jun 29 2019 at 14:52, on Zulip):

it's all about where the pointers "come from" being well-nested

RalfJ (Jun 29 2019 at 14:52, on Zulip):

for your program, it's more like a tree... hm not sure how to draw that here

Lokathor (Jun 29 2019 at 14:52, on Zulip):

yes, pointers next to references is the mental model that most people using unsafe code that I've spoken to subscribe to

RalfJ (Jun 29 2019 at 14:53, on Zulip):

the root is v. ptr is one child of v and y is another child of v. they exist next to each other and thus are in conflict

RalfJ (Jun 29 2019 at 14:53, on Zulip):

whereas if you make y a child of v, the situation is very different

RalfJ (Jun 29 2019 at 14:53, on Zulip):

yes, pointers next to references is the mental model that most people using unsafe code that I've spoken to subscribe to

but then what happens when you create a reference again? you have to "put it somewhere"

RalfJ (Jun 29 2019 at 14:53, on Zulip):

and that requires tracking raw pointers as well

RalfJ (Jun 29 2019 at 14:54, on Zulip):

like... let me see if I can make an example

RalfJ (Jun 29 2019 at 14:55, on Zulip):
let x = &mut 42;
let y = x as *mut _; // y derived from x
let z = &mut *y; // z derived from y.
*y = 14; // this has to make z invalid! if we used z again there was an access between creating it and using it, a clear aliasing violation.
// however, x remains valid, because y was "created from" x so x "knows" that mutations can happen through y.
*x = 14; // however, now x may again assume that no mutations through anything else can happen.
RalfJ (Jun 29 2019 at 14:56, on Zulip):

the only way I know to explain this is to say that y is "between" x and z

Lokathor (Jun 29 2019 at 15:02, on Zulip):

okay i see what you're saying, but, if i had my way, *y=14 is UB since z is live, and the rest of the program is unimportant once that happens

rkruppe (Jun 29 2019 at 15:04, on Zulip):

That seems even worse for unsafe code authors?

Lokathor (Jun 29 2019 at 15:04, on Zulip):

not that there couldn't be another model, there is apparently another model already, but you should post BUG HUGE signs about this sort of thing because most people are currently living under a model where addresses (only) are what matter, not any kind of source chain

Lokathor (Jun 29 2019 at 15:08, on Zulip):

@rkruppe no it's a really easy model for unsafe code authors: "is anyone else referencing this address right now? (no) is it aligned and a valid bit pattern and all that? (yes) Okay I can make a &mut pointing at it and once I'm sure that &mut is gone nothing else about my program is different."

RalfJ (Jun 29 2019 at 15:14, on Zulip):

okay i see what you're saying, but, if i had my way, *y=14 is UB since z is live, and the rest of the program is unimportant once that happens

that's weird, because this is safe:

let x = &mut 42;
let y = &mut *x; // y derived from x
let z = &mut *y; // z derived from y.
*y = 14;
*x = 14;
RalfJ (Jun 29 2019 at 15:14, on Zulip):

not that there couldn't be another model, there is apparently another model already, but you should post BUG HUGE signs about this sort of thing because most people are currently living under a model where addresses (only) are what matter, not any kind of source chain

well most people are wrong in many ways ;)

RalfJ (Jun 29 2019 at 15:14, on Zulip):

and even your model makes the source matter

RalfJ (Jun 29 2019 at 15:15, on Zulip):

if I used z instead of y I assume you are fine, even though both point to the same address

rkruppe (Jun 29 2019 at 15:15, on Zulip):

@Lokathor That is not the direction I'm worried about. You seem to suggest that creating raw pointers from a mutable reference and using them always causes UB, even if the mutable reference is not used while the raw pointer(s) are in use. On its face that seems to outlaw lots of unsafe code that should obviously work, e.g.:

let some_memory: &mut Foo = ...;
ptr::write(some_memory as *mut Foo, Default::default());
RalfJ (Jun 29 2019 at 15:15, on Zulip):

rkruppe no it's a really easy model for unsafe code authors: "is anyone else referencing this address right now? (no) is it aligned and a valid bit pattern and all that? (yes) Okay I can make a &mut pointing at it and once I'm sure that &mut is gone nothing else about my program is different."

what does it mean "referencing right now"? in my safe example we have three references to the same thing, so it should be UB under your rules!

Lokathor (Jun 29 2019 at 15:19, on Zulip):

oh i see, i read Ralf's example wrong

Lokathor (Jun 29 2019 at 15:19, on Zulip):

i should never look at code on my phone, my brain doesn't format it right somehow

RalfJ (Jun 29 2019 at 15:20, on Zulip):

^^

Lokathor (Jun 29 2019 at 15:20, on Zulip):

I'll have to review and think again later today

RalfJ (Jun 29 2019 at 15:20, on Zulip):

Stacked Borrows tries to formalize a notion that is like "liveness", but we cannot use literal liveness because that's a static thing

RalfJ (Jun 29 2019 at 15:20, on Zulip):

at the *y = 14 we dont know yet if z is live as we cannot predict if it gets used in the future

RalfJ (Jun 29 2019 at 15:21, on Zulip):

and I'd rather not make UB at one point depend on what the program does in the future^^ that route lies madness

RalfJ (Jun 29 2019 at 15:21, on Zulip):

(I am aware that UB is atemporal, but this would be far worse)

Lokathor (Jun 29 2019 at 15:25, on Zulip):

so, what I've seen many people do is imagine it all as being similar to how drop works

Lokathor (Jun 29 2019 at 15:25, on Zulip):

if &mut is in scope, even if it's not used later, it's illegal to violate the rules, it hasn't "dropped" yet

RalfJ (Jun 29 2019 at 15:40, on Zulip):

that's incompatible with NLL though

RalfJ (Jun 29 2019 at 15:40, on Zulip):

since NLL, reference lifetime has nothing to do with scopes any more

RalfJ (Jun 29 2019 at 15:41, on Zulip):

I tried implementing something "drop-like" some years ago: https://www.ralfj.de/blog/2017/07/17/types-as-contracts.html

RalfJ (Jun 29 2019 at 15:41, on Zulip):

it didnt work

gnzlbg (Jun 29 2019 at 16:37, on Zulip):

I find the current model intuitive. If I look at a pointer or a reference, I just need to walk up its ancestors, checking if they had other children that might have invalidated the reference i'm studying

gnzlbg (Jun 29 2019 at 16:38, on Zulip):

(deleted)

Lokathor (Jun 29 2019 at 19:12, on Zulip):

Well once it's explained it mostly makes sense

Lokathor (Jun 29 2019 at 19:26, on Zulip):

but this might need a PSA post, is I guess what I'm saying

"Hey, a lot of yall think it works like X, but really the rules are more like Y. This isn't even a change, it's been like this for a while now."

Matt Taylor (Jun 29 2019 at 19:29, on Zulip):

definitely agree. seems to be a lot of confusion

gnzlbg (Jun 29 2019 at 19:32, on Zulip):

The rules aren't RFCed yet, so until that happens, its only interesting if you are using miri to find UB in your code

gnzlbg (Jun 29 2019 at 19:33, on Zulip):

Even if the rules are RFC'ed, many users don't really need to know them

gnzlbg (Jun 29 2019 at 19:34, on Zulip):

And from those who need to, listening to miri might be enough

gnzlbg (Jun 29 2019 at 19:40, on Zulip):

Once the rules are written somewhere, learning all of this will be easier

RalfJ (Jun 29 2019 at 20:46, on Zulip):

right, these are the current WIP rules

RalfJ (Jun 29 2019 at 20:47, on Zulip):

the relevant posts are on my blog, they did get announced on reddit + IRLO

RalfJ (Jun 29 2019 at 20:47, on Zulip):

but nothing is official at this point

RalfJ (Jun 29 2019 at 20:47, on Zulip):

and it seems like anyway the rules will have to change again...

Lokathor (Jun 30 2019 at 02:41, on Zulip):

If people are triggering UB because they're writing code according to a different memory model than is the actual one, everyone cares about that.

Let me put it another way: Lots of people write unsafe code, few of them use miri on that code because it's "obviously" already correct.

Even if the current rules are only temporary and subject to change later, people deserve to know what the current rules are at all times. Like, the usual Rust policy of saying little until one final ultimate stability push is over and done with, and then announcing however it ended, is a very bad plan in this situation.

Yall are great, but people are out there footgunning themselves today, they don't wait for you to do anything, they just push unsound code like there's no tomorrow. Educational materials should be spread sooner, not later.

RalfJ (Jun 30 2019 at 09:11, on Zulip):

well I don't know what to do beyond writing blog posts and helping answer questions here. it's not like the rules are secret or anything.

the usual Rust policy of saying little until one final ultimate stability push is over and done with

this is unfair. first of all that's not at all the usual Rust policy, there is so much discussion everywhere, but of course if you wait until it appears on the official blog you won't see it. that's not what the official blog is for.
and secondly if we were to assign Stacked Borrows into the usual Rust process, it would be pre-RFC or pre-pre-RFC. there's not usually anything in the official channels or docs at that stage for any proposal, and rightly so. the official channels would drown in noise otherwise.

we do have educational material that says "references have strict aliasing rules, we dont know the rules yet so be conservative". I don't think we can do much more on the official docs at this point in time. if people insist on ignoring that and instead doing things that are "obviously correct", I don't know what we can do about that.

RalfJ (Jun 30 2019 at 09:12, on Zulip):

if we say anything like "the following thing is okay to do with raw pointers and references", that's a big commitment

RalfJ (Jun 30 2019 at 09:14, on Zulip):

I guess we could have a "negative list" somewhere -- "the following things might seem obvious but they are not, don't do that". and maybe hopefully that list wont just be ignored because it contains so many things that people want to do...

RalfJ (Jun 30 2019 at 09:15, on Zulip):

but I also have to add that I am at max capacity with what I am doing with Stacked Borrows and Miri and the Unsafe Code Guidelines currently. there's only so much one person can do in a day. (Not saying I am doing all of that alone, but Stacked Borrows so far has been mostly me -- it's also very complicated to get into so I am not blaming anyone. And some people have helped tremendously developing the model.)

RalfJ (Jun 30 2019 at 09:25, on Zulip):

Help writing more educational material is certainly welcome though. :-)
Of course, that all needs to be updated whenever things change, and they will... writing such things early costs a lot of resources that we currently just don't have.

Lokathor (Jun 30 2019 at 15:11, on Zulip):

I don't think that you need to say "you can do this with raw pointers (now and for all time)". I think you just need to say "as of precisely June 30th of 2019, this is what you can do with raw pointers (and this is all "unstable, implementation defined" and can all change at any moment)".

I'm fairly good at putting out very large word counts of materials as necessary. If you were interested in setting up a place for "what we think the rules currently mean" to be written down I could probably fill in the prose parts in between some select code examples like the little 4 line example we had above. Vetting it for accuracy would still fall on the WG members. I'm not thinking that it has to be too complex, probably a single markdown file could cover it until things move from at least "pre-pre-RFC" up to "pre-RFC" or better.

We could have great examples such as "aliasing or not actually depends on the parentage of where the pointer/reference comes from" and "no, you currently cannot make a slice of uninitialized memory as a buffer to write to, please stop doing that"

RalfJ (Jun 30 2019 at 15:15, on Zulip):

in terms of placement, sounds like a nomicon thing to me?

"no, you currently cannot make a slice of uninitialized memory as a buffer to write to, please stop doing that"

that's unrelated to Stacked Borrows, but I guess that distinction is not very meaningful "from the outside"

Lokathor (Jun 30 2019 at 15:46, on Zulip):

I mean that stacked borrows aren't the only thing that people regularly do wrong :)

And yeah, maybe a "grab bag" page in the Nomicon would fit this situation.

RalfJ (Jun 30 2019 at 15:57, on Zulip):

I mean that stacked borrows aren't the only thing that people regularly do wrong :)

sure. and while we have a lot of docs about what you can do, its much less clearer where to put docs about what you cannot do

RalfJ (Jun 30 2019 at 15:57, on Zulip):

these days, at least the docs for MaybeUninit and mem::uninitialized call out that you cant just run around with uninitialized memory

RalfJ (Jun 30 2019 at 15:57, on Zulip):

(on beta currently, will hit stable next week)

RalfJ (Jun 30 2019 at 15:58, on Zulip):

those might be good docs for you to look over and suggest ho they could be improved, maybe?

Lokathor (Jun 30 2019 at 16:25, on Zulip):

Sure, I'll have a look. I think my usual writing style is a bit more like the Nomicon or Too Many Linked Lists style than like the libstd style.

The other problem is that some of this stuff is currently one way but as you say not really set in stone. For example, https://github.com/rust-lang/unsafe-code-guidelines/issues/71 might make advice against uninit &[u8] obsolete eventually. In this case, people might use old compiler versions of course (eg: debian stable will be 1.34 for a long while), so those versions will need the old info of course. I'm not super clear how the libs/docs team feels about supplimental docs that need to change over time like that.

RalfJ (Jun 30 2019 at 16:38, on Zulip):

In this case, people might use old compiler versions of course (eg: debian stable will be 1.34 for a long while), so those versions will need the old info of course.

actually, at least in this case, that's not true. the compiler is already compatible with a spec that says "integers may be uninitialized as long as you dont do any arithmetic or so on them"

RalfJ (Jun 30 2019 at 16:38, on Zulip):

we just dont want to commit to that yet

RalfJ (Jun 30 2019 at 16:38, on Zulip):

but once we do, this becomes retroactively correct for old compilers

Matt Taylor (Jun 30 2019 at 16:41, on Zulip):

i would personally like a bit more clarity on these two pages in particular https://doc.rust-lang.org/reference/behavior-considered-undefined.html https://doc.rust-lang.org/nomicon/aliasing.html

Matt Taylor (Jun 30 2019 at 16:41, on Zulip):

the nomicon page currently only talks about references

Matt Taylor (Jun 30 2019 at 16:46, on Zulip):

and the reference basically says ‘pointers are like C, check llvm noalias docs for references’ - which is a problem because 1) noalias suggests there’s only a problem on accesses through noalias’d pointers, yet as per my original example it can be UB just by creating a &mut T and not using it at all and 2) it doesn’t detail the interaction between pointers and refs

Matt Taylor (Jun 30 2019 at 16:47, on Zulip):

if i knew all the details in my head of C’s model and LLVM’s noalias i could probably figure that out for myself but people shouldn’t have to dig that far to see what’s allowed

RalfJ (Jun 30 2019 at 16:55, on Zulip):

I agree

Lokathor (Jun 30 2019 at 17:41, on Zulip):

the reference page seems good already

Lokathor (Jun 30 2019 at 17:41, on Zulip):

i mean, it's clearly needing more info later, but at the moment, the things it does say are relatively clear

Lokathor (Jun 30 2019 at 17:42, on Zulip):

the nomicon page... leaves a bit to be desired i think

Lokathor (Jun 30 2019 at 17:43, on Zulip):

If i have time later today, and I think I will, then I'll make an attempt at expanding that page a bit as my first programming priority of the day

RalfJ (Jun 30 2019 at 17:59, on Zulip):

thanks @Lokathor :)
I can help reviewing, though I dont have any r+ powers for the reference or the nomicon. but we'll find someone who does.^^

Lokathor (Jun 30 2019 at 18:02, on Zulip):

I would have guessed the Nomicon to be under the direct control of the unsafe wg

RalfJ (Jun 30 2019 at 18:07, on Zulip):

no, the nomicon is waay older than we are

RalfJ (Jun 30 2019 at 18:07, on Zulip):

and we dont have anything directly to do with it.

RalfJ (Jun 30 2019 at 18:07, on Zulip):

but that sounds like a thing we might want to change :D

RalfJ (Jun 30 2019 at 18:08, on Zulip):

the problem is, the WG doesnt really have any leftover bandwidth. we can barely keep our discussions going...

Lokathor (Jun 30 2019 at 21:04, on Zulip):

assimilate more members

Lokathor (Jun 30 2019 at 23:00, on Zulip):

@RalfJ https://github.com/rust-lang-nursery/nomicon/pull/144 is a very small edit to just the readme, just to get my feet wet since I haven't really PR'd at repo other than my own in a while. But it's a start

RalfJ (Jul 01 2019 at 17:22, on Zulip):

thanks :)

Ehsan M. Kermani (Jul 19 2019 at 15:29, on Zulip):

May I ask what is "niches" that was discussed here?

rkruppe (Jul 19 2019 at 15:39, on Zulip):

See https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#niche

Ehsan M. Kermani (Jul 19 2019 at 16:33, on Zulip):

Ah, thanks!
Just to make sure about the linked doc, type repr includes size, alignment, niches and abi and having these 4 defines a valid type? In other words, is it always a one-to-four relation?

rkruppe (Jul 19 2019 at 16:49, on Zulip):

repr(esentation) is overloaded and not currently defined in the glossary. I'm not sure what exactly the underlying question is to answer it directly. Certainly there is more a "type" than just these four things. Even for data layout considerations, the location (offsets) of the fields contained in it are important as well.

Ehsan M. Kermani (Jul 19 2019 at 16:53, on Zulip):

Ok, just want to have a correct mental model of the constituents. So anything more than (size, align, offset, niche, abi) + underlying validity requirments?

rkruppe (Jul 19 2019 at 17:05, on Zulip):

Hm. I don't want to commit to that being exhaustive but it's roughly all I can think of that comes up in the current discussions.

RalfJ (Jul 20 2019 at 11:03, on Zulip):

@Ehsan M. Kermani there's also the validity invariant! What exactly that invariant is for various types is currently being discussed.

RalfJ (Jul 20 2019 at 11:03, on Zulip):

see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/active_discussion/validity.md and https://github.com/rust-lang/unsafe-code-guidelines/issues?q=is%3Aissue+is%3Aopen+label%3Atopic-validity

Ehsan M. Kermani (Jul 23 2019 at 17:25, on Zulip):

In light of niches discussion, I don't think the name is representative enough. In cases like boolian, it's not "niches" per say!

RalfJ (Jul 23 2019 at 19:51, on Zulip):

I do not understand what you are trying to say?

Ehsan M. Kermani (Jul 23 2019 at 20:20, on Zulip):

@RalfJ I'm trying to see if there's a better name instead of "niche" :sweat_smile:

RalfJ (Jul 23 2019 at 21:01, on Zulip):

ah. well there probably is but that'd be a discussion to have with @eddyb and t-compiler -- "niche" is what rustc calls it internally ;)

Tom Phinney (Jul 23 2019 at 21:14, on Zulip):

IMO as a native American English speaker, "niche" is about as good a term as we are likely to find.

Jake Goulding (Jul 24 2019 at 01:39, on Zulip):

Just try to get two people to agree on a pronunciation of "niche"

Tom Phinney (Jul 24 2019 at 02:33, on Zulip):

It's like "cache" :rolling_on_the_floor_laughing:

Tom Phinney (Jul 24 2019 at 02:35, on Zulip):

Of course, it really isn't; the terminal consonant is pronounced differently. But just as there are many pronunciations of "cache", there certainly will be similar variety with "niche".

Andreas Molzer (Jul 24 2019 at 07:48, on Zulip):

Pronouncing it 'nitch' or 'nish' instead of neesh is passing on a perfectly justified opportunity for speaking French :wink:

Tom Phinney (Jul 24 2019 at 15:32, on Zulip):

You're right, of course. That's also how it's pronounced in the UK. However, in much of the US it's pronounced "nitch". See https://www.quickanddirtytips.com/education/grammar/how-do-you-pronounce-niche and https://www.google.com/search?q=Dictionary#dobs=niche. If we want to encourage a uniform pronunciation for Rustaceans, the it would be best to include parenthetical pronunciation, such as "(preferably pronounced nēSH)", at the first occurrence of the word in various major Rust teaching documents.

gnzlbg (Jul 24 2019 at 16:29, on Zulip):

FWIW I always found it funny at conferences to hear how different people pronounce the different technical words

gnzlbg (Jul 24 2019 at 16:30, on Zulip):

I don't recall the differences in pronunciation ever being a problem, more like a curiosity

gnzlbg (Jul 24 2019 at 16:31, on Zulip):

e.g. some people pronounce std as "stood", while others say "standard" and others say "es-tee-dee"

gnzlbg (Jul 24 2019 at 16:33, on Zulip):

if we choose "niche" I don't really have a problem with people pronouncing it however they want, i really can't see the different pronunciations leading to communication problems, at most some people will get a laugh

RalfJ (Jul 24 2019 at 17:09, on Zulip):

e.g. some people pronounce std as "stood", while others say "standard" and others say "es-tee-dee"

I don't have any vowel in my version of std, its just "shtd"

Last update: Nov 19 2019 at 19:05UTC