Stream: t-lang/wg-unsafe-code-guidelines

Topic: meeting 2018-10-11


nikomatsakis (Oct 11 2018 at 16:32, on Zulip):

So I think it is too late to reasonably have this meeting

nikomatsakis (Oct 11 2018 at 16:32, on Zulip):

but I would like to make progress here

nikomatsakis (Oct 11 2018 at 16:32, on Zulip):

I was going over the struct+tuple PR this morning and realized that there were some unclear points to me

Alan Jeffrey (Oct 11 2018 at 16:32, on Zulip):

@nikomatsakis well @RalfJ and I already had a meeting about memory invariants :)

nikomatsakis (Oct 11 2018 at 16:33, on Zulip):

=)

nikomatsakis (Oct 11 2018 at 16:33, on Zulip):

I was trying to make edits, it seemed like there were a few "contenious" points and I was debating how to handle it

nikomatsakis (Oct 11 2018 at 16:33, on Zulip):

I think my feeling before has been: no need to settle every question, at least let's describe the unknowns and move on, and circle back later

nikomatsakis (Oct 11 2018 at 16:33, on Zulip):

but it's always tempting to try and "fix" a few things

nikomatsakis (Oct 11 2018 at 16:33, on Zulip):

(to be clear, I view all of this as making "recommendations" that can become RFCs anyway, and maybe that framing helps)

nikomatsakis (Oct 11 2018 at 16:36, on Zulip):

to that end I was debating about writing a section that explicitly said something like "We recommend the following semantics" around these points

nikomatsakis (Oct 11 2018 at 16:37, on Zulip):

but really we don't have to answer those specific questions here necessarily, I'd rather get feedback on the meta question of how we handle this sort of question -- how much should we try to make a decision --

nikomatsakis (Oct 11 2018 at 16:37, on Zulip):

that and how we are going to get progress on the other write-ups we talked about :)

nikomatsakis (Oct 11 2018 at 16:39, on Zulip):

thoughts? :)

Alan Jeffrey (Oct 11 2018 at 16:42, on Zulip):

If "struct Foo { x: T } with one field (and repr(Rust)) has the same memory layout (not necessarily other ABI details) as T" then does this make repr(transparent)unncessary?

nikomatsakis (Oct 11 2018 at 16:43, on Zulip):

no, because repr(transparent) is specifically about the ABI details

Alan Jeffrey (Oct 11 2018 at 16:43, on Zulip):

what's an example? alignment?

nikomatsakis (Oct 11 2018 at 16:44, on Zulip):

no

nikomatsakis (Oct 11 2018 at 16:44, on Zulip):

e.g. do you use a register

nikomatsakis (Oct 11 2018 at 16:45, on Zulip):

or pass on the stack

Alan Jeffrey (Oct 11 2018 at 16:45, on Zulip):

ah, that makes sense.

Nicole Mazzuca (Oct 11 2018 at 16:47, on Zulip):

Imo, the rules around tuple structs for a type struct X(Tn...); should be exactly as if you wrote struct X { fn...: Tn... }

Nicole Mazzuca (Oct 11 2018 at 16:49, on Zulip):

also, we agreed struct layout should be non-deterministic, afaik.

Alan Jeffrey (Oct 11 2018 at 16:50, on Zulip):

I don't think anyone's objecting to (T,T,T) having the same layout as [T;3], is the issue whether x.i should have the same semantics as x[i]?

nikomatsakis (Oct 11 2018 at 16:50, on Zulip):

I feel like the order of named fields should not be significant

nikomatsakis (Oct 11 2018 at 16:51, on Zulip):

for all the details that came up on the thread

nikomatsakis (Oct 11 2018 at 16:51, on Zulip):

the order of anonymous fields is already significant from a semver POV

nikomatsakis (Oct 11 2018 at 16:51, on Zulip):

i.e., if you expose public anonymous fields

nikomatsakis (Oct 11 2018 at 16:51, on Zulip):

your clients are already relying on the order

nikomatsakis (Oct 11 2018 at 16:52, on Zulip):

for all the details that came up on the thread

for all the reasons I meant

nikomatsakis (Oct 11 2018 at 16:52, on Zulip):

so that was my concern: if we just say that "all structs of homogenous type are laid out as an array" the question naturally becomes "which index is which field"

Alan Jeffrey (Oct 11 2018 at 16:52, on Zulip):

@nikomatsakis this is in conflict with the goals of a) having tuples and structs agree on layout, b) having tuples and arrays agree on layout, and c) having array indexing and field access agree. :(

nikomatsakis (Oct 11 2018 at 16:52, on Zulip):

maybe we should just ditch the whole thing

nagisa (Oct 11 2018 at 16:52, on Zulip):

@Alan Jeffrey read this for more details (not of direct relevance, but still pretty relevant)

nikomatsakis (Oct 11 2018 at 16:53, on Zulip):

and you should use the type [T; N] instead :)

nikomatsakis (Oct 11 2018 at 16:53, on Zulip):

I agree @Alan Jeffrey there is some conflict — this is what I was attempting to resolve by saying "the mapping from named fields to indices is undefined"

nikomatsakis (Oct 11 2018 at 16:53, on Zulip):

i.e., it does have the layout of [T; N], but you can't reliably tell which field is where

nikomatsakis (Oct 11 2018 at 16:53, on Zulip):

so then the only difference between tuple structs and named structs is that exact case

nikomatsakis (Oct 11 2018 at 16:54, on Zulip):

(that is, in one case there is a mapping)

nikomatsakis (Oct 11 2018 at 16:54, on Zulip):

and that difference is sort of pre-existing and banked into the pattern matching syntax anyway

nikomatsakis (Oct 11 2018 at 16:54, on Zulip):

anyway I don't know how much I care about this. I think it falls under that "useful" guarantee

nikomatsakis (Oct 11 2018 at 16:54, on Zulip):

perhaps I will move all the "useful guarantees" to some sort of "recommendation" section of "stuff we are 90% sure is a good idea"

Alan Jeffrey (Oct 11 2018 at 16:55, on Zulip):

@nagisa ta, that looks like a related but probably orthogonal issue?

nagisa (Oct 11 2018 at 16:56, on Zulip):

@nagisa ta, that looks like a related but probably orthogonal issue?

It is related in a sense that similar differences apply as for with #[repr(transparent)] structs and #[repr(Rust)] ones.

nikomatsakis (Oct 11 2018 at 16:56, on Zulip):

in any case I definitely think this should not be specific to tuples

nagisa (Oct 11 2018 at 16:56, on Zulip):

Its just the context in which this occurs that’s different.

nagisa (Oct 11 2018 at 16:57, on Zulip):

Tuples cannot guarantee in-memory order of its elements anyway, because of the layout optimisation

nagisa (Oct 11 2018 at 16:58, on Zulip):

I see no reason to make a special case for (T, T, ..., T)

Alan Jeffrey (Oct 11 2018 at 16:58, on Zulip):

@nikomatsakis so is the proposal to drop (c)? or to change the representation of structs such that order does matter in certain cases?

Alan Jeffrey (Oct 11 2018 at 16:59, on Zulip):

(where (c) is "having array indexing and field access agree")

Alan Jeffrey (Oct 11 2018 at 16:59, on Zulip):

(since that itemized list has scrolled off the page already)

nikomatsakis (Oct 11 2018 at 17:00, on Zulip):

I am confused what proposal you are talking about :)

nikomatsakis (Oct 11 2018 at 17:00, on Zulip):

I guess you mean the thing I sketched above?

Alan Jeffrey (Oct 11 2018 at 17:00, on Zulip):

@nikomatsakis yes

nikomatsakis (Oct 11 2018 at 17:00, on Zulip):

I would prefer to focus on a meta question: my proposal for the RFC

nikomatsakis (Oct 11 2018 at 17:00, on Zulip):

I think I settled on this courses of action:

nikomatsakis (Oct 11 2018 at 17:00, on Zulip):

I will enumerate the things I think might be contentious in the RFC

nikomatsakis (Oct 11 2018 at 17:00, on Zulip):

(which include this point)

RalfJ (Oct 11 2018 at 17:00, on Zulip):

so that was my concern: if we just say that "all structs of homogenous type are laid out as an array" the question naturally becomes "which index is which field"

and the answer naturally is "the order as written"? so yes, in this special case the order does matter

nikomatsakis (Oct 11 2018 at 17:00, on Zulip):

I will open issues to discuss them

nikomatsakis (Oct 11 2018 at 17:01, on Zulip):

we will close the main struct + tuples issue

nikomatsakis (Oct 11 2018 at 17:01, on Zulip):

and we can merge the write-up as is (with "open questions" linking to the issues in question)

nikomatsakis (Oct 11 2018 at 17:01, on Zulip):

then we can (once we seem to have some agreement) close down those issues with an open PR

nikomatsakis (Oct 11 2018 at 17:01, on Zulip):

seem reasonable?

Alan Jeffrey (Oct 11 2018 at 17:01, on Zulip):

@nikomatsakis sounds sensible

nikomatsakis (Oct 11 2018 at 17:01, on Zulip):

the idea being that we should be refining the set of open issues to narrow questions

nikomatsakis (Oct 11 2018 at 17:02, on Zulip):

(to circle back to your point, @Alan Jeffrey, I would say that "field access and array indexing agree" is not a clear notion when you have something like foo.bar -- to what index does bar map? It seems much clearer for foo.1 =)

nikomatsakis (Oct 11 2018 at 17:02, on Zulip):

and hence I don't feel I am abandoning c by saying that the indices assigned to "named fields" are compiler determined

nikomatsakis (Oct 11 2018 at 17:03, on Zulip):

it's sort of like saying "what is 1/0" and answering undefined :P

nikomatsakis (Oct 11 2018 at 17:04, on Zulip):

OK, so, I will make those edits to the struct PR

nikomatsakis (Oct 11 2018 at 17:04, on Zulip):

question then becomes: what is the process for merging :)

nikomatsakis (Oct 11 2018 at 17:05, on Zulip):

I propose we add a "final comment period" label and post an annoucement somewhere

nikomatsakis (Oct 11 2018 at 17:05, on Zulip):

then next meeting we can merge?

nikomatsakis (Oct 11 2018 at 17:05, on Zulip):

(I don't really want to go the whole rfcbot way because I didn't want to have to enumerate who the people are that have to check their names)

nikomatsakis (Oct 11 2018 at 17:05, on Zulip):

(but we could do that, I've just been hoping to keep this process more "open" feeling)

RalfJ (Oct 11 2018 at 17:08, on Zulip):

makes sense to me

nikomatsakis (Oct 11 2018 at 17:09, on Zulip):

OK

nikomatsakis (Oct 11 2018 at 17:09, on Zulip):

so

nikomatsakis (Oct 11 2018 at 17:09, on Zulip):

what about the other topics :)

nikomatsakis (Oct 11 2018 at 17:09, on Zulip):

/me goes to look for the meeting minutes from last time

nikomatsakis (Oct 11 2018 at 17:09, on Zulip):

link to minutes in repo

nikomatsakis (Oct 11 2018 at 17:10, on Zulip):
nikomatsakis (Oct 11 2018 at 17:10, on Zulip):

I think packed/align I kind of described in my PR

nikomatsakis (Oct 11 2018 at 17:10, on Zulip):

the minutes also say

nikomatsakis (Oct 11 2018 at 17:10, on Zulip):

We’re still looking for a consensus/further discussion on:

nikomatsakis (Oct 11 2018 at 17:10, on Zulip):

not sure if that is still true

nikomatsakis (Oct 11 2018 at 17:10, on Zulip):

I have some unread notifications

nikomatsakis (Oct 11 2018 at 17:11, on Zulip):

but I would imagine we can at least do the "write up consensus and then open subissue" thing

Alan Jeffrey (Oct 11 2018 at 17:11, on Zulip):

@nikomatsakis this is for memory layouts, not semantic invariants, yes?

nikomatsakis (Oct 11 2018 at 17:12, on Zulip):

confirm

nikomatsakis (Oct 11 2018 at 17:12, on Zulip):

I am amused that integers turned out to be one of the more controversial issues

Alan Jeffrey (Oct 11 2018 at 17:12, on Zulip):

We are experts at punting issues :)

nikomatsakis (Oct 11 2018 at 17:12, on Zulip):

but this is what C has done to us ;)

nikomatsakis (Oct 11 2018 at 17:12, on Zulip):

"how dare you think you can say how a u32 is represented"

Alan Jeffrey (Oct 11 2018 at 17:12, on Zulip):

@nikomatsakis this will return, at the point that we're discussing invariants.

Alan Jeffrey (Oct 11 2018 at 17:13, on Zulip):

Do scalar types carry any shadow state? Er um, don't know.

nikomatsakis (Oct 11 2018 at 17:14, on Zulip):

/me hides their head in the sand

nikomatsakis (Oct 11 2018 at 17:14, on Zulip):

well, so @Nicole Mazzuca, still think you have time to write up fn types?

nikomatsakis (Oct 11 2018 at 17:15, on Zulip):

else I can do it

Alan Jeffrey (Oct 11 2018 at 17:15, on Zulip):

BTW, I realized talking to @RalfJ that although we appear to be comfortable with the phrase "shadow state", I'm not sure what to call the "real state", that is the bit of the semantics that's really kept in memory at run-time.

nikomatsakis (Oct 11 2018 at 17:15, on Zulip):

lol

nikomatsakis (Oct 11 2018 at 17:15, on Zulip):

what is the opposite of a "shadow"

nikomatsakis (Oct 11 2018 at 17:15, on Zulip):

(reified state?)

Alan Jeffrey (Oct 11 2018 at 17:15, on Zulip):

"sunshine state" :)

nikomatsakis (Oct 11 2018 at 17:16, on Zulip):

we'll just call it the FL

nikomatsakis (Oct 11 2018 at 17:16, on Zulip):

or the FL state

Alan Jeffrey (Oct 11 2018 at 17:16, on Zulip):

:)

nikomatsakis (Oct 11 2018 at 17:16, on Zulip):

/me wonders if Florida is the Sunshine State. Or is that CA?

Alan Jeffrey (Oct 11 2018 at 17:16, on Zulip):

FL IIRC

nikomatsakis (Oct 11 2018 at 17:16, on Zulip):

no way I am naming our state after california

nikomatsakis (Oct 11 2018 at 17:17, on Zulip):

/me has no real idea why they are trashing CA right now, either

nikomatsakis (Oct 11 2018 at 17:17, on Zulip):

mostly jealousy for all the people who hang out in the SF and MV offices and have friends and stuff

nikomatsakis (Oct 11 2018 at 17:17, on Zulip):

ok, anyway, back on topic

Alan Jeffrey (Oct 11 2018 at 17:17, on Zulip):

anyway, this is bikeshedding.

nikomatsakis (Oct 11 2018 at 17:18, on Zulip):

I like the word reify, for the record. It always makes me feel very educated.

nikomatsakis (Oct 11 2018 at 17:18, on Zulip):

so who will tackle unions

nikomatsakis (Oct 11 2018 at 17:18, on Zulip):

I nominate @RalfJ :)

Alan Jeffrey (Oct 11 2018 at 17:18, on Zulip):

It has an air of 1990s object-orientation about it though :/

nikomatsakis (Oct 11 2018 at 17:18, on Zulip):

anyway, maybe the thing to do is to post some links in internals and see if we can get someone excited to do it

nikomatsakis (Oct 11 2018 at 17:18, on Zulip):

I'd like to grow the group here

nikomatsakis (Oct 11 2018 at 17:18, on Zulip):

and we're not all here chatting and making stupid jokes (that's just me, really)

Alan Jeffrey (Oct 11 2018 at 17:19, on Zulip):

me too!

nikomatsakis (Oct 11 2018 at 17:19, on Zulip):

your jokes are funnier

Alan Jeffrey (Oct 11 2018 at 17:20, on Zulip):

Thank you.

Alan Jeffrey (Oct 11 2018 at 17:20, on Zulip):

I think trying to recruit from internal is A Good Idea.

Alan Jeffrey (Oct 11 2018 at 17:21, on Zulip):

We can't just volunteer @RalfJ for everything while his back is turned.

nikomatsakis (Oct 11 2018 at 17:22, on Zulip):

oh I thought they were around

nikomatsakis (Oct 11 2018 at 17:22, on Zulip):

but I suppose you're right:)

nikomatsakis (Oct 11 2018 at 17:23, on Zulip):

so there was this comment on the PR:

Maybe it might be worth spelling out which properties of repr(C) are platform dependent (implementation defined) and cannot be relied on by portable code.

nikomatsakis (Oct 11 2018 at 17:23, on Zulip):

they went on to say:


I meant properties like this:

Assuming the struct is not packed, each field's offset is aligned[^aligned] to
the ABI-mandated alignment for that field's type, possibly creating
unused padding bits.

For example, I can hardcode the address offset of the first field w.r.t. to a value of the struct type to 0, and that will work on all platforms. But doing the same for any other field, even if it is correct for the platform being targeted, is not portable in general.

nikomatsakis (Oct 11 2018 at 17:23, on Zulip):

I kind of have no idea what they are talking about

nikomatsakis (Oct 11 2018 at 17:23, on Zulip):

I guess #[repr(C)] in principle leaves more freedom than that?

nikomatsakis (Oct 11 2018 at 17:23, on Zulip):

this comes back to a meta point

nikomatsakis (Oct 11 2018 at 17:24, on Zulip):

where I would really like it if we can separate "theoretical portability concerns that apply to a few esoteric platforms" from things "everybody relies on all the time" (though it's good to highlight them both I think)

nikomatsakis (Oct 11 2018 at 17:25, on Zulip):

I will adjust the language to

For structs tagged #[repr(C)], the compiler will apply a C-like
layout scheme. See section 6.7.2.1 of the [C17 specification][C17] for
a detailed write-up of what such rules entail. For most platforms,
however, this means the following:

[C17]: http://www.open-std.org/jtc1/sc22/wg14/www/abq/c17_updated_proposed_fdis.pdf

nikomatsakis (Oct 11 2018 at 17:25, on Zulip):

which I think is accurate

nikomatsakis (Oct 11 2018 at 17:34, on Zulip):

ha, actually, writing that up made me realize: clearly we should guarantee that struct Foo; is zero-sized, right..?

Alan Jeffrey (Oct 11 2018 at 17:36, on Zulip):

Presumably. Ditto enum Foo;?

nikomatsakis (Oct 11 2018 at 17:44, on Zulip):

presumably

nikomatsakis (Oct 11 2018 at 17:44, on Zulip):

ok, I've done that work, and I'm going to post now on internals and try to draw some attention

nikomatsakis (Oct 11 2018 at 17:44, on Zulip):

also write up some minutes

Nicole Mazzuca (Oct 11 2018 at 17:47, on Zulip):

@nikomatsakis yeah, I'll do it this weekend

nikomatsakis (Oct 11 2018 at 17:59, on Zulip):

oh, great! :)

nikomatsakis (Oct 11 2018 at 17:59, on Zulip):

I'll put that back to writeup-assigned then

RalfJ (Oct 11 2018 at 20:11, on Zulip):

unions? what do I know about unions?^^

RalfJ (Oct 11 2018 at 20:11, on Zulip):

I guess I can try writing something, so as I said I generally dont care much about these layout questions. In my work I assume they are answered and go from there, which leaves enough stuff open^^ my knowledge of arcane platforms and the C rules for this is pretty limited.

nikomatsakis (Oct 11 2018 at 20:30, on Zulip):

no worries ;) I was poking a stick at random

nikomatsakis (Oct 11 2018 at 20:30, on Zulip):

you keep your eye on stacked borrows!

RalfJ (Oct 11 2018 at 20:47, on Zulip):

whatever you prefer. feel free to assign someone else in that issue. I can do it, but of course that means I have less time for miri ;)

Last update: Nov 20 2019 at 12:35UTC