Stream: t-lang/wg-unsafe-code-guidelines

Topic: how much determinism


nikomatsakis (Oct 23 2018 at 16:28, on Zulip):

@rkruppe I'm just getting to comments on #31, and I'd like to know what you think of this?

rkruppe (Oct 23 2018 at 16:38, on Zulip):

My general position is that it's cool if it's some deterministic function, but if the inputs to that function are as expansive as they have to be to accommodate the current (lack of) consensus, that determinism isn't useful in practice. For example, Cargo passes very comprehensive hashes of the crate metadata and absolute paths, so it seems basically impossible to get the exact same command line twice unless you go to even more effort than typical reproducible build setups do

nikomatsakis (Oct 23 2018 at 16:42, on Zulip):

OK. I don't disagree with that, I agree the set of inputs is pretty expansive. The phrasing I gave is basically "all compiler input"

nikomatsakis (Oct 23 2018 at 16:42, on Zulip):

I suppose I said sort of two contradictory things

nikomatsakis (Oct 23 2018 at 16:43, on Zulip):

maybe I'll just cut the opening paragraph more-or-less and run with this follow-on paragraph:

As of this writing, we have not reached a full consensus on what
limitations should exist on possible field struct layouts. Therefore,
the default layout of structs is considered undefined and subject to
change between individual compilations. This implies that (among other
things) two structs with the same field types may not be laid out in
the same way (for example, the hypothetical struct representing tuples
may be laid out differently from user-declared structs).

nikomatsakis (Oct 23 2018 at 16:43, on Zulip):

which is basically what you were advocating @rkruppe

rkruppe (Oct 23 2018 at 16:44, on Zulip):

I don't mind giving specific (non-exhaustive) examples of things that might influence layout

rkruppe (Oct 23 2018 at 16:45, on Zulip):

But either way it should be clear that something like abomonation isn't covered by it

nikomatsakis (Oct 23 2018 at 16:49, on Zulip):

what do you think of this:

The default layout of structs is not specified. As of this
writing, we have not reached a full consensus on what limitations
should exist on possible field struct layouts, so effectively one must
assume that the compiler can select any layout it likes on each
compilation. This implies that (among other things) two structs with
the same field types may not be laid out in the same way (for example,
the hypothetical struct representing tuples may be laid out
differently from user-declared structs).

Known things that can influence layout (non-exhaustive):

rkruppe (Oct 23 2018 at 16:52, on Zulip):

One thing I'd call out in the first paragraph is choosing different layout for the same type across different compilations. It's heavily implied by everything else but I think it's worth calling out in those words because it's just as subtle and important as the case of two distinct but structually-identical types in one compilation.

rkruppe (Oct 23 2018 at 16:56, on Zulip):

Come to think of it, this seems a bit out of place now. It's an important observation as soon as we try to guarantee that two structs with (in some precise sense) equivalent fields are laid out the same, but since the current text doesn't do anything like that...

nikomatsakis (Oct 23 2018 at 16:56, on Zulip):

yes, but

nikomatsakis (Oct 23 2018 at 16:57, on Zulip):

I opted to include it because

nikomatsakis (Oct 23 2018 at 16:57, on Zulip):

it just seemed like something worth knowing

nikomatsakis (Oct 23 2018 at 16:57, on Zulip):

I guess because in practice if you are trying to understand what the compiler really does

nikomatsakis (Oct 23 2018 at 16:57, on Zulip):

you would like to know it

nikomatsakis (Oct 23 2018 at 16:57, on Zulip):

perhaps I'll move it down to the "Compiler's current behavior" section

rkruppe (Oct 23 2018 at 16:57, on Zulip):

yes that seems better

rkruppe (Oct 23 2018 at 16:58, on Zulip):

understanding the current behavior tends to be an obstacle to knowing what one can rely on :)

nikomatsakis (Oct 23 2018 at 17:00, on Zulip):

I also added

A note on determinism. The definition above does not guarantee
determinism between executions of the compiler -- two executions may
select different layouts. Naturally, in practice, the compiler aims to
produce deterministic output. However, it is difficult to produce a
comprehensive summary of the various factors that may affect the
layout of structs, and so for the time being we have opted for a
conservative definition.

nikomatsakis (Oct 23 2018 at 17:02, on Zulip):

did you see this? I wasn't aware that #[repr(C)] struct Foo { } had zero-size...

nikomatsakis (Oct 23 2018 at 17:02, on Zulip):

tbh I hadn't really thought about it :P

rkruppe (Oct 23 2018 at 17:04, on Zulip):

yeah, it's annoying for hand-written bindings but otherwise probably not an issue. tbh even if one messes that up it seems hard to write a program that will actually go wrong because of that difference (I expect that accesses to struct Foo {} are nops in C too)

rkruppe (Oct 23 2018 at 17:08, on Zulip):

yeah by-value passing and load/stores of empty structs are nops in C https://godbolt.org/z/Y02oVi

rkruppe (Oct 23 2018 at 17:20, on Zulip):

in the x86 and x86_64 SysV ABIs, I should say

nikomatsakis (Oct 23 2018 at 17:24, on Zulip):

I'd be more concerned about the layout of things that embed empty structs being different in subtle ways

rkruppe (Oct 23 2018 at 17:41, on Zulip):

oh good point

RalfJ (Oct 23 2018 at 21:54, on Zulip):

But either way it should be clear that something like abomonation isn't covered by it

What is abomonation doing that you are referring to here?

rkruppe (Oct 23 2018 at 21:59, on Zulip):

I am referring to assuming the same program compiled twice by the same rustc version will have the same layout for everything. That isn't inherent in abomonation itself but using it generally requires that because e.g. you're communicating between a cdylib and a binary that include the same source files

RalfJ (Oct 23 2018 at 22:00, on Zulip):

ah you mean essentially dumping the raw memory representation into a file, and reading it back later with just some fixes for ptrs?

rkruppe (Oct 23 2018 at 22:01, on Zulip):

that's basically what abomonation does, yes

RalfJ (Oct 23 2018 at 22:11, on Zulip):

well, for me the core thing was always that it also plays crazy tricks with lifetimes and references, that's why I was confused at first^^

rkruppe (Oct 23 2018 at 22:13, on Zulip):

oh right, there's plenty there to be proven sound there, I was only thinking about layout stuff

Nicole Mazzuca (Oct 24 2018 at 22:35, on Zulip):

@rkruppe to be clear, that is C++

Nicole Mazzuca (Oct 24 2018 at 22:35, on Zulip):

C does not have empty structs

rkruppe (Oct 24 2018 at 22:38, on Zulip):

My godbolt link was running clang in c++ mode, yes. clang's and gcc's c mode also accept empty structs though, so we'll want interop with that no matter what the c standard says

Nicole Mazzuca (Oct 24 2018 at 22:38, on Zulip):

clang and gcc's C mode have empty structs with size 0.

rkruppe (Oct 24 2018 at 22:39, on Zulip):

huh, indeed

rkruppe (Oct 24 2018 at 22:39, on Zulip):

thanks for pointing that out

Nicole Mazzuca (Oct 24 2018 at 22:40, on Zulip):

https://godbolt.org/z/AS2gdC

Nicole Mazzuca (Oct 24 2018 at 22:44, on Zulip):

no worries, it's a common bit of confusion :)

Nicole Mazzuca (Oct 24 2018 at 22:44, on Zulip):

especially among C++ programmers who think it's shitty that Bar takes up more than four bytes of space, cough cough :<

Last update: Nov 20 2019 at 11:25UTC