Stream: t-lang/wg-unsafe-code-guidelines

Topic: layout arrays


gnzlbg (Feb 22 2019 at 09:18, on Zulip):

@RalfJ the example you posted already works like you expect with stride > size

gnzlbg (Feb 22 2019 at 09:19, on Zulip):

I'm still not sure which extension you mean with respect to how arrays already work in Rust

gnzlbg (Feb 22 2019 at 09:20, on Zulip):

Like currently stride is always >= size.

RalfJ (Feb 22 2019 at 09:22, on Zulip):

did you check your claim?

RalfJ (Feb 22 2019 at 09:22, on Zulip):

because it doesn't seem correct: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f8263bc23052159dcc30470216064c17

RalfJ (Feb 22 2019 at 09:23, on Zulip):

A currently has size 4

RalfJ (Feb 22 2019 at 09:23, on Zulip):

AFAIK currently we always have stride == size. but maybe with repr(packed) there is a way around that? I'd be curious for an example

gnzlbg (Feb 22 2019 at 09:23, on Zulip):

I checked, it does have size 4 indeed

RalfJ (Feb 22 2019 at 09:24, on Zulip):

I specifically asked for stride > size

gnzlbg (Feb 22 2019 at 09:24, on Zulip):

but what has that to do with arrays? If you want A to have size 3, you have to packed it

RalfJ (Feb 22 2019 at 09:24, on Zulip):

but then it will still have stride == size

RalfJ (Feb 22 2019 at 09:24, on Zulip):

as in, then both are 3

gnzlbg (Feb 22 2019 at 09:24, on Zulip):

So do you have an example in which stride > size makes sense ?

RalfJ (Feb 22 2019 at 09:25, on Zulip):

yes, the example I gave in the post

RalfJ (Feb 22 2019 at 09:25, on Zulip):

there is no reason that A should have size 4

gnzlbg (Feb 22 2019 at 09:25, on Zulip):

In that example A has size 4,

RalfJ (Feb 22 2019 at 09:25, on Zulip):

adding a byte of padding at the end, what good is that supposed to make?

RalfJ (Feb 22 2019 at 09:25, on Zulip):

it'd make much more sense to make A have size 3, such that (A, u8) can have size 4

RalfJ (Feb 22 2019 at 09:25, on Zulip):

the only reason we add padding at the end is to make stride == size

gnzlbg (Feb 22 2019 at 09:26, on Zulip):

when using arrays you mean ?

gnzlbg (Feb 22 2019 at 09:26, on Zulip):

as in, the only reason A has size 4 is because arrays don't support stride > size ?

RalfJ (Feb 22 2019 at 09:26, on Zulip):

well, rustc doesn't. or maybe LLVM, I don't know

RalfJ (Feb 22 2019 at 09:26, on Zulip):

but stride is an array-notion

RalfJ (Feb 22 2019 at 09:26, on Zulip):

so I guess yes?

RalfJ (Feb 22 2019 at 09:27, on Zulip):

I am confused by your question

gnzlbg (Feb 22 2019 at 09:27, on Zulip):

sure, but in these examples stride == size because A has size 4

RalfJ (Feb 22 2019 at 09:27, on Zulip):

yes and I am saying that's silly, and we should keep open the option to do better

gnzlbg (Feb 22 2019 at 09:27, on Zulip):

so your question is to me that we don't want to force repr(rust) types to be larger than they need to be just because they could be put in arrays and arrays don't support stride > size

RalfJ (Feb 22 2019 at 09:27, on Zulip):

yes

gnzlbg (Feb 22 2019 at 09:28, on Zulip):

gotcha, i was confused because i thought that you somehow wanted, for a given size, choose a stride > size

RalfJ (Feb 22 2019 at 09:28, on Zulip):

well I kind of do, but those size-alignment pairs dont exist currently :P

gnzlbg (Feb 22 2019 at 09:28, on Zulip):

well that's what you want, i guess i was confused because I assumed that we wanted to keep padding at the end of types for other reasons

RalfJ (Feb 22 2019 at 09:28, on Zulip):

basically, stride is always size rounded up to the next multiple of the alignment

RalfJ (Feb 22 2019 at 09:29, on Zulip):

I am saying we should NOT hard-code that that will always be the same as size

gnzlbg (Feb 22 2019 at 09:29, on Zulip):

so... i think the main issue is FFI. we currently allow arrays there, they coerce to raw pointers, and C and C++ arrays don't do that

RalfJ (Feb 22 2019 at 09:29, on Zulip):

I want to remove padding at the end of types

RalfJ (Feb 22 2019 at 09:29, on Zulip):

well this can only happen if you use a repr(Rust) type in FFI

RalfJ (Feb 22 2019 at 09:29, on Zulip):

repr(C) types will always have size be a multiple of the alignment

gnzlbg (Feb 22 2019 at 09:31, on Zulip):

I guess we could require arrays used in C FFI to be repr(C)

gnzlbg (Feb 22 2019 at 09:31, on Zulip):

but right now, using a non-repr(C) array with a non-repr(C) element type in C FFI does what C does

RalfJ (Feb 22 2019 at 09:32, on Zulip):

uh, we dont give any guarantees when you use non-repr(C) types in FFI, I thought?

RalfJ (Feb 22 2019 at 09:32, on Zulip):

like, to do this you need to have a repr(Rust) struct, and a matching struct in C. which is not possible, already right now.

gnzlbg (Feb 22 2019 at 09:32, on Zulip):

no, and we do warn about this

RalfJ (Feb 22 2019 at 09:33, on Zulip):

we reorder fields, and we may change the order between rust releases

gnzlbg (Feb 22 2019 at 09:33, on Zulip):

(there is an improper ctypes warning)

RalfJ (Feb 22 2019 at 09:33, on Zulip):

so, we already break such code

gnzlbg (Feb 22 2019 at 09:33, on Zulip):

we can't use #[repr(C)] on arrays yet, but that could be fixed

gnzlbg (Feb 22 2019 at 09:34, on Zulip):

we should definetely document repr(Rust) and repr(C) arrays differently, and leave the door open to doing something better for repr(rust)

RalfJ (Feb 22 2019 at 09:34, on Zulip):

well, we wouldnt really need that. if the array element type is repr(C), the array will be compatible

RalfJ (Feb 22 2019 at 09:34, on Zulip):

we should definetely document repr(Rust) and repr(C) arrays differently, and leave the door open to doing something better for repr(rust)

no need

RalfJ (Feb 22 2019 at 09:35, on Zulip):

if we define arrays the way I suggested, they'll do the right thing regardless

gnzlbg (Feb 22 2019 at 09:35, on Zulip):

ah because repr(C) types already have padding at the end

gnzlbg (Feb 22 2019 at 09:36, on Zulip):

because in C they have to, because of arrays

gnzlbg (Feb 22 2019 at 09:36, on Zulip):

i think it is still worth calling out that if the element type of the array is repr(C), then stride == size

RalfJ (Feb 22 2019 at 09:39, on Zulip):

i think it is still worth calling out that if the element type of the array is repr(C), then stride == size

fair

gnzlbg (Mar 06 2019 at 16:21, on Zulip):

@RalfJ the definition of stride is in the first sentence

RalfJ (Mar 06 2019 at 16:29, on Zulip):

yes, I know? me suggesting a structure does not mean that nothing was at the right (in my view) place -- just a lot of it wasn't. The rest of this text is somewhat confusing (didn't get to read the latest version yet, I'll come back to you)

gnzlbg (Mar 06 2019 at 16:42, on Zulip):

@RalfJ i've reworded it trying to address both yours and @rkruppe 's comment

gnzlbg (Mar 06 2019 at 16:43, on Zulip):

I needed to remove the mention that arrays elements are laid out contiguously in memory
that only holds for stride == size

gnzlbg (Mar 06 2019 at 16:43, on Zulip):

but i clarified that below

gnzlbg (Mar 06 2019 at 17:45, on Zulip):

@rkruppe for me personally the biggest downside of stride > size is that it makes arrays non-contiguous

gnzlbg (Mar 06 2019 at 17:46, on Zulip):

but it is probably better to raise those concerns in the issues discussing such proposals

rkruppe (Mar 06 2019 at 17:46, on Zulip):

you need an odd definition of "contiguity" to make that statement

rkruppe (Mar 06 2019 at 17:46, on Zulip):

but yeah, UCG is not the place to decide about those proposals

gnzlbg (Mar 06 2019 at 17:46, on Zulip):

For me contiguity is that you can iterate over the array using ptr.add(elem_size).

gnzlbg (Mar 06 2019 at 17:47, on Zulip):

But I see how ptr.add(arr_stride) is very close.

rkruppe (Mar 06 2019 at 17:48, on Zulip):

If stride was independent from type size, as it is e.g. in matrix views, clearly it wouldn't be contiguous. But here the array is "as contiguous as it can be", and the stride is basically the element size anyway.

gnzlbg (Mar 06 2019 at 17:50, on Zulip):

right, i don;t know how much code changing this assumption would break - anyways have to go bake a cake

RalfJ (Mar 06 2019 at 18:05, on Zulip):

the point of mentioning strie > size was precisely because UCG is not the place to exclude such proposals

rkruppe (Mar 06 2019 at 18:18, on Zulip):

Sure. I did a poor job at this initially but I hope by now I am clear that I'm not arguing for excluding this possibility now, just against putting it front and center instead of mentioning it at the end as a possibility of resolving the unspecified aspects.

RalfJ (Mar 06 2019 at 18:18, on Zulip):

fair

gnzlbg (Mar 07 2019 at 08:30, on Zulip):

So @RalfJ @rkruppe i'll make the SIMD Vector thing clearer, and I'll explain what packed means.

@RalfJ While Cray vectors are old, they are new again. IIRC actually @rkruppe was working on the LLVM implementation. He had a talk about it last year at EuroLLVM.

RalfJ (Mar 07 2019 at 08:31, on Zulip):

is there a good document that explains this that you can refer to?

RalfJ (Mar 07 2019 at 08:31, on Zulip):

this repo seems like the wrong place for such an explanation

gnzlbg (Mar 07 2019 at 08:44, on Zulip):

i think the portable packed SIMD RFC explained that, let me check

gnzlbg (Mar 07 2019 at 08:45, on Zulip):

https://github.com/gnzlbg/rfcs/blob/ppv/text/0000-ppv.md#guide-level-explanation
https://github.com/gnzlbg/rfcs/blob/ppv/text/0000-ppv.md#interaction-with-cray-vectors

gnzlbg (Mar 07 2019 at 08:46, on Zulip):

I think I can add the definition of the guide level explanation of that RFC to the packed-simd-vector.md document

gnzlbg (Mar 07 2019 at 08:46, on Zulip):

as a note, it is short enough

gnzlbg (Mar 07 2019 at 08:46, on Zulip):

maybe without mentioning cray vectors

gnzlbg (Mar 07 2019 at 08:47, on Zulip):

packed vectors -> arrays, cray vectors -> slices

gnzlbg (Mar 07 2019 at 08:57, on Zulip):

@RalfJ I've added a footnote to the vector document explaining what packed means, and linking to the RFC

rkruppe (Mar 07 2019 at 10:33, on Zulip):

To give credit where credit is due, the architecture I work on (RISC-V) is just one of two recent architectures that try to bring this programming model back. Arm beat RISC-V to the punch with SVE, and I'm building on stuff Arm engineers have done previously for SVE.

RalfJ (Mar 07 2019 at 16:20, on Zulip):

@gnzlbg that was in the wrong topic^^

Last update: Nov 20 2019 at 13:25UTC