@Nicole Mazzuca one can communicate with C without specifying the representation of slices and trait objects.

I can query the slice pointer and length and pass those to C directly, or put them in a repr(C) struct.

that's of no help of a &[T] is embedded in a larger data structure you'd otherwise want to share with C

you'd have to create a struct on the C side of things anyways, given that C does not have slices or trait objects :/

ah, no, i see what you mean

still, you could memcpy it into a struct that's repr(C), except for the &[T] pieces, which you translate to pointer+length

i mean, you would only be able to use &[T] in C FFI as long as the C code defines it in the exact same way than we do

this is correct

but I'd also argue it's useful to have the repr(C) guarantee for ease of FFI

not necessary, but helpful

(as opposed to saying that the first word is the pointer, which isn't)

You should always be using cbindgen/rust-bindgen for ffi so keeping the repr right shouldn’t be a burden

?

I mean for C APIs written in Rust.

I want to be able to declare

template <typename T>
struct slice {
  T* ptr;
  size_t length;
};

extern "C" {
  void print_int_slice(slice<int const> x);
}

and then in rust

#[no_mangle]
pub extern "C" fn print_int_slice(x: &[i32]) {
  ...
}

or w/e

I think for slices and trait objects we could commit to something. (Though we shouldnt fix vtable layout.) The original statement was about fat pointers in general, there we should be careful.

I don't know why I'd want to be careful here.

didn't you propose custom DST? then we could have fat ptrs with nobody-knows-what as metadata

sure

why's that make a difference?

uh, if we say all fat ptrs are two pointer sizes large, then that restricts your custom DST proposal to ptr-sized metadata?

I didn't say that?

I didnt say you did

right, but why should we be careful about committing to the thing I said

obviously we can't commit to two pointers, that's a non-starter

okay I guess I have some context here that I thought we share but don't... one sec

https://github.com/rust-rfcs/unsafe-code-guidelines/issues/16#issuecomment-439106988

What you proposed sounds absolutely reasonable, that's why I said that for slices (and trait objects) we can commit to something

at least for layout. I will leave ABI questions to the experts.^^

right, but we don't need to worry about "just for slices and trait objects"

I would argue we should define it forall Metadata

I'd rather not define anything for custom DST

just to keep all options open

hmm

I personally disagree, but not super strongly

I dont care strongly either^^

As I already wrote on the issue I think we can't define layout in terms DynamicallySized because it doesn't even exist yet

(I dont care strongly about basically any layout stuff, except where it interacts with validity invariants^^)

Unless of course you propose waiting to define anything until we have it

oh, if you want to say that, sure

I mean right when custom DSTs are implemented, this should be defined.

I want to be able to use custom DSTs in C APIs

yeah i agree

eddyb is reluctant to define {ptr, len} because we actually always pass it as two values in the rust abi, even on x86 which passes structs on the stack

we could just... not do that... in the C calling convention

alternatively we could "just" teach cbindgen that extern fn hello(data: &[T]) is void hello(T* data_ptr, size_t data_len);

Which honestly seems like more idiomatic C code

but is also easier to mess up your usage of

There is also the matter of to what extent we are guaranteeing, accidentally or not, whether {*T, usize} is type-punnable with &[T]

I don't think it's a huge concern?

Basically I think this is the first time we're saying "hey these are the same everywhere" instead of "hey this is specifically the valid FFI bridging, but in actual Rust code don't rely on this"

i think that type-punability is something we should be intentionally guaranteeing, as it's more useful than just passing immediate slices across the FFI boundary -- you can already do that today with a small wrapper that disassembles the slice, but today you can't share a big repr(C) data structure with C if it contains slice pointers

to be clear this is not really about type punning within Rust, but that type punning becomes legal as a side effect of allowing C to manipulate slice pointers natively

Correct me if I'm wrong but we try to keep the Rust ABI close to the C one so native tooling will be happier, right?

Uh

I at least haven't ever heard that articulated as a goal

I might have hallucinated it, I'll check on #compiler discord

@Nicole Mazzuca do you think DSTs that do not contain a pointer are useful ?

I don't understand why so many people want to specify a layout for custom DSTs right now, when nobody has agreed yet on what custom DSTs actually are

say I have a global array, and write a custom DST that has a pointer to this array as an associated const, and only stores a u8 index to index into it

that would be a &T that has the same size as an u8 - if we specify that &T is at least pointer sized, and that for custom DSTs that they are all (ptr, Metadata), then these types cannot be written.

if there's no data pointer component, why do you want to bludgeon it into the shame of a reference to an dynamically sized value? in your example, why not just have a newtype around an u8?

there is a data pointer component, it is just not stored in the DST

out of all the features of pointers, pretty much only the ability to return such a thing from Index::index (and variants) seems relevant, everything else (e.g. different kinds of pointers like raw vs reference vs Rc vs ..., the lifetime parameter some of these have, etc.) seems irrelevant

i don't really know if supporting this use case is worth it, or even useful, hence why I asked

and even the Index::index thing seems dubious because you'd artificially constrain yourself to returning &'a YourCustomThingy, when you could return &'static YourCustomThingy

It really sounds like you want a custom pointer type, not a custom DST

You could write a custom pointer type than then has size == 1, but if you then &* you would get a pointer-sized reference

why is that a problem?

you want to store it somewhere, and would prefer to store a reference type with size == 1.

if that place where you store it is under your control, there are ways to be reasonably generic over different pointer types (e.g. for starters P: Copy+Deref). if that place is not under your control and e.g. hardcodes &T, it could already be doing a ton of things to this pointer that cannot be generalized to things which claim to be pointers but aren't actually an address

report back from discord: I had misunderstood the ABI concerns. eddyb's concerns seem to all revolve around wanting to better understand the custom DST plan to avoid any special-casing of the implementation

I really don't understand why custom DSTs have to be constrained to have a pointer and meta-data members internally, in any particular order

why do they even need a metadata: () member at all, if its just (), can't it be omitted in the type ?

and same for the pointer, given that we don't even have a merged RFC for custom DSTs, why do we have to specify right now that custom DSTs need to always have a pointer, and that it has to be the first member

even if we were to never support DSTs without a memory address, why a pointer and not an usize

or a NonNull<*()>

ok some terminology for starters: if it doesn't have a thin data pointer component and some other components that help interpret that data pointer, it's not a generalization of the concept of DSTs, it's a generalization of the concept of pointers period

now your question might be, why generalize DSTs specifically and not pointers in general

and one reason is, there's a ton of things that can be done generically over all DSTs but not necessarily over arbitrary things that are in some shape or form "like pointers". for example, ThinBox<T: ?Sized>

why do they even need a metadata: () member at all, if its just (), can't it be omitted in the type ?

this is just so we can talk about all pointers without special casing for pointers to sized types

even if we were to never support DSTs without a memory address, why a pointer and not an usize

irrelevant but a raw pointer is slightly more natural since most of the things you'd do with it are pointer things not address things

("irrelevant" because raw pointers and usizes are equivalent for layout and ABI so it's just a matter of where you cast, if at all)

if I can write my own DSTs, I can provide APIs that expose them and completely hide where they come from

what wouuld ThinBox<YourU8IndexIntoGlobalArray> or Rc<YourU8IndexIntoGlobalArray> even mean?

that's the question

for custom DSTs as defined in the various DSTs it's completely clear what Rc, ThinBox, etc. mean. for your example i don't even think it can be made to make sense

@rkruppe those imply some sort of ownership, but I don't have to expose a way to create T, just expose &T

that's why I say you want a new kind of pointer, instead of a new kind of type that can be referenced by any pointer

ah I see

I thought custom DSTs would also allow implementing &T

or that it would be an aim of that at some point

first I heard of that

no, just as the existing DSTs are str and [T] and dyn Bounds and so on, not &str, &[T], &dyn Bounds

so yeah, i'd like a way to create custom pointers then

thinking about DSTs, I guess it also wouldn't be possible to have DSTs where the meta-data is encoded in the pointer then ?

as in, the metadata has to be a distinct field of the DST

this seems very off-topic

yeah best take this question to a specific proposal about custom DSTs

so specifying custom DSTs as (*T, Metadata) does not prevent this?