Stream: project-safe-transmute

Topic: changes to pre-RFC proposal


Ryan Levick (Dec 17 2019 at 12:06, on Zulip):

I took the time to write out what I believe are the limitations of the pre-RFC proposal:

To address these issues I propose the following changes:

This addresses some of the concerns listed above, though not all, but I believe it leaves things open for those concerns to be addressed in the future in a way that makes sense and is a natural progression of this RFC.

These changes do not _guarantee_ that casts are zero cost because the API also needs to handle when there is a cost. Though it should be clearer to be people that the optimizer will handle the Compatible case. In the future with chalk, people can move to using safe_transmute if they are casting types that are Compatible. The same goes for the fact that infallible casts need to be unwraped, but again these can be move to using the safe_transmute API in the future. Thoughts?

gnzlbg (Dec 17 2019 at 12:09, on Zulip):

Do you mean something like Compatible<[u8]> instead of Compatible<u8> ?

Ryan Levick (Dec 17 2019 at 12:11, on Zulip):

Yes, sorry. Will fix the typo

gnzlbg (Dec 17 2019 at 12:14, on Zulip):

Sounds like a good idea. I think the very first Compatible<T> discussion included an incremental approach to implementing it, that kind of aligns with what you have suggested.

gnzlbg (Dec 17 2019 at 12:15, on Zulip):

See this comment:

A first version could just be this, requiring manual implementation of the trait.

A second version could add compiler support for automatically deriving the trait in addition to doing so manually and for performing transitive closure so that if A: Compatible<B> and B: Compatible<C> (whether manually or automatically), then A: Compatible<C>.

A further addition could be to provide a safe way to manually implement the trait for private types in the current crate.

gnzlbg (Dec 17 2019 at 12:17, on Zulip):

I think that the Compatible<T> design direction most appealing advantage is the transitive closure part, so I'd be wary of stabilizing a less powerful version without at least some nightly prototype supporting transitive closure.

gnzlbg (Dec 17 2019 at 12:18, on Zulip):

If when somebody tries to implement this, for whatever reason, we discover that transitive closure is a bad idea, then maybe Compatible<T> isn't worth it and there is a better solution that could be pursued instead.

Ryan Levick (Dec 17 2019 at 12:21, on Zulip):

If when somebody tries to implement this, for whatever reason, we discover that transitive closure is a bad idea, then maybe Compatible<T> isn't worth it and there is a better solution that could be pursued instead.

This is the only thing I worry about. While I agree that Compatible<T> is probably the most appealing approach, it does rely on a bit of faith that the implementation is possible.

Ryan Levick (Dec 17 2019 at 12:24, on Zulip):

But I think we encode that in the alternatives section of the RFC. I personally am not familiar enough with rustc development to have an opinion here

Ryan Levick (Dec 17 2019 at 15:49, on Zulip):

After doing some thinking I think it would be best if we kept Compatible as an unstable Feature and just implemented FromAnyBytes and ToBytes in terms of it. That way we don't have to 1000% sure about compatible before stabilizing the feature.

Lokathor (Dec 17 2019 at 16:32, on Zulip):

I think that all changes being zero cost isn't a realistic goal in the first place.

Daniel Henry-Mantilla (Dec 18 2019 at 11:38, on Zulip):

I think that having changes be zero-cost AND have a cheap type-level compilation overhead is indeed overly optimistic: we may indeed be able to move many checks to const / compile-time through derives, marker traits, and const_asserts, but so doing will impact compile-time, especially the derives. Imho the benefits (safe, quasi-zero-cost type punning) would clearly outweigh such cost, but it is something to be aware of.

(by the way I'd love to contribute to this project :slight_smile:)

gnzlbg (Dec 18 2019 at 15:21, on Zulip):

@Ryan Levick so one thing i'm not sure how well it would work with FromAnyBytes and ToBytes is packed SIMD vectors.

Ryan Levick (Dec 18 2019 at 15:24, on Zulip):

Can you expand on this?

gnzlbg (Dec 18 2019 at 15:24, on Zulip):

I can see the use case of going from &__m128 to &__m128d when working with slices, but often one constructs SIMD vectors on the stack, and just wants to "transmute" __m128 to __m128d.

gnzlbg (Dec 18 2019 at 15:26, on Zulip):

For this later use case, it is kind of weird to have to create a &__m128 to the stack (feels like a reference to a register), then transmute that into a &__m128d, and then load from it, which creates a copy.

gnzlbg (Dec 18 2019 at 15:28, on Zulip):

Does that make sense?

Ryan Levick (Dec 18 2019 at 15:29, on Zulip):

I'm not sure I fully understand. It sounds like you're saying you don't believe the current proposal handles "owned to owned" casting. Is that what you mean?

gnzlbg (Dec 18 2019 at 15:30, on Zulip):

Yes, thanks, that's a much better way to put it.

gnzlbg (Dec 18 2019 at 15:31, on Zulip):

The main use case of the FromBits/IntoBits proposal was SIMD, and for that use case, "owned to owned" feels more idiomatic.

Ryan Levick (Dec 18 2019 at 15:31, on Zulip):

I believe it should. There should be a cast_intomethod in addition to the other casts.

gnzlbg (Dec 18 2019 at 15:32, on Zulip):

Oh, I don't recall seeing cast_into in the v2

gnzlbg (Dec 18 2019 at 15:32, on Zulip):

Is it new in the repo ?

Ryan Levick (Dec 18 2019 at 15:34, on Zulip):

It was removed in the v2 proposal when Josh posted it, and I'm not sure why. I've added back on the RFC draft I'm working on now. I haven't push that work to the repo yet unfortunately. I need to ask Josh if that was on purpose, and if so, why.

gnzlbg (Dec 18 2019 at 15:34, on Zulip):

Ah ok, thanks, i'll check the v1 version.

Ryan Levick (Dec 18 2019 at 15:57, on Zulip):

@gnzlbg one question I never got answered about Compatible is how it interacts with the orphan rule. If I have a type B and some other crates has type A and I want to express that it is safe to read B when you have a A (and perhaps not the other way around), how do you encode this? I don't believe it would be possible to impl Compatible<B> for A because type A lives outside your crate.

gnzlbg (Dec 18 2019 at 16:05, on Zulip):

Yes good question. I think the orphan rules allow impl Compatible<A> for B but i'm not sure if impl Compatible<B> for A is allowed

gnzlbg (Dec 18 2019 at 16:05, on Zulip):

Probably not, since Compatible is outside your crate, and so is A.

Ryan Levick (Dec 18 2019 at 16:06, on Zulip):

Yes the first is allowed because the current crate "owns" that type, but the second is not allowed I believe.
Is this a deal breaker? Seems pretty serious to me

gnzlbg (Dec 18 2019 at 16:06, on Zulip):

I'm not sure if it is possible to relax the orphan ruls to allow that impl, but I recall there were proposals to relax the orphan rules in directions similar to this (e.g. for impl From<B> for A maybe?)

gnzlbg (Dec 18 2019 at 16:07, on Zulip):

It's definetely not good, and i'm not sure if it is a deal breaker.

gnzlbg (Dec 18 2019 at 16:07, on Zulip):

Simple example would be impl Compatible<B> for [u8; size_of::<B>()]

gnzlbg (Dec 18 2019 at 16:08, on Zulip):

since the array type is outside all crates but libcore. It would be very bad if that can't work, since then nothing works. Nice catch.

Ryan Levick (Dec 18 2019 at 16:08, on Zulip):

hehe but then we're back to using bytes as intermediary which is very similar to the "v2" proposal

gnzlbg (Dec 18 2019 at 16:09, on Zulip):

Nono, that array was just an example of some A type

gnzlbg (Dec 18 2019 at 16:09, on Zulip):

notice that [u8; N], [T], etc. are all declared in libcore

gnzlbg (Dec 18 2019 at 16:09, on Zulip):

so that's the crate they belong to

Ryan Levick (Dec 18 2019 at 16:09, on Zulip):

So then we're back to needing to change the orphan rules to get Compatible<T> to work

gnzlbg (Dec 18 2019 at 16:10, on Zulip):

If you have a crate with a type B, and you want to express that [u8; N] can be constructed from B, then you need such an impl to work, and that's somehting everybody has to be able to impl

gnzlbg (Dec 18 2019 at 16:10, on Zulip):

@Ryan Levick are you sure that you need to change the orphan rules for that ?

gnzlbg (Dec 18 2019 at 16:10, on Zulip):

E.g. does impl From<B> for u8` do not work ?

gnzlbg (Dec 18 2019 at 16:11, on Zulip):

IIRC the "trait" isn't From, but ""From<B>"", and since B is defined in your crate, then From<B> is kind of like defined in your crate as well, so you are like implementing a trait from your crate to an external type, and that does not violate orphan rules.

gnzlbg (Dec 18 2019 at 16:11, on Zulip):

Let me check if I can find that

gnzlbg (Dec 18 2019 at 16:12, on Zulip):

Yeah, see this issue: https://github.com/rust-lang/rfcs/issues/1856

gnzlbg (Dec 18 2019 at 16:13, on Zulip):

And this RFC: https://github.com/rust-lang/rfcs/pull/2451

gnzlbg (Dec 18 2019 at 16:13, on Zulip):

More concretely, it allows impl<T> ForeignTrait<LocalType> for ForeignType<T> to be written.

gnzlbg (Dec 18 2019 at 16:14, on Zulip):

For example, impl From<Foo> for Vec<i32> is something any crate can write, even though From is a foreign trait, and Vec is a foreign type.

gnzlbg (Dec 18 2019 at 16:14, on Zulip):

So it appears that this was allowed even before that RFC

gnzlbg (Dec 18 2019 at 16:14, on Zulip):

That RFC extended that to the impl<T> case

Ryan Levick (Dec 18 2019 at 16:14, on Zulip):

Ah ok then maybe we don't have a problem

gnzlbg (Dec 18 2019 at 16:15, on Zulip):

I think that if this is a problem, then it would be kind of a killer

Ryan Levick (Dec 18 2019 at 16:18, on Zulip):

This does bring up the issue that new types cannot help you if a piece of your Compatible type is a foreign type and it does not impl Compatible

Lokathor (Dec 18 2019 at 19:44, on Zulip):

Why would you load data as __m128 and then transmute to __m128d? There's functions to load directly into __m128d already.

Lokathor (Dec 18 2019 at 19:50, on Zulip):

As to orphaning, I know that in beryllium I've written impls like

impl core::From<beryllium::ControllerAxisEvent> for fermium::SDL_ControllerAxisEvent

(not quite real, I'm putting the crate origin in front of each type for clarity)

So in this case I'm using another crate's trait (core) and for another crate's type (fermium), but since the generic of the other crate's trait is a type from my own crate (beryllium), the overall instance is mine to write or not.

Lokathor (Dec 18 2019 at 19:56, on Zulip):

So anyone defining their own type should be able to similarly write impl Compatible<MyType> for [u8; 4] or whatever.

gnzlbg (Dec 19 2019 at 09:52, on Zulip):

Why would you load data as __m128 and then transmute to __m128d?

Some intrinsics accept one type and not the other.

gnzlbg (Dec 19 2019 at 09:53, on Zulip):

So you need to go from one to the other somehow.

gnzlbg (Dec 19 2019 at 09:53, on Zulip):

This does bring up the issue that new types cannot help you if a piece of your Compatible type is a foreign type and it does not impl Compatible

Do you have an example of what you mean ?

Lokathor (Dec 19 2019 at 17:20, on Zulip):

@gnzlbg there's intrinsics for those conversions, and probably they will perform better than whatever cast mechanism we make.

I don't mean to say that they shouldn't be supported by a casting API, but just that a casting API is the fallback case already.

gnzlbg (Dec 19 2019 at 17:26, on Zulip):

Care to elaborate? Those intrinsics just call transmute, the transmutes are safe, so a safe transmute API should be able to perform them if it aims to be complete , which his one of the constraints mentioned.

Last update: Apr 05 2020 at 01:40UTC