Stream: project-safe-transmute

Topic: Slimmed Down Proposal


Ryan Levick (Feb 24 2020 at 17:09, on Zulip):

Hey all. Here's a rough draft of a slimmed down proposal similar to the one I posted last (going through a bytes intermediary). This proposal does not attempt to solve the general transmute problem (i.e., transmuting between any two compatible types), but tries to focus on the use case that I'm coming from which is efficient parsing (i.e., you have some buffer of bytes and you need to safely view types inside of that buffer). This will likely not please everyone as there are a lot of valid use cases this does not address, but I hope it seems minimal enough to be a valuable first step. Please let me know what you think: https://hackmd.io/h7a2sEioRF69wBUhsvEfwQ

Shnatsel (Feb 24 2020 at 18:21, on Zulip):

Typic is a bit of an abuse of the type system after all, and the compilation time concern can be alleviated by turning it into actual Rust code. I feel compiler subsystem that does the same thing as Typic ought to be mentioned as an alternative, for completeness.

Shnatsel (Feb 24 2020 at 18:22, on Zulip):

Also I don't see bytemuck mentioned as prior art. It's more minimal and focused than zerocopy. Was it investigated?
https://docs.rs/bytemuck/1.2.0/bytemuck/

Shnatsel (Feb 24 2020 at 18:23, on Zulip):

But these are all minor nits. Thank you so much for driving this!

Ryan Levick (Feb 24 2020 at 19:59, on Zulip):

Yes, I have looked at bytemuck. I'll add that to the list, and I'll make a note about how something like typic can be built into the compiler directly

Lokathor (Feb 24 2020 at 23:59, on Zulip):

Just parsing is kinda a boring sub-type of the range of casts a person can do :/

Lokathor (Feb 25 2020 at 00:14, on Zulip):

Like to the point where bytemuck-1.0.0 literally didn't special case it at all, and simonsapin threw a PR at me to have a 1.1 with to_bytes and from_bytes and all that jazz added

Ryan Levick (Feb 27 2020 at 11:46, on Zulip):

@Lokathor I disagree. I've added some use cases that just bytes can get you. It covers basically 90% of the cases that people on my team need. In any case I believe that my proposal is a strict subset of the functionality to have bytemuck in the std lib. Do you agree?

Lokathor (Feb 27 2020 at 15:19, on Zulip):

I would agree that it does cover some cases that are of use, yes.

However, my own experience with data casting usage is that the people kept wanting slice casting and owned casting. Once you have that, doing to and from slices of bytes is just a one liner. That's why the "to and from bytes" is the 1.1 afterthought, because you could do it for free in the 1.0 version it just wasn't entirely obvious, and essentially all that the 1.1 version did is add a function to make it more clear how to do that.

Also, I'm still genuinely unsure of how useful having both to_any_bytes and from_any_bytes as separate traits is. It basically means you can to_any_bytes with references to char and bool, but you can't do it with mut references, and you can't from_any_bytes them. It seems like a very small amount of utility for the complexity of having two traits.

Shnatsel (Mar 14 2020 at 19:15, on Zulip):

I've found yet another crate that provides an abstraction to reinterpret bytes as types: https://crates.io/crates/plain

Shnatsel (Mar 14 2020 at 19:15, on Zulip):

This is getting out of hand - now there are six of them!

Yato (Mar 15 2020 at 00:22, on Zulip):

What am I missing???

Jack Wrenn (Mar 15 2020 at 00:23, on Zulip):

zerocopy!

Jack Wrenn (Mar 15 2020 at 00:24, on Zulip):

And this one: https://crates.io/crates/safe-transmute

HeroicKatora (Mar 15 2020 at 15:01, on Zulip):

There's also https://crates.io/crates/lazy-bytes-cast

Shnatsel (Mar 18 2020 at 00:21, on Zulip):

also https://crates.io/crates/scroll

Shnatsel (Mar 18 2020 at 00:21, on Zulip):

so seven

Philip Craig (Mar 18 2020 at 12:53, on Zulip):

scroll copies data, it doesn't transmute. However, I'm sure there's more crates than just those (e.g https://crates.io/crates/pod, https://crates.io/crates/zero, and more that I can't recall). Plus there's probably a few that have private implementations instead (https://github.com/gimli-rs/object/blob/master/src/pod.rs).

Last update: Apr 03 2020 at 18:30UTC