Stream: t-lang/wg-unsafe-code-guidelines

Topic: Can validity invariants be violated if we fix them...


Lokathor (Dec 08 2019 at 09:00, on Zulip):

validity is LLVM-level invariant

Lokathor (Dec 08 2019 at 09:00, on Zulip):

safety is Rust API-level invariant

rkruppe (Dec 08 2019 at 09:05, on Zulip):

Memory is not typed in Rust. A region of memory has to fulfill the validity invariant for T when (vaguely speaking) something looks at that memory as a T, not at any other times. So your code is fine AFAICT.

RalfJ (Dec 11 2019 at 17:56, on Zulip):

I'm wondering if the constraint it has is that it must leave the usize as non-zero when it returns, or if the usize must never be zeroed.

Very good questions! We have an open issue about this: https://github.com/rust-lang/unsafe-code-guidelines/issues/84

RalfJ (Dec 11 2019 at 17:56, on Zulip):

@Thom Chiovoloni could you add your usecase in that issue? I think so far we didnt really have one.

RalfJ (Dec 11 2019 at 17:57, on Zulip):

We could in principle decide either way, though I see no good way to actually precisely specify things in a way that your code is not legal.

RalfJ (Dec 11 2019 at 17:57, on Zulip):

(of course you need to be careful with panics, but that is a separate concern)

Thom Chiovoloni (Dec 12 2019 at 04:50, on Zulip):

@RalfJ sorry, I missed this. I'll add a comment to that issue now. Also, good catch, there was a case where a panic was in-principal possible which I hadn't considered. the code looked like:

// secretively read from our first field to see if that was the final byte
// needed to zero it out..
if unlikely!(*(self as *const _ as *const usize) == 0) {
    // oh no! overwrite it with something else quickly then move the string
    // onto heap allocated storage.
    *end = 1;
    self.handle_degenerate_push(c);
    return
}

but the unlikely macro is secretive, in that if cfg(count_branch_hints) is enabled, it records how many times it's taken/missed. that code could have probably paniced! (that said, that's a internal thing i'm doing for profiling, which end users aren't intended to enable).

i think it's probably possible to manage to use some bitwise int trickery to fold the equivalent to || c == '\0' into a slow-path check earlier in the function, which would mean i could be certain this code never has to deal with writing a null, and both eliminate this test and the 'temporary invalid state'. (that said, i still think it's better in general if, as others put it, rust doesn't have typed memory -- in general that makes this kind of unsafe much easier to reason about than in c++ -- the lack of strict-aliasing is <3)

Thom Chiovoloni (Dec 12 2019 at 06:16, on Zulip):

posted, but i'm sorry it's so long (i tried to shorten it ended up longer, and with some parts moved into footnotes...).

it should give an idea of what i want to do though. even if i fix this issue, essentially i'd like to be able to interpret any memory as any other type of memory, given the constraints of:

The last bit is the relevant one here, and it's error prone, but in hot code my experience has often been that it can be much faster to 'do then check' vs 'check than do' -- the 2nd case has to check both if the thing we're considering doing could cause a problem (for any state -- e.g. c == '\0'), and that we're currently in a state where it would cause a problem for us (e.g. len == 7 && first 7 bytes were also NUL -- for a 64 bit system)

And so i'm glad to hear you don't think there's a reason to disallow it. (now, there are plenty of things i'd like to do that are even dodgier, but that are disallowed, but i guess i can wait until inline assembly stabilizes to, say, read a little past the end of objects for cases where i know a page fault is impossible... :p)

gnzlbg (Dec 12 2019 at 12:36, on Zulip):

@Thom Chiovoloni you assume that you can't use an union or an enum

gnzlbg (Dec 12 2019 at 12:36, on Zulip):

go for a struct instead

Thom Chiovoloni (Dec 12 2019 at 12:36, on Zulip):

that's what i did

gnzlbg (Dec 12 2019 at 12:36, on Zulip):

run into validity issues, and your post in the ucgs asks whether these are issues or not

gnzlbg (Dec 12 2019 at 12:37, on Zulip):

AFAICT you can just use an enum as long as you give it the right niches

gnzlbg (Dec 12 2019 at 12:37, on Zulip):

so none of this matters

Thom Chiovoloni (Dec 12 2019 at 12:37, on Zulip):

i actually need to read from inactive variants.

gnzlbg (Dec 12 2019 at 12:37, on Zulip):

it is hard to tell, since you haven't actually explained what layout you'd like to have, and why you can't have it with an enum or union

gnzlbg (Dec 12 2019 at 12:37, on Zulip):

all variants of an union are always active

gnzlbg (Dec 12 2019 at 12:37, on Zulip):

there is no concept of "inactive" variants for an union

Thom Chiovoloni (Dec 12 2019 at 12:38, on Zulip):

right, a union would be possible but i'd lose NonNull niche which i deliberately want to keep

gnzlbg (Dec 12 2019 at 12:38, on Zulip):

I don't understand why that is the case

gnzlbg (Dec 12 2019 at 12:38, on Zulip):

you can just give the union a niche

gnzlbg (Dec 12 2019 at 12:38, on Zulip):

and then you get it back

Thom Chiovoloni (Dec 12 2019 at 12:38, on Zulip):

so that Option<String> is the same as String. i talked more about what i'm doing and the layout in the issue i was asked to post on

gnzlbg (Dec 12 2019 at 12:38, on Zulip):

this is exactly what i'm telling you

Thom Chiovoloni (Dec 12 2019 at 12:38, on Zulip):

hmm

gnzlbg (Dec 12 2019 at 12:39, on Zulip):

you are assuming that this is impossible, but maybe it isn't

Thom Chiovoloni (Dec 12 2019 at 12:39, on Zulip):

i can give a union a niche?

gnzlbg (Dec 12 2019 at 12:39, on Zulip):

please, explain what it is that you'd actually want

gnzlbg (Dec 12 2019 at 12:39, on Zulip):

i can give a union a niche?

Should be as simple as #[rustc_layout_scalar_valid_range_start(1)]

gnzlbg (Dec 12 2019 at 12:40, on Zulip):

and if it isn't, we can just allow it

Thom Chiovoloni (Dec 12 2019 at 12:40, on Zulip):

i want union { struct { NonNull<u8>, usize, usize }; [u8; size_of::<usize>() * 3] } while still keeping the NonNull's niche

Thom Chiovoloni (Dec 12 2019 at 12:40, on Zulip):

oh i'm not using nightly

gnzlbg (Dec 12 2019 at 12:40, on Zulip):

that does not make sense

gnzlbg (Dec 12 2019 at 12:41, on Zulip):

since the second variant first size_of::<usize>() u8 can be null

gnzlbg (Dec 12 2019 at 12:41, on Zulip):

(according to those types)

Thom Chiovoloni (Dec 12 2019 at 12:41, on Zulip):

i'm willing to ensure that the nonnull's invariant stays respected

Thom Chiovoloni (Dec 12 2019 at 12:42, on Zulip):

https://github.com/rust-lang/unsafe-code-guidelines/issues/13#issuecomment-419140218 "My personal stanza on this is that if someone really wants their union layout optimized, we should provide attributes to let them do that" that's really what i would want

gnzlbg (Dec 12 2019 at 12:42, on Zulip):
union U {
    heap: (NonNull<u8>, usize, usize),
    sso: (NonZeroUsize, [u8; 2 * size_of::<usize>()]),
}
Thom Chiovoloni (Dec 12 2019 at 12:42, on Zulip):

barring that, i can just maintain validity invariant explicitly

Thom Chiovoloni (Dec 12 2019 at 12:43, on Zulip):

hmmm

Thom Chiovoloni (Dec 12 2019 at 12:43, on Zulip):

i guess that would work

gnzlbg (Dec 12 2019 at 12:43, on Zulip):

That union _might_ have a niche for the first usize

Thom Chiovoloni (Dec 12 2019 at 12:43, on Zulip):

assuming it does it would work but i'm not sure it buys me anything

Thom Chiovoloni (Dec 12 2019 at 12:44, on Zulip):

over just treating the struct like an array when it's sso

gnzlbg (Dec 12 2019 at 12:44, on Zulip):

I'm just giving you what you ask for, you wanted an union with a niche at null, that is it

gnzlbg (Dec 12 2019 at 12:44, on Zulip):

I'm not sure why you can't use an enum though

gnzlbg (Dec 12 2019 at 12:44, on Zulip):

I suppose you have a discriminant somewhere to tell between sso and heap

gnzlbg (Dec 12 2019 at 12:45, on Zulip):

So you could just use an enum, and give that enum a niche

Thom Chiovoloni (Dec 12 2019 at 12:45, on Zulip):

my layout a little more suble than that actually. i could use an enum if i had exact control over how descriminants are chosen i guess

Thom Chiovoloni (Dec 12 2019 at 12:45, on Zulip):

err, where the bits are placed

gnzlbg (Dec 12 2019 at 12:45, on Zulip):
enum E {
    Heap(NonNull<u8>, usize, usize),
    Sso(NonZeroUsize, [usize; 2]),
}
gnzlbg (Dec 12 2019 at 12:46, on Zulip):

That enum will be 32 bytes on 64-bits, because the discriminant must be stored extra

Thom Chiovoloni (Dec 12 2019 at 12:46, on Zulip):

yeah that's a loss.

gnzlbg (Dec 12 2019 at 12:46, on Zulip):

but you'd need to tell Rust that there is a way to store it inside the enum

Thom Chiovoloni (Dec 12 2019 at 12:46, on Zulip):

and i mean, it's not that this can't be made to work in an enum without changing my layout

Thom Chiovoloni (Dec 12 2019 at 12:46, on Zulip):

it could

Thom Chiovoloni (Dec 12 2019 at 12:46, on Zulip):

i'd just lose a couple optimizations

gnzlbg (Dec 12 2019 at 12:47, on Zulip):

as mentioned, if you are able to fit this in an union, it will be because you are storing the discriminant inline

gnzlbg (Dec 12 2019 at 12:47, on Zulip):

you can do that for an enum to

Thom Chiovoloni (Dec 12 2019 at 12:47, on Zulip):

it is

Thom Chiovoloni (Dec 12 2019 at 12:47, on Zulip):

well

gnzlbg (Dec 12 2019 at 12:47, on Zulip):

but you haven't explained how you would do it for the union

gnzlbg (Dec 12 2019 at 12:48, on Zulip):

there is probably a bit somewhere that's 1 for SSO and 0 for heap or viceversa

Thom Chiovoloni (Dec 12 2019 at 12:48, on Zulip):

it is

gnzlbg (Dec 12 2019 at 12:48, on Zulip):

So just have two types, one with a niche for 1, and one with a niche for 0

Thom Chiovoloni (Dec 12 2019 at 12:48, on Zulip):

there are 2 bits, individually they have specific meanings (heap vs notheap, mutable vs const). small is notheap + mutable

gnzlbg (Dec 12 2019 at 12:48, on Zulip):

and rustc should just make the enum small automatically

Thom Chiovoloni (Dec 12 2019 at 12:49, on Zulip):

no, i have four variants, but my descriminant layout and being consitent in how the others are represented means functions only need to check 1 case generally (small vs non-small) for fast paths

gnzlbg (Dec 12 2019 at 12:50, on Zulip):
enum E {
    Heap(NonNull<u8>, usize, [u8; size_of::<usize>() - 1], TagHeap),
    Sso(NonZeroUsize, [u8; size_of::<usize>() * 2 - 1], TagSso),
}
gnzlbg (Dec 12 2019 at 12:50, on Zulip):

And then your TagHeap and TagSso use the #[rustc_layout_scalar_valid_range_start(N)] and #[rustc_layout_scalar_valid_range_end(N)] attributes to encode their ranges of valid values

Thom Chiovoloni (Dec 12 2019 at 12:51, on Zulip):

right, the tag bits are in the same byte as the small-string length, and they're the most/least (depends on endian) singificant bits of the heap/arc/static string capacity

gnzlbg (Dec 12 2019 at 12:51, on Zulip):

e.g.

#[repr(transparent)]
#[rustc_layout_scalar_valid_range_start(2)]
#[rustc_layout_scalar_valid_range_end(255)]
struct TagSso(u8);
Thom Chiovoloni (Dec 12 2019 at 12:51, on Zulip):

lol i mean, that seems very stable

gnzlbg (Dec 12 2019 at 12:52, on Zulip):

Once you get something like that working on nightly, with an enum, then the only question would be how to make it work on stable Rust

Thom Chiovoloni (Dec 12 2019 at 12:52, on Zulip):

why?

gnzlbg (Dec 12 2019 at 12:52, on Zulip):

A lot of people want to be able to define their own NonZeroXY types, so we might just stabilize a way to do that

gnzlbg (Dec 12 2019 at 12:53, on Zulip):

Like, NonMaxU8 where the niche is not at zero, but at 255, for example, and well, everything in between

Thom Chiovoloni (Dec 12 2019 at 12:53, on Zulip):

yeah but why are you saying the way i'm doing it now is wrong?

Thom Chiovoloni (Dec 12 2019 at 12:53, on Zulip):

fwiw this is just a proof of concept and about 50% exists to prove a bet with someone who claimed rust couldn't do this sort of thing as well as c++.

gnzlbg (Dec 12 2019 at 12:53, on Zulip):

because as you say, you actually want an enum

Thom Chiovoloni (Dec 12 2019 at 12:53, on Zulip):

well

Thom Chiovoloni (Dec 12 2019 at 12:54, on Zulip):

i really want precise control

Thom Chiovoloni (Dec 12 2019 at 12:54, on Zulip):

conceptually it's an enum

gnzlbg (Dec 12 2019 at 12:54, on Zulip):

sure, so why shouldn't you be able to tell the enum how the discriminant is stored ?

gnzlbg (Dec 12 2019 at 12:55, on Zulip):

why should you have to drop down to a struct or an union, deal with a lot more unsafe code, validity invariants, etc. ?

Thom Chiovoloni (Dec 12 2019 at 12:55, on Zulip):

i feel like by defining the range of the enum i'm not actually telling it how to store things though. it still is kind of up to it whetehr or not it puts it in one place or another

Thom Chiovoloni (Dec 12 2019 at 12:55, on Zulip):

i'd feel uncomforatble having unsafe code rely on anything less than a hard guarantee of a specific layout opt. from rustc

gnzlbg (Dec 12 2019 at 12:56, on Zulip):

You can make your enum repr(C), to get defined layout

gnzlbg (Dec 12 2019 at 12:56, on Zulip):

you can control how many bits the discriminant of an enum takes already as well

Thom Chiovoloni (Dec 12 2019 at 12:56, on Zulip):

that disables layout optimizations though no? turns into tagged union

gnzlbg (Dec 12 2019 at 12:58, on Zulip):

yep, but if you do:

#[repr(C)] struct Heap(NonNull<u8>, usize, ...);
#[repr(C)] struct Sso(...);
struct Enum { Heap(Heap), Sso(Sso) }

then the layout of the variant fields is fixed

Thom Chiovoloni (Dec 12 2019 at 12:58, on Zulip):

anyway i'm not really interested in the most clean way to write it, i think the stdlib's string (and vec+rawvec) is very clean. i'm intersted in seeing what optimizations i can make work in rust to prove a point

Thom Chiovoloni (Dec 12 2019 at 12:59, on Zulip):

additionally i'm interested in what kinds of optimizations rust can do that c++ can't, layout optimizations are one of them, but honestly all bitpatterns in this are fully occupited

gnzlbg (Dec 12 2019 at 12:59, on Zulip):

Rust enums have guaranteed layout optimizations, if you have an enum with the right niches, which you can have, then it is possible for Option<SSoString> and SsoString to have the same size

Thom Chiovoloni (Dec 12 2019 at 12:59, on Zulip):

yeah. that's true for my current implementation

gnzlbg (Dec 12 2019 at 13:00, on Zulip):

without validity issues, and with the safety of enums

gnzlbg (Dec 12 2019 at 13:00, on Zulip):

If you don't care about that, just write struct SsoString(NonZeroU8, [u8; ...]); and deal with raw bytes, but then you are kind of on your own.

Thom Chiovoloni (Dec 12 2019 at 13:01, on Zulip):

i don't have validity issues according to everybody else who's said anything about this. and i could remove the case that's dodgy at the cost of an extra test if it turns out that it's not leglal

Thom Chiovoloni (Dec 12 2019 at 13:02, on Zulip):

i mean literally my string is

#[repr(C)]
pub(crate) struct String {
    data: NonNull<u8>,
    len: usize,
    cap_flags: CapFlags,
}

as i said in the issue. when it's small i just treat differently

gnzlbg (Dec 12 2019 at 13:03, on Zulip):

As I said, you can do that, in the same way that you can treat all your types as a [u8; N] buffer and disable all type safety.

Thom Chiovoloni (Dec 12 2019 at 13:04, on Zulip):

yeah, i mean i didn't say it was great, it's also not actually an issue for most of the code

gnzlbg (Dec 12 2019 at 13:04, on Zulip):

But to me the use case you posted read like an XY problem. You have an enum, that you don't know how to express in Rust. AFAICT it is possible to express it in Rust, so the usecase itself isn't that interesting to me at least, like, you should just use an enum instead.

Thom Chiovoloni (Dec 12 2019 at 13:05, on Zulip):

yeah i can tell you think it's an XY problem. honestly, it's obvious you think i don't know what i'm doing :/. i've written plenty of rust, been following UCG, and done similar things in c++ many times.

Thom Chiovoloni (Dec 12 2019 at 13:07, on Zulip):

i agree that it's not very clean to interpret a struct as a bag of bytes. i'm also not interested in fighting with the compiler to attempt it to lay something out the way i've proven it out on paper

gnzlbg (Dec 12 2019 at 13:07, on Zulip):

you don't need to talk about SSO at all to ask whether transmute::<&mut NonNull<u8>, &mut usize> allows you to write zero through the &mut usize or not

Thom Chiovoloni (Dec 12 2019 at 13:08, on Zulip):

honestly it was an attempt at heading off suggestions of XY problems :sigh:

Thom Chiovoloni (Dec 12 2019 at 13:08, on Zulip):

but also to give context why it might be useful for my case

Thom Chiovoloni (Dec 12 2019 at 13:11, on Zulip):

anyway since i have you here, re:mimallocator (i'm the one that filed the big pr that you don't want to review lol), i could reduce the size of the PR by doing separate things for the heap-local api and the global one, but i'm not sure that would actually make "does this api seem reasonable" any easier to answer.

Thom Chiovoloni (Dec 12 2019 at 13:12, on Zulip):

(also if you want to talk in PM or another channel that might better, i guess, )

centril (Dec 16 2019 at 10:45, on Zulip):

@gnzlbg rustc_layout_scalar_calid_range_start is for internal use only. It's perma-unstable

gnzlbg (Dec 16 2019 at 10:45, on Zulip):

so?

gnzlbg (Dec 16 2019 at 10:46, on Zulip):

Its the only way to specify niches for scalar values

gnzlbg (Dec 16 2019 at 10:46, on Zulip):

If you want a NonMaxU32 its the only feature you can use

gnzlbg (Dec 16 2019 at 10:46, on Zulip):

and that's a very reasonable thing to want

gnzlbg (Dec 16 2019 at 10:47, on Zulip):

I find the Small String optimization is also a very reasonable use case of user-defined niches

gnzlbg (Dec 16 2019 at 10:49, on Zulip):

And I find it more reasonable to use that feature than to workaround it using completely illogical types to try to approximate the same effect

gnzlbg (Dec 16 2019 at 10:50, on Zulip):

You should choose types that give meaning to your bits in memory, and not puzzle types with wrong meanings to get some particular combinations of niches, and then unsafe-code your way out of that situation

centril (Dec 16 2019 at 10:55, on Zulip):

@gnzlbg I don't think the attribute should be recommended for general use. We might change how it behaves at any time or even remove it entirely

gnzlbg (Dec 16 2019 at 10:56, on Zulip):

I'm not recommending it for general use, I'm pointing out that it is the best tool we currently have for the job

gnzlbg (Dec 16 2019 at 10:56, on Zulip):

If you don't agree with that, suggest a better tool

gnzlbg (Dec 16 2019 at 10:57, on Zulip):

Worse tools had already been suggested

centril (Dec 16 2019 at 10:57, on Zulip):

@gnzlbg I would be happy if you just noted that it was perma-unstable

centril (Dec 16 2019 at 10:57, on Zulip):

When mentioning it

gnzlbg (Dec 16 2019 at 10:57, on Zulip):

So if we remove it, how will all NonZero / NonNull types work ?

gnzlbg (Dec 16 2019 at 10:58, on Zulip):

Do all of them become lang items? Or does this tool get replaced with a different one ?

gnzlbg (Dec 16 2019 at 10:58, on Zulip):

I don't see any mid term (~4 years) future were this tool is replaced with something less powerful

centril (Dec 16 2019 at 10:59, on Zulip):

I'm not saying it will be changed because there is no need to from a compiler perspective

gnzlbg (Dec 16 2019 at 10:59, on Zulip):

And well its unstable, and requires the rustc_attrs features, so those using it already know that it isn't stable, and rustc-only

centril (Dec 16 2019 at 11:02, on Zulip):

Well if you want to take the risk of an attribute which we are free to remove or fundamentally change then by all means.

gnzlbg (Dec 16 2019 at 11:04, on Zulip):

I mean, that's technically true for all nightly features / attributes.

gnzlbg (Dec 16 2019 at 11:05, on Zulip):

No matter how far a nightly feature is on the path towards stabilization, before its stabilized a 180 turn can always happen.

gnzlbg (Dec 16 2019 at 11:05, on Zulip):

Like, why do we have specialization and not rustc_specialization

gnzlbg (Dec 16 2019 at 11:05, on Zulip):

A lot of nightly crates use that feature, and chances are that if the feature ever sees more progress, most of them will break.

centril (Dec 16 2019 at 11:08, on Zulip):

Technically yes but in practice no. Specialisation is unstable for very particular reasons whereas rustc_attrs features are never intended to be stabilised

gnzlbg (Dec 16 2019 at 11:09, on Zulip):

If a feature isn't stable, we can break it, and that's a risk. But trying to quantify how big is the risk in using the different nightly features in some code is quite hard. Some people think that the closer the feature is to stabilization the lesser the risk. Yet for my code, the closer a feature is to stabilization, the higher the chances that it gets iterated, and breaks.

centril (Dec 16 2019 at 11:16, on Zulip):

@gnzlbg all I really want is that it be mentioned that a feature is perma-unstable to give users a fair warning, that's all

centril (Dec 16 2019 at 11:17, on Zulip):

I think I'm going to exit this conversation for now

gnzlbg (Dec 16 2019 at 11:17, on Zulip):

@Thom Chiovoloni this feature is perma-unstable, which means that there is no path for it to stabilization

gnzlbg (Dec 16 2019 at 11:19, on Zulip):

Its been available for years, it hasn't changed at all, there aren't AFAIK any plans to change it either, and it is required to support stable functionality that cannot be removed, which means that if this feature is removed, it needs to be replaced with something else that is at least able to support the current stable functionality, yet might not be able to support current unstable use cases (like the SSO).

Lokathor (Dec 16 2019 at 16:45, on Zulip):

@centril Why is it permanently unstable? It seems extremely useful. I understand it would need an RFC and such, but is there any other reason to not make it available to the public?

Thom Chiovoloni (Dec 16 2019 at 17:29, on Zulip):

Yeah, I was familiar with the attribute before it was mentioned and was aware it was perma-unstable.

centril (Dec 16 2019 at 17:57, on Zulip):

@Lokathor because the form it exists in is very suboptimal and const generics is necessary to provide something good

gnzlbg (Dec 16 2019 at 21:50, on Zulip):

@centril i thought about it a bit, and i'm not sure const generics suffices to provide something good

gnzlbg (Dec 16 2019 at 21:51, on Zulip):

If we want to say that an integer only is valid in a particular range, we can emulate a quite good wrapper with const generics (e.g. struct Int<T: Int, const From: T, const To: T>(T);), but we can't give the type a niche with const generics I think

gnzlbg (Dec 16 2019 at 21:51, on Zulip):

Or at least I wouldn't know how.

gnzlbg (Dec 16 2019 at 21:52, on Zulip):

we'd need to be able to specify that the T in that wrapper only takes values in range [From; To), and that feels quite a bit more like full-fledged dependent types

Lokathor (Dec 16 2019 at 21:57, on Zulip):

I don't think that generics need to be involved at all to make it a useful ability to have, but I suppose I see why would want to do nothing in the hope that you don't trip over a forward compatibility problem.

comex (Dec 17 2019 at 06:56, on Zulip):

With NonZeroU32, you have fn new(n: u32) -> Option<NonZeroU32>, which does a runtime check, and unsafe fn new_unchecked(n: u32) -> NonZeroU32, which omits it at the cost of unsafety. No reason you can't do the same thing for @gnzlbg's Int.

comex (Dec 17 2019 at 06:56, on Zulip):

What dependent types would give you is the ability to safely skip the runtime check in more situations.

comex (Dec 17 2019 at 06:56, on Zulip):

(fixed)

comex (Dec 17 2019 at 06:57, on Zulip):

Which I personally think would be quite nice, but that's hardly necessary if we just want to replicate the API of NonZeroU32 and friends in a way that lets you specify any range.

gnzlbg (Dec 17 2019 at 11:39, on Zulip):

@comex I wasn't worried about avoiding run-time checks, what the use cases discussed here require is for Option<T> to have the same size as T, which is something that the const generics solution by itself doesn't give you, because Int(T) has the same niches as T, which for i32 would be none.

gnzlbg (Dec 17 2019 at 11:40, on Zulip):

What I claimed is that if we wanted to give struct Int<T, const From: T, const To: usize>(T); niches outside [From, To), we'd need something more than just "const generics".

rkruppe (Dec 17 2019 at 17:40, on Zulip):

I don't believe anyone proposes such a radical and general mechanism (at least, not specifically for replacing scalar_valid_range), one idea I've seen floated around is that there would would be a lang item or something that looks more or less like Int and has a scalar valid range defined by the values of its two const parameters.

Lokathor (Dec 17 2019 at 19:06, on Zulip):

Yeah I think for most purposes people need only the simple version. In fact just having "NonMaxU32" and similar would often be enough

comex (Dec 17 2019 at 20:51, on Zulip):

Yeah, I was assuming it would just be a lang item.

gnzlbg (Dec 18 2019 at 15:10, on Zulip):

There is a library called optional that lets the user customize the niche of a T by allowing users to specify a Noned bit-pattern (a niche)

gnzlbg (Dec 18 2019 at 15:11, on Zulip):

So you can, for example, implement Noned for struct U32(u32); and give it 42 as the None bit-pattern.

gnzlbg (Dec 18 2019 at 15:15, on Zulip):

I've no idea if the general problem of allowing users to specify niches on, e.g., the U32 type itself is a problem worth solving. but at least as currently written, this approach does not support specifying an invalid bit-pattern as the None one (cc @llogiq ), since the Optional<T> type actually stores a T, and thus the "None" bit pattern must be valid for T.

gnzlbg (Dec 18 2019 at 15:16, on Zulip):

The API constraints users to pick a valid value of T as the None value, so this value must be valid.

gnzlbg (Dec 18 2019 at 15:17, on Zulip):

I've seen similar types being written in C++, so apparently it appears to be something that some users want (e.g. see https://akrzemi1.wordpress.com/2015/07/15/efficient-optional-values/).

RalfJ (Dec 26 2019 at 17:34, on Zulip):

@Thom Chiovoloni thanks; I think that kind of code is a good argument for leaving memory untyped as I'd prefer anyway. The main concern here of course is that this will cost us some optimizations. But if C taught us anything it's that being too greedy with optimizations leads to a langauge noone can reason about...

Thom Chiovoloni (Dec 26 2019 at 17:39, on Zulip):

Sure, and those optimizations can be somewhat "made up" by leaving libraries more free to perform different optimizations, which can often have at least as substantial of an impact (if not more) than compiler-aided ones. The "typed memory" notion of C and C++ is, without a doubt, one of the biggest problems with those languages (Take a look at how many large c++ projects have -fno-strict-aliasing enabled; it's more than one might naively think, even ones with relatively high code quality like web browsers).

Last update: Jan 21 2020 at 08:25UTC