Stream: t-compiler/const-eval

Topic: `ConstSafe` and bitwise copies


ecstatic-morse (Nov 05 2019 at 18:03, on Zulip):

ConstSafe and ConstRefSafe seem to be the framework we're planning to use to enable heap allocation in constants. Am I correct in saying that ConstSafe is a requirement on types? Does ConstSafe imply that something is bitwise copyable?

ecstatic-morse (Nov 05 2019 at 18:05, on Zulip):
const V: Vec<i32> = Vec::new();

fn main() {
    let x = V;
    let y = V;
}

This code will compile on 1.39 stable, but obviously not all Vecs are ConstSafe, just the one returned from Vec::new

ecstatic-morse (Nov 05 2019 at 18:06, on Zulip):

How do we differentiate Box::new(1) from Vec::new() when both would be const fn under most proposals for heap allocation in consts? We still want to be able to do const B: &Box<i32> = &Box::new(1); after all.

ecstatic-morse (Nov 05 2019 at 18:07, on Zulip):

cc @WG-const-eval @RalfJ

ecstatic-morse (Nov 05 2019 at 18:09, on Zulip):

Do we need an additional effect here? i.e. const(ref) fn new(x: T) -> Box<T>

RalfJ (Nov 05 2019 at 19:20, on Zulip):

uh... maybe we were a bit too quick to stabilize Vec::new as a const fn? Cc @oli

ecstatic-morse (Nov 05 2019 at 19:27, on Zulip):

I think this became a problem as soon as we allowed types with custom Drop impls to be returned from a const fn while allowing Drop types in the final value of a const. Users can write stable const constructors for their own container types today.

RalfJ (Nov 05 2019 at 19:31, on Zulip):

I dont see a relation to Drop here TBH?

RalfJ (Nov 05 2019 at 19:31, on Zulip):

one could have a silly Vec with heap allocations that doesnt drop

ecstatic-morse (Nov 05 2019 at 19:31, on Zulip):

Uh yeah, I guess I really mean non-Copy?

RalfJ (Nov 05 2019 at 19:32, on Zulip):

I mean for Copy types its trivial that we can bit-copy them^^

ecstatic-morse (Nov 05 2019 at 19:33, on Zulip):

NeedsDrop is what's talked about in qualify_consts

RalfJ (Nov 05 2019 at 19:33, on Zulip):

I thought thats for other purposes?

RalfJ (Nov 05 2019 at 19:33, on Zulip):

it (a) protects against calling non-const-drop impls during const eval, and (b) affects promotion

RalfJ (Nov 05 2019 at 19:33, on Zulip):

but the final value of a const is explicitly exempt from it

ecstatic-morse (Nov 05 2019 at 19:36, on Zulip):

Basically, by writing const fn, you are implicitly stating that the returned value is bitwise copyable, even if the returned type is not Copy.

ecstatic-morse (Nov 05 2019 at 19:40, on Zulip):

Well, even by constructing something in a const at all. E.g., if I have a Fd(u32) that closes a file descriptor when dropped (and thus can't be copy) and I put it in a const, bad things will happen.

RalfJ (Nov 05 2019 at 19:46, on Zulip):

well in a const you cant open a file to get a file descriptor^^

ecstatic-morse (Nov 05 2019 at 19:48, on Zulip):

Right, my point is that being in a const allows you to bitwise copy things that are not Copy. While this is fine for now, it means we need additional machinery to enable things that are really really not Copy in a const context. We can't just reuse const fn for this.

ecstatic-morse (Nov 05 2019 at 19:51, on Zulip):

Alternatively, we could do less bitwise copying of consts, and have it really act like you copy-pasted the initializer into your code. Like you, I think those semantics may be unworkable, but it should be discussed at least.

ecstatic-morse (Nov 05 2019 at 19:54, on Zulip):

How would my idea work inside a loop for example? loop { x = CONST_BOX; }

ecstatic-morse (Nov 05 2019 at 19:54, on Zulip):

(it doesn't XD)

RalfJ (Nov 05 2019 at 19:55, on Zulip):

^^

ecstatic-morse (Nov 05 2019 at 19:56, on Zulip):

Well I mean, we could make it do something, but like you said, nothing about it would be const

RalfJ (Nov 05 2019 at 20:08, on Zulip):

well, the plan might be to first inspect the final value of the constant, and only if that contains any heap pointers ConstSafe becomes relevant

RalfJ (Nov 05 2019 at 20:09, on Zulip):

with const changing the value of the constant already is semver-breaking anyway as there could be arbitrary compile-time dependencies

ecstatic-morse (Nov 05 2019 at 20:16, on Zulip):

Delaying all ConstSafe errors until MIRI is run seems ungreat to me. Also this means generic consts can be ConstSafe for some T but not others? I think being explicit is better here.

RalfJ (Nov 05 2019 at 20:39, on Zulip):

well it's kind of too late for an entirely type-based solution, it seems...

ecstatic-morse (Nov 05 2019 at 20:43, on Zulip):

Like I said, I think we would need an effect, a modifier on top of const that says "this code can execute at compile time, but its result is not necessarily bitwise copyable". If we don't wish to do this at compile-time.

oli (Nov 06 2019 at 08:44, on Zulip):

well, the plan might be to first inspect the final value of the constant, and only if that contains any heap pointers ConstSafe becomes relevant

as @eddyb noted on the const heap issue that won't work for associated constants or other generic situtations. Maybe we need to explore the exposed-const-fn-bodies strategy again by bubbling up information like whether it allocates or not

RalfJ (Nov 06 2019 at 10:21, on Zulip):

Like I said, I think we would need an effect, a modifier on top of const that says "this code can execute at compile time, but its result is not necessarily bitwise copyable". If we don't wish to (edit) detect this in the interpreter.

We can automatically determine if the result is bitwise copyable once we have a concrete const value, though

RalfJ (Nov 06 2019 at 10:22, on Zulip):

as @eddyb noted on the const heap issue that won't work for associated constants or other generic situtations. Maybe we need to explore the exposed-const-fn-bodies strategy again by bubbling up information like whether it allocates or not

Then we'll just have to be conservative. Though I am not sure how.

oli (Nov 06 2019 at 10:24, on Zulip):

Right, we can just go for a post monomorphization error, but we don't want to unless we really have to

RalfJ (Nov 06 2019 at 10:27, on Zulip):

So just to be sure I got it, the current proposal is that the final value of a const must be either free of heap ptrs (or even free of ptrs that point to not-yet-interned memory?), or their type must beConstSafe. Is that right? Is that allowing strictly more than we do right now?

RalfJ (Nov 06 2019 at 10:29, on Zulip):

hm, consts can point to not yet-interned memory right now I think, with things like const FOO: &Vec<i32> = &Vec::new();. This is not promotion.

oli (Nov 06 2019 at 10:31, on Zulip):

free of heap pointers is the only condition

RalfJ (Nov 06 2019 at 10:31, on Zulip):

this is the "enclosing scope" stuff

RalfJ (Nov 06 2019 at 10:31, on Zulip):

I wonder if we can backpadel on that for consts...

oli (Nov 06 2019 at 10:31, on Zulip):

not-yet-interned memory does not exist

RalfJ (Nov 06 2019 at 10:32, on Zulip):

not-yet-interned memory does not exist

hu?

oli (Nov 06 2019 at 10:32, on Zulip):

well, it isn't checked at least

oli (Nov 06 2019 at 10:32, on Zulip):

we intern everything that is referenced, but there may be dangling pointers, so we error on these

oli (Nov 06 2019 at 10:32, on Zulip):

but everything that is not dangling is interned

RalfJ (Nov 06 2019 at 10:32, on Zulip):

right but in a const it's actually hard to reference anything

oli (Nov 06 2019 at 10:32, on Zulip):

right

RalfJ (Nov 06 2019 at 10:33, on Zulip):

at least assuming @eddyb goes through with changing promotion in consts to do "MIR extraction" like it does in runtime contexts

oli (Nov 06 2019 at 10:33, on Zulip):

anyway, that seems orthogonal

RalfJ (Nov 06 2019 at 10:33, on Zulip):

then the only remaining way to do that is to use the "enclosing scope" rule, I think. so if we could somehow move into a place where consts never contain any not-yet-interned pointers (or else we error), that would be good. it would make the heap story much easier, anyway.

oli (Nov 06 2019 at 10:34, on Zulip):

the only problem with that proposal is that it has post monomorphization errors just like if you do 0 - T::ASSOC_CONST it may error post monomorphization

oli (Nov 06 2019 at 10:34, on Zulip):

how would that make the heap story easier?

RalfJ (Nov 06 2019 at 10:35, on Zulip):

we'd know that all currently defined const are good

RalfJ (Nov 06 2019 at 10:35, on Zulip):

the problem with the proposed check is that if we define a const using some generic const, we cannot check if the return value contains any heap ptrs

oli (Nov 06 2019 at 10:35, on Zulip):

I don't know how that could be done without significant overcommitting to the interned memory

RalfJ (Nov 06 2019 at 10:35, on Zulip):

so we have to demand ConstSafe

oli (Nov 06 2019 at 10:36, on Zulip):

const FOO: () = { let x = &42; }; should not intern

RalfJ (Nov 06 2019 at 10:36, on Zulip):

const FOO: () = { let x = &42; }; should not intern

we do intern for dead run-time code, what is worse about dead compile-time code?

oli (Nov 06 2019 at 10:36, on Zulip):

if we eagerly intern, that would happen though

oli (Nov 06 2019 at 10:36, on Zulip):

hmm...

oli (Nov 06 2019 at 10:37, on Zulip):

the problem is less about "dead" and more about "not referenced by final constant"

RalfJ (Nov 06 2019 at 10:37, on Zulip):

vs "not referenced by remaining code", for the run-time case. seems symmetric to me.

oli (Nov 06 2019 at 10:38, on Zulip):

ok, so assuming overcommit is fine. how do we know when to intern?

RalfJ (Nov 06 2019 at 10:38, on Zulip):

but I think you are right that it doesn't really help... we cannot just start demanding ConstSafe on opaque consts as we already allow using them without any such bound

oli (Nov 06 2019 at 10:39, on Zulip):

yea, that's the problem with the generic situation and it was a problem at 1.0

RalfJ (Nov 06 2019 at 10:40, on Zulip):

yeah

oli (Nov 06 2019 at 10:40, on Zulip):

we could treat generic constants as !ConstRefSafe, but that won't help for the value of the constant

oli (Nov 06 2019 at 10:40, on Zulip):

oh, no we can't do that either

RalfJ (Nov 06 2019 at 10:40, on Zulip):

that's not backcompat though?

oli (Nov 06 2019 at 10:40, on Zulip):

yea I just realized that, too

oli (Nov 06 2019 at 10:41, on Zulip):

basically we need a new flag similar to the cell checks

RalfJ (Nov 06 2019 at 10:41, on Zulip):

but even if we could break everything, what would we even do? it looks like you basically want all generic and assoc consts to be ConstSafe

oli (Nov 06 2019 at 10:41, on Zulip):

but it also needs to look at function bodies :D

RalfJ (Nov 06 2019 at 10:41, on Zulip):

that's a severe restriction for their usefulnes

RalfJ (Nov 06 2019 at 10:41, on Zulip):

but it also needs to look at function bodies :D

I hope you mean "return value", we should never inspect the code

oli (Nov 06 2019 at 10:42, on Zulip):

well... we have precedent for that in terms of Send and Sync marker traits

oli (Nov 06 2019 at 10:42, on Zulip):

except that noone ever did something like that for functions

oli (Nov 06 2019 at 10:42, on Zulip):

only types

RalfJ (Nov 06 2019 at 10:43, on Zulip):

so you want to basically infer "returns const safe" for functions? might as well infer const fn then. :P

oli (Nov 06 2019 at 10:43, on Zulip):

but even so... that would still be overly restrictive, so we don't want that

oli (Nov 06 2019 at 10:44, on Zulip):

I'm still for the post-monomorphization-errors if they occur rarely

RalfJ (Nov 06 2019 at 10:45, on Zulip):

well we don't know how rare they will be

RalfJ (Nov 06 2019 at 10:46, on Zulip):

maybe Box::new cannot be a const fn after all, just a const(heap) fn...

oli (Nov 06 2019 at 10:47, on Zulip):

that won't help, because then Vec::push will also be const (heap) and a function using Vec::push, even if not putting that thing into its return value will also be const (heap) even if it doesn't need to be

RalfJ (Nov 06 2019 at 10:47, on Zulip):

and values into which const(heap) computations flow may only be used as "full" const values of their type is ConstSafe

RalfJ (Nov 06 2019 at 10:47, on Zulip):

like, ConstSafe is the thing that "masks" the heap effect

oli (Nov 06 2019 at 10:47, on Zulip):

you'll need to look at function bodies to figure out const (heap)

oli (Nov 06 2019 at 10:48, on Zulip):

how is that proposal different from mine?

RalfJ (Nov 06 2019 at 10:48, on Zulip):

functions need to set it

RalfJ (Nov 06 2019 at 10:48, on Zulip):

it's explicit, not inferred

RalfJ (Nov 06 2019 at 10:49, on Zulip):

even if we end up inferring things I think it is good to start with the explicit calculus and work out its rules^^

oli (Nov 06 2019 at 10:49, on Zulip):

hm

RalfJ (Nov 06 2019 at 10:50, on Zulip):

but anyway, first consts should be able to call generic fns before we take on another big project^^

RalfJ (Nov 06 2019 at 10:50, on Zulip):

it seems like Vec::new doesnt make anything worse, so there is no emergency here

oli (Nov 06 2019 at 10:50, on Zulip):

ok this seems nice... it's opt-in, just like if everything were const and we'd have notconst functions

oli (Nov 06 2019 at 10:51, on Zulip):

yea, users can already do that for their own types

oli (Nov 06 2019 at 10:51, on Zulip):

Vec::new just does it for a libstd type

RalfJ (Nov 06 2019 at 10:51, on Zulip):

yeah

RalfJ (Nov 06 2019 at 11:05, on Zulip):

I summarized this at https://github.com/rust-lang/const-eval/issues/20#issuecomment-550260232

RalfJ (Nov 06 2019 at 11:05, on Zulip):

though "effect" seems like a wrong term to me now, this is purely a property of the return value, not the computation

RalfJ (Nov 06 2019 at 11:06, on Zulip):

it's more like "refinement types"

RalfJ (Nov 06 2019 at 11:06, on Zulip):

Vec::new() returns a "Vec that is ConstSafe" (even though the type in general is not)

RalfJ (Nov 06 2019 at 11:06, on Zulip):

while Box::new(0) makes no such extra statement

eddyb (Nov 06 2019 at 14:22, on Zulip):

at least assuming eddyb goes through with changing promotion in consts to do "MIR extraction" like it does in runtime contexts

@ecstatic-morse informed me that we can't with the current setup, but should be trivial once promotion is its own pass

eddyb (Nov 06 2019 at 14:24, on Zulip):

speaking of which, I should go review their PRs :P

eddyb (Nov 06 2019 at 14:25, on Zulip):

@RalfJ I love refinement types and the Vec vs Box distinction makes perfect sense to me but I'm a bit worried there's no "simple" solution in general

Christian Poveda (Nov 06 2019 at 14:55, on Zulip):

Vec::new() returns a "Vec that is ConstSafe" (even though the type in general is not)

Neat!

RalfJ (Nov 06 2019 at 15:40, on Zulip):

@oli tis might also be a good way to think about some of our other type-based analyses... CopyValueOf<Option<T>> refers to those values of type Option<T> that can be copied, like None, etc

Last update: Nov 15 2019 at 20:20UTC