Stream: t-compiler

Topic: REPL: Cranelift backend


Alexander Regueiro (Nov 29 2019 at 16:06, on Zulip):

discussion scheduled for 17:00 GMT with @bjorn3

bjorn3 (Nov 29 2019 at 16:50, on Zulip):

I am back. No problem if you want to wait till 17:00 GMT.

bjorn3 (Nov 29 2019 at 16:50, on Zulip):

Context: https://github.com/rust-lang/compiler-team/issues/213#issuecomment-559055312 and following comments.

Alexander Regueiro (Nov 29 2019 at 17:06, on Zulip):

Hi @bjorn3

bjorn3 (Nov 29 2019 at 17:07, on Zulip):

Hi

bjorn3 (Nov 29 2019 at 17:07, on Zulip):

I think the most important part is saving the user variables

Alexander Regueiro (Nov 29 2019 at 17:08, on Zulip):

Right

Alexander Regueiro (Nov 29 2019 at 17:08, on Zulip):

Debug-printing also to some degree

bjorn3 (Nov 29 2019 at 17:08, on Zulip):

One possibility for the variables is heap allocating them and telling cg_clif to look at a certain location for the locals of the main closure.

Alexander Regueiro (Nov 29 2019 at 17:10, on Zulip):

Hmm

bjorn3 (Nov 29 2019 at 17:10, on Zulip):

The as_debug intrinsic is easily implementable in cg_clif or any other backend I think.

Alexander Regueiro (Nov 29 2019 at 17:10, on Zulip):

Oh great

Alexander Regueiro (Nov 29 2019 at 17:11, on Zulip):

Ideally we don’t heap allocate, since the main point of the cranelift backend is performance.

bjorn3 (Nov 29 2019 at 17:11, on Zulip):

Heap allocation would also make stack pinning automatically work, as the stack variables are never moved

Alexander Regueiro (Nov 29 2019 at 17:11, on Zulip):

But still, there are worse things.

Alexander Regueiro (Nov 29 2019 at 17:11, on Zulip):

True

Alexander Regueiro (Nov 29 2019 at 17:12, on Zulip):

Is there support for stack-pinning at present?

bjorn3 (Nov 29 2019 at 17:12, on Zulip):

A bump allocator should work for the heap

bjorn3 (Nov 29 2019 at 17:13, on Zulip):

Stack-pinning should just work in cg_clif, as it only depends on the stack not moving (why would it) and the MIR being correctly evaluated (duh)

Alexander Regueiro (Nov 29 2019 at 17:13, on Zulip):

Execution would be using SpiderMonkey I presume?

Alexander Regueiro (Nov 29 2019 at 17:14, on Zulip):

Yes fair

bjorn3 (Nov 29 2019 at 17:14, on Zulip):

Cranelift can be used as compiler backend for SpiderMonkey when compiling wasm. It doesn't depend on SpiderMonkey itself.

Alexander Regueiro (Nov 29 2019 at 17:14, on Zulip):

I mean, we could run using one of several engines, but SpiderMonkey by default makes sense right?

Alexander Regueiro (Nov 29 2019 at 17:15, on Zulip):

Yep of course.

bjorn3 (Nov 29 2019 at 17:15, on Zulip):

In cg_clif I use simplejit for JIT execution

Alexander Regueiro (Nov 29 2019 at 17:15, on Zulip):

We want to avoid assumptions in the REPL about the execution engine I suppose.

Alexander Regueiro (Nov 29 2019 at 17:15, on Zulip):

I see.

bjorn3 (Nov 29 2019 at 17:16, on Zulip):

Is is called "simple" as it doesn't support on stack replacement of function for deoptimization and many other things.

Alexander Regueiro (Nov 29 2019 at 17:20, on Zulip):

hmm

bjorn3 (Nov 29 2019 at 17:20, on Zulip):

How is panicking/intrinsics::abort handled by the REPL? I assume it rolls-back all possible changes. However for example written files can't be rolled-back.

Alexander Regueiro (Nov 29 2019 at 17:20, on Zulip):

I haven't worried about that until now. It's not really a concern for the MVP, though I admit more thought will need to be given to it.

Alexander Regueiro (Nov 29 2019 at 17:21, on Zulip):

at the moment I believe it just ignores the current evaluation session.

Alexander Regueiro (Nov 29 2019 at 17:21, on Zulip):

like it never happened (modulo I/O)

bjorn3 (Nov 29 2019 at 17:21, on Zulip):

This would be even worse for cg_clif as intrinsic::abort and SIGSEGV would kill the REPL.

Alexander Regueiro (Nov 29 2019 at 17:21, on Zulip):

true

Alexander Regueiro (Nov 29 2019 at 17:22, on Zulip):

BTW, what do you mean "stack replacement of function"?

Alexander Regueiro (Nov 29 2019 at 17:23, on Zulip):

I am far from expert when it comes to low-level languages / bytecodes / etc.

bjorn3 (Nov 29 2019 at 17:24, on Zulip):

That the execution gets reverted back to an interpreter with all locals available to that interpreter. And when function returns the interpreter returns to the parent. I believe the other way around (interpreter -> jit) is also possible.

Alexander Regueiro (Nov 29 2019 at 17:25, on Zulip):

ah

Alexander Regueiro (Nov 29 2019 at 17:27, on Zulip):

So, can we maybe discuss a high-level overview of how this should work?

Alexander Regueiro (Nov 29 2019 at 17:27, on Zulip):

and differ from the miri backend, in particular?

Alexander Regueiro (Nov 29 2019 at 17:27, on Zulip):

that would help me, I think.

bjorn3 (Nov 29 2019 at 17:28, on Zulip):

ok

Alexander Regueiro (Nov 29 2019 at 17:29, on Zulip):

ta

Alexander Regueiro (Nov 29 2019 at 17:30, on Zulip):

so, the REPL currently drives the compilation itself (doesn't really use rustc_driver)

bjorn3 (Nov 29 2019 at 17:30, on Zulip):

By the way

This would be even worse for cg_clif as intrinsic::abort and SIGSEGV would kill the REPL.

The https://github.com/google/evcxr jupyter kernel decided to thrash all non serde::Serializable values on crashes.

Alexander Regueiro (Nov 29 2019 at 17:31, on Zulip):

I don't know how cg_clif integrates with rustc right now (do you have to drive compilation yourself?)

bjorn3 (Nov 29 2019 at 17:31, on Zulip):

cg_clif uses the same interface as cg_llvm currently does

Alexander Regueiro (Nov 29 2019 at 17:31, on Zulip):

but regardless, because of the way the REPL is set up, I guess I can just depend on rustc_codegen_cranelift an replace the normal codegen step with this.

Alexander Regueiro (Nov 29 2019 at 17:31, on Zulip):

okay nice

bjorn3 (Nov 29 2019 at 17:32, on Zulip):

Basically: You pass -Zcodegen-backend=/path/to/librustc_codegen_cranelift.so to rustc

Alexander Regueiro (Nov 29 2019 at 17:32, on Zulip):

evcxr is a nice idea, but a terrible hacky thing I believe :-) hopefully this new REPL will lend itself towards a better Jupiter kernel in time.

Alexander Regueiro (Nov 29 2019 at 17:32, on Zulip):

oh right

Alexander Regueiro (Nov 29 2019 at 17:32, on Zulip):

and it outputs files, necessarily, or it can output to memory too?

Alexander Regueiro (Nov 29 2019 at 17:32, on Zulip):

the Cranelift bytecode

bjorn3 (Nov 29 2019 at 17:33, on Zulip):

Rustc then loads it, and calls __rustc_codegen_backend, which returns a Box<dyn CodegenBackend>

bjorn3 (Nov 29 2019 at 17:34, on Zulip):
  1. Cranelift doesn't have a bytecode, only a text format.
  2. When using cranelift-simplejit, the executable code gets generated on the fly into memory and a pointer to a function can then be retrieved
Alexander Regueiro (Nov 29 2019 at 17:35, on Zulip):

aha, shows me ignorance about Cranelift, sorry :-)

Alexander Regueiro (Nov 29 2019 at 17:35, on Zulip):

interesting

bjorn3 (Nov 29 2019 at 17:35, on Zulip):

evcxr is a nice idea, but a terrible hacky thing I believe :-)

Indeed. It even parses the rustc error messages.

Alexander Regueiro (Nov 29 2019 at 17:35, on Zulip):

cranelift-simplejit sounds great. we should use this by default, at least (maybe the only executor for now)

Alexander Regueiro (Nov 29 2019 at 17:35, on Zulip):

yep I saw that hah (error message parsing)

bjorn3 (Nov 29 2019 at 17:36, on Zulip):

aha, shows me ignorance about Cranelift, sorry :-)

I didn't expect you to know everything about Cranelift :)

Alexander Regueiro (Nov 29 2019 at 17:37, on Zulip):

so, simplejit is somehow passed as an option for the rustc_codegen_cranelift backend?

Alexander Regueiro (Nov 29 2019 at 17:37, on Zulip):

as a .so too, perhaps?

bjorn3 (Nov 29 2019 at 17:38, on Zulip):

Yes, you set the "SHOULD_RUN" env var: https://github.com/bjorn3/rustc_codegen_cranelift/blob/65f69d10853be9e386749b08c26abd4e84193f25/src/driver.rs#L22

Alexander Regueiro (Nov 29 2019 at 17:38, on Zulip):

aha

bjorn3 (Nov 29 2019 at 17:39, on Zulip):

cranelift-simplejit sounds great. we should use this by default, at least (maybe the only executor for now)

Miri is nice too, as that will show you many cases of UB if you are prototyping something.

bjorn3 (Nov 29 2019 at 17:40, on Zulip):

And it has better fault-tolerance (rollback instead of crashing the repl)

Alexander Regueiro (Nov 29 2019 at 17:40, on Zulip):

exactly my thoughts

Alexander Regueiro (Nov 29 2019 at 17:41, on Zulip):

also, we may build in advanced debugging capabilities that are only possible in the miri backend.

Alexander Regueiro (Nov 29 2019 at 17:41, on Zulip):

but that's post-MVP obviously

Alexander Regueiro (Nov 29 2019 at 17:41, on Zulip):

I have plans for various features after the MVP, like value and type holes, but I think they should be backend-independent.

bjorn3 (Nov 29 2019 at 17:42, on Zulip):

Yeah, priroda support would be nice. (disclaimer: I got write perm)

Alexander Regueiro (Nov 29 2019 at 17:42, on Zulip):

okay nice

Alexander Regueiro (Nov 29 2019 at 17:42, on Zulip):

so, the idea is, once JIT execution finishes, cg-clif can return a set of live variables? this is already possible, right?

Alexander Regueiro (Nov 29 2019 at 17:43, on Zulip):

and we know their types, so we can recursively serialise all the memory they reference?

bjorn3 (Nov 29 2019 at 17:43, on Zulip):

How would you implement value holes in cg_clif? There may not be an invalid value for the type. (MaybeUninit<u8>)

Alexander Regueiro (Nov 29 2019 at 17:44, on Zulip):

it would be done at the MIR level in fact, as I envisage, so by the time it gets to cranelift, there is nothing uninitialised, no holes, etc., until they get filled later :-)

bjorn3 (Nov 29 2019 at 17:44, on Zulip):

so, the idea is, once JIT execution finishes, cg-clif can return a set of live variables? this is already possible, right?

That should be on the MIR level I think.

bjorn3 (Nov 29 2019 at 17:45, on Zulip):

and we know their types, so we can recursively serialise all the memory they reference?

No, cough ptr-int casts cough

Alexander Regueiro (Nov 29 2019 at 17:46, on Zulip):

heh

Alexander Regueiro (Nov 29 2019 at 17:46, on Zulip):

but wait: is the idea is the idea to keep all the memory around after JIT finishes, so serialisation isn't needed at all in theory? (sorry for my naivety)

Alexander Regueiro (Nov 29 2019 at 17:46, on Zulip):

obviously we can't really do this with miri

Alexander Regueiro (Nov 29 2019 at 17:47, on Zulip):

but simplejit...

bjorn3 (Nov 29 2019 at 17:47, on Zulip):

but wait: is the idea is the idea to keep all the memory around after JIT finishes, so serialisation isn't needed at all in theory? (sorry for my naivety)

Yes, except for crash survival

bjorn3 (Nov 29 2019 at 17:48, on Zulip):

Except that that would need to serialize way more (c memory, open fds, memmaps, ...)

bjorn3 (Nov 29 2019 at 17:48, on Zulip):

fork() should work I think

Alexander Regueiro (Nov 29 2019 at 17:50, on Zulip):

yep true

Alexander Regueiro (Nov 29 2019 at 17:50, on Zulip):

serialisation would also be nice to have feature-parity with using the miri backend (where you can actually save REPL sessions to disk and load them back up again later!)

Alexander Regueiro (Nov 29 2019 at 17:50, on Zulip):

but maybe not an MVP thing for the Cranelift backend

bjorn3 (Nov 29 2019 at 17:51, on Zulip):

serde::Serialize would be the most reliable

bjorn3 (Nov 29 2019 at 17:52, on Zulip):

How does the REPL handle compilation of extern crates?

bjorn3 (Nov 29 2019 at 17:52, on Zulip):

Would it be possible to force compilation as dylib?

Alexander Regueiro (Nov 29 2019 at 17:52, on Zulip):

well, I use rustc's in-built serialisation framework right now, for obvious reasons... but I believe that is migrating over to serde anyway, at some point

Alexander Regueiro (Nov 29 2019 at 17:52, on Zulip):

I've thought about that a bit

Alexander Regueiro (Nov 29 2019 at 17:52, on Zulip):

but not in any depth

bjorn3 (Nov 29 2019 at 17:53, on Zulip):

cg_clif in jit mode can currently only load dylibs, as it doesn't link rlibs.

Alexander Regueiro (Nov 29 2019 at 17:53, on Zulip):

right now it's all miri-evaluated (there's actually no explicit support for extern crates in the REPL, but in theory that's how it would work)

Alexander Regueiro (Nov 29 2019 at 17:53, on Zulip):

right

Alexander Regueiro (Nov 29 2019 at 17:55, on Zulip):

anyway, is there something special we need to do to tell simplejit where the memory for the previous compilation session was?

bjorn3 (Nov 29 2019 at 17:57, on Zulip):

I don't think that is necessary. Only cg_clif needs to know that to emit clif ir to load the locals from the right place.

Alexander Regueiro (Nov 29 2019 at 17:58, on Zulip):

aha

Alexander Regueiro (Nov 29 2019 at 17:58, on Zulip):

makes sense

Alexander Regueiro (Nov 29 2019 at 17:58, on Zulip):

so how do we tell it?

bjorn3 (Nov 29 2019 at 18:00, on Zulip):

Given that you are already using a custom driver, it should be possible to export another function like __rustc_codegen_backend, but which returns a struct, whose codegen_crate method accepts a map between main closure locals and addresses

bjorn3 (Nov 29 2019 at 18:00, on Zulip):

Or something like that

Alexander Regueiro (Nov 29 2019 at 18:02, on Zulip):

right, this makes sense

Alexander Regueiro (Nov 29 2019 at 18:04, on Zulip):

so, re liveness, there's a slight issue:

Alexander Regueiro (Nov 29 2019 at 18:04, on Zulip):

I currently compute liveness dynamically, with the miri backend

Alexander Regueiro (Nov 29 2019 at 18:04, on Zulip):

which is less conservative than static liveness calculations

Alexander Regueiro (Nov 29 2019 at 18:04, on Zulip):

obviously

Alexander Regueiro (Nov 29 2019 at 18:05, on Zulip):

maybe the solution is to use static liveness for both backends, and just accept the conservativeness?

bjorn3 (Nov 29 2019 at 18:05, on Zulip):

You mean:

let a = String::new();
if rand_bool () {
    drop(a);
}
Alexander Regueiro (Nov 29 2019 at 18:06, on Zulip):

exactly

bjorn3 (Nov 29 2019 at 18:07, on Zulip):

When a variable is possibly uninitialized, it is impossible to use it. However I believe the following works:

let a = String::new();
let b = String::new();
let c = if rand_bool () {
    drop(b);
    &a
} else {
    drop(a);
    &b
};

Which means that the possibly uninitialized variable must not be dropped yet.

Alexander Regueiro (Nov 29 2019 at 18:08, on Zulip):

hah, I wondered what was going on there until you edited it!

Alexander Regueiro (Nov 29 2019 at 18:08, on Zulip):

hmm

bjorn3 (Nov 29 2019 at 18:09, on Zulip):

I think just keeping all variables allocated on the heap would work.

Alexander Regueiro (Nov 29 2019 at 18:09, on Zulip):

what's the default allocator for simplejit, a bump allocator?

bjorn3 (Nov 29 2019 at 18:11, on Zulip):

For code it just mmap's. For stack, it uses the native stack. For codegened code Box::new it uses the same allocator as normally.

Alexander Regueiro (Nov 29 2019 at 18:11, on Zulip):

wait, mmap's? I thought there was nothing on disk...

bjorn3 (Nov 29 2019 at 18:12, on Zulip):

mmap can also be used for creating a large chunk of memory out of the void, not just for files.

Alexander Regueiro (Nov 29 2019 at 18:12, on Zulip):

ah, sorry

Alexander Regueiro (Nov 29 2019 at 18:12, on Zulip):

I see

Alexander Regueiro (Nov 29 2019 at 18:12, on Zulip):

anyway

Alexander Regueiro (Nov 29 2019 at 18:13, on Zulip):

I agree with you now: heap-allocating all locals just makes sense.

bjorn3 (Nov 29 2019 at 18:13, on Zulip):

I made a issue for cg_clif to do all this: https://github.com/bjorn3/rustc_codegen_cranelift/issues/817

Alexander Regueiro (Nov 29 2019 at 18:13, on Zulip):

but you're thinking of using a separate allocator for that, right? a bump allocator?

bjorn3 (Nov 29 2019 at 18:14, on Zulip):

yes, if it has a measurable impact on the execution time. (even ~10ms) as a REPL has to be as fast as possible

Alexander Regueiro (Nov 29 2019 at 18:16, on Zulip):

yep

bjorn3 (Nov 29 2019 at 18:16, on Zulip):

I will post a summary of this discussion on the issue later

Alexander Regueiro (Nov 29 2019 at 18:17, on Zulip):

incidentally, I will probably try to optimise the miri backend anyway, but I doubt we can get any better than 100x native speed.

Alexander Regueiro (Nov 29 2019 at 18:17, on Zulip):

which is fine for many cases (especially considering certain benefits of miri), but equally a big enough slowdown that a Cranelift backend is really worth the effort.

Alexander Regueiro (Nov 29 2019 at 18:17, on Zulip):

okay cool. a summary would be great!

bjorn3 (Nov 29 2019 at 18:18, on Zulip):

is there anything else to discuss?

Alexander Regueiro (Nov 29 2019 at 18:19, on Zulip):

okay, just one other thing came to mind:

Alexander Regueiro (Nov 29 2019 at 18:20, on Zulip):

as part of this struct we could pass into codegen_crate, we would specify both the "user fn" locals <-> memory addresses mapping but also which locals should be kept live (not deallocated), which is calculated ahead of time in some MIR pass, yes?

bjorn3 (Nov 29 2019 at 18:21, on Zulip):

I think it would be the easiest to let the REPL handle all allocation and deallocation of the locals

Alexander Regueiro (Nov 29 2019 at 18:22, on Zulip):

okay

Alexander Regueiro (Nov 29 2019 at 18:22, on Zulip):

so the REPL would have its own bump allocator (there are already one or two crates out there for this, I believe), and just feed in the addresses?

bjorn3 (Nov 29 2019 at 18:22, on Zulip):

yes

bjorn3 (Nov 29 2019 at 18:23, on Zulip):

I will have to leave

Alexander Regueiro (Nov 29 2019 at 18:23, on Zulip):

sorry

Alexander Regueiro (Nov 29 2019 at 18:23, on Zulip):

thanks for your time

Alexander Regueiro (Nov 29 2019 at 18:23, on Zulip):

it's been very useful

bjorn3 (Nov 29 2019 at 18:23, on Zulip):

no problem

Alexander Regueiro (Nov 29 2019 at 18:24, on Zulip):

@bjorn3 I guess Cranelift will have to know not to deallocate any local that's specified in that struct, and just let the caller (REPL) handle it, but that shouldn't be hard to implement I guess.

Alexander Regueiro (Nov 29 2019 at 18:24, on Zulip):

I think we've covered all the main points

Alexander Regueiro (Nov 29 2019 at 18:24, on Zulip):

the reset will be details as we go along

Alexander Regueiro (Nov 29 2019 at 18:24, on Zulip):

anyway, see you later.

bjorn3 (Nov 29 2019 at 18:44, on Zulip):

yes, cg_clif shouldn't dealloc those locals

bjorn3 (Nov 29 2019 at 18:48, on Zulip):

i forgot to mention one thing: currently any jitted function will remain allocated forever and be jitted again during the next compilation round.

bjorn3 (Nov 29 2019 at 18:48, on Zulip):

i think i should teach simplejit how to dealloc specified functions and do that from cg_clif for all functions not potentially reachable from a fn ptr or vtable.

bjorn3 (Nov 29 2019 at 18:49, on Zulip):

as for the recompilation, that will need something else. maybe a static hashmap between instance hash and address?

Alexander Regueiro (Nov 29 2019 at 20:26, on Zulip):

@bjorn3 I see. the problem is, knowing whether a given fn pointer is still hanging around is not something that can be statically analysed, obviously. this would be a very hard problem to solve in the general case, I think. I'm not sure how. However, since only "new" code is sent to Cranelift (including functions), I think you'll only get this problem if the fn definition is changed. (And in that case you may want the old definition of the fn to hang around anyway, although not necessarily.)

Alexander Regueiro (Nov 29 2019 at 20:26, on Zulip):

Still, we can be conservative and keep all versions of a fn around, throughout redefinitions.

bjorn3 (Nov 29 2019 at 20:32, on Zulip):

Actually all functions which would normally end up in a rlib are all codegened by cg_clif, even if unchanged. the incremental cache is currently not used for codegen artifacts. also for simplejit it would be impossible to use the incremental cache, as no files are written. instead it must be kept in memory.

bjorn3 (Nov 29 2019 at 20:33, on Zulip):

for fn pointer liveness i was thinking about assuming that every function which gets turned into a fnptr is assumed to be alive forever.

Alexander Regueiro (Nov 29 2019 at 20:33, on Zulip):

the latter could work yes

Alexander Regueiro (Nov 29 2019 at 20:34, on Zulip):

ah, I see about the incremental compilation...

bjorn3 (Nov 29 2019 at 20:34, on Zulip):

for function redefinitions, i think it is the safest to keep the old function as target of function pointers. somebody might have spawned a thread which called it. (though cg_clif doesnt yet support threads)

Alexander Regueiro (Nov 29 2019 at 20:36, on Zulip):

this seems reasonable I think. this way only new code getting evaluated (current session) which turns a fn into a fn-ptr will make that fn stay around forever in memory

Alexander Regueiro (Nov 29 2019 at 20:37, on Zulip):

so this will probably need to be a MIR pass in rustc itself, which is enabled only in interpreter mode. that can come down the line...

bjorn3 (Nov 29 2019 at 20:39, on Zulip):

it can also be done at the cg_clif end, which has to implement FnDef -> FnPtr casts anyway.

Alexander Regueiro (Nov 29 2019 at 20:40, on Zulip):

hmm

Alexander Regueiro (Nov 29 2019 at 20:41, on Zulip):

maybe this makes more sense, since although checking at MIR level should be equivalent, doing it at Cranelift level is guaranteed to be more reliable, I suppose.

bjorn3 (Nov 29 2019 at 20:59, on Zulip):

Yes, and mir passes are done before monomorphization, so you may not yet know what type a vtable is created for.

Alexander Regueiro (Nov 29 2019 at 21:06, on Zulip):

yeah fair point

Alexander Regueiro (Nov 29 2019 at 21:07, on Zulip):

basically, cg-clif should take care by itself of when to deallocate functions and when not to.

bjorn3 (Nov 29 2019 at 21:11, on Zulip):

yes

bjorn3 (Nov 29 2019 at 21:19, on Zulip):

Opened https://github.com/bytecodealliance/cranelift/issues/1260 for the simpleJIT part of function dealloc.

Alexander Regueiro (Nov 29 2019 at 21:21, on Zulip):

good stuff, cheers

Last update: Dec 12 2019 at 00:45UTC